In our article ‘Ensure your market research is spot on’ we’ve provided a list of things you need to think through to set the parameters for your research project. If you’ve decided that you need quantitative research and that you’re going to do it yourself, then use the following guide to designing, testing and analysing your survey to ensure it provides results you can count on.
Define and shape the questions
Start designing your questionnaire by going back to the main purpose of the survey and what you want to do with the results. The questionnaire you design will need to take people logically through the areas you need to cover. It’s important to ensure that the questions you ask are not biased in any way, that you aren’t leading respondents to give you the answers you want, as doing this will invalidate the results.
It may sound obvious, but if you need to filter by different characteristics, such as age, gender, buying or consumption profile, then remember to include questions which give you the responses you need to do this.
Resist every impulse to add the ‘nice to know’ questions to any research you’re doing. Respondents will be more likely to complete a survey if it’s short and you won’t be left with a chunk of information which you won’t know how to use.
You have two main options for question types: open and closed questions.
Closed questions have pre-set response options. Open questions allow respondents to say anything they like. While closed questions are easier to analyse, they only work well if you have a good idea of the sort of responses people are likely to want to give.
There are a range of ways you can shape closed questions. These can include
- single response – e.g. yes/no; the most important; your favourite; etc
- multiple response – e.g. which of these factors do you consider
- rating scales in numbers or words – e.g. how important
- ranking – e.g. in order of preference or importance
With closed questions which include a list of options, it can be helpful to add a box for ‘Other’. This can be a good check as to whether the predetermined items in the list are the right ones… if you’ve got lots of people choosing other, then your original list wasn’t right. If you do add this option, then it’s helpful to include a place for people to write in what that ‘other’ option is.
Including a rating scale requires you to set the rating options and this brings up the question of the level of sensitivity you want to include. With a numeric scale, do you want 1-10 or 1-5? With word scales, do you include a neither/nor midpoint? To decide on what you need, go back to thinking about how you would interpret the results and which responses will give you a meaningful answer to act on.
Open questions can be invaluable in gaining a new understanding into thoughts, feelings and actions. As the responses are freeform, it will be hard to put numbers or percentages on the responses, but they can show you areas which need looking in to further. Adding in some of the responses as quotes can also help lift your survey report and help readers quickly grasp what the survey is telling you.
If you are using open questions, then try to point people at the sort of things you’d like them to comment on. As an example, rather than using ‘Please give the reason for your response?’ ask them ‘What was it about the product, packaging or the service you received that made you say that?’
It can be important to have some open questions in your survey as it can allow respondents to get something off their chest or bring up something you haven’t even considered.
It’s worth remembering that people can get bored easily, so try to use different ways of asking your questions as well as keeping the questionnaire as short as you can.
Finally on this, ensure the language you are using is right for the audience you are surveying. Depending on who they are, it can be worth considering using illustrative images in place of words.
Sometimes you need to ask people about things they might consider to be sensitive or where they might be ‘shy’ of sharing their real opinions… just ask those who have worked on election opinion polls over the last couple of years!
Emphasising the anonymity of responses can be helpful here and it can also be better to provide a tick box closed question than to ask for an open response.
There is no way of guaranteeing you will get honest answers, so if you have asked questions about sensitive topics, bear that in mind when reviewing the results and flag this up in your reporting.
Take time to test the questionnaire
Before putting your questions to your audience, it’s worth getting a few people to do a trial run. This can help you pick up any issues with your questionnaire, such as incorrectly routing people through it or using language where the meaning is unclear to respondents in either the question itself or the response options.
Asking your testers to trial the survey using fictional circumstances can be particularly illuminating.
If you’re hoping to get people to complete your survey on-line, test how well it views on mobile and tablet as well as larger screens.
Conducting your survey on-line, using one of the many commercial packages* available, means you won’t need to worry about this aspect as the data will go straight into the system, ready to be analysed. If you do need to enter a lot of data by hand, however, then it’s sensible to conduct some random tests on the data prior to undertaking the analysis to ensure that the information has been input correctly.
Interpreting the results
Whenever you report on research results or discuss the findings with other people, always remind them that you are looking at a sample of the population which might not be truly representative of the audience it is drawn from.
Interpret the results with a sensitivity to the limitations of the data you’ve managed to collect and bearing in mind what the survey was set up to find. If you’ve only got a small sample to work with then resist quoting percentages, use raw numbers instead e.g. 7 of the 12 respondents. If you do use percentages, highlight the greater potential for error due to the small number of respondents.
Use appropriate charts to illustrate the findings. Interspersing these with quotes and relating the findings back to what they might mean in relation to the questions you are seeking to answer.
Finally, bear in mind some of the issues I identified in my article on ‘The Wrong Numbers’, which highlights mistakes that can be made when interpreting data of any type. Use your gut feel. If the results feel wrong in some way, treat them with caution and consider repeating the survey to see if they are repeated.
Using a commercial package
There are lots of great software packages available which can make it very easy to set up, distribute and analyse your own quantitative market research. For questionnaire design, they can offer preformatted question types and advice on wording, as well as different options for sharing the survey with respondents. Many of them have low or no-cost price plans which are suitable for small scale use by SMEs.
The best one for you to use depends on what you’re anticipating doing. Things to consider before deciding which one to go with include: limits on the number of questions you can include in any one survey, the number of responses you can analyse, the ability to export results into another appropriate package, options for the design and branding of the questionnaire and the pre-set question formats available.
In summary, undertaking your own quantitative market research can be a cost-effective way to gain insights which will inform your decision making. Just ensure you ask the right questions of enough of the right people to provide you with a sound base.
If you would like any help with your market research then please get in touch