You have now sent out your survey and the results have started to come in. It is important to get the number of responses required to obtain a representative measurement. If you are not satisfied with the response rate you can send out reminders to those who have not yet replied. This often leads to you getting around a further 30 per cent of the replies, so if you feel that the number of responses you have received is not sufficient for a survey, reminders are a good way of boosting this.
Once some time has passed since the final reminder it is time to start analysing the results. However, if the responses continue to drop in after the deadline you should wait for a further period of time so as not to irritate those who are late and who have spent time responding.
Even if you have sent out reminders a number of times you can never expect a 100 per cent response rate; nor, indeed, do you usually need one. The response rate can differ greatly depending on how you put the questions, and to whom you put them.
Personal contact produces a higher response rate
The closer your relationship to the person to whom you are putting the question, the greater the chance of receiving a response. Conversely, if the question arrives unannounced, or through a pop-up window, the chances are reduced. If your question relates personally to your respondent, or if the respondent can see a personal benefit from replying, the chances will increase.
To increase the response rate, always be as service-minded as possible. If the question is sent to a company that operates throughout the Nordic region, for example, respondents should be able to choose in which language they wish to reply.
Another commonly used and effective way of attracting respondents is to offer scratch cards, or to send a gift to a charitable cause if people take part. However, it is important to check the laws and regulations that apply in the country where the question is to be published.
There is no basic rule as to what is considered a representative sample; it differs from subject to subject. However, if you are worried that your group has a lopsided distribution, you can conduct a so-called non-response analysis, where you check which group has had a low response rate. You then perform matching, where you seek a number of people from the missing target group and conduct a separate supplementary survey.
Link the analysis to the aim
– but be open to surprises
As we discussed in the first part, the analysis part must be linked to the whole aim of the survey. Depending on what you wish to get out of the survey, you may either require extensive statistical calculations or just a simple frequency presentation. With surveys that contain both qualitative and subjective parts, the responses may need to be interpreted separately.
Sometimes, the respondents may need to be divided into sub-groups; bear in mind, though, that the number of respondents in each group should not be too low. A rule of thumb is that there should not be fewer than five.
Conducting a survey can produce interesting spin-off effects, and the most interesting conclusions do not always tie in with the aims of your survey. You may perhaps discover a new and exciting behaviour pattern in your customers that you can make use of in your marketing. When you present your survey it can be interesting to the audience to be told these new facts. However, you should be careful to focus on the main aim of the survey and to provide the answers relating to the relevant results that the audience is expecting. One good approach may be to mention the interesting new findings at the end and leave the matter open for discussion.
You should also be careful at the presentation to make sure that your results are intelligible and easy to interpret. Never-ending rows of figures are of no benefit to anyone if they do not give rise to an interesting discussion. You should also avoid being tempted by the opportunities provided by technology to create exciting animations or three-dimensional graphs – or, at least, you should consider how these would help the listener to better understand your reasoning.
Increasing the response rate with good wording and design
The actual design of the questions and of the survey are very important factors in increasing the response rate, but they also ensure that you obtain the information that you and your company need to make the correct strategic decisions.
Problems analysing survey responses can almost always be traced all the way back to the actual design of the questions. The key to good design is having well-defined aims for your survey. If the aim of the survey can be expressed in just a few clear, readily understandable sentences, then it becomes much easier to design accurate questions.
The best way to clarify the aim of your survey is to give thought at an early stage to how you intend using the information obtained. You should do this before you start to design the survey. Why carry out a survey if the results are not going to be put to some use?
It is a good idea to write down the aims of the survey in point form. Every time you are unsure about a question, take a look at the aims and see whether they fit in with the question.
Only design questions that are relevant based on the aims. Avoid the temptation to ask questions just because it would be ‘nice to know’. Distinguish between ‘need to know’ and ‘nice to know’.
An important rule when designing forms is that long ones receive a lower response rate than short ones. A high response rate is the very best way of ensuring that the statistics are credible. A low response rate can give misleading results and ruin the survey. You must therefore do everything you can to maximise the response rate, and keeping the questionnaire short is one of the most effective ways of doing this.
If your survey is long – perhaps running to several pages – try to delete some questions. Having said that, it can be difficult choosing which questions to delete. Read through all your questions and keep the following question in your mind: ‘How will I use this information?’ If you think that the answers will provide important strategic guidance to your company, keep the question. If not, you can discard it.
When designing the questions, by all means involve people who have some form of strategically important position in the company. Their suggestions may improve the questions in terms of hitting the mark. However, you should also be responsive to what they consider is ‘nice to know’ and ‘need to know’ information. Another advantage of involving these people is that, presumably, they will feel more confident about the results if they have been able to take part in the design themselves.
When designing the questions, you should also consider how you will analyse the results afterwards. If you cannot think of a way of analysing the results, or how you will use the information, delete the question.
We all know the importance of making a good first impression. This applies to a great extent to survey forms. The welcome page itself is therefore very important, both in terms of design and choice of words. Try to make the first page as graphically attractive as possible, and choose words that do not feel distant and bureaucratic.
Give plenty of thought to the name of your survey; it should summarise the spirit of the entire survey while at the same time being short. A survey that has a name radiates more credibility than one without.
Be careful to ensure that you provide simple, intuitive instructions for completing the survey. These need to be very simple to understand, so use short sentences and normal (subject-verb-object) word order.
If the survey is too complicated, many people will ignore it.
Increasing the response rate
– getting them to complete the survey
In this part, we will explain what you do to get the people taking part in a survey to want to start the questionnaire in the first place, as well as how to get them to complete it right to the end.
Almost everyone who takes part in a survey wants to know about the time aspect first. You therefore need to tell the participants how long the survey is expected to take to complete. For example, you could write that the ‘survey will take no more than ten minutes to complete’. The respondent then knows how much time to set aside. Be honest with the times – if the time drags on the respondent may stop filling in the survey out of pure spite.
Do not scare away the respondents
Take care to ensure that you begin with some simple questions that are easy to answer and awaken the interest without being embarrassing or over-familiar. If the questions seem boring or positively degrading, many people will not bother to answer them right from the start.
People often check the first few questions in order to decide whether there is any point in completing the survey. Try, therefore, to put some interesting questions in at the start to persuade them to continue.
Use simple language, with normal (subject-verb-object) word order and without using subordinate clauses. It must be possible to read and understand the question quickly. Avoid using any complex or bureaucratic words. If people do not understand the question they may feel both stupid and bored – and, of course, no-one wants to feel like that! Using simpler language also avoids misinterpretations, which can lead to misleading results.
By all means use typographic aids such as bold text, italics or underlining to emphasise important words. However, avoid overusing these as it can give a muddled impression.
Leave an amount of space for the reply that is appropriate to the question. If you want the answer to be as illuminative as possible you must leave enough room for this. You can obtain a lot of valuable ‘bonus information’ if you allow a large reply field. On the other hand, if you do not need a wordy reply you should keep the reply field small; otherwise it just takes up unnecessary additional space.
The important things first
Place the questions that you consider it most important to get replies to in the first half of the survey. Many respondents may be interrupted for some reason (a telephone call or visit to the toilet) and may leave it half-completed. At least then you have answers to the most important questions.
Keep the interest burning! Ideally, of course, we want them to complete the whole survey. One way of achieving this is to vary the nature of the questions so that the questions do not seem monotonous. At the same time, however, you need to be careful to ensure that the survey does not feel too sprawled out. This is where a certain balancing act and a sure instinct are required.
Try to think up some important incentives to get respondents to complete the survey. Is there anything you can offer those who complete the entire survey? A scratch card lottery tends to work extremely well as a motivator. However, if the budget does not allow this you may perhaps offer respondents the chance to get a copy of the report afterwards (assuming that they can be expected to be interested in the results).
Last but not least – test your survey on a few people, ideally within the target group you are aiming at. It is a good idea to sit in when they are completing the survey as it enables you to pick up on the difficulties right away. Feel free to ask the trial subject to tell you when they are having problems. The questions they ask tend to indicate the type of problem that it might be. Make a note of them as they arise and do something about them. Then repeat the test until it appears that there are no further problems.