Sending out a survey to all your email contacts is a simple thing to implement in practical terms. And asking questions that will give the answers you want is usually not a problem either. Doesn't it feel good to get confirmation that you have the best product, that you have put on a great event or that you have the tastiest food of all the restaurants in town?
But getting the answers you really need to develop your business is another matter entirely. Such questions require reflection, planning and accuracy. To then analyse and use the results in the best possible way is another important and delicate task. Wrong questions or incorrect conclusions from your answers can permanently ruin the trust you have built up in your environment, while asking the right questions and correctly analysing the responses can take your business to new levels.
So read carefully through the following list of tips before you send out a survey to customers, guests, members, stakeholders, the public, or whoever it may be.
Before you begin your survey
1. Decide why you want to do a survey. Answering the question "why?" is step one. The most common reason is that information is wanted that will help you as a company, organisation, association or person to develop further in the future. If this is the reason, you should always keep this in mind throughout the survey process. Remind yourself from time to time during the process.
2. Decide on the type of survey. Depending on how you want to develop your business, you can choose between a number of different types of surveys. Decide on one of these and stick with it to avoid giving a fragmented and confusing impression.
3. Do a qualitative survey when you want to target a limited sample that gives in-depth and well-reasoned answers.
4. Do a quantitative survey when you want to provide predetermined answer options and you don't need such nuanced answers. In return, you get a much larger amount of data where you can see what a larger population thinks and trends within different populations.
5. Decide on the form. A digital survey sent via email usually gives the highest response. But in certain circumstances, surveys sent by traditional letter post or in-depth interviews might work better. This is very dependent on the target group that will be answering.
6. Don't be tempted by simplicity. It has become easy and cheap to send out digital surveys. But don't let this be at the expense of quality. Don't ruin your chances by doing something rash.
7. Limit the survey. One of the most common mistakes is wanting to have the answers to all the questions at once. Be aware that you are asking your target group to invest time in you. If you take too much of their time, they will not want to answer. Limit the number of questions to a minimum.
8. Assess feasibility. Think carefully about how workable/difficult your method is. If you see that it will require too many resources, you should back up a few steps.
9. Don't underestimate the time needed. A well-executed survey takes time in all its stages: preparation, implementation, summary of results and analysis. Be sure that you have plenty of time available for every stage.
10. Estimate the cost. A well-executed survey can (but need not) involve costs. Make an estimate of what costs there might be. These may apply to graphic design, writing assistance, consulting service for data analysis, the purchase of email addresses, gifts to participants, and so on. Don't let the cost come as a surprise.
Identify the target group for the survey
11. Decide on the target group. It is easier to design the questions when you know who you want to question. If you are addressing potential customers, for example, the market survey's questions will be designed differently than for existing customers.
12. Do a random sample if you want to be absolutely certain that totally safe conclusions can be drawn for the entire population you want to examine.
13. Don't be fooled by the sample size. Just because you use a large number of individuals, it doesn't necessarily mean that the survey is representative. Spend time on getting the right sample.
14. Choose sampling method. Random sampling means that everyone in the population has the same probability of being selected. Systematic sampling can involve every 50th person in a register. Stratified sampling can mean that the population is divided into a certain age, or a particular gender. Cluster sampling involves natural groupings, such as a school class or a workplace.
15. Use non-random sampling with care. There may be situations where you can use non-random sampling for reasons of cost or where you have no need of totally accurate answers. In such a case, be aware of the following:
16. For telephone directory sampling, you should be aware that only people with phone subscriptions will respond. In addition, only a certain type of working person has the opportunity to answer the phone at certain times of the day. There may be more variables.
17. For a typical sample, you select a number of individuals who you think are typical of the population. But then you are involving your own values, and that should be done with caution.
18. Convenience sampling is done when you select people who are easy to get hold of. They may be family members, neighbours, colleagues, and so on. It is a method that rarely wins scientific acceptance. Only use it if you know that the answers cannot be affected by your relationship to the respondent.
19. Voluntary sampling means that you ask people if they want to participate. Only those who agree to be surveyed respond. Most surveys fall into this category, and can thus be regarded as "non-random".
20. Don't forget the non-respondents. Be careful with drawing conclusions about conditions that you know nothing about. Analysis can only be done on available observations, not on that for which there is no data. Do what you can to minimise non-response.
21. Find out the target group's frame of reference. Questions may need to be formulated differently, depending on whether the market survey is targeted solely at women or solely at men. Similarly, if for example it is aimed solely at people interested in boats, people interested in leisure time, or people in general.
Test the survey
22. Designate a test group. A test group may be needed to detect if the survey contains some hidden problems in its implementation (which it almost always does). The test group should be representatively selected in relation to the entire final group.
23. Do a pilot test. The pilot test is carried out on the group in accordance with the previous point.
24. Evaluate the pilot test and make adjustments. If the evaluation was shown to contain problems, the questions, the questionnaire or some technical detail in the survey tool has to be adjusted. When that is done, you repeat the pilot test.
25. Treat the pilot test with respect. If the pilot test contains abnormalities, these must be corrected. The abnormalities will be even more evident in the main survey.
26. Repeat the pilot test until it is watertight. The last pilot test should function completely without error before it is time to launch the main survey. This is especially true if you are a beginner in the area of surveys. Errors can occur in the most unusual places.
Prepare the respondents
27. Tell them about the time needed. Make it clear to the respondent how long the survey will take. Then he or she can judge for themselves if it's worth spending time on the survey.
28. Let the respondent remain anonymous if you want honest answers. Don't let yourself be tempted by curiosity to know who thinks what.
29. Let the respondent identify him- or herself if you think that you will want to have more information about them after the survey.
30. Make sure that the right person answers. You can't be completely sure that the respondent is the same person you sent the survey to. A manager can, for example, have asked one of his employees to fill in the survey. Always have a space where the respondent fills in their name, and possibly their position.
32. Remind the respondents in advance. You can send out emails in advance to tell them that they are going to be in a survey. The chances are then greater that they will take the time to do the survey when it comes.
33. Ask understandable questions. The questions must be easy to understand. Otherwise the survey will not be completed. A common mistake is to ask questions so complicated that the respondent can't be bothered to complete the survey.
Asking the right questions in your survey
34. Avoid complicated words. This is definitely not the time to show off with difficult words. You then run the risk of putting off the respondents. Use as simple words as possible so that everyone can understand.
35. Ask one thing at a time. A question should only be able to result in one answer. If your question could be interpreted in different ways, choose to ask two questions instead. Otherwise you risk both confusing the respondent and getting a misleading answer.
36. Read the question out loud. Maybe to a friend. You or your friend will then notice if it is strangely worded.
37. Provide adequate response options. If the response options are not sufficient to give a correct answer, you risk antagonising the respondent who might leave the survey half completed. Always make sure that the response options cover the question.
38. Don't ask leading questions. Your questions should be of a "need to know", not "nice to know", nature. You might like to get a response that points in a certain direction. But you should not try to influence the respondent to answer in a certain way. The answers would then be difficult to analyse and misleading.
39. Don't make any groundless assumptions. The question must be based on a neutral assumption. The classic journalist question "Have you stopped beating your wife?", is, for example, unanswerable if you have never beaten your wife.
40. Provide help for difficult questions. If the question is difficult, you should include a "don't know" option or explain that just a quick appraisal is required. Otherwise, the respondent may leave the survey unanswered.
41. Use "don't know" sparingly. Only include the "don't know" option where it is needed. Otherwise, the respondent may choose this option without proper consideration.
42. Avoid value-laden words. Use neutral words in your questions so that there will be as little emotional impact as possible on the respondent.
43. Avoid negation. Questions of the type "Wouldn't it be better..." contain a negation. This can confuse the respondent and also steer the answer in a particular direction.
44. Avoid imagery. When you ask, "Do you feel that you are on your knees because of your work situation?", it can be misinterpreted. Especially if it is directed to a new arrival to Sweden who has not fully mastered the language.
45. Avoid yes/no options in attitude questions. Answers concerning attitudes may need to be qualified.
46. Ask short and concise questions. Once again, remember that you are taking up your respondents' time. Be polite and ask as short questions as possible.
47. Look for precise answers. Try to get the respondent to respond as accurately as possible to your survey questions. It simplifies your analytical work and helps you avoid doubt.
48. Make the questionnaire easy to fill in. The respondent should not have to click back and forth to get to the form. The form should then be as simple and intuitive to complete as possible.
49. Use open-ended questions if you need a balanced and insightful answer. An open-ended question cannot be answered briefly, so be aware that it can be time-consuming to go through the answers.
50. Use closed questions, which give a yes or no answer, if you want to get precise answers from a large target group.
51. Use simple sentences. SVO (subject-verb-object) word order without subordinate clauses or inserted sentences is always preferable.
52. Highlight important words. Write the keywords in the question in bold print. This helps give an understanding of the question.
53. Use questions with response options if you have a large number of respondents and you want to have a high response rate. It's quicker for the respondents to answer these questions.
54. Start your survey with the easy questions. The respondent is then lulled into a sense of security and will be pleased to continue with the following questions. If the first question is too difficult, the respondent may not continue on to the next question.
55. Let the questions follow each other in a natural order. A good question is linked with the previous one. Don't throw the respondent back and forth from one subject to another. Try to gather the questions in thematic groups.
56. Be consistent with rating scales. If the survey is going to contain rating scales, they should be consistent. If the scale is 1-5, stick to this for the entire survey to avoid confusing the respondent.
57. Use an odd number scale if you want to give the respondent an opportunity to select the middle option. On a scale of one to five, three is the middle option. The respondent doesn't then have to choose sides.
58. Use an even number scale if the respondent is to be forced to pick a side. On a scale of 1 to 4, it isn't possible to choose a middle option. The respondent must in that case choose 2, which leans to the left, or 3, which leans to the right.
59. Vary the questions. This may seem contradictory to what has been stated above. But a certain variation makes the survey a little more entertaining. But it must not vary so much so that it appears disjointed.
60. End with personal questions. More in-depth questions about the respondent as a person should be put at the end of the survey. If they are asked too early the respondent may feel the survey is too intrusive.
61. Finish the survey with a comment box. A final comment box can obtain additional information and answers to questions you have not asked in the survey. But remember that processing and evaluating these comments can be time-consuming.
62. Wait for all the answers. Don't begin analytical work until all answers have been received. It can irritate those that have not yet been able to respond and you also risk making a faulty analysis.
63. Use personal contacts. The closer the relationship you have with the respondent, the greater the chance you have of getting the survey answered. Send to the contacts you have, but also keep in mind that the answer can be influenced by your relationship.
64. Use other language versions. If the market survey is sent to the Nordic countries, you should have it translated into the respective languages. If the questionnaire is sent to different European countries, don't use English for all of them. Have them translated into the respective languages.
65. Tempt them with a gift. It can be effective to promise a scratch card or something similar to those participating in the survey. But keep an eye on the laws in the respective countries.
66. Don't forget to say thanks for participating. A survey that doesn't contain a "thank you for your help" when it has been completed is likely to undermine the confidence of the respondents.
67. Send an extra thank you. It doesn't hurt to be extra grateful. After a while, a "thank you" email can be sent saying how important the respondent's answer is for the development of the company or organisation.
68. Don't abuse the target group's patience by sending out too many surveys in a short period of time. Focus on quality rather than quantity, and think carefully through each survey.
Study the results of your survey
69. Remind about the survey. If you don't get as many responses as you want, you can send out a reminder about the survey after a while. You will often get an additional 30 percent of responses.
70. Carry out a non-response analysis if you notice that one of the target groups has a lower response rate than other target groups. Then try to do a separate survey with this target group.
71. Analyse the results. The information you got from the market survey should be examined in an unbiased way. Especially the qualitative parts of it.
72. Let a neutral party examine it. It may be beneficial if an impartial third party compiles the results of the survey so that the analysis is not affected by your wishful thinking.
73. Determine how the results should be used. Go through with your management team how the survey results will be used for your business.
74. Link the analysis to the aim. The aim should always be kept "top of mind" when you start analysing the results.
75. Be careful with percent terms. Percent and percentage points are two concepts that can result in big misinterpretations if you can't distinguish between them. An increase or decrease of 10 percent or 10 percentage points can be a huge difference.
76. Keep track of cause and effect. Just because two variations occur simultaneously, it doesn't necessarily mean that one is dependent on the other.
77. Avoid over-interpretation. Just because a curve is going in a certain direction, it doesn't necessarily mean that it will always continue in this direction.
78. Be open to surprise. Even though the survey should always focus on the aim, you should be receptive to any additional information that it may provide. Sometimes this information can be difficult to spot at first sight.
79. Have respect for the subject of statistics. Interpreting statistics is a scientific discipline along with mathematics. A fully-fledged statistician has gone through many years of university studies. As a layman, it's easy to draw wrong conclusions that have far-reaching consequences. If you are not sure, you should consult a professional.
Presenting the results of the survey
80. Make a report. Don't rest on your laurels because you have been told what you want to know. Everyone in your business must have access to the information in a readily comprehensible form. A well-written report will facilitate this. Save the results in a database of experience.
81. Present the analysis results clearly. The results you report should be clear and intelligible, so that all parties involved understand the information given by the analysis. Remember to adapt your presentation to your audience.
82. Use Excel or a similar programme. Put the data from the survey in an Excel document or something similar. The results can then be experimented with by changing the variables to give a picture of the future.
83. Construct a bell curve if you want to show how the distribution of response options has turned out. The mid-point of the bell curve is the average.
84. Make a line chart if you want to study how different phenomena evolve over a period of time.
85. Make a scatter plot if you want to show the analytical units, such as the people in a survey, as points spread over the graph. In such a graph, a straight trend line can be added that provides a measure of developments.
86. Use percentiles if you want to show how, for example, income is distributed in a population or in a group.
87. Use choropleth maps if you want to show how, for example, election results, average income or unemployment is distributed in different geographical areas. The geographical areas are then marked in different colours.
88. Make a bar graph if you want to show how different categories of respondents differ in their responses.
89. Make a time series chart if you want to show how a characteristic of the observed object develops over time at repeated intervals.
90. Make a histogram if you want to show how a characteristic varies in a population or in a sample.
91. Make a pie chart if you want to show proportions of a whole.
92. Present interesting side-effects last of all. When you make your analysis report you should always begin with the real aim of the survey and the results of this. But, towards the end, you can bring up the other interesting results that came from the survey.
93. Use the principle of simplicity. It should be as easy as possible to keep up with and understand the presentation. Choose the presentation method which will enable your audience to understand the results in the fastest possible way.
94. Beware of pitfalls. A graph can drastically change appearance by changing the scale on the axes. A graph where the points are clustered on the y-axis when the scale values on the x-axis are far apart can give an impression of a weak trend, but we get the opposite impression if we plot the same series on a graph where the scale values on the y-axis are far apart and on the x-axis are close together.
Adding to the survey
95. Supplement with personal interviews. If the analysis shows strange, ambiguous or incomprehensible results, you may need to supplement the survey with a sample of personal interviews.
96. Interviews involving sensitive information. Supplement the survey with face-to-face interviews if the questions are of an emotional nature and where subtleties and attitudes are important to examine.
97. Pay attention to your visual senses. If you use interviews as a supplement, you should take note of the communication that takes place outside language - gestures, facial expressions and other visual clues.
98. Save the survey for the future. A survey is not only of short-term interest. It can often be of great value several years after the survey period.
99. Save the survey in several formats. If you want to be able to read and study the results in thirty or forty years time, it can be useful to save the survey in a variety of formats. Who knows what type of data storage we will have in the future? Print out the survey on permanent paper if you want to be sure.