Survey analysis software is only as good as the data you feed it, which is why survey data quality starts before fieldwork. Most of the work that decides whether a project goes smoothly happens before the first respondent sees a question. If you are also deciding how to check that a questionnaire measures what you intend, our guide to validating a survey covers the wider process; this article covers the four pre-survey steps that make analysis fast.
Key takeaways
- Pre-survey steps are the planning work you finish before fielding: objectives and an analysis plan, questions built for your tables, a pilot, and a data map.
- Write your hypotheses with thresholds and the expected result for every table cell before fielding, so you can compare your intuition with the real output.
- Draft each question by asking which table it feeds, cut it if you cannot name one, and include classification questions such as age group and region.
- Pilot with 10 to 30 people from your real audience, check that the data fills your table shells, and exclude pilot responses from the final data.
- Build a data map before fielding that records each variable’s name, label, type and codes, with multi-select options stored as separate 0/1 columns.
What should you do before you launch a survey?
Four things, in this order: write down your objectives and your analysis plan, design the questionnaire around the tables you will need, pilot it with a small group, and build a data map that says how every answer will be stored. Each step is cheap before fielding and expensive after.
We will use one running example. You want to focus your marketing on the social media channels your target audience actually uses, so you will survey that audience about their social media habits. All numbers below are illustrative.
How do you set objectives and an analysis plan?
Start with the decision the survey supports. Common purposes include gathering feedback, learning more about a target audience, tracking company performance and collecting market research. In the example, the decision is which two or three channels get the marketing budget.
Next, turn the objective into hypotheses, which are statements the data can support or contradict. You hold some hunches already: Facebook is widely used, Instagram is growing, and LinkedIn matters less because you sell to consumers, not businesses. Write them as testable claims with a threshold:
- More than 75 percent of the target audience uses Facebook at least once a week.
- Fewer than 25 percent use LinkedIn at least once a week.
- At least 50 percent use Instagram at least once a week.
Then add the part most teams skip, the analysis plan. List the tables and charts you expect to produce, and write your expected result in each cell before fielding. With 400 completes, your guesses become counts you can compare with the real output:
| Channel (used weekly) | Hypothesis | Expected count of 400 | Actual |
|---|---|---|---|
| More than 75% | More than 300 | ||
| Fewer than 25% | Fewer than 100 | ||
| At least 50% | At least 200 | ||
| Snapchat | About 10% | About 40 | |
| X (formerly Twitter) | About 10% | About 40 |
When the results arrive, you can see where your intuition was right and where it was off. Percentages need not sum to 100, since respondents can use more than one channel.
Also decide the comparisons now. If you will want results by age group or region, those become banner columns later, and each needs enough respondents. At 400 completes, a result near 50 percent carries roughly plus or minus 5 points of sampling error (1.96 x the square root of 0.5 x 0.5 / 400 is about 0.049), and each subgroup has a wider margin than the total.
How do you design questions with the final tables in mind?
Draft each question by asking what table it feeds. If you cannot name the table, cut the question. Straightforward questions work best, and one well-built question can replace several.
The first draft of our example might have three questions: how often do you use Facebook, LinkedIn and Instagram, each with the answers at least once a day, at least once a week, at least once a month, or less than once a month. That gives you more detail than your decision needs. Your marketing plan only cares about channels used weekly or more, so a revised version is a single multiple-select question:
Which social media channels do you use at least once a week? (Select all that apply)
- Other (please specify)
- None of these
The trade-off is worth stating plainly: you lose daily versus weekly detail. If frequency would change your decision, for example whether to post daily, keep the frequency question. Simplicity should follow your analysis plan, not replace it.
A few habits keep the tables clean. Make answer options mutually exclusive and complete, add “None of these” or “Other” so nobody is forced into a wrong answer, and ask one thing per question. Put your classification questions, such as age group and region, in the questionnaire, because you cannot cross-tabulate by something you did not ask.
Why should you pilot a survey before you launch it?
A pilot, or pre-test, is a small run with people from your real audience before the full launch. It finds problems you cannot see from the inside: a question that two people read two different ways, an answer list with a missing option, a screen that fails on a phone.
Run it in two parts. First, field the survey to a small group, often 10 to 30 people. Then look at the data they produced, not only their comments, and check that it flows into your table shells without surprises. Second, talk to a few of them for five minutes and ask what each question meant to them, in their own words.
Watch for long completion times, questions where people stop, answers piling into “Other” (a missing option), and skip logic that sends people to the wrong question. Fix what you find, then retest if the changes were large. Exclude pilot responses from the final data, since the questionnaire changed.
How do you build a data map so analysis is fast?
A data map, sometimes called a codebook, records how each answer will be stored. Build it before fielding and test it with the pilot data. It is the step that turns raw exports into tables in minutes instead of days.
For every question, record the variable name, the full question text, the question type, and the value codes. Our example looks like this:
| Variable | Label | Type | Codes |
|---|---|---|---|
| Q1_1 | Uses Facebook weekly | Multi-select, one column per option | 1 = selected, 0 = not selected |
| Q1_2 | Uses LinkedIn weekly | Multi-select | 1 = selected, 0 = not selected |
| Q1_3 | Uses Instagram weekly | Multi-select | 1 = selected, 0 = not selected |
| Q1_4_text | Other channel, text | Open-ended | Recode into Snapchat, X, other |
| Age_grp | Age group | Single choice | 1 = 18-34, 2 = 35-54, 3 = 55+ |
Three conventions save the most time. Store each option of a multi-select question as its own 0/1 column, so counts add up cleanly. Reserve one code for “not asked” or missing and use it everywhere, so a skipped question is never confused with a “no”. And plan how you will code open-ended answers: if you expect Snapchat and X to show up under “Other (please specify)”, write the code frame now and update it when real answers arrive.
How do you do this in practice?
With a plan, a questionnaire and a data map in place, tabulation becomes mechanical. In Halo Reports you can load the data, build crosstabs and banner tables with weighting and significance testing, chart the results, and share an interactive report page. The table shells you wrote before fielding tell you which tables to build first. For wide, complex studies and trackers, mTab for insights and research teams keeps them on one governed platform.
What are the most common pre-survey mistakes?
- Starting with questions instead of a decision. A questionnaire that collects “interesting” data produces tables nobody can act on.
- Skipping expected results. Without a prior guess, every number looks plausible and you learn nothing about your own assumptions.
- Leaving out the classification questions. Age, region or customer type are cheap to ask and impossible to add after the fact.
- Writing options that overlap or miss cases. “Once a week” and “a few times a week” in the same list leave respondents guessing.
- Skipping the pilot because of time. A one-day pilot is far cheaper than relaunching a survey.
What should you do next?
Take a survey you are planning and write the table shells first: the rows, the banner columns, and your expected number in each cell. Then draft only the questions those tables need, pilot them with a handful of real respondents, and finish the data map before you launch. If your table shells and your questionnaire disagree, you have found the problem while it is still free to fix.