Your survey ended with “What is the main reason for your score?” and now you have 600 answers in a spreadsheet column. Reading them gives you a feel, not a finding you can take to a product team. This guide shows how to analyze open-ended survey responses so they count like closed questions. It is the survey side of thematic analysis, the wider method for finding patterns in what people say.
Key takeaways
- Clean the answers and read a sample before you write a code, so the code frame reflects what people said.
- A code frame lists defined codes. Nets group related codes so you can report at two levels.
- Report the share of respondents mentioning each code. One answer can carry several codes, so the total passes 100%.
- A net counts each respondent once, so it is usually smaller than the sum of its codes.
- Crosstab the codes by a closed question, such as detractors versus promoters, to explain the score.
Why are open-ended survey responses worth analyzing?
Closed questions tell you how many people feel something. Open ends tell you why, in words you did not offer as an answer option. A score that drops four points is a number, while a jump in comments about late deliveries is a reason you can act on. The benefits and challenges of open-ended survey questions covers when to ask them.
The hard part is scale. At 600 answers, people skim, remember the vivid comments and forget the common ones, and two analysts file the same comment in different places. Verbatim analysis needs a structure that counts every response once, the same way.
How to analyze open-ended survey responses in six steps
- Clean. Remove blanks, “n/a”, keyboard mashing and off-topic answers. Decide whether “nothing” is a removal or a code; for “What would you improve?” it is a real answer. Leave spelling alone, since you will quote from this file.
- Read a random sample. Read 100 to 200 responses and note each distinct idea. Stop when new answers keep landing on ideas you already have.
- Build the code frame. Turn the ideas into codes, each with a one-line definition and an example, then group related codes into nets. Add an “Other” code. Our guide to qualitative coding shows how to write codes two people apply the same way.
- Code every response. Give each answer every code that applies. You can code by hand, accept or correct a model’s suggestions, or let a model apply the frame and review a sample. AI thematic analysis covers where automated coding needs a human check.
- Check quality. Have a second person code 50 to 100 responses and compare. Disagreement usually means a vague definition, so fix it and recode. A growing “Other” means a missing code.
- Quantify. Count the respondents mentioning each code and net, divide by the right base, and crosstab against your closed questions.
What is a code frame, and how is it different from themes?
A code frame (or codebook) is the fixed list of codes you apply to every response so you can count them. A code is one concrete idea, such as “late or missed delivery”. A net groups related codes under a heading, such as “Delivery”.
A theme is an interpretation of what sits underneath many codes, such as “customers feel they have lost control of their subscription”. Codes are counted, and they should stay stable across waves. Themes are argued from evidence and can change as you learn more. Most survey projects need both: codes to size each issue, themes to explain how the issues connect.
How do you turn coded responses into numbers?
Here is a worked example with illustrative data. A meal-kit service asked customers the 0–10 likelihood-to-recommend question, then “What is the main reason for your score?” After cleaning, 600 respondents gave a usable answer. The team coded them into 8 codes in 3 nets, 840 codes in total (1.4 per respondent).
| Net or code | Respondents | % of 600 |
|---|---|---|
| Food (net) | 312 | 52.0% |
| Recipe variety | 186 | 31.0% |
| Ingredient freshness | 132 | 22.0% |
| Portion size | 72 | 12.0% |
| Delivery (net) | 156 | 26.0% |
| Late or missed delivery | 114 | 19.0% |
| Damaged or excess packaging | 66 | 11.0% |
| Price and service (net) | 222 | 37.0% |
| Price or value for money | 168 | 28.0% |
| Hard to skip or cancel | 54 | 9.0% |
| Customer service | 48 | 8.0% |
Use respondents as the base, not mentions
Divide each count by the 600 respondents: recipe variety is 186 ÷ 600 = 31.0%. The eight codes add to 140% (840 ÷ 600), which is correct for a multi-response question. Dividing by the 840 mentions would give the share of comments, not of customers.
Count each respondent once in a net
The Food codes add to 186 + 132 + 72 = 390 mentions, or 65%. But 78 of those mentions come from people who named more than one food code, so only 312 people mentioned food: 312 ÷ 600 = 52.0%. Build nets from respondents with at least one code in the net, never by adding percentages.
How do you compare themes between detractors and promoters?
Coded open ends earn their keep in a crosstab. The 600 split into 250 promoters (9–10), 170 passives (7–8) and 180 detractors (0–6): 41.7% minus 30.0%, a Net Promoter Score of about +12. Each column below uses its own group as the base.
| Net or code | Detractors (180) | Promoters (250) |
|---|---|---|
| Food (net) | 25.0% (45) | 70.0% (175) |
| Delivery (net) | 50.0% (90) | 8.0% (20) |
| Late or missed delivery | 40.0% (72) | 4.0% (10) |
| Price and service (net) | 55.0% (99) | 24.0% (60) |
| Hard to skip or cancel | 20.0% (36) | 1.2% (3) |
Late delivery is 72 ÷ 180 = 40.0% of detractors but 10 ÷ 250 = 4.0% of promoters. Promoters talk about the food. Detractors talk about getting it, paying for it and trying to leave. Recipe variety, the top code overall, is mostly praise (120 of its 186 mentions come from promoters), so it is not the thing to fix.
Price appears on both sides: “too expensive” from detractors, “worth it” from promoters. Many frames split codes like this by sentiment. Coded verbatims also feed voice of customer analysis, alongside reviews and support tickets. Teams that report these drivers every quarter can keep satisfaction drivers, verbatims and loyalty in one continuous view.
How do you choose verbatims to quote?
Numbers size an issue, and quotes make people believe it. Pick quotes from the largest codes in the group you are reporting, prefer typical answers to colorful ones, and say how many people each quote stands for.
An illustrative quote for late deliveries might read: “Box came a day late twice this month and the chicken was warm.” Show it next to the 40%, not instead of it. Remove anything that identifies a person.
How do you keep codes consistent across tracker waves?
In a tracker, the code frame is part of the questionnaire. Redefine a code in wave 3 and nobody can tell whether customers changed or the coding did.
- Keep a codebook with each code’s definition, inclusions, exclusions and example answers.
- Add new codes for new issues and note the wave they started. Do not quietly widen old ones.
- Before each wave, recode a sample from the last one to check the coders, or the model, still agree.
Keep the frame and coded data in a research repository. When you reuse coded findings across studies and waves, each wave starts from what the last one learned.
What are the common mistakes in survey text analysis?
- Coding before reading. A frame written from the questionnaire misses what people said and fills “Other”.
- Vague codes. “Service issues” means different things to different coders.
- Using mentions as the base. Share of comments is not share of customers.
- Adding codes to build a net. Summing double-counts people who mention two codes.
- Reporting the total only. The top code overall may be praise. Cut by your key closed question first.
What should you do next?
Take the open end from your last survey and run the six steps: clean, read a sample, write a short frame, code, check, then crosstab by your main score. Once the coded data sits with your other research, colleagues can ask questions in plain language and get answers from your own research, with the source shown.