You have a satisfaction score that dropped four points, and the closed questions cannot tell you why. That is the job of open-ended survey questions. They are also the part of a questionnaire most likely to be rushed, skipped or left unread. This article covers what they give you, what they cost, how to write and place them, and how to analyze the answers. For the wider picture of survey design and response quality, see our guide to customer surveys that produce decisions.
Key takeaways
- Open-ended survey questions let respondents answer in their own words, giving you reasons behind a rating at the cost of slower analysis and more respondent effort.
- Ask one neutral question about one thing, tied to something the respondent just experienced, such as the main reason for the rating they gave.
- Place open questions after the closed question they explain, never first, and in a survey of about ten minutes, ask one to three.
- Code the answers by reading 50 to 100 responses, drafting a coding frame, coding every response, and checking a random 10 percent with a second coder.
- Use text analytics for a first pass on large volumes, then have a person review samples from each theme and the Other bucket.
What are the benefits and challenges of open-ended survey questions?
Both sides come from the same feature: respondents write whatever they like. That freedom produces richer answers and a heavier workload, for them and for you.
| Benefits | Challenges |
|---|---|
| Reasons and context behind a rating, in the customer’s own words | Respondents must think and type, so they take longer than clicking |
| Ideas and problems you did not think to list as answer options | Fatigue and abandonment rise if there are too many |
| Wording you can reuse in future closed questions, personas and messaging | Answers vary in length, quality and relevance; many are one word |
| Fewer answer choices imposed by the survey designer | Analysis is slow and needs a consistent coding method |
| A place for candid comments that a fixed list would not allow | Results are harder to compare across segments until they are coded |
Open-ended questions reduce the bias of a limited answer list, but they do not remove bias. Wording still steers the answer, and the people who bother to write are not a random cross-section of your sample.
What are examples of good and weak open-ended questions?
A good open-ended question asks about one thing, is neutral, and is tied to something the respondent has just experienced. A weak one is vague, leading or asks for too much.
| Weak question | Why it fails | Better version |
|---|---|---|
| ”Any other comments?” (as the only open question) | Invites “no” or “none”; no focus | ”What is the main reason for the rating you gave?" |
| "Why did you love our fast, friendly service?” | Leading; assumes the answer | ”How would you describe the service you received?" |
| "What do you think about our brand, products, pricing and support?” | Four topics in one box | Ask one question per topic, or only the most important one |
| ”Why did you give a low score?” (to everyone) | Many respondents did not give one | Show it only to respondents who rated 6 or lower: “What would have made your experience better?" |
| "Tell us everything about your experience.” | Too broad; produces long, unfocused text | ”What is one thing we should change first?” |
Where should you place open-ended questions in a survey?
Do not open the survey with one. A blank text box as the first thing a respondent sees is a quick way to lose people before they have started. Place open questions after the closed question they explain, once the respondent is warmed up.
Three placements work well:
- Right after a rating. Ask a satisfaction or likelihood-to-recommend question, then “What is the main reason for your answer?” The context is fresh and the answer is tied to a number you can segment. This pairing is the backbone of a customer satisfaction survey.
- As an “Other, please specify” option. When a multiple-choice list may be incomplete, add a text box so respondents are not forced into the nearest wrong answer.
- Once near the end. A single closing question about anything you missed catches issues you did not anticipate. Keep it optional.
How many open-ended questions should you ask?
There is no formula, so treat this as a rule of thumb: ask as few as the decision needs. In a survey of about ten minutes, one to three is usually enough, each answering a specific “why” a closed question cannot. If you are writing eight, you probably have an interview guide, not a survey.
Every extra text box adds effort, and effort is one reason people quit partway through, which feeds into your survey response rate. Make most open questions optional and show each only to the respondents it applies to.
How do you analyze open-ended survey responses?
The goal is to turn text into categories you can count, compare and explain. That process is called coding, and the list of categories is a coding frame (some teams say “buckets” or a codebook).
- Read a sample first. Read 50 to 100 responses before designing anything. Look for recurring ideas, not quotes.
- Draft the coding frame. Create a category for each recurring idea and write a one-line definition and an example for each. Add “Other” and “Unclear”.
- Code every response. Allow more than one code per response when it mentions more than one idea. Revise the frame if a new theme keeps appearing.
- Check consistency. Have a second person code a random sample of about 10 percent and compare. A simple agreement rate works, or Cohen’s kappa if you want a figure that adjusts for chance. Disagreements usually mean a category definition is vague.
- Count and cross. Report the share of respondents mentioning each code, and cross it with the closed questions: by segment, by rating or by wave.
- Pull representative quotes. Choose quotes typical of a theme, not just the most colorful.
What does a coding example look like?
Suppose 240 respondents who rated their experience 6 or lower on a 0 to 10 scale answered “What would have made your experience better?” These numbers are illustrative. Here is how three responses might be coded:
| Response | Codes |
|---|---|
| ”My order came a week late and nobody told me.” | Delivery delays; Slow support |
| ”Checkout kept rejecting my card.” | Hard-to-use checkout |
| ”Fine product, but too expensive for what it is.” | Price |
After coding all 240 responses, the counts look like this:
| Code | Responses | Share of 240 |
|---|---|---|
| Delivery delays | 66 | 27.5% |
| Hard-to-use checkout | 48 | 20.0% |
| Price | 42 | 17.5% |
| Slow support | 30 | 12.5% |
| Product quality | 24 | 10.0% |
| Other or unclear | 36 | 15.0% |
The 246 codes exceed 240 because six responses carried two codes, so the shares add to more than 100 percent. That is expected with multi-coding. The finding is now a statement a team can act on: delivery delays came up in more than a quarter of low-score comments, ahead of checkout and price.
Should you use text analytics or code by hand?
Use both. Text analytics tools can group thousands of comments into themes and score sentiment in minutes, which is worth it when volume rules out reading everything. They are weaker on sarcasm, one-word answers, industry terms and comments that mention two ideas at once.
So use software for the first pass, then add human checks. Read a random sample from every theme, review the “Other” bucket by hand, and have a person sign off on any theme that goes into a report. Reading raw text yourself still builds a feel for the data that no summary gives you. Candor matters too; see why people lie on customer surveys.
How do you do this in practice?
Once responses are coded, each code behaves like any other survey variable. In Halo Reports, you can build crosstabs and banner tables with weighting and significance testing, so you can see whether “Delivery delays” is mentioned more by one segment than another, and whether a difference is large enough to trust. For comments that arrive all year, mTab for customer experience teams gives a continuous view of emerging issues across verbatims, reviews and support themes.
What mistakes should you avoid?
- Asking too many. Three well-aimed questions beat ten broad ones.
- Leading the respondent. Wording like “How did our excellent team help you?” plants the answer. Stay neutral.
- Coding with no written definitions. Without definitions, two coders will sort the same comment differently, and your counts will not be reliable.
- Reporting only vivid quotes. One memorable comment is not a finding. Say how many respondents raised the theme.
- Trusting automated themes unchecked. A label produced by software is a hypothesis until a person has read examples behind it.
What should you do next?
Keep only the open-ended questions that explain a closed answer you will act on. Put each directly after that closed question, write its coding frame before you field the survey, and decide who will check the coding. mTab Pulse monitors trackers and studies and surfaces meaningful movement, so a growing theme gets noticed.