You have two or three ad routes, a media budget, and a launch date. Before that money is spent, someone will ask which route works best and whether any of them will move the brand at all. Ad testing answers those questions with a survey of your target audience. It applies the same logic as concept testing to finished or near-finished creative, and this guide covers what to measure, how to set up the test and how to read the numbers.
Key takeaways
- Ad testing (or creative testing) shows an ad to target buyers and measures attention, branding, message takeaway, likeability and persuasion.
- Survey pre-testing explains why an ad works. In-market A/B testing shows which version wins on clicks or sales.
- Always include a control cell that sees no ad, or you cannot tell what the ad caused.
- Branding is the measure teams most often skip. A persuasive ad that people credit to a competitor helps the competitor.
- After launch, brand tracking shows whether the effect holds in market.
What is ad testing, and how is it different from A/B testing?
Ad testing in research usually means a survey pre-test. Target buyers see the ad inside a questionnaire, and you learn whether they noticed it, who it was for, what it said and whether it changed their view of the brand.
In-market A/B testing runs two live versions to real audiences and compares clicks, sign-ups or sales. It is grounded in behavior, but it tells you which version won, not why.
| Survey pre-testing | In-market A/B testing | |
|---|---|---|
| When | Before launch (or before scaling spend) | After launch, with live media |
| Who sees the ad | A recruited sample of target buyers | Whoever the media reaches |
| What it measures | Recall, branding, message, likeability, persuasion, diagnostics | Clicks, conversions, sales |
| Explains why | Yes, through diagnostic and open-ended questions | Rarely |
| Works for | Any format, including rough cuts and TV | Digital formats you can split |
| Main risk | Forced exposure overstates attention | Optimizes short-term response, not brand |
Most teams need both: ad testing to pick and fix a route, then live testing to tune versions of the winner.
What should you measure in a creative test?
A good creative test covers six areas, asked in roughly this order so later questions do not prompt earlier answers.
- Attention and recall. Did the ad register? Ask which ads people remember from a short reel, before naming the brand.
- Branding. Ask “Which brand was this ad for?” as an open question. Correct brand attribution is the share who name the right brand unprompted.
- Message takeaway. Ask what the ad was trying to say, in their own words, then code the answers against the intended message. Our guide to message testing covers how to test the claim itself before it goes into creative.
- Likeability. A simple 5-point scale is enough, plus “what, if anything, did you dislike?”
- Persuasion. The change in purchase intent or consideration compared with the control cell. This is where the ad earns its budget, and it maps onto the stages of the purchase funnel.
- Diagnostics. Is it believable, relevant, different, confusing? These explain the scores above.
Write the questions once and reuse them for every test. The concept testing questions guide has wording you can adapt for purchase intent and believability.
Which ad testing design should you use?
Monadic test versus a control cell
The most reliable design splits a matched sample into cells. The control cell sees no ad (or an unrelated filler ad), and each test cell sees one ad. Everyone answers the same brand questions, so the gap between a test cell and the control is the effect of the ad.
One ad per person is a monadic design. To compare several routes on a tight budget, a sequential monadic design shows each person two or three ads in rotated order; see monadic vs sequential monadic testing for the trade-offs.
Pre/post exposure
A pre/post design asks the same person about the brand, shows the ad, then asks again. It needs fewer respondents, but people can guess what you want and raise their answer. Use it for quick reads, and prefer a control cell when the result will decide a media plan.
How do you know if an ad lifted consideration?
Here is an illustrative test. The numbers are invented to show the arithmetic. A brand recruits 500 category buyers and splits them evenly: 250 in a control cell and 250 who see the new ad.
| Cell | Respondents | Would consider the brand | Consideration |
|---|---|---|---|
| Control (no ad) | 250 | 80 | 32.0% |
| Test (saw the ad) | 250 | 103 | 41.2% |
The gap is 9.2 points. A two-proportion z test checks whether it is bigger than chance would produce:
- Pooled proportion: (80 + 103) / 500 = 0.366.
- Standard error of the difference: the square root of 0.366 × 0.634 × (1/250 + 1/250) = the square root of 0.001856 = 0.04309.
- Test statistic: 0.092 / 0.04309 = 2.14.
The cutoff for 95% confidence is 1.96, so the lift is significant (two-sided p of about 0.03). It is not precise, though. The 95% interval on the gap runs from about 0.8 to 17.6 points, so the true lift could be small. To detect a 9-point lift reliably (80% power), you would need about 450 people per cell.
Now check branding. All 250 people in the test cell were asked, unprompted, which brand the ad was for.
| Answer | Respondents | Share |
|---|---|---|
| Correct brand | 145 | 58% |
| A competitor | 55 | 22% |
| Not sure | 50 | 20% |
More than one in five credited a competitor. That is the finding to act on before launch: make the brand appear earlier, longer or more distinctly, then retest. A lift in consideration means less if a fifth of the ad’s memory goes to someone else.
How do you run ad testing across markets?
Use the same design in every market: the same cells, the same questions, translated and checked by a native speaker. Give each market its own control cell. Baseline consideration differs by country, so a lift is only meaningful against that market’s own control.
Read the results market by market before you pool them. An ad can persuade in one country and confuse in another, and an average hides both. Size the cells in each market you need to read on its own, not just the total.
How do you do this in practice?
Standardize the questionnaire, the control cell and the report, so each ad test is comparable with the last. Cut every measure by cell and segment with crosstabs that test significance, which runs the z test above across the whole table. Over time, connecting findings across studies and waves lets you build your own norms for a good branding or persuasion score.
Once the ad is live, pre-test results are a forecast, not proof. Field a brand tracking wave before launch and another after the campaign, and watch whether consideration moves in market. To follow that continuously, track brand health and consideration alongside competitors.
What are the common mistakes in creative testing?
- No control cell. Without one, a 41% consideration score has nothing to be compared with. You cannot tell what the ad added.
- Testing animatics as if they were finished ads. Rough cuts tend to score lower on likeability and attention. Say so in the report, and compare rough material only with rough material.
- Ignoring branding. Teams celebrate high likeability and persuasion, then find that people remember the ad but not the brand. Always ask attribution unprompted.
- Different questionnaires per cell. Every cell needs identical questions in identical order, or the gap reflects the questionnaire, not the ad.
- Reading small gaps without a test. With 250 per cell, a 5-point gap is well inside the noise.
Before your next ad test, write down the smallest lift in consideration that would change your media decision, then size the cells to detect it. Add an unprompted branding question if you do not have one. Those two steps fix most of the creative tests we see go wrong.