Synthetic Ad Testing: What a Persona Cannot See

Methodology · 11 min read

TL;DR: An ad test measures two different things at once. Reception is whether the creative is seen, attended to and encoded, and it depends on the exposure conditions and the human visual system. Response is what a viewer makes of the ad once it has landed: comprehension, relevance, brand fit, message take-out. A synthetic panel stages no exposure and has no visual system, so it cannot reach the reception layer at all. It can carry parts of the response layer, under conditions. This guide sets out the two-layer test, the four response measures that survive, a six-step protocol that keeps a synthetic creative test honest, and the four cases that still need live fieldwork.

Can synthetic personas test ad creative?

Partly. An ad test measures reception and response at the same time. Reception depends on exposure conditions and the human visual system, neither of which a synthetic persona has. Response covers comprehension, relevance, brand fit and message take-out, and a census-grounded panel can carry parts of it. Reading a response answer as a reception prediction is the common failure.

Ask a synthetic persona what it thinks of your ad and you will get an answer. It will be articulate, it will reference the persona's stated circumstances, and it will read like someone who watched your ad.

Nobody watched your ad.

That gap is the whole subject. A creative test in a live panel does not only collect opinions. It stages an exposure, and the exposure does measurement work that the opinions cannot do on their own. Remove the exposure and some measures survive intact while others become fiction.

The split is clean enough to use as a rule, and it is a different check from the one you run before screening concepts on a synthetic panel. A concept is a proposition written in words. An ad is an execution with visual, temporal and contextual properties, and those properties are where creative testing earns its budget.

What does an ad test actually measure?

Two layers. The reception layer asks whether the creative wins attention, gets recognised as your brand, and lands in memory. The response layer asks what a viewer makes of it once it has been received. A live test measures both because it stages an exposure. A synthetic test stages nothing, so it only ever addresses the second layer.

Most creative-testing arguments go wrong because both layers arrive in the same report, on the same five-point scales, under the same heading.

Separate them and the question stops being whether synthetic ad testing works. It becomes which row of the table you are standing on.

The table below splits a standard creative test into its six working measures. The third column names what does the measuring in a live study. Where that thing is the viewing itself, a synthetic panel has nothing to substitute.

Why can a persona not judge whether an ad breaks through?

Because breakthrough is a property of the viewing, not of the file. Attention is counted in seconds against a memory threshold, and where a person looks inside an ad is predicted by dedicated models trained on eye-tracking data. A language model reads an image as described content. It has no glance, no fovea and no competing feed.

Attention research treats the creative and the viewing as separate objects. Amplified Intelligence's work on the attention memory threshold argues that attention has to clear a duration threshold before anything encodes at all, and that the duration owes as much to the placement as to the creative.³ An asset has no attention score on its own.

Where someone looks inside an advertising image is its own research problem with its own data. The fixation prediction benchmark published in the Journal of Visual Communication and Image Representation in 2021 built a dedicated dataset of advertising images and human fixations, because general-purpose saliency models did not carry over to ads.² That measurement comes from eye-tracking, or from a model trained on eye-tracking. It does not come from a respondent, synthetic or human.

The deeper limit sits in the model. A 2024 benchmark study asked directly whether multimodal models see the way the human visual system does, comparing model behaviour against established properties of human vision.¹ The framing matters more than any single score. Model vision is tested against human vision because the two are not assumed to be the same process.

So when a persona calls an ad eye-catching, it is describing the content of an image. It is not reporting a glance. Those two things look identical in a slide and mean different things.

Which parts of a creative test survive on a synthetic panel?

Four. Comprehension of the intended message, whether a wrong message is also available, relevance to a stated segment, and fit with what the brand is understood to stand for. All four are judgements about text and meaning, which is the layer a language model is built on. Report them as rankings between executions, never as levels.

Start with what does not survive, because the temptation is strongest there. Predicting whether an ad will be remembered is a modelling problem with its own literature. Work on long-term ad memorability set out to understand and generate memorable ads by learning from human memory data.⁵ MindMem, published in 2025, predicts advertisement memorability with multimodal models trained for that task.⁶

Both need ground truth about how real ads performed with real people. A persona has none, and asking it to guess is a different method wearing the same clothes.

The response layer has a different problem, and it is quality rather than absence. A 2026 psychometric audit of language models used as synthetic survey respondents reported response patterns that were plausible but not psychometrically valid.⁷ Plausible is the operative word. Scale scores can look entirely reasonable and still fail the checks you would run on a human scale.

That is why ranking survives and levels do not. If execution A sits above execution B on comprehension across replicate runs, the ordering carries information even when neither absolute number means much. A top-two-box appeal percentage carries nothing you can defend.

PersonaHive is built for the response layer rather than the reception layer. Personas are grounded in national census data, country by country across nine countries, and validated against real surveys, so an execution is read by the segment you intend to reach rather than by a free-text sketch of one. Every response ships with a written rationale, which is what makes a comprehension test readable: you can see which line of copy the persona took the message from, not only that it scored four out of five. Flat, hedged ratings are a known failure of prompted personas, and the controls for neutrality bias matter more here than almost anywhere, because the differences between two executions of the same brief are small.

The smallest useful next step is a message take-out test on two executions you have already made. Ask each persona what the ad is asking it to do, in its own words, before you ask for any rating at all. If both executions return the same take-out, your creative difference is not a message difference, and you found that out in an hour.

How do you run a synthetic creative test that holds up?

Six steps. Describe each execution neutrally, ask for open take-out before any scale, run replicates, compare executions instead of reading absolute scores, treat the rationales as the finding, and state in the report that no exposure was staged. That last step is what stops a stakeholder reading a response result as a media prediction.

1. Write a neutral description of each execution. Same structure, same length, same level of detail. Adjectives in the stimulus become findings in the output, which is the fastest way to test your own copywriting instead of your ad.

2. Ask for open message take-out first. What is this ad asking you to do, and who is it for. Any scale you ask before this contaminates it.

3. Ask the wrong-message question. What else could someone reasonably think this is saying. Miscomprehension is the response-layer risk that costs the most in market, and it is one a synthetic panel finds cheaply.

4. Run the same test three times with fresh persona draws. If the ranking between executions flips across runs, you do not have a creative difference. You have run-to-run noise, and the rules for wording a synthetic questionnaire usually explain why.

5. Read the rationales, not the means. A rationale that quotes the headline back at you is comprehension. A rationale that invents a benefit the ad never claimed is a warning about the concept, not about the execution.

6. Put one line in the report: no exposure was staged, so nothing here speaks to attention, branding or recall. Write it before anyone asks, because someone will read a comprehension score as a breakthrough forecast if you leave the space empty.

When does creative testing still have to use people?

Four cases. Any decision about media weight or placement, any go or no-go on a large production spend, any claim about recall or brand linkage, and any execution whose effect rests on tone, music, pacing or humour. Those sit in the reception layer, or in reactions a written description cannot carry.

The fourth case is the one teams underrate. A written description of a joke is not a joke. Timing, performance, music and edit pace are the parts of an execution that a neutral text summary flattens completely, and they are often the reason one execution outperforms another that reads identically on paper.

The first three are reception-layer decisions, and the industry treats them as validation problems for good reason. Kantar's own account of whether ad testing predicts sales impact is built on linking pre-test measures to in-market outcomes across a large database of real campaigns.⁴ That is the evidence base a pre-test rests on. A synthetic panel has no equivalent, and no vendor can honestly claim one.

The useful sequencing is boring and it works. Use the synthetic panel while executions are cheap to change, on the response measures it can carry. Take two or three survivors to live fieldwork for the reception measures it cannot. You will buy less fieldwork and buy it later, which is a smaller claim than replacement and a true one.

The general boundary conditions are worth reading alongside this: where synthetic evidence is not valid covers the failure modes that apply across methods, and this post is the creative-testing case of one of them.

If you want to try the response half, start with the two-execution take-out test described above. PersonaHive's free tier includes 250 credits and needs no card.

Frequently asked questions about synthetic ad creative testing

Short answers to five questions that come up when a creative test lands on a synthetic panel: whether uploading the asset changes anything, whether static and video behave the same way, whether a synthetic test can stand in for a pre-test before a media buy, how many personas a creative test needs, and how to present the result to a CMO.

**Can I upload the actual ad instead of describing it?** It helps with comprehension and it changes nothing in the reception layer. A model reading your asset still produces a description of its content, not a record of a viewing. Treat an uploaded file as a more faithful stimulus, not as an exposure.

**Do static and video behave the same way?** No. Video loses more. Pacing, edit rhythm, music and performance carry a large share of a video execution's effect, and all of them thin out in any description a model works from. Static ads survive the translation better.

**Can this replace a pre-test before a large media buy?** No. A pre-test earns its place by linking its measures to in-market outcomes, which is a claim built on real campaign histories. Use the synthetic round to decide which executions deserve the pre-test.

**How many personas does a creative test need?** Enough that the ranking between executions is stable across replicate runs, which is a different question from statistical power. Add runs before you add personas.

**How do I present this to a CMO?** As a ranked shortlist with the reasons attached, and one sentence naming what was not measured. The credibility comes from the exclusion, not from the sample size.

Sources

• Do Multimodal Large Language Models See Like Humans? — arXiv
• Fixation prediction for advertising images: Dataset and benchmark — Journal of Visual Communication and Image Representation
• Why does the attention memory threshold matter — Amplified Intelligence, Amplified Intelligence
• Can ad testing really predict sales impact? — Kantar, Kantar
• Long-Term Ad Memorability: Understanding and Generating Memorable Ads — arXiv
• MindMem: Multimodal for Predicting Advertisement Memorability Using LLMs and Deep Learning — arXiv
• Plausible but Not Valid: A Psychometric Audit of LLMs as Synthetic Survey Respondents — arXiv