> ## Content Index
> Fetch the complete content index at: https://pulsecorp.co/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Synthetic respondents vs real customers: when AI personas help and when they mislead
- URL: https://pulsecorp.co/blog/synthetic-respondents-vs-real-customers/
- Published: 2026-11-14T12:00:00.000Z
- Updated: 2026-11-15T02:56:49.000Z
- Description: Synthetic respondents vs real customers: what the benchmarks show, where AI personas help, where they mislead, and how to combine them with interviews.
- Author: Khyati Ojha

Imagine you're on the research team at a juice company. The company wants to launch a new flavor, and a vendor offers you thousands of survey answers by lunchtime, all written by AI. Can those answers stand in for real customers?

On synthetic respondents vs real customers, the plain answer is no. AI answers can help you draft ideas worth testing. But real customers should make the call on the new flavor.

## What are synthetic respondents?

A synthetic respondent is a language model that has been told to answer as a member of a target group. Many of them together form a synthetic sample. So the juice company could ask a model to answer as a young parent in a big city.

Synthetic respondents are built in two main ways. The simple way gives a model a profile, such as age, job and city, and then asks the survey question. The richer way also feeds the model real survey data from that group, so its answers lean on what real people said before.

Neither is the same as an AI-moderated interview, where the AI asks and a real person answers.

### The short version

- Synthetic samples can match some overall answers well, mostly on attitudes surveys ask about often.
- For single people and for smaller groups of customers, published tests find them less accurate than simple statistics.
- Use them to draft ideas to test and to try out your questions before a real survey.
- Use real customers for prices, reasons, small customer groups and any decision someone will check.
- Label synthetic output so nobody mistakes it for what customers said.

## What does the evidence say about accuracy?

Before you trust AI answers on a flavor, you'd want to know how close they get. The evidence is mixed in a useful way. Synthetic answers can follow the overall shape of opinion on some questions that surveys ask often. They fall short when you need to know what one group or one person thinks, and they fall further behind across cultures.

The table compares two published tests that set synthetic answers against real survey data.

| Study                                                             | Real data compared against                                                                                             | What it found                                                                                                                                                               |
| ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| González-Bustamante, Verelst and Cisternas, arXiv, September 2025 | Chile's CEP survey No. 92: 1,482 face-to-face interviews, fielded August to September 2024                             | 128 prompt, model and question combinations produced 189,696 synthetic profiles. The best trust item passed 0.90 accuracy and F1, with large differences from item to item. |
| Chen, Zhu and Zheng, arXiv, July 2026                             | World Values Survey wave 7 (91,774 people, 63 countries) and the US General Social Survey 2016 to 2024 (14,704 people) | On world values, every model tested was 11 to 22 percentage points less accurate than a simple lookup table. On US attitudes, the best model only tied it.                  |

The Chilean study is the hopeful case ([González-Bustamante et al., 2025](https://arxiv.org/html/2509.09871v1#:~:text=Shrestha%20et%20al.%2C%20%282024%29,accuracy%20of%20synthetic%20data.)). F1 is a score from 0 to 1\. It rewards getting both the yes and the no answers right, so 0.90 is strong. But economic questions were much harder to copy, and results varied a lot from one question to the next.

The second study is the worrying one ([Chen, Zhu and Zheng, 2026](https://arxiv.org/html/2607.26348v1#:~:text=WorldValuesBench%20derives%20a%20large,struggling%20to%20generalize%20%5B17%5D.)). A lookup table here predicts each person's answer from the average answer of people with the same demographics. Demographics are basic facts about a person, such as age. The models also made the gaps between groups two to four times bigger than they really were. Asked which group a team should target, the models picked the wrong group in about half of the US cases. A larger model did not fix either problem.

So a synthetic sample can sketch the crowd, but it guesses badly about the people in it.

## When do synthetic respondents help?

Synthetic respondents help when a wrong answer costs little and the output is only a starting point. Three jobs fit well.

The first is early thinking before any research. Say the juice company asks a synthetic sample how different groups might react to a mango and ginger flavor. In an hour it gets a list of likely reactions. That list is a set of ideas to test, which researchers call hypotheses.

The second is trying out a survey before it goes live. Run the draft past a synthetic sample to spot confusing wording or questions everyone answers the same way. Then you fix them before you pay for a real survey.

The third is planning an interview guide, the list of questions an interviewer follows. Seeing the range of answers a question might get helps you write better follow-ups for real interviews.

## When do they mislead?

Synthetic respondents mislead when the decision depends on what real people would actually do, feel or pay. Four areas stand out.

Prices are the first. Even real people's answers about price differ from what they later do in the shop. A model that copies those answers adds a second layer of guessing.

Feelings and reasons come next. Why someone stopped buying a juice depends on a taste they disliked, a price that jumped at their local shop or a friend's advice. A model has no life of its own to draw on, so it gives a likely-sounding average reason.

Small groups that surveys rarely reach are third. Models know least about the groups that appear least in the text they learned from. The World Values Survey result above shows the gap growing across cultures.

The last is any decision where your manager will ask "who said this?" Synthetic output has no person behind it, so there is nothing to trace.

![Diagram: synthetic respondents help with early ideas, survey wording and interview guides, and mislead on prices, feelings and reasons, small groups, and decisions someone will check.](https://pulsecorp.co/blog/content/images/2026/11/synthetic-respondents-vs-real-customers-cg0fci.webp "Use synthetic answers where a wrong answer is cheap, and real customers where it is not.")

Use synthetic answers where a wrong answer is cheap, and real customers where it is not.

## How do synthetic respondents compare with real customers?

The difference is what each one can show. Synthetic output can suggest what might be true. Interviews with real customers can show what is true for the people you spoke to, and why.

The table sets the two side by side.

| Criterion                   | Synthetic output                        | Real customer interviews                             |
| --------------------------- | --------------------------------------- | ---------------------------------------------------- |
| Speed to first answer       | Minutes                                 | Depends on finding people and running the interviews |
| Main cost driver            | Model use and setup                     | Recruitment, incentives and analysis time            |
| Depth on reasons            | Likely-sounding, averaged reasons       | Specific reasons from real life                      |
| Accuracy on small groups    | Weakest where data is thinnest          | As good as your screener and sample                  |
| Follow-up on a vague answer | Generates more text from the same model | A real person explains what they meant               |
| Traceable to a real person  | No                                      | Yes, through the transcript                          |
| Fit for a final decision    | Ideas to test only                      | Yes, with the right sample                           |

## How do you combine them?

Say the juice company wants speed early on and evidence it can trust at the end. Then use AI answers first, real customers second, and decide last.

### 1\. Use synthetic output to draft ideas and try out your questions

Ask the synthetic sample your questions, collect the range of answers and fix any wording that confuses it. Watch for answers that look too tidy. That tidiness comes from the model averaging, and it doesn't mean customers agree.

### 2\. Label it so nobody mistakes it for what customers said

Mark every slide or note that uses synthetic output as "synthetic, not customer evidence". Watch for made-up quotes slipping into a report as if a customer said them.

### 3\. Test the ideas in interviews with real customers from the target group

Turn each idea into an interview question, and recruit people who match the group. Watch for ideas that real customers don't recognize. Those are often the most useful result.

### 4\. Make the decision on the real evidence, traced to its source

Base the decision on themes from real interviews, each linked to the passages behind it. One good test is to ask, for every finding in the report, to see the transcript it came from.

![Diagram: four steps. Draft ideas with synthetic output, label it as not customer evidence, test with real customers from the target group, then decide on real evidence traced to transcripts.](https://pulsecorp.co/blog/content/images/2026/11/synthetic-respondents-vs-real-customers-h7pyad.webp "Steps 1 and 2 give you ideas to test. Only steps 3 and 4 give you evidence.")

Steps 1 and 2 give you ideas to test. Only steps 3 and 4 give you evidence.

## A juice company tests a new flavor

Say the research team at the juice company is shaping a new flavor for young parents in two countries. Your manager wants a direction within a month.

The team first runs a synthetic sample to draft ideas. Parents might want less sugar, might not trust unusual fruit, and might respond to messages about health. The team also uses the synthetic sample to try out its screener and its interview guide.

The synthetic output can show which questions are clear and which reactions are worth testing. It can't show which reasons are real, how strongly each group holds them, or how the two countries differ. The study across cultures above warns about that last point.

So the team interviews real parents in both countries, in their own languages. It weighs the interview themes, each traced to its transcript, above anything the synthetic sample said. The next decision, which flavor to make first, rests only on what real parents said.

## Where does Pulse fit?

Pulse, the AI-moderated research platform, interviews real people. It recruits participants to your profile, with fraud checks on every participant. Or it interviews your own customers through an uploaded list or a shared link.

The AI moderator follows each answer. Its follow-ups build on what the person said earlier, and a vague reply gets a second question. Pulse then groups what people said into themes. Every theme carries its quotes, and every quote links to the transcript it came from. Studies run across several markets at once, in 240+ languages, enabled on request. See how [AI-moderated interviews with real customers](https://pulsecorp.co/product/ai-interviews#:~:text=In%20a%20pre%2Dregistered%2C%20between%2Dsubjects,across%20three%20industry%20partners.) work.

Pulse offers a free trial, and it also offers a free study. To see it on your own question, [book a demo of Pulse](https://pulsecorp.co/demo/).

## Questions readers ask

### 1\. What are synthetic insights?

Synthetic insights are findings made from synthetic respondents, which are language models told to answer as members of a target group. They can sketch how a group might answer. But no real person stands behind them, so test them with real customers before any decision.

### 2\. Can synthetic respondents replace a survey panel?

Not for decisions about particular groups of customers. The 2026 study found models less accurate than a simple lookup by demographics for single people. It also found they made the gaps between groups look bigger. They can help a panel with trying out questions and drafting ideas.

### 3\. What value do synthetic respondents bring to research teams?

They bring speed at the start of a project. They help a team explore possible reactions, try out wording and plan an interview guide before real interviews begin.

### 4\. How do you check whether synthetic output is accurate?

Compare it with real data on the same questions. Run a few real interviews or survey answers from the target group. Then see where the synthetic answers agree, where they differ and whether they make the gaps between groups look bigger.

### 5\. Is an AI-moderated interview the same as a synthetic respondent?

No. In an AI-moderated interview, the AI asks the questions and a real person answers, so every finding traces to a real transcript. A synthetic respondent is the AI answering. Our guide to [AI-moderated versus human-moderated interviews](https://pulsecorp.co/blog/ai-moderated-interviews-vs-human-moderated-interviews-whats-the-difference) and our piece on how to [judge AI interview quality](https://pulsecorp.co/blog/good-probe-or-leading-question-how-to-judge-ai-interview-quality-in-one-transcri) go further.