Synthetic User Research Skill
AI personas are the most misused research tool of the decade — and genuinely useful inside a narrow lane. The difference is the question you ask them. Synthetic panels can catch comprehension failures, confusing flows, and survey defects before you spend real participants on them; they cannot tell you what people will pay for, feel, or do. This skill enforces the lane, then runs the method properly.
What This Skill Produces
- A fit verdict: is this question answerable synthetically at all? (Sometimes the deliverable is "no — here's the human study instead")
- A persona-panel design grounded in real data you already have, with provenance per persona
- Findings, labelled synthetic throughout, with confidence calibrated to the method's floor
- The human follow-up plan — what the synthetic pass earned you the right to test properly
The Lane (checked before anything runs)
Synthetic methods CAN usefully probe — because the answer lives in the artifact, not in human hearts:
- Comprehension: is this copy/onboarding/explanation understandable? Where does a reader stumble?
- Instrument defects: leading questions, double-barrelled items, missing answer options in a survey before fielding it
- Information architecture: can a goal-holder find the thing? Where does the nav mislead?
- Message differentiation: do these three positionings even read as different?
- Edge-case generation: what user situations did the design forget? (Personas as brainstorm, not oracle)
Synthetic methods CANNOT establish — refuse these, and say why:
- Willingness to pay, purchase intent, or price sensitivity (models have no budget and infinite agreeableness)
- Emotional response, delight, trust (simulated feeling is fluent and empty)
- Discovery of unknown needs (personas remix known data; discovery is precisely the unknown)
- Behavioural prediction (what people say is already unreliable; what a model says they'd say is worse)
- Validation for a launch/investment decision (synthetic evidence is not evidence of demand)
Required Inputs
Ask for (if not already provided):
- The research question (runs through the lane check first — verdict before method)
- Real data to ground personas: interview notes, support tickets, reviews, analytics segments. No real data → no panel: ungrounded personas are the model's stereotypes wearing name tags
- The artifact under test (the copy, flow, survey, IA)
- What decision this feeds — and its stakes (higher stakes shrink the lane)
Method (when the lane check passes)
- Build personas from data, with provenance. Each persona cites its sources ("from the 14 churn interviews: SMB admin, low technical confidence, evaluates in <10 min"). 4-6 personas spanning the real segment axes, including at least one hostile/low-attention profile — synthetic panels skew cooperative unless you force otherwise.
- Fight the agreeableness. Instruct personas to struggle where their profile would struggle; ask for failure ("where do you stop reading? what would make you give up?") rather than opinions ("do you like this?"); never ask satisfaction or intent questions — the lane forbids the questions models answer most fluently.
- Run artifact-grounded tasks. Give the persona the actual artifact and a goal; capture where it misreads, stalls, or takes the wrong path. Quote the artifact in every finding.
- Triangulate across personas and runs. A stumble that appears across 4/6 personas and repeated runs is a signal; a single eloquent complaint is noise wearing insight's clothes.
- Label relentlessly and hand off. Every output says SYNTHETIC at the top and per-finding. Findings convert to: fixes to the artifact (cheap, do now) and hypotheses for the human study (the follow-up plan names method, n, and what would confirm/refute).
Output Format
Synthetic Research Pass: [artifact] — ⚠️ SYNTHETIC SIGNAL, NOT USER EVIDENCE
Lane check: [question] → [in-lane ✅ / out-of-lane 🔴 with the human method to use instead]
Panel: [persona → grounded in → key traits] (provenance per persona)
Findings (each labelled synthetic)
| # |
Finding |
Artifact evidence (quoted) |
Personas affected |
Confidence |
Fixes now: [artifact changes the synthetic pass justifies — comprehension/IA/instrument defects]
For real humans: [hypothesis → method → n → what confirms/refutes] — the synthetic pass bought sharper questions, not answers
Quality Checks
Anti-Patterns
Example Trigger Phrases
- "Run synthetic user testing."
- "Simulate user reactions with AI personas."
- "Pretest a survey."
- "Decide whether synthetic research is appropriate at all."
1---2name: synthetic-user-research3description: "Use AI personas for early-stage research signal — with hard guardrails on what synthetic methods can and cannot validate. Use when asked to run synthetic user testing, simulate user reactions with AI personas, pretest a survey or message before fielding it, or decide whether synthetic research is appropriate at all. Produces a fit verdict for the question at hand, a persona-panel design grounded in real data, the findings labelled as synthetic throughout, and the follow-up plan with real humans. Never a substitute for discovery interviews — see discovery-interview-guide and user-research-synthesis for the real thing."4disable-model-invocation: true # skill-optimizer5metadata:6 source: https://skillmd.com/skills/slipcast-dev/synthetic-user-research7---89# Synthetic User Research Skill1011AI personas are the most misused research tool of the decade — and genuinely useful inside a narrow lane. The difference is the question you ask them. Synthetic panels can catch comprehension failures, confusing flows, and survey defects *before you spend real participants on them*; they cannot tell you what people will pay for, feel, or do. This skill enforces the lane, then runs the method properly.1213## What This Skill Produces1415- A **fit verdict**: is this question answerable synthetically at all? (Sometimes the deliverable is "no — here's the human study instead")16- A **persona-panel design** grounded in real data you already have, with provenance per persona17- **Findings, labelled synthetic throughout**, with confidence calibrated to the method's floor18- The **human follow-up plan** — what the synthetic pass earned you the right to test properly1920## The Lane (checked before anything runs)2122**Synthetic methods CAN usefully probe** — because the answer lives in the artifact, not in human hearts:23- **Comprehension**: is this copy/onboarding/explanation understandable? Where does a reader stumble?24- **Instrument defects**: leading questions, double-barrelled items, missing answer options in a survey *before* fielding it25- **Information architecture**: can a goal-holder find the thing? Where does the nav mislead?26- **Message differentiation**: do these three positionings even *read* as different?27- **Edge-case generation**: what user situations did the design forget? (Personas as brainstorm, not oracle)2829**Synthetic methods CANNOT establish** — refuse these, and say why:30- Willingness to pay, purchase intent, or price sensitivity (models have no budget and infinite agreeableness)31- Emotional response, delight, trust (simulated feeling is fluent and empty)32- Discovery of unknown needs (personas remix known data; discovery is precisely the unknown)33- Behavioural prediction (what people *say* is already unreliable; what a model says they'd say is worse)34- Validation for a launch/investment decision (synthetic evidence is not evidence of demand)3536## Required Inputs3738Ask for (if not already provided):39- **The research question** (runs through the lane check first — verdict before method)40- **Real data to ground personas**: interview notes, support tickets, reviews, analytics segments. *No real data → no panel*: ungrounded personas are the model's stereotypes wearing name tags41- **The artifact under test** (the copy, flow, survey, IA)42- **What decision this feeds** — and its stakes (higher stakes shrink the lane)4344## Method (when the lane check passes)45461. **Build personas from data, with provenance.** Each persona cites its sources ("from the 14 churn interviews: SMB admin, low technical confidence, evaluates in <10 min"). 4-6 personas spanning the *real* segment axes, including at least one hostile/low-attention profile — synthetic panels skew cooperative unless you force otherwise.472. **Fight the agreeableness.** Instruct personas to struggle where their profile would struggle; ask for failure ("where do you stop reading? what would make you give up?") rather than opinions ("do you like this?"); never ask satisfaction or intent questions — the lane forbids the questions models answer most fluently.483. **Run artifact-grounded tasks.** Give the persona the actual artifact and a goal; capture where it misreads, stalls, or takes the wrong path. Quote the artifact in every finding.494. **Triangulate across personas and runs.** A stumble that appears across 4/6 personas and repeated runs is a signal; a single eloquent complaint is noise wearing insight's clothes.505. **Label relentlessly and hand off.** Every output says **SYNTHETIC** at the top and per-finding. Findings convert to: fixes to the artifact (cheap, do now) and hypotheses for the human study (the follow-up plan names method, n, and what would confirm/refute).5152## Output Format5354### Synthetic Research Pass: [artifact] — ⚠️ SYNTHETIC SIGNAL, NOT USER EVIDENCE5556**Lane check:** [question] → [in-lane ✅ / out-of-lane 🔴 with the human method to use instead]5758**Panel:** [persona → grounded in → key traits] *(provenance per persona)*5960**Findings** *(each labelled synthetic)*61| # | Finding | Artifact evidence (quoted) | Personas affected | Confidence |62|---|---|---|---|---|6364**Fixes now:** [artifact changes the synthetic pass justifies — comprehension/IA/instrument defects]6566**For real humans:** [hypothesis → method → n → what confirms/refutes] — *the synthetic pass bought sharper questions, not answers*6768## Quality Checks6970- [ ] The lane check ran first, and out-of-lane questions were refused with the alternative named71- [ ] Every persona cites the real data it's built from — no data, no persona72- [ ] The panel includes hostile/low-attention profiles73- [ ] No finding reports simulated emotion, intent, or willingness to pay74- [ ] SYNTHETIC labelling survives copy-paste (it's in the findings, not just the header)75- [ ] The human follow-up plan exists — this method ends in better questions, never in validation7677## Anti-Patterns7879- [ ] Do not run synthetic "validation" for launch or investment decisions — that's laundering a model's agreeableness into evidence80- [ ] Do not build personas from vibes or market-report archetypes — stereotypes in, stereotypes out81- [ ] Do not ask personas how they *feel* or what they'd *pay* — the fluent answer is the false one82- [ ] Do not report synthetic findings in the same register as real research — a stakeholder who can't tell the difference wasn't told loudly enough83- [ ] Do not let a synthetic pass replace the discovery interview it was supposed to prepare — the lane is *before* human research, never instead of it8485## Example Trigger Phrases8687- "Run synthetic user testing."88- "Simulate user reactions with AI personas."89- "Pretest a survey."90- "Decide whether synthetic research is appropriate at all."