Customer Survey Design
Design surveys that produce answers you can act on, not vanity data. The discipline is starting from the decision the survey must inform and writing questions that don't lead the respondent — most surveys fail by asking biased questions that confirm what the author hoped.
The second, quieter failure mode is fielding a survey that can't answer its own question: 60 responses split across five segments, then decisions made on ±13-point noise. This skill computes what a readable survey actually requires before anything is sent — and refuses to bless segment cuts the sample can't support.
When to use
The user is about to field a survey and needs the questions, structure, and sample plan to be right before it goes out.
Before you start
- Name the decision. What will you do differently based on the results? If a question doesn't inform a decision, cut it.
- Identify the respondents and how you'll reach them (existing customers, churned users, prospects, market panel) — this shapes wording and length. Get the reachable audience size and a realistic response rate if known.
- List the cuts you'll report. If the readout says "by segment," the sample must be sized for the smallest cell, not the total.
- Pick the survey type to match: PMF ("how would you feel if you could no longer use this?"), NPS + why, onboarding/CSAT, churn reason, or discovery.
Method
The engine (resources/survey-design.js, Node, stdlib only) computes:
- Required completes — n₀ = z²·p(1−p)/e², with finite-population correction n = n₀/(1+(n₀−1)/N) when the audience is finite. Worked check: ±5pp at 95% (z=1.96, p=0.5) → n₀ = 384.16 → 385; corrected for N=2,000 → 323.
- Invites — completes ÷ response rate, with a feasibility verdict against the audience size (loosen the margin, lift response, or run a census and report the achieved ±).
- Segment readability — worst-case margin per cell, e = z·√(0.25/n_cell), labelled READ (≤±5pp), CAUTION (±5–8pp, directional), DON'T READ (wider). Thresholds are working labels, not laws.
Modes: sample '<json>' and segments '<json>'. --demo runs both worked examples with no arguments; --help prints the schema.
Workflow
- Size the survey first — run
node resources/survey-design.js sample '<json>' with the margin, confidence, audience, and response rate. If it's not feasible, fix the plan before writing a single question.
- Check the cuts — run
segments mode with planned segment shares. Merge or oversample any DON'T READ cell now, not after fielding.
- Map questions to decisions — list the decisions, then write the minimum questions that inform each.
- Choose question types and scales using
resources/survey-template.md (open-ended for discovery, closed/scale for measurement; consistent scale direction).
- Write unbiased wording — neutral, one idea per question, no leading or loaded phrasing, balanced options.
- Order for flow and low drop-off — easy/engaging first, sensitive/demographic last, group by topic, keep it short.
- Add the key open-ended "why" after rating questions — that's where the actionable insight lives.
Present the result
Lead with the sample plan (completes, invites, feasibility, and the segment-readability table from the engine), then the survey in order (question, type, scale/options) with the decision each informs, an estimated completion time, and the recommended channel/timing.
Guardrails & common mistakes
- Don't promise segment reads the sample can't support. If a cell says DON'T READ, either resize or drop the cut from the readout.
- No leading questions. "How much did you love our amazing onboarding?" yields noise. Ask neutrally.
- One idea per question. Double-barreled questions ("Was it fast and easy?") can't be answered cleanly.
- Keep it short. Every extra question lowers completion and data quality.
- Balance the scale. Offer as many negative as positive options; include a neutral where appropriate.
- Pilot first. Test with a few people to catch confusing wording before fielding.
- Survey sampling ≠ experiment significance. Testing whether variant B beat variant A belongs in
ab-test-significance; this engine sizes reads, not experiments.
Related AAJ resources
Related skills
persona-builder (turn answers into personas) · positioning-statement (questions that test messaging) · lifecycle-and-retention (churn-reason surveys feed the retention diagnosis) · ab-test-significance (experiments, not survey reads).
Credits
Original AAJ skill. The Agent Skills format and Corey Haines' coreyhaines31/marketingskills (MIT) were references for structure and coverage; this skill is independently written. See the repository README.
1---2name: customer-survey-design3description: Use when the user wants to design a customer or market survey — choosing the questions, structure, scales, and sample size to get unbiased, decision-useful answers. Also use when the user mentions survey, questionnaire, NPS, customer feedback, market research survey, product-market fit survey, sample size, margin of error, "how many responses do I need," or "what should I ask my customers." Produces a structured survey with unbiased questions mapped to decisions, plus computed sample-size, invite, and segment-readability numbers from a runnable engine.4license: MIT5---67# Customer Survey Design89Design surveys that produce **answers you can act on**, not vanity data. The discipline is starting from the decision the survey must inform and writing questions that don't lead the respondent — most surveys fail by asking biased questions that confirm what the author hoped.1011The second, quieter failure mode is fielding a survey that can't answer its own question: 60 responses split across five segments, then decisions made on ±13-point noise. This skill computes what a readable survey actually requires **before** anything is sent — and refuses to bless segment cuts the sample can't support.1213## When to use1415The user is about to field a survey and needs the questions, structure, and sample plan to be right before it goes out.1617## Before you start18191. **Name the decision.** What will you do differently based on the results? If a question doesn't inform a decision, cut it.202. **Identify the respondents** and how you'll reach them (existing customers, churned users, prospects, market panel) — this shapes wording and length. Get the reachable audience size and a realistic response rate if known.213. **List the cuts you'll report.** If the readout says "by segment," the sample must be sized for the smallest cell, not the total.224. **Pick the survey type** to match: PMF ("how would you feel if you could no longer use this?"), NPS + why, onboarding/CSAT, churn reason, or discovery.2324## Method2526The engine (`resources/survey-design.js`, Node, stdlib only) computes:2728- **Required completes** — n₀ = z²·p(1−p)/e², with finite-population correction n = n₀/(1+(n₀−1)/N) when the audience is finite. Worked check: ±5pp at 95% (z=1.96, p=0.5) → n₀ = 384.16 → **385**; corrected for N=2,000 → **323**.29- **Invites** — completes ÷ response rate, with a feasibility verdict against the audience size (loosen the margin, lift response, or run a census and report the achieved ±).30- **Segment readability** — worst-case margin per cell, e = z·√(0.25/n_cell), labelled READ (≤±5pp), CAUTION (±5–8pp, directional), DON'T READ (wider). Thresholds are working labels, not laws.3132Modes: `sample '<json>'` and `segments '<json>'`. `--demo` runs both worked examples with no arguments; `--help` prints the schema.3334## Workflow35361. **Size the survey first** — run `node resources/survey-design.js sample '<json>'` with the margin, confidence, audience, and response rate. If it's not feasible, fix the plan before writing a single question.372. **Check the cuts** — run `segments` mode with planned segment shares. Merge or oversample any DON'T READ cell now, not after fielding.383. **Map questions to decisions** — list the decisions, then write the minimum questions that inform each.394. **Choose question types and scales** using `resources/survey-template.md` (open-ended for discovery, closed/scale for measurement; consistent scale direction).405. **Write unbiased wording** — neutral, one idea per question, no leading or loaded phrasing, balanced options.416. **Order for flow and low drop-off** — easy/engaging first, sensitive/demographic last, group by topic, keep it short.427. **Add the key open-ended "why"** after rating questions — that's where the actionable insight lives.4344## Present the result4546Lead with the sample plan (completes, invites, feasibility, and the segment-readability table from the engine), then the survey in order (question, type, scale/options) with the decision each informs, an estimated completion time, and the recommended channel/timing.4748## Guardrails & common mistakes4950- **Don't promise segment reads the sample can't support.** If a cell says DON'T READ, either resize or drop the cut from the readout.51- **No leading questions.** "How much did you love our amazing onboarding?" yields noise. Ask neutrally.52- **One idea per question.** Double-barreled questions ("Was it fast and easy?") can't be answered cleanly.53- **Keep it short.** Every extra question lowers completion and data quality.54- **Balance the scale.** Offer as many negative as positive options; include a neutral where appropriate.55- **Pilot first.** Test with a few people to catch confusing wording before fielding.56- **Survey sampling ≠ experiment significance.** Testing whether variant B beat variant A belongs in `ab-test-significance`; this engine sizes *reads*, not experiments.5758## Related AAJ resources5960- Interactive tool: https://aajconsult.com/tools/survey-studio6162## Related skills6364`persona-builder` (turn answers into personas) · `positioning-statement` (questions that test messaging) · `lifecycle-and-retention` (churn-reason surveys feed the retention diagnosis) · `ab-test-significance` (experiments, not survey reads).6566## Credits6768Original AAJ skill. The Agent Skills format and Corey Haines' `coreyhaines31/marketingskills` (MIT) were references for structure and coverage; this skill is independently written. See the repository README.
sarojkjha