# Survey Design

> When to activate: survey design, questionnaire design, Likert scale, survey bias, sampling strategy, response rate, NPS, CSAT, survey analysis, survey questions

- Skill: `mattakushi432/survey-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add mattakushi432/survey-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/mattakushi432/survey-design/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: Mattakushi432 (https://skillmd.com/u/mattakushi432)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/mattakushi432/survey-design

---


# Survey Design Patterns

## Survey Architecture

### Survey Structure
```
1. Introduction (required)
   - Who you are, why you're asking, how long it takes
   - Anonymity/confidentiality statement
   - Incentive (if any)

2. Screener questions (optional)
   - Qualify respondents before going deep
   - Use skip logic to exit non-qualifiers early

3. Warm-up questions (2-3)
   - Easy, non-threatening
   - Related to topic but not the core issue

4. Core questions (main body)
   - Most important questions first
   - Most sensitive/demographic last

5. Open-ended (1-3)
   - "Anything else you'd like to share?"
   - Keep optional

6. Demographic questions (end)
   - Age, role, company size, etc.
   - Only ask what you'll actually use
```

## Question Types

### Closed Question Types
| Type | Format | Best For |
|------|--------|---------|
| Likert | Agree/Disagree 5 or 7 point | Attitudes, satisfaction |
| Rating scale | 1-10 | NPS, CSAT, effort |
| Multiple choice | Single select | Categorical, mutually exclusive |
| Multi-select | Check all that apply | Features, channels |
| Ranking | Order items | Priority, preference |
| Dichotomous | Yes/No | Screening, facts |
| Matrix | Multiple questions, same scale | Efficiency for related items |

### Likert Scale Design
```
5-point: Strongly Disagree / Disagree / Neutral / Agree / Strongly Agree
7-point: Adds "Somewhat" variants for more nuance
Use 7-point for research requiring finer distinctions

Label ALL points (not just ends) — anchors reduce variability

Balanced scale (equal + and - options):
Strongly Disagree — Disagree — Neither — Agree — Strongly Agree

Avoid:
- Unbalanced: Poor / Fair / Good / Very Good / Excellent (positive skew)
- Unless measuring satisfaction (acceptable to use positive-anchored scale)
```

### NPS Question
```
"How likely are you to recommend [product] to a friend or colleague?"
Scale: 0 (Not at all likely) — 10 (Extremely likely)

Segmentation:
0-6: Detractors
7-8: Passives
9-10: Promoters

NPS = % Promoters − % Detractors
Range: −100 to +100

Benchmarks (SaaS):
< 0: Danger zone
0-30: Room for improvement
30-70: Good
> 70: Excellent (Apple ~72, Slack ~51)
```

## Bias Prevention

### Common Survey Biases
| Bias | Description | Fix |
|------|-------------|-----|
| **Leading questions** | "Don't you agree that...?" | Neutral phrasing |
| **Double-barreled** | "Is the product fast and reliable?" | Split into 2 questions |
| **Social desirability** | Answering to look good | Anonymous survey, indirect phrasing |
| **Acquiescence bias** | Tendency to agree | Mix positively + negatively worded items |
| **Order effect** | Earlier answers influence later ones | Randomize item order |
| **Recency effect** | Last options chosen more often | Randomize response options |
| **Primacy effect** | First options chosen more often | Randomize or use grid |
| **Framing bias** | "Save 9 of 10 patients" vs "1 in 10 die" | Test both frames or use neutral |

### Question Writing Rules
```
✅ Good:
"How often do you use [feature]?"
  ○ Daily   ○ Weekly   ○ Monthly   ○ Rarely   ○ Never

❌ Bad:
"Don't you think [feature] makes things easier?"  ← leading
"How often do you use this great feature?"  ← loaded
"How often do you use [feature] and would you recommend it?"  ← double-barreled

✅ Good (sensitive topic):
"Some people feel [behavior A] while others feel [behavior B]. 
 Which is closer to your experience?"

❌ Bad:
"Do you ever [embarrassing behavior]?"
```

## Sampling Strategy

### Probability Sampling (for generalization)
| Method | Description | When to Use |
|--------|-------------|-------------|
| Simple random | Every member equal chance | Homogeneous population |
| Systematic | Every Nth person | Lists of customers |
| Stratified | Random within subgroups | Need segment representation |
| Cluster | Random groups, survey all | Geographically dispersed |

### Non-Probability Sampling (for exploration)
| Method | Description | When to Use |
|--------|-------------|-------------|
| Convenience | Whoever is available | Quick pulse checks |
| Purposive | Select by criteria | Qualitative follow-up |
| Snowball | Referrals | Hard-to-reach populations |
| Quota | Fill predefined cells | Ensure segment coverage |

### Sample Size Calculator
```python
import math

def sample_size(population, confidence=0.95, margin_error=0.05, p=0.5):
    """
    population: total population size (use 1e9 for unknown)
    confidence: 0.90, 0.95, or 0.99
    margin_error: desired margin of error (0.05 = ±5%)
    p: expected proportion (0.5 = most conservative)
    """
    z_scores = {0.90: 1.645, 0.95: 1.96, 0.99: 2.576}
    z = z_scores[confidence]
    
    n_inf = (z**2 * p * (1-p)) / margin_error**2
    n = n_inf / (1 + (n_inf - 1) / population)
    
    return math.ceil(n)

# Examples
print(sample_size(10000))    # 370 for 95% CI ±5%
print(sample_size(100000))   # 383 for 95% CI ±5%
print(sample_size(1e9))      # 385 (large population asymptote)
```

## Response Rate Optimization

### Response Rate Benchmarks
| Channel | Typical Rate |
|---------|-------------|
| Email (customers) | 10-30% |
| Email (cold/list) | 1-5% |
| In-app survey | 15-40% |
| SMS | 20-45% |
| Pop-up (web) | 2-10% |
| Panel (incentivized) | 60-80% |

### Improving Response Rate
- **Keep it short** — 5 min target, show progress bar
- **Time it right** — after successful interaction, not mid-task
- **Mobile-optimize** — >60% open email on mobile
- **Personalize subject** — "[Name], 2 min to improve [product]"
- **Pre-notify** — "Next week we'll ask you about [topic]"
- **Incentivize** — gift card, donation, exclusive content
- **Follow up once** — single reminder 3-5 days later
- **Executive signature** — CEO name increases open rate for B2B

## Analysis and Reporting

### Likert Analysis Approaches
```
Approach 1: Treat as ordinal (conservative)
- Use median and IQR
- Non-parametric tests (Mann-Whitney, Kruskal-Wallis)

Approach 2: Treat as interval (common in practice)
- Use mean and std dev
- Visualize with stacked bar (% favorable vs unfavorable)

Top-2-box score:
% who chose top 2 responses (Agree + Strongly Agree)
Most actionable for tracking over time
```

### Survey Report Template
```
## Key Findings

### Executive Summary
[3 bullet points: what you found, what it means, what to do]

### NPS / Satisfaction Score
Current: X | Previous: Y | Change: +Z pts
Benchmark: Industry avg XX

### Top Pain Points
1. [Theme] — mentioned by X% of respondents
2. [Theme] — mentioned by X% of respondents

### Verbatim Highlights
"[Best positive quote]" — [Persona type]
"[Most actionable critical quote]" — [Persona type]

### Recommendations
1. [Action] — addresses [% of respondents who cited this]
2. [Action]

### Methodology
- N = X respondents
- Dates: [range]
- Channel: [email/in-app/etc]
- Response rate: X%
```