Growth Experiment Planner
When to invoke
- "Plan an A/B test for the new onboarding flow."
- "How long do we need to run this experiment?"
- "Draft an experiment brief for the pricing page test."
Inputs needed
- Hypothesis — what change, expected effect, why.
- Primary metric — name, baseline rate or mean, variance if known.
- Traffic — daily users hitting the surface.
- MDE — minimum detectable effect (relative).
- Variants — control + N treatments.
- Guardrails — metrics that must NOT regress (revenue, errors, latency).
Workflow
- Frame — restate hypothesis in one sentence.
- Size — call
plan.py to compute sample size and runtime.
- Spec — generate experiment brief: metrics, segments, allocation, stopping rules, guardrails.
- Checklist — pre-launch QA, holdout, instrumentation, rollback path.
- Hand off — output a Markdown brief ready for LaunchDarkly/Optimizely.
Output format
A complete experiment brief with: Hypothesis, Variants, Metrics, Sample size, Runtime, Allocation, Guardrails, Stopping rules, QA checklist, Rollback plan.
Guardrails
- Always require a primary metric defined before launch (no metric fishing).
- Require explicit guardrails — at minimum: error rate, p95 latency, revenue per user.
- Flag if runtime exceeds 4 weeks (novelty + seasonality risk).
Reference code
plan.py computes two-proportion sample size (Evan Miller formula) or two-sample t-test sample size.
1---2name: growth-experiment-planner3description: Use when planning A/B tests in LaunchDarkly, Optimizely, or similar platforms. Sizes the experiment (sample size, MDE, runtime), drafts hypothesis + success metrics + guardrails, and produces a launch checklist + rollback plan.4---56# Growth Experiment Planner78## When to invoke9- "Plan an A/B test for the new onboarding flow."10- "How long do we need to run this experiment?"11- "Draft an experiment brief for the pricing page test."1213## Inputs needed141. **Hypothesis** — what change, expected effect, why.152. **Primary metric** — name, baseline rate or mean, variance if known.163. **Traffic** — daily users hitting the surface.174. **MDE** — minimum detectable effect (relative).185. **Variants** — control + N treatments.196. **Guardrails** — metrics that must NOT regress (revenue, errors, latency).2021## Workflow221. **Frame** — restate hypothesis in one sentence.232. **Size** — call `plan.py` to compute sample size and runtime.243. **Spec** — generate experiment brief: metrics, segments, allocation, stopping rules, guardrails.254. **Checklist** — pre-launch QA, holdout, instrumentation, rollback path.265. **Hand off** — output a Markdown brief ready for LaunchDarkly/Optimizely.2728## Output format29A complete experiment brief with: Hypothesis, Variants, Metrics, Sample size, Runtime, Allocation, Guardrails, Stopping rules, QA checklist, Rollback plan.3031## Guardrails32- Always require a primary metric defined before launch (no metric fishing).33- Require explicit guardrails — at minimum: error rate, p95 latency, revenue per user.34- Flag if runtime exceeds 4 weeks (novelty + seasonality risk).3536## Reference code37`plan.py` computes two-proportion sample size (Evan Miller formula) or two-sample t-test sample size.
sisodiabhumca