# Ab Testing Design

> A/B test design, experiment setup, variant design, and statistical analysis for designers.

- Skill: `dragoon0x/ab-testing-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add dragoon0x/ab-testing-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/dragoon0x/ab-testing-design/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: dragoon0x (https://skillmd.com/u/dragoon0x)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/dragoon0x/ab-testing-design

---


# Ab Testing Design

## Experiment Design

### What to Test
- **High impact, easy to change:** Copy, color, layout, CTA.
- **Clear hypothesis:** "Changing the CTA from 'Sign Up' to 'Start Free Trial' will increase conversions because it communicates zero commitment."

### What Not to Test
- Preferences ("Do users prefer blue or green?").
- Everything at once (test one variable).
- Insignificant elements (button border-radius doesn't matter).

## Variant Design
- Control (A) = current experience.
- Variant (B) = one change with a hypothesis.
- Don't create wildly different variants (you won't know what caused the difference).

## Statistical Rigor
- **Sample size calculator** before starting.
- **Run for full weeks** (not 2 days — behavior varies by day).
- **95% confidence minimum** before declaring a winner.
- **Watch for novelty effects** (new things get more clicks initially).

## Reporting
- State the hypothesis.
- Show the numbers (conversion rate, confidence interval).
- State the winner with context.
- Document learnings for future tests.