# Experiment Design

> Analytics and experimentation skill for product managers.

- Skill: `sethdford/experiment-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add sethdford/experiment-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sethdford/experiment-design/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Data & Analytics
- Author: sethdford (https://skillmd.com/u/sethdford)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/sethdford/experiment-design

---


# experiment-design

Measure product performance and customer behavior to drive data-informed decisions.

## Context

You are defining metrics or running experiments. If you have product data, usage patterns, or business goals, use them. Your goal is clear measurement frameworks.

## Domain Context

- **Metrics hierarchy**: Strategic (north star) → operational → diagnostic
- **Leading vs. lagging indicators**: Early signals vs. business results
- **Causation vs. correlation**: Be careful about assuming cause
- **Cohort analysis**: Track specific user groups over time
- **Experimentation velocity**: Fast learning beats big bets

## Instructions

1. **Define success metrics**: What measures success?
2. **Identify leading indicators**: What predicts future success?
3. **Set targets**: What's good performance?
4. **Establish baseline**: What's performance today?
5. **Monitor regularly**: Daily/weekly reviews to detect changes
6. **Investigate changes**: When metrics move, understand why

## Output Artifact

Metrics framework or experiment plan including:
- Key metrics and definitions
- Baseline and targets
- Data collection method
- Review cadence
- Success criteria

## Anti-Patterns

- **Vanity metrics**: Tracking volume without quality/retention
- **Too many metrics**: Tracking 50+ metrics dilutes focus
- **No leading indicators**: Only looking at lagging metrics
- **Not investigating changes**: Metrics move without understanding why

## Further Reading

- *Lean Analytics* (Croll & Yoskovitz) — metrics frameworks
- *Inspired* (Marty Cagan) — product metrics