itseffi Experiment Design
Design robust experiments to test product hypotheses.
When to Use
When you have a hypothesis and need to design an experiment to validate it.
The Process
1. Factor Breakdown
Analyze your goal and system:
- What factors could influence the outcome?
- Which are controllable vs. environmental?
- What are the key variables?
2. Experiment Structure
For each factor, define:
- Hypothesis: What you expect to happen
- Independent variable: What you're changing
- Dependent variable: What you're measuring
- Control group: Baseline comparison
- Experimental group: Who gets the change
- Measurement method: How you'll collect data
- Confounding variables: What else could affect results
3. Sample Design
- Who participates?
- How many needed for statistical significance?
- How will you recruit/select?
4. Timeline
- How long to run?
- When to check results?
- What's the minimum detectable effect?
5. Stop/Scale Rules
- What results mean "stop"?
- What results mean "scale"?
- What's inconclusive?
Output Format
For each experiment:
- Hypothesis statement
- Variables (independent, dependent)
- Groups (control, experimental)
- Sample size and selection
- Measurement approach
- Success criteria
- Stop/scale rules
When Not to Use
Do not use this skill when the request is unrelated, low-stakes, or better handled by a simpler direct response.