# Experimental Design

> Best practices for designing reproducible ML experiments. Use when planning ablations, baselines, or controlled experiments.

- Skill: `enuno/experimental-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add enuno/experimental-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/enuno/experimental-design/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: enuno (https://skillmd.com/u/enuno)
- Updated: 2026-09-09
- Page: https://skillmd.com/skills/enuno/experimental-design

---


## Experimental Design Best Practice
1. ALWAYS include meaningful baselines (not just random):
   - At least one classical method baseline
   - At least one recent SOTA method baseline
   - A simple-but-strong baseline (e.g., linear probe, k-NN)
2. Use MULTIPLE random seeds (minimum 3, ideally 5)
3. Report mean +/- std across seeds
4. Design ablations that isolate EACH key component:
   - Remove one component at a time
   - Each ablation must be meaningfully different from baseline
5. Control variables: change only ONE thing per comparison
6. Use standard splits (train/val/test) — never test on training data
7. Report wall-clock time and memory usage alongside accuracy