# Experiment

> Measure performance or tuning changes with a baseline, controlled comparison, correctness check, and recorded verdict. Use for speed, memory, latency, size, "ทดลอง", or any claim that one approach is better. Not for correctness bugs.

- Skill: `ohm41321/experiment` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ohm41321/experiment`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ohm41321/experiment/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: ohm41321 (https://skillmd.com/u/ohm41321)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ohm41321/experiment

---


# Experiment — no claim without a measurement

Optimization needs numbers; null results get recorded too.

## 1. Define the metric before touching code

Choose one command that prints the number: runtime, RSS, latency, size, or count.
Decide **now** what improvement would count, before seeing results. If no metric
command exists, build it first.

## 2. Baseline

Run at least 3 times and record all values, not only the mean. Pin what you can:
seed, input, cache state, and environment; state what remains uncontrolled.
Correctness verify must be green before and after.

## 3. One variable per experiment

Change one thing. Multiple changes make the result uninterpretable.

## 4. Measure again

Use the Same command, same repetitions, same conditions. A result must beat the
baseline spread; Inside the noise = **null result**.

## 5. Verdict and record

Follow the repository's existing experiment log; otherwise create and append to
`docs/experiments.md`:

```
## <date> — <hypothesis>
change: <one variable>
baseline: <all values> | result: <all values>
verdict: WIN <n%> | NULL (inside noise) | LOSS
decision: <kept or reverted + reason>
```

Losers and nulls are reverted immediately; the log preserves the finding, not
the bad diff. A null result is a finding—record it so it is not retried without
new evidence. Never delete a previous entry; append a correction when later
evidence overturns it. Route load-bearing nulls through `/retro`.

