# Experiment

> Use when the plan rests on an unproven assumption that could be tested cheaply before committing serious effort — will people turn up, will anyone donate, does this message land, will the partner actually deliver. Designs the smallest test that could falsify the assumption, with a pass/fail line set in advance. Writes to the goal's experiments key.

- Skill: `skyf0xx/experiment` (Agent Skill)
- Install (CLI): `npx get-skill add skyf0xx/experiment`
- Raw SKILL.md: https://api.skillmd.com/api/skills/skyf0xx/experiment/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: skyf0xx (https://skillmd.com/u/skyf0xx)
- Updated: 2026-11-19
- Page: https://skillmd.com/skills/skyf0xx/experiment

---


# Skill: experiment

**Trigger**: The plan depends on something unproven that a small test could settle. Before
a big commitment of time, money, or credibility. Also use when `forecast` produces a
load-bearing estimate with low confidence, or when two people disagree about what will
happen and the disagreement is cheap to resolve empirically.

**Purpose**: Buy information before buying commitment.

The instinct on an unproven assumption is to argue about it or to proceed and find out.
There's usually a third option: a small, fast test that settles it for a fraction of the
cost. The discipline is designing the test so it can actually come back negative —
otherwise it's a demonstration, not an experiment, and it will confirm whatever was
already believed.

---

## Voice & Tone

Empirical and economical. You are buying information, and information has a price — the
question is always whether this test is worth what it costs.

Ruthless about falsifiability. A test that cannot fail is worthless and worse than
worthless, because it manufactures confidence. If the proposed test would look like
success under every outcome, say so and redesign it.

Never let a test become the work. A test that takes three weeks to run on a plan with a
six-week horizon isn't a test.

---

## Execution Sequence

### 1. Load Context

Read the goal — the goal statement, the `plan` key, the `forecasts` array if present, and any assumption flagged
as unverified by `systems`, `threat`, `premortem`, or `plan`.

### 2. Name the Assumption

One sentence, stated so it could be false:

```
ASSUMPTION: [the belief the plan rests on]
  Plan depends on it via: [which step or steps fail if this is wrong]
  Cost if we find out late: [what's already spent by then]
  Current confidence: high | moderate | low — [basis]
```

If the cost of being wrong is low, say so and skip the test. Not every assumption is worth
buying information about, and testing a cheap-to-be-wrong assumption is itself waste.

### 3. Design the Smallest Test

```
TEST: [what you actually do]
  Measures: [the specific observable]
  Cost: [time, money, effort, credibility]
  Duration: [how long until you have an answer]
  Who's involved: [and what they're asked to do]
```

Push for the smallest version. Common compressions worth suggesting:

- **Ask before building.** A conversation with five people in the target group often
  settles what a full campaign would.
- **A commitment, not an opinion.** "Would you come?" is nearly worthless; "will you put
  your name down for the 30th?" is a real signal. Costless agreement predicts nothing.
- **Test one segment**, one suburb, one channel, one list — before all of them.
- **A manual version first.** Do by hand what the plan proposes to do at scale, once, for
  a few people.
- **A precedent search** instead of an experiment — if someone has already run this test,
  a web search (where the tool is present) is faster and free.

### 4. Set the Pass/Fail Line — Before Running

**This is the step that makes it an experiment.** Set it in advance, in writing:

```
PASS IF: [specific threshold]
FAIL IF: [specific threshold]
AMBIGUOUS IF: [the range that settles nothing]

If it passes, we will: [the specific next action]
If it fails, we will: [the specific alternative — and it must be a real one]
```

Two tests to apply before running anything:

- **If the answer wouldn't change what you do, don't run it.** Both branches must lead
  somewhere different, or the test is theatre.
- **If you can't state a failing result, the test is unfalsifiable.** Redesign it.

Set the threshold before seeing data. A threshold set afterwards will be set wherever the
result landed — reliably, and without anyone noticing they've done it.

### 5. Check for Contamination

```
Would this test's result be misleading because:
  - the sample is people who already agree?
  - the user's own effort is doing what the plan assumes will happen naturally?
  - the timing is unrepresentative — a news cycle, a holiday, a one-off?
  - running the test changes the thing being tested?
```

The first is by far the most common. Testing a message on people who already support the
cause tells you nothing about people who don't, and it feels like validation.

### 6. Elicit

```
Two questions:

  - What result are you hoping for? Say it out loud — it's the bias to watch.
  - If it comes back negative, will you actually change course, or look for a reason it
    doesn't count?
```

The second question is uncomfortable and worth asking. If the honest answer is no, the
test is a waste of time and the decision has already been made — better to name that and
proceed deliberately (`decide`) than to dress a commitment up as an enquiry.

### 7. Record and Run

Add new experiments to the `experiments` array, or update existing entries once they complete — this is the array `write_section` writes in step 8 below. Each entry must have `assumption`, `test`, `passIf`, `by` (YYYY-MM-DD), and `done` (boolean). Once complete, set `done: true`, `result` (outcome), and optionally `changedAsResult` (what changed because of this result). `detail` (optional, max 280 chars) is a hover tooltip in the visual layer — non-obvious context on why this assumption or test matters. Fill in only when worth preserving beyond the fields above.

New unstarted experiment:
```json
{
  "experiments": [
    {
      "assumption": "People in the target group will commit if asked directly",
      "test": "Ask five people from the target group whether they will attend if scheduled",
      "passIf": "Three or more say yes and confirm availability",
      "by": "2026-10-15",
      "done": false
    }
  ]
}
```

Completed experiment (update the same entry):
```json
{
  "experiments": [
    {
      "assumption": "People in the target group will commit if asked directly",
      "test": "Ask five people from the target group whether they will attend if scheduled",
      "passIf": "Three or more say yes and confirm availability",
      "by": "2026-10-15",
      "done": true,
      "result": "pass — four of five confirmed",
      "changedAsResult": "proceed to booking; commit now rather than build interest first"
    }
  ]
}
```

Ambiguous is a legitimate outcome and must be recorded as such. Either design a sharper test or proceed knowing the assumption is still open.

### 8. Update the Goal and Name the Next Step

Call `write_section` on `experiments` with the array from step 7. When one resolves, name the affected assumption in the reply and
load `systems` or `plan` to carry it through. A falsified assumption sitting unchallenged in
the systems notes or the plan is worse than one never tested.

If the write returns { ok: false, errors }, fix the reported fields and retry before ending the turn.

```
Next: [run the test, or the first step of it]

Or:
  - It passed — commit and sequence → plan
  - It failed and the assumption sits in the systems read → systems
  - It failed — the focus may be wrong → strategy
  - Ambiguous — sharpen the test, or decide without it → decide
  - Someone's already run this test → web search
```

