alaimo-labs Falsifiable Hypotheses
A conviction is something the team believes ("users want faster reports"). A hypothesis is a claim that evidence could prove wrong. Prototypes built without a falsifiable hypothesis produce demos, not learning.
Language
Write hypotheses and experiment docs in the language of the conversation.
Hypothesis structure
# Hypothesis: {short name}
**Tests:** {the `product/overview.md` belief this hypothesis puts at stake, quoted — omit when no belief registry exists}
**We believe that** {specific user segment, characterized by behavior or role}
**will** {observable behavior}
**because** {the underlying need or pain}.
**We will know this holds if** {behavioral signal + concrete threshold}
**within** {timeframe}.
**We will know this failed if** {behavioral signal that refutes it}.
Quality bar
- Falsifiable. Describable evidence could make the team say "this didn't hold". If no imaginable result would change the next decision, it's not a hypothesis — it's a plan wearing a costume.
- One claim per hypothesis. "Users will sign up AND invite teammates AND pay" is three hypotheses; test the load-bearing one first.
- Behavioral signals over opinions. "Users say they like it" is not a signal; "40% of users who open the report export it within the first session" is. Design the prototype to capture the behavior (events, persisted actions, completion of a real task) — see the course principle: signals beyond opinion-based feedback.
- Segment characterized by behavior or role, never "everyone" and never a traffic percentage.
- Threshold set before the experiment runs. Deciding what "good" looks like after seeing the data is the most common way teams fool themselves.
- Time-boxed. An experiment without an end date never concludes.
From hypothesis to experiment
For each hypothesis, define before building:
- Smallest exposure that tests it — what's the least product needed to elicit the behavior? (An exposure plan orders these when there are several layers — see the
exposure-plansskill.) - Instrumentation — which events/records capture the signal, and where they'll be stored (analytics events, database rows, form submissions).
- Decision rule — advance / iterate / stop, mapped to signal outcomes, written down in advance.
Save to product/hypotheses/{slug}.md. When the experiment concludes, append an Outcome section: observed signal, decision taken, and what was learned — this becomes the evidence trail for the final presentation. If the hypothesis cites a product/overview.md belief, the Outcome is also that belief's verdict: propose annotating it on its own line — — confirmed/contradicted/weakened by [hypotheses file] (date), fixed English keywords — with the user's approval, never silently.
Anti-patterns
- Vanity metrics as signals (page views, sign-ups without activation).
- "Validate the prototype" as a goal — prototypes aren't validated, beliefs are.
- Moving the threshold after seeing results.
- Testing a hypothesis whose failure wouldn't change anything you'd do next.