ashermahonin Hypothesis Validator
Purpose
Stop the agent from treating opinions as decisions. Force every load-bearing claim through a falsifiable-hypothesis form, then run the cheapest useful test to settle it before commitment.
Inputs
- Read
references/hypothesis-format.md. - List every load-bearing claim attached to the current decision: stack choice, framework choice, model choice, scope inclusion, performance promise, market segment, user behavior assumption.
- For each claim, ask: "what would prove this wrong?" If no answer, refactor the claim until one exists.
- Use Context7 MCP for any claim that depends on current external technology, market, or platform behavior.
Decision process
- For each candidate claim, write a hypothesis card: name, claim, why we believe it, kill criterion, cheapest useful test, owner skill, status.
- Rank hypotheses by impact × uncertainty. The riskiest survives — test it first.
- Choose the cheapest useful test per hypothesis: spike, paper review, eval set, prototype, telemetry probe, A/B, expert interview, vendor docs lookup, regulatory check.
- Run or hand off the test to the matching skill (
service-implementationfor spikes,research-domainfor market,qa-evalfor evals,security-owasp-*for safety claims,cve-zero-day-scannerfor dependency claims). - Record the outcome: supported, disproved, deferred (with reason), or still-open (with planned next test).
- Update the hypothesis register and notify
ai-pdlcof phase-boundary impact.
Decision boundaries
- Use Context7 MCP whenever an external fact gates the test design.
- Keep a decision trace: claim, candidate tests, chosen test, outcome, residual uncertainty.
- Refuse to mark a hypothesis Supported without evidence the test method was sound, not only that the result was favorable.
- Escalate when a "kill criterion" cannot be defined; that is itself the finding.
Decision record
- Hypothesis register entries (one card per claim)
- Ranked test list with cheapest-useful-test rationale
- Test outcomes per hypothesis
- Hypothesis-derived risks pushed into
07-risk-register - Phase-boundary signal to
ai-pdlc
Ready when
- No claim survives without a kill criterion.
- No kill criterion accepted that is not measurable.
- No test promoted to "Supported" from a single sample where multiple were possible.
- No silent collapse of two hypotheses into one.
Handoff
Hand off settled hypotheses to ai-pdlc and architecture-review. Hand the unsettled ones back into the register and to the owner skill that runs the next-cheapest test.
References
references/hypothesis-format.md: hypothesis card schema, ranking method, cheapest-useful-test catalogue, anti-patterns.