Fuzz Before You Claim Done

Fuzz before you claim "done"

Purpose

A hand-picked test list is a mirror: it verifies the cases its author imagined, and the author of the tests is usually the author of the change. In someone else's repo that gap lands on the maintainer — the reviewer becomes your fuzzer. A seeded property sweep is the cheapest adversarial reviewer you can hire before the human one: randomized structures, deterministic per seed, checked against invariants ("no writes lost", "exactly once", "resume equals uninterrupted") rather than expected outputs. When it fails, the seed is the whole repro.

When to use

• Any change to execution semantics: scheduling, barriers/joins, retries, caching, interrupts/resume, checkpointing — anything where inputs combine.
• Before writing "verified" / "all cases covered" in a PR body or a handoff.
• When your deterministic suite is green and you notice every test was written by the same mind that wrote the code.

When NOT to use

Pure data transforms with enumerable inputs — a table-driven test is clearer. And don't ship the 10,000-seed run in CI: sweep large once, commit a small seeded subset, record the large run's result where the reviewer can see it.

Two adjacent adversaries answer different doubts, and a fuzz run answers neither: red-team-your-own-diff attacks the diff with a named reviewer's own recorded rules, and run-the-bots-review-before-it-does pre-runs the automated reviewer's knowable checklist. Volume in one of the three is not coverage of the other two.

The practice (checklist)

• Generate structures, not just values: random graph/schedule shapes — branches of different depths, loops with bounded exits, dynamic dispatch, 1–2 instances of the feature mixed with defaults.
• Deterministic per seed (e.g. mulberry32): a failure message carrying only its seed must reproduce the run exactly.
• Assert invariants, not outputs: "target runs exactly once iff a source ran", "a completed run leaves nothing armed", "interrupt+resume produces the uninterrupted result". Derive the oracle from the contract, not from the code.
• Instrument the mechanism (TRAP): log every state transition of the thing under test (each barrier write with before/after) — buffered per seed, dumped only on failure. A failing seed without a trace is a second investigation.
• Differential where possible: run the same seed uninterrupted vs interrupted-and-resumed, or feature-on vs feature-off, and compare.
• Classify non-done outcomes (recursion limit, error) instead of ignoring them — "limit hit on a loop-free graph" is a finding, not noise.
• Chase the literature's named pathologies as targeted cases (for joins: the OR-join vicious circle) — fuzz finds the unknown unknowns, the literature names the known ones.
• When a failure is a semantics surprise rather than a bug, pin it as a deterministic test and document the behavior — the fuzz just wrote your docs.
• Vary the axes the battery holds constant, not only the inputs: any flag that changes a persisted shape gets a flip test — create state under configuration A, resume under configuration B, both directions. 2,400 seeds missed a silent thread-corruption bug because every seed compiled the graph once; the bug lived in recompiling with a different flag over a live thread.

Rationalizations

RECEIPT

SOURCING-receipt, internal (pending publication of the branch it guarded). 2026-08-14, langgraphjs inclusive-waiting-edge work: a 2,400-seed sweep over random graph shapes (conditional entry subsets, chains at different depths, bounded loops, Send dispatchers) with every barrier write TRAP-logged, run after 12 hand-picked tests were green on the first attempt. Round 1: 29 failing seeds → one real defect the hand-picked list could not have expressed — a run interrupted exactly at the release point reported the documented "run is over" signal (next: []), so a client following the docs never resumed and the release never happened — plus one undocumented semantics (once-per-arming on double-triggered sources) that became JSDoc and a pinned test. Round 2, after the fix: 0 of 2,400. The same day, targeted literature cases (OR-join vicious circle) and a kill -9 crash test rode the same harness. Evidence: archived probes with as-run results in the practice's private flight recorder; ripens into a public receipt when the branch ships.

Lifecycle

Sharpen after each use: record in the LEDGER which invariant caught something, which generated shape never fired (delete it), and every claim the sweep proved wrong. Retire if two consecutive uses on real work produce zero findings beyond the deterministic suite — that would mean the suites got better, which is the goal.