robzilla1738 Fuzz & Property Harness
Static hunting finds suspicious code. This skill proves it — or finds what static review missed — by executing the code against generated inputs and checking that invariants hold. It's the dynamic counterpart to the bughunt lenses.
Use it standalone to harden a function, or as the confirm step of a hunt: convert a
Probable finding into Confirmed with a failing test.
This skill runs code. The loop is: discover → pick technique → generate harness → execute → shrink → emit. It writes only test/harness code, never product fixes.
The loop
| Step | Do |
|---|---|
| 1 Discover | Find fuzzable targets. For a whole-module pass, run python3 ${CLAUDE_PLUGIN_ROOT}/skills/bughunt/scripts/bughunt.py census --functions — it shortlists pure-ish functions (no obvious side effects) with their params, the best fuzz candidates. For a Probable finding from a hunt, the target is the cited function. |
| 2 Pick technique | Choose property / fuzz / differential / metamorphic (table below) and design the oracle. |
| 3 Detect & generate | Detect the project's test runner and property/fuzz library (table below). Generate a harness in the project's own conventions. If the library isn't installed, ask before installing it — never add a dependency silently. |
| 4 Execute | Run it via the project's runner. A real run that fails is the proof; a green run is evidence the property holds for the explored space. |
| 5 Shrink | Let the framework minimize the failing input (most shrink automatically), or minimize by hand to the smallest case that still fails. |
| 6 Emit | Keep or present the shrunk case as a named regression test, and record a findings-JSON entry with verified: {by:"fuzz", method:"property-test", verdict:"upheld"} so triage/merge can fold it into the report as Confirmed. |
When there's no test infrastructure
If the project has no test runner or the property library can't be installed (offline, user
declines), degrade gracefully: generate the harness and the concrete boundary inputs,
hand-trace the most suspicious case, and present the ready-to-run harness with a one-line
"install X and run Y to confirm" — clearly marked not executed. Never report a fuzz
finding as Confirmed unless you actually ran it.
Pick the technique
| Technique | Use when | What you need |
|---|---|---|
| Property-based testing | A function has a stated property/invariant (round-trip, idempotence, ordering, bounds) | An oracle: a property that must always hold |
| Fuzzing | A surface parses/decodes untrusted or complex input and must never crash/hang | A target function + a "does not crash/leak/hang" oracle |
| Differential testing | Two implementations should agree (old vs new, fast vs reference, two libs) | Both impls + an equality oracle |
| Metamorphic testing | No oracle exists, but transformed inputs have predictable relations | A relation (e.g. sort(reverse(x)) == sort(x)) |
Choosing the oracle (the hard part)
The bug-finding power is in the property, not the input generator. Good oracles:
- Round-trip:
decode(encode(x)) == x;parse(render(x)) == x. - Invariants: output always sorted; balance never negative; size within bounds.
- Idempotence / commutativity:
f(f(x)) == f(x); order independence. - Never-crash / never-hang: no panics, no unhandled exceptions, terminates under a timeout.
- Agreement: matches a slow reference impl or the previous version (differential).
- Conservation: counts/sums preserved across a transform.
Seed the generator with the boundary values the numeric/boundaries lens cares about: empty, single, max, negative, zero, NaN, huge, unicode, nested, duplicate.
Per-ecosystem harness sketches
These are starting points — match the project's existing test runner and conventions.
- Python —
hypothesis(@given(strategies...));atherisfor coverage-guided fuzzing. - JS/TS —
fast-check(fc.assert(fc.property(...)));jazzer.js/jsfuzzfor fuzzing. - Go — native
testing/quickandfunc FuzzXxx(f *testing.F)withgo test -fuzz. - Rust —
proptest/quickcheck;cargo-fuzz(libFuzzer) forfuzz_target!. - C/C++ — libFuzzer (
LLVMFuzzerTestOneInput) or AFL++; run under ASan/UBSan. - Swift —
SwiftCheckfor properties; libFuzzer via-sanitize=fuzzerfor C-interop surfaces. - JVM —
jqwikfor properties;Jazzerfor coverage-guided fuzzing.
Run fuzzers under sanitizers (ASan/UBSan/TSan) where available — they turn silent corruption into loud, locatable failures.
From crash to regression test
- Reproduce — capture the exact failing input.
- Shrink — let the framework minimize it (or minimize by hand) to the smallest input that still fails. Most property frameworks shrink automatically.
- Pin it — preserve the minimized case as a normal, named regression test so it can never come back silently. Note the invariant it violated.
- Report — hand the proven case back to triage to record as
Confirmed, with the failing test as the repro.
Report-only by default
This skill writes test/harness code, not product fixes. It surfaces and proves defects;
fixing the underlying code is handed to the human or the autoreview skill (/review).
Don't leave throwaway fuzz scaffolding behind — keep the shrunk regression test, remove the
rest unless the user wants a permanent fuzz target.