Beyond The Fix

Beyond the Fix

A patch is the highest-signal artifact in any repository. Someone already did the expensive part: they found the class, located the file, and wrote down what they believed was wrong. What they did not do — because nobody does — is keep looking after the tests went green.

The thesis, in one line:

That someone fixed it means they fixed that, and nothing else.

A fix is evidence about the past: something was wrong here, and a human confirmed it. It is not evidence about the present. Those are different claims, and the entire discipline of this skill lives in the gap between them.

Three biases produce that gap, and all three are structural rather than personal:

1. Attention collapses at green. The search terminates the moment the reporter's PoC stops reproducing. That oracle — my one input no longer works — is the weakest oracle in existence: it tests an input, not the property (falsifiable-testing).
2. The frame came from the reporter. The fixer inherited the demonstrated path and fixed the demonstrated path. Whatever the reporter did not think of, the fix does not cover.
3. A fixed area repels review. The file now carries a mark that says already looked at, usually by the person who understood it best. It becomes the safest-looking code in the repo, and it is the code most likely to hold the next bug. Security fixes ship under time pressure and get less review than ordinary features, not more.

This is not a hypothesis about how people work. Project Zero's tracking of exploited-in-the-wild 0-days has repeatedly found that a large share of them each year are variants of previously patched bugs. The exact fraction moves; the pattern does not.

Composes with the library:

• beyond-the-sink — the sibling; its Part 3 (a root class is a template) is this skill's engine, applied systematically to fix history
• invariant-hunting — name the property the fix was attempting to establish, then ask whether it holds
• discriminating-proof — the pre-fix / post-fix control in Part 4
• software-archaeology — reconstructing intent from a diff whose author is unavailable
• audit-before-patch — the fix you are auditing must be confirmed present in the live code before anything is built on it
• remediation-driven-reporting — the report that gets the class fixed this time, referencing their own prior fix
• falsifiable-testing — the regression test that stops the fix from decaying

Scope: your own repositories, authorized targets, and public patches of open source. Reading a public diff is research; probing someone's production because the diff suggested something is not, and needs the same authorization as any other test.

Part 1 — Reconstruct the bug from the diff, not from the advisory

The advisory is the vendor's framing, written to be reassuring and to reveal as little as is polite. The diff is the fact. Start there, always, and derive what must have been possible before the change.

Then extract the thing that makes this skill work:

The fix encodes the author's mental model of the bug. The gap between that model and the real system is where the next bug is.

Read the fix for its model, and the model predicts what else was missed:

This is the abductive step (abductive-engineering): the patch is a sign, its object is the author's belief, and the interpretant you want is the region their belief did not cover.

Part 2 — The eight ways a fix falls short

Name the family before searching. Each has a distinct search.

1. It covers the input, not the property. They rejected the reporter's exact payload, encoding, or id. Search: the next member of the class — another encoding layer, a different normalization, an equivalent value.
2. It covers one path, not every path. Sync patched, async not. The v2 handler patched, v1 still routed. The interactive path patched; the batch, import, retry, streaming, replay, admin, and CLI paths untouched. Search: every path that reaches the same sink.
3. It is at the wrong layer. Patched where the bug was observed rather than where the property should be enforced — so every other caller of the same primitive still has it. This is the highest-yield family, because the fix is genuinely correct and genuinely local.
4. It suppresses the symptom. A catch, a retry, a clamp, a default. The operation no longer crashes; it now fails silently, and the corrupt state it leaves behind is a new problem (honest-degradation). Search: what does the system do after the suppressed failure?
5. It is correct but unenforced. The right check exists, and nothing stops the next contributor from adding a caller that skips it. No test, no type, no chokepoint. Search: how many callers exist, and what makes a new one safe?
6. It introduced a new bug. The diff was written fast, under pressure, by someone tired, and reviewed deferentially because they were the expert. Off-by-one in the new bound, a null path that did not exist before, a TOCTOU between the new check and the old use, a lock now held across I/O. Read the fix diff adversarially as new code, because that is what it is.
7. Siblings were left unpatched. Other implementations, other languages, other services, forks, vendored copies, the same function copy-pasted twice. Coordinated fixes almost never reach every implementation (beyond-the-sink Part 3's existence and behavior gates).
8. It decayed. The fix landed and no longer exists: a refactor moved the check to the wrong side of a branch, a performance patch added a fast path that skips it, a merge dropped it, a rewrite reimplemented the function without it. Search: does the check still exist at HEAD, and is it still on every path it used to cover?

Family 8 deserves a habit of its own: for every historical fix you care about, verify it is still present today. Fixes with no regression test are the ones that decay, and a decayed fix is a live bug with a closed ticket in front of it.

Part 3 — Where fixes announce themselves

The fix history is a map of where the authors already bled. Mine it:

• Commit messages: fix, security, CVE, bypass, sanitize, escape, validate, harden, overflow, injection, regression, hotfix.
• Advisories, CVE records and their patch commits; changelog and release notes; issues closed as fixed; your own past reports.
• Suspiciously terse messages on sensitive files — "minor cleanup" on a parser, "refactor" on an auth path. A silent security fix looks exactly like this, and it is the highest-value commit in the log.
• Clusters. Three fixes in one file over two years is not three incidents, it is one structural problem that has been patched three times. Fix density is the single best predictor of the next bug's location.
• Velocity tells. A fix landed within hours of a report, in a file its author does not usually own, is a rushed fix by definition.

And the rule inherited from beyond-the-sink: dig adjacent to old fixes, not on top of them. The exact bug is the one place that is genuinely well tested now. The neighbors are not.

Part 4 — The pre-fix control makes the verdict unambiguous

When you have a candidate, run it against three versions, not one. This is the discriminating experiment for this whole skill:

That table is why the pre-fix build is worth the trouble of checking out: it separates never fixed from newly introduced, which are different findings, with different severities, reported to different people, with different fixes. Without it you have an observation and a guess about its origin.

Then apply the epistemic ladder as usual — a surviving path found by reading is PLAUSIBLE until it is run (discriminating-proof).

Part 5 — The report writes itself, and the blue fix is a chokepoint

An incomplete-fix finding is often stronger than a novel bug report, and it is certainly easier to land:

• The vendor already agreed it is a security issue. They fixed it once. The entire "is this really a vulnerability" argument is pre-settled — cite their own commit or advisory.
• Name the family from Part 2 and show the specific surviving path. "The fix is incomplete" without a demonstrated survivor is an opinion.
• Propose the chokepoint, not another point fix — otherwise you are filing the same report again in six months, which is precisely the loop this skill exists to break (remediation-driven-reporting Part 3).

The defensive deliverable, for your own repos:

1. A regression test per fix, asserting the property, not the reporter's input. A fix with no test is a fix with an expiry date.
2. A decay check: a test that fails if the check is removed or bypassed — the only thing that catches family 8 before an attacker does.
3. A post-fix review pass as policy. The security fix diff gets more scrutiny than a feature diff, not less. Right now the opposite is true everywhere.
4. A fix-cluster review. When one file collects its third fix, stop patching it and redesign the boundary.

Deliverable

## Fix audit — <component> — fixes reviewed: <n>
| fix / commit / CVE | invariant it attempted | families checked | outcome |

### Findings
<id> — family (1–8) — the surviving path — pre-fix/post-fix control result
     — epistemic level — the prior fix it references

### Fixes verified complete
<recorded as terrain: what was traced, which paths, what enforces it>

### Decayed fixes
<fix, when it landed, what removed or bypassed it, whether it is live now>

### Structural proposal
<chokepoint + regression tests + decay checks>

### Not reviewed

Anti-patterns

• Reading the advisory instead of the diff, and inheriting the vendor's framing of what the bug was.
• Re-running the original PoC against the patched build and concluding the fix is complete — that is the fixer's own weak oracle, borrowed.
• Assuming the fix's location is the correct location for the property.
• Never checking whether a historical fix still exists at HEAD.
• Treating the fix diff as reviewed code. It is the least-reviewed code in the repository.
• Skipping the pre-fix control, and so being unable to distinguish "never covered" from "introduced by the fix".
• Digging on top of the exact bug that was fixed — the one spot now covered by a test — instead of adjacent to it.
• Reporting "the fix is incomplete" without a demonstrated surviving path.
• Accepting a second point fix for the same file, and scheduling your own next report.
• Recording a verified-complete fix nowhere, so the next auditor re-does the work you already did.