Learn From Fixes

Learn From Fixes

A project's git history is a labelled dataset of what its first pass gets wrong. A fix commit landing on a file a feat commit touched days earlier is not maintenance — it is the feature having shipped broken, with the diagnosis written in the commit message.

This turns that history into a ranked list of what to gate.

1. Measure

node "${CLAUDE_PLUGIN_ROOT}/scripts/mine-fixes.js" .

Add --json for machine-readable output, --window-days=7 for slower-moving repos. It is read-only and never writes to the repo.

If it reports no conventional fix: commits, say so and stop — the analysis needs conventional subjects, and inventing a ranking without them would be exactly the unverified guess this skill exists to prevent.

2. Read the top classes against the real commits

The tool ranks by subject-line keywords, which is a proxy — and measured against a read of the commit bodies, a weak one. On three production repos the two passes disagreed on magnitude by roughly 8x (ordering/async: 41% / 32% / 40% by reading, 5% / 6% / 3% by regex) and on which class ranked first. A body saying "the phone home raced boot and lost" ships under the subject fix(now): first-paint; the regex only ever sees the subject.

So: the fix:feature ratio, the rework window and the hot-file list are counts and can be quoted. The class ranking is a starting point, and its number is a floor, not a share. Before drawing conclusions, read the actual commits behind the top two or three classes:

git log --format='%h %s%n%b' --grep='^fix' -30 -- <hot file from the report>

You are looking for the stated cause, not the label: "because …", "was never …", "only fired when …", "in two places". That sentence is what a gate has to catch.

3. Report

For each of the top classes, give:

• How often, with the count and the share of fixes.
• A representative commit, quoted.
• Why existing gates missed it — typecheck, build, console, tests. If they could have caught it, the finding is that they were not run, which is a different and more fixable problem.
• What would catch it next time, concretely.

Rank by frequency × how expensive each instance was to find. A class that only manual QA can catch outranks a more frequent one that a typecheck catches.

4. Propose gates, do not write them yet

For the top two or three classes, propose an executable check — something that runs in preflight or CI and fails the build:

Then hand the chosen ones to /preflight add <class>, which owns the gate file and the four laws that keep it honest. Show the user the list and let them choose. Do not generate six gates nobody asked for — an unwanted gate gets disabled, and a disabled gate teaches the team that gates are noise.

5. Two rules about gates themselves, both learned the hard way

A gate nobody runs is not a gate. Wire every gate into one command that runs automatically. In a repo audited for this, sixty harness scripts existed and nothing ran them; two had been failing for eight days and the only thing that objected was a script nobody executed.

A gate that can go stale must fail when it does. Keep known failures in an explicit list keyed to open work items, and fail the build when a known-red gate starts passing — otherwise a stale excuse is how a real failure gets waved through.

Verify any gate you do write by reintroducing the original defect and confirming the gate goes red. A gate never seen to fail is not known to work.

6. Write it down

Append the confirmed classes to .claude/project-rules.md under a ## What this project keeps getting wrong heading, each with its count and date. /autodev-init owns that file; this skill adds a section to it rather than creating a competing one.

That file is what review and audit read, so a class recorded there is checked on every future change — which is the entire point of the exercise.

Running it on a schedule

The loop above only closes when someone remembers to ask. A nightly or weekly routine can run the measurement half unattended and propose the rest:

node "${CLAUDE_PLUGIN_ROOT}/scripts/mine-fixes.js" <repo> --json

Report-only rules for the unattended run:

• Quote the tool's counts (fix:feature ratio, rework window, hot files) as counts. The class ranking is a floor, not a share — the calibration in step 2 applies doubly when no human is reading the commit bodies.
• When a repo's numbers look worth a human's time, log a proposal to run /learn-from-fixes there. Never write gates or edit project-rules.md unattended — an unwanted gate teaches the team that gates are noise, and an unreviewed rule is a guess wearing a rule's clothes.
• End the run by touching the scheduled task's .last-run heartbeat, clean or not, so drift-audit can tell a quiet week from a dead schedule.

The other half: what went wrong IN the session

mine-fixes reads git, so it can only see failures that survived long enough to be committed and then fixed. The failures that cost the most time never get there — an Edit refused because the file was never read, a browser call made before its precondition existed, a query naming a column that does not exist. They are paid for in retries inside a session and leave no trace in history.

node "${CLAUDE_PLUGIN_ROOT}/scripts/analyze-session-patterns.js" --days 7 --json

Two differences from mine-fixes that change how it is run and read:

• It is machine-wide, not per-repo. It reads the transcript tree, so run it ONCE per routine rather than once per repo — looping it over repos reports the same fleet numbers N times and makes a single stuck session look systemic.
• Rank by sessions, not count. A class hitting twenty sessions once each is a fleet problem worth a rule; one hitting a single session forty times is that session having a bad day, and the output flags the second case as concentration so it cannot be misread as the first.

Report-only, with the same rules as above, plus two specific to this tool:

• Quote the population, never a bare percentage. The output leads with files scanned, lines skipped as outside the window, tool results and the error rate for a reason: this tool has already produced two confidently wrong readings — a denominator that counted only error-bearing lines (so "783 of 783 failed"), and a window that filtered by file mtime while counting events months older. A share with no denominator beside it is how both survived review.
• Check --by-day before proposing anything. A class that is already falling needs no new rule; something has fixed it. The Bash denylist removal shows the shape to look for — 40, 34, 2, 1 across four days while the daily error total held, so the fall was the change and not a quiet weekend. Propose work for classes that are flat or rising, and say which day the series starts.

A class whose fix is already written down and which is still flat is the useful finding: it means the rule exists and is not reaching anyone, so the answer is a gate or a hook rather than another paragraph.