Investigate: root cause before any fix
Start here, before reading code. In the repository (cd into it first: the shell starts in the workspace root), run with execute, adding the failing file or directory if you know it:
git status --short
git log --oneline -20 -- <path>
Then reproduce the symptom with the project's own test or run command, and write down what you saw, before you form a hypothesis.
Iron Law
NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST.
Fixing symptoms creates whack-a-mole debugging. Every fix that doesn't address root cause makes the next bug harder to find. Find the root cause, then fix it.
How to work
Find the root cause, fix it with the smallest change, prove the fix, and finish with the DEBUG REPORT below. The order, tools and depth are yours. Three lessons that cost real time before:
- Before calling two queries, helpers or code paths duplicates, read their call sites: what looks identical usually differs in a caller's guard, transaction or scope, and the "duplicate" is the one that is right.
- Before timing anything that spawns git in a benchmark or bisect worktree, run
git update-index --refresh there: a stale index makes the first git status/git diff pay a full refresh and the measurement lies.
- Two fixes that shared one premise failed the same gate — write the premise down in one sentence and test it before a third fix. The evidence already points away from it.
Three guards:
- 3-strike rule: if 3 hypotheses fail, STOP and ask the user: A) continue with a new hypothesis (describe it), B) escalate for human review, C) add logging and wait.
- Blast radius: if the fix touches more than 5 files, ask the user: A) proceed, the root cause spans these files, B) split, fixing the critical path now, C) rethink for a more targeted approach.
- Web searches: search the generic error class, library and version, never the raw error text, hostnames, IPs, file paths, SQL, customer data or secrets.
Race recipe
A race is two actors touching shared state with an ordering nobody enforces.
Make it deterministic before fixing it.
- Timing probes. Log a monotonic timestamp (
performance.now()) and the
actor id at every read and write of the shared state (file, row, map, lock).
A race shows as interleaved writes or a read between another actor's
check and its write. Keep the probe output as evidence.
- Forced interleaving. Inject an
await or sleep at the suspected point
in one actor (behind a test-only hook or env var) so the other actor runs in
the window. The bug must now appear every time; if it does not, the window is
elsewhere.
- Concurrent-writer template. Drive N writers at once and assert the
invariant, not the absence of an error:
const results = await Promise.allSettled(Array.from({ length: N }, (_, i) => writer(i)));
// exactly one winner | all N applied | file or row intact | lock released and owner recorded
Run with N ≥ 8 under the forced interleaving; the template becomes the
regression test with the interleaving still injected.
- Fix shape. An atomic primitive: create with
wx, compare-and-swap,
UPDATE … WHERE old_value = ?, a lock with an owner token and a stale-owner
rule. A check-then-act with a smaller window is not a fix. Rerun step 3
after the change and paste both outputs.
Report
Output a structured debug report:
DEBUG REPORT
════════════════════════════════════════
Symptom: [what the user observed]
Root cause: [what was actually wrong]
Fix: [what was changed, with file:line references]
Evidence: [test output, reproduction attempt showing fix works]
Regression test: [file:line of the new test]
Related: [TODOS.md items, prior bugs in same area, architectural notes]
Status: DONE | DONE_WITH_CONCERNS | BLOCKED
════════════════════════════════════════
Important Rules
- 3+ failed fix attempts → STOP and question the architecture. Wrong architecture, not failed hypothesis.
- Never apply a fix you cannot verify. If you can't reproduce and confirm, don't ship it.
- Never say "this should fix it." Verify and prove it. Run the tests.
- If fix touches >5 files → ask the user about blast radius before proceeding.
- Completion status:
- DONE — root cause found, fix applied, regression test written, all tests pass
- DONE_WITH_CONCERNS — fixed but cannot fully verify (e.g., intermittent bug, requires staging)
- BLOCKED — root cause unclear after investigation, escalated
1---2name: gstack-investigate3description: "Use when a test fails, code errors or output is wrong: find and prove the root cause before any fix, then fix it with a regression test. (gstack)"4metadata:5 source: https://skillmd.com/skills/tuyv/gstack-investigate6---78<!-- Generated by scripts/gen-qm-edition.ts from hosts/qm.ts and investigate/SKILL.md.tmpl, investigate/sections/race.md.tmpl. Do not edit. -->910# Investigate: root cause before any fix1112**Start here, before reading code.** In the repository (`cd` into it first: the shell starts in the workspace root), run with `execute`, adding the failing file or directory if you know it:1314```sh15git status --short16git log --oneline -20 -- <path>17```1819Then reproduce the symptom with the project's own test or run command, and write down what you saw, before you form a hypothesis.2021## Iron Law2223**NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST.**2425Fixing symptoms creates whack-a-mole debugging. Every fix that doesn't address root cause makes the next bug harder to find. Find the root cause, then fix it.2627## How to work2829Find the root cause, fix it with the smallest change, prove the fix, and finish with the DEBUG REPORT below. The order, tools and depth are yours. Three lessons that cost real time before:3031- Before calling two queries, helpers or code paths duplicates, read their call sites: what looks identical usually differs in a caller's guard, transaction or scope, and the "duplicate" is the one that is right.32- Before timing anything that spawns git in a benchmark or bisect worktree, run `git update-index --refresh` there: a stale index makes the first `git status`/`git diff` pay a full refresh and the measurement lies.33- Two fixes that shared one premise failed the same gate — write the premise down in one sentence and test it before a third fix. The evidence already points away from it.3435Three guards:3637- **3-strike rule:** if 3 hypotheses fail, **STOP** and ask the user: A) continue with a new hypothesis (describe it), B) escalate for human review, C) add logging and wait.38- **Blast radius:** if the fix touches more than 5 files, ask the user: A) proceed, the root cause spans these files, B) split, fixing the critical path now, C) rethink for a more targeted approach.39- **Web searches:** search the generic error class, library and version, never the raw error text, hostnames, IPs, file paths, SQL, customer data or secrets.4041---4243## Race recipe4445A race is two actors touching shared state with an ordering nobody enforces.46Make it deterministic before fixing it.47481. **Timing probes.** Log a monotonic timestamp (`performance.now()`) and the49 actor id at every read and write of the shared state (file, row, map, lock).50 A race shows as interleaved writes or a read between another actor's51 check and its write. Keep the probe output as evidence.522. **Forced interleaving.** Inject an `await` or sleep at the suspected point53 in one actor (behind a test-only hook or env var) so the other actor runs in54 the window. The bug must now appear every time; if it does not, the window is55 elsewhere.563. **Concurrent-writer template.** Drive N writers at once and assert the57 invariant, not the absence of an error:58 ```ts59 const results = await Promise.allSettled(Array.from({ length: N }, (_, i) => writer(i)));60 // exactly one winner | all N applied | file or row intact | lock released and owner recorded61 ```62 Run with N ≥ 8 under the forced interleaving; the template becomes the63 regression test with the interleaving still injected.644. **Fix shape.** An atomic primitive: create with `wx`, compare-and-swap,65 `UPDATE … WHERE old_value = ?`, a lock with an owner token and a stale-owner66 rule. A check-then-act with a smaller window is not a fix. Rerun step 367 after the change and paste both outputs.6869## Report7071Output a structured debug report:72```73DEBUG REPORT74════════════════════════════════════════75Symptom: [what the user observed]76Root cause: [what was actually wrong]77Fix: [what was changed, with file:line references]78Evidence: [test output, reproduction attempt showing fix works]79Regression test: [file:line of the new test]80Related: [TODOS.md items, prior bugs in same area, architectural notes]81Status: DONE | DONE_WITH_CONCERNS | BLOCKED82════════════════════════════════════════83```8485---8687## Important Rules8889- **3+ failed fix attempts → STOP and question the architecture.** Wrong architecture, not failed hypothesis.90- **Never apply a fix you cannot verify.** If you can't reproduce and confirm, don't ship it.91- **Never say "this should fix it."** Verify and prove it. Run the tests.92- **If fix touches >5 files → ask the user** about blast radius before proceeding.93- **Completion status:**94 - DONE — root cause found, fix applied, regression test written, all tests pass95 - DONE_WITH_CONCERNS — fixed but cannot fully verify (e.g., intermittent bug, requires staging)96 - BLOCKED — root cause unclear after investigation, escalated