ai-agents Failure Archaeology
This repo's rules are fossils of incidents. Before you challenge a gate, weaken
a guard, or propose a "simpler" approach, check whether that battle was already
fought and what it cost. The canon lives in .agents/retrospective/ and
.serena/memories/; the provenance table below carries the commands that count
both. Full local history is
present (git rev-list --count HEAD = ~1471 as of 2026-07-03), but retro-cited
short SHAs (e.g. ddb76e0, 01e76615a) may not resolve locally, so archaeology
still routes through the retros and memories as primary sources, not git log.
Depth per incident lives in references/incidents.md (read it when a table row
below is not enough). This file is the index and the verdict list.
Triggers
has this failed beforewhy does this rule existfailure archaeologywhat happened with issue 2205are we re-litigating a settled decision
Process
Phase 1: Check the Major Incidents Table
If your question touches hooks, generators, drift, review iteration, escape
hatches, or silent defaults, it is probably one of these eight. Read the
matching subsection in references/incidents.md before doing anything else.
| Incident |
One-line summary |
Primary retro (in .agents/retrospective/) |
Status |
| #2205 customer wedge |
Bare ./hooks/... paths + Copilot CLI running hooks from the USER's cwd wedged every plugin customer for 33 days (v0.3.0 to v0.5.6); recovery was uninstall. First fix added 3 new defects; session 1873 fixed it with an empirical probe of Copilot CLI 1.0.57 |
2026-06-02-pr-2205-customer-wedge-incident.md |
Settled; gated by scripts/validation/validate_hook_anchoring.py + tests/build_scripts/test_generate_hooks_runtime_contract.py |
| #2290 payload casing |
Copilot CLI payload field names depend on event-key casing: camelCase sends toolName/toolArgs (toolArgs is a JSON string); PascalCase sends tool_name/tool_input. FM-11 second occurrence |
2026-06-02-issue-2290-copilot-hook-payload-format.md |
Settled (PascalCase + dual-format shim); exit-143 timeout flagged P0, unresolved in that retro |
| #1887 iteration paradox |
Guard framework built to cut review iteration took 69 commits / 254 conversations; Phase-6 audit showed the guards would have prevented 0 of its own 35 fix commits |
2026-05-05-pr-1887-iteration-paradox.md |
Settled diagnosis: bot concurrency (60-70% overlap) + reviewThreads(first: 100) pagination cliff drove the cost |
| #1989 recursive failure |
The mitigation PR reproduced the failure modes it mitigated; M1 built on a false premise (pagination already existed), M4 threshold 6 vs repo max 4 could never fire, M5 guard never run on its own branch |
2026-05-10-pr-1989-recursive-failure.md |
Settled: 3 process rules (self-application, memory contradiction check, threshold calibration) |
| Session 1187 trust incident |
SKIP_PREPUSH abused 3x within hours of creation; git checkout --ours corrupted main's session log; user: "You can't be trusted in the least bit." |
2026-02-08-session-1187-skip-prepush-abuse.md |
Settled: SKIP_PREPUSH removed; session-merge rule binding |
| PR #908 scope explosion |
59 commits / 95 files; unscoped markdownlint --fix reformatted 53 unrelated memory files into the diff |
2026-01-15-pr-908-comprehensive-retrospective.md |
Settled: birthed scripts/validation/pre_pr.py, commit caps, scoped lint |
| 2025-12-15 drift inversion |
Agent edited the SOURCE (Claude agents) to match the GENERATED (templates); commit ddb76e0 reverted |
2025-12-15-drift-detection-disaster.md |
Settled: drift shows difference, never direction; always ask which side is canonical |
| PR #1965 silent defaults |
Verdict parser defaulted a missing VERDICT: line to non-blocking; 3 fix rounds because parser, exit-code translator, and workflow gate each had their own silent default |
2026-05-10-pr-1965-review-axes-convergence.md + FM-10 in FAILURE-MODES.md |
Settled: "there is no neutral default for a missing signal" |
Also settled, no dedicated retro: PR #1942 stale plugin cache. It deleted the
deprecated workflow skill from .claude/skills/ but left plugin.json at
0.3.0; installed plugin caches key off the version, so installs kept shipping
the dead /workflow until the gap was hand-caught in PR #2114. Record and gate:
build/scripts/validate_plugin_version_bump.py docstring (lines 1-20).
Phase 2: Check the Settled Battles List
These are decided. Re-opening one requires new evidence plus the change-control
path in ai-agents-change-control, not a fresh opinion.
| Settled position |
Verdict |
Evidence |
| Launcher-level fail-open wrapper |
REJECTED. Exiting 0 on a broken launcher silently disables the hook; prevent the bad launcher at generation time, fail closed and loud if one escapes. Issue #2230 closed addressed-by-prevention |
2026-06-02-pr-2205-customer-wedge-incident.md:289-297, :411 |
| Self-referential tests |
BANNED for runtime contracts. A test asserting the generator's own output passes when the generator is consistently wrong; it shipped 2 of the 3 session-1872 defects |
.claude/rules/canonical-source-mirror.md; 2026-06-02-pr-2205-customer-wedge-incident.md:143 |
| Copilot CLI plugin-root env contract |
SETTLED EMPIRICALLY (CLI 1.0.57, probe + env dump): COPILOT_PLUGIN_ROOT, CLAUDE_PLUGIN_ROOT, and bare PLUGIN_ROOT are all set, though the public docs list none of them. Anchor form: ${COPILOT_PLUGIN_ROOT:-${CLAUDE_PLUGIN_ROOT}} |
.serena/memories/decision-copilot-cli-hook-plugin-root-contract.md |
| Session-file merge conflicts |
Always git checkout --theirs (keep main's file), rename yours to the next number. Main's session files are immutable audit records |
2026-02-08-session-1187-skip-prepush-abuse.md:334; merge-resolver agent |
| Threshold-based detectors |
MUST ship with a calibration table replaying the last ~5 real merged PRs. A detector that cannot fire on real history is not calibrated |
2026-05-10-pr-1989-recursive-failure.md:149-157 |
| Drift-gate failures |
The output shows a difference, not a direction. Identify the canonical side before editing anything |
2025-12-15-drift-detection-disaster.md:283-286 |
| Silent defaults |
No neutral default for a missing signal: raise or block, never assume PASS |
.agents/governance/FAILURE-MODES.md:387 (FM-10) |
| Frictionless escape hatches |
Get teeth (logging, guards, approval) or get abused within hours. SKIP_PREPUSH is the proof |
2026-02-08-session-1187-skip-prepush-abuse.md:706; catalog in ai-agents-config-catalog |
| Guards shipped without self-application |
A guard PR must show the guard's output run against its own branch |
2026-05-10-pr-1989-recursive-failure.md:129-137 |
| CLI subcommands proposed from analogy |
BANNED. Run --help first; two hallucinated install commands cost user trust in the #2290 session |
2026-06-02-issue-2290-copilot-hook-payload-format.md:74-82 |
Phase 3: Map to the Failure-Mode Catalog
.agents/governance/FAILURE-MODES.md is the canonical taxonomy (11 patterns as
of 2026-07-02). Classify any new incident against it BEFORE proposing a new
class; FM-CONTRACT was once invented in a retro and had to be corrected to
FM-11 (2026-06-02-issue-2290-copilot-hook-payload-format.md:311-321).
| FM |
Name |
Anchor incident |
| FM-1 |
Context reading failure (95.8% session-start non-compliance in the anchor sample) |
2025-12-20-session-protocol-mass-failure.md |
| FM-2 |
Continuation reset after compaction |
2026-01-09-session-protocol-violation-analysis.md |
| FM-3 |
Ambiguous instruction inversion |
2025-12-17-protocol-compliance-failure.md |
| FM-4 |
False completion markers |
2026-01-13-pr894-test-coverage-failure.md |
| FM-5 |
Premature merge and deploy |
2025-12-22-pr-226-premature-merge-failure.md |
| FM-6 |
Multi-agent rubber-stamping |
2025-12-24-parallel-pr-review-session.md |
| FM-7 |
Self-contained agent delegation failure |
2025-12-19-self-contained-agents.md |
| FM-8 |
Security drift |
2026-01-04-pr760-security-suppression-failure.md |
| FM-9 |
Confident-incorrectness recurrence (claims of "matches/mirrors" without quoting the canonical source) |
2026-05-05-pr-1887-iteration-paradox.md, 2026-05-08-pr-1897-confident-incorrectness-recurrence.md |
| FM-10 |
Silent defaults and guard-clause suppression |
PR #1965 rounds 9-11; FM-10 section of FAILURE-MODES.md |
| FM-11 |
Customer-facing generated artifact shipped without runtime verification |
#2205 (first), #2290 (second) |
FM-10 is a mechanism that produces FM-4 symptoms; fix at the FM-10 layer
(harden the parser), not by lecturing the agent about honesty
(FAILURE-MODES.md:391).
Phase 4: Do Your Own Archaeology
When the tables above do not answer the question:
- Search retros by keyword, not the index:
grep -rli "<term>" .agents/retrospective/. Do NOT trust
.agents/retrospective/INDEX.md: it indexes a small fraction of the retro
files. The provenance table's coverage command prints both numbers.
- Search memories:
grep -rli "<term>" .serena/memories/ or the
memory-search skill. Decision memories (decision-*.md) record settled
contracts; root-cause-*.md record why gates exist;
*-observations.md record per-domain corrections.
- Resolve ADR references by content, never by number. Numbers collided
historically (8 ADRs renamed in issue #474) and retros cite numbers that
later moved: the #2205/#2290 retros say "ADR-063" for the runtime contract,
but the shipped ADR is
ADR-071-plugin-hook-runtime-contract-verification.md;
the real ADR-063 is memory-skill decomposition. Use
grep -rl "<topic>" .agents/architecture/.
- Do not lean on
git log for incident history. Full local history is
present (~1471 commits as of 2026-07-03), but SHAs cited in retros (e.g.
01e76615a, ddb76e0) do not resolve locally; look them up on GitHub and
treat retros and memories as the primary record.
- Auto-retros are shallow. Files named
*-auto-retro.md are unfilled
Stop-hook skeletons; a later hand-written incident retro may supersede them
(the #2205 retro explicitly supersedes three of them, :452-458).
- Write what you find. If you settle a new question, record it: a retro
per
.claude/rules/retros.md (classify against FAILURE-MODES.md, evidence
links mandatory), a decision memory for contracts, or both. Use the
retrospective skill for the structured workflow and chestertons-fence
when the question is "why does this existing thing exist".
Anti-Patterns
| Anti-pattern |
Why it fails |
Instead |
| Proposing a fail-open wrapper "so hooks never break the user" |
Re-litigates #2230; silently disabled protection is the worst outcome |
Prevent at generation time; fail closed and loud (see references/incidents.md, Incident 1) |
| Testing a generator by asserting its own output string |
Self-referential; passed while #2205 shipped broken for 33 days |
Runtime-contract test with foreign cwd/env and a negative control (ai-agents-empirical-probe-toolkit) |
| Trusting vendor docs for runtime contracts |
Docs were wrong by omission twice (#2205 env vars, #2290 payload) |
Probe the pinned CLI version empirically (agent-harness-reference for the settled contract table) |
| "Fixing" a drift gate by editing whichever side is easier |
2025-12-15 inversion; edited the source of truth to match generated output |
Identify the canonical side first (ai-agents-architecture-contract) |
| Citing an ADR by number from an old document |
Numbers collided and moved (ADR-063 vs ADR-071) |
Grep architecture dir by topic, verify the title |
| Shipping a detector with an intuition-chosen threshold |
#1989 M4 threshold could never fire on real PRs |
Calibrate against the last ~5 merged PRs, show the table |
| Adding a quick env-var bypass to reduce friction |
Session 1187: abused 3x within hours; ended in a trust failure |
Route new flags through ai-agents-config-catalog (define semantics, guard, telemetry) |
Mining git log for incident history |
Retro-cited SHAs do not resolve locally even with full history present |
Retros + memories + GitHub |
Verification
Before you rely on or extend this chronicle:
Provenance and Maintenance
Compiled 2026-07-02 from primary sources; every claim was verified against the
working tree on that date. Volatile facts and their re-verification commands:
| Fact |
Source |
Re-verify |
| Retro file count and INDEX.md coverage |
.agents/retrospective/ and .agents/retrospective/INDEX.md |
python3 -c "import pathlib;d=pathlib.Path('.agents/retrospective');f={p.name for p in d.glob('*.md')}-{'INDEX.md'};t=(d/'INDEX.md').read_text();print(len(f),'retro files,',sum(n in t for n in f),'indexed')" |
| Memory file count |
.serena/memories/ |
python3 -c "import pathlib;print(len(list(pathlib.Path('.serena/memories').rglob('*.md'))))" |
| Full history present (~1471 commits) but retro-cited SHAs unresolvable |
local clone |
git rev-list --count HEAD; git cat-file -t ddb76e0 (expect a count near 1471 and "Not a valid object name") |
| 11 failure modes |
.agents/governance/FAILURE-MODES.md:16-28 |
python3 -c "print(sum(1 for l in open('.agents/governance/FAILURE-MODES.md') if l[:2]=='\x7c ' and l[2].isdigit()))" |
| Historical SKIP_PREPUSH removal |
Session 1187 retrospective |
Confirm current Git hook jobs in lefthook.yml; do not reintroduce a global bypass |
| Anchoring gate + runtime-contract test + e2e exist |
repo tree |
ls scripts/validation/validate_hook_anchoring.py tests/build_scripts/test_generate_hooks_runtime_contract.py tests/e2e/test_cli_hook_e2e.py |
| ADR-071 is the runtime-contract ADR; ADR-063 is memory decomposition |
.agents/architecture/ |
head -1 .agents/architecture/ADR-071*.md .agents/architecture/ADR-063*.md |
| Plugin-root env contract (CLI 1.0.57) |
.serena/memories/decision-copilot-cli-hook-plugin-root-contract.md |
open the memory; re-probe per ai-agents-empirical-probe-toolkit if the CLI version moved |
| Exit-143 timeout status "unresolved" |
2026-06-02-issue-2290-copilot-hook-payload-format.md:59 |
check open issues before assuming; ai-agents-portability-campaign Phase 3 owns verification |
Primary sources: the eight retros named in Phase 1;
.agents/governance/FAILURE-MODES.md;
.serena/memories/decision-copilot-cli-hook-plugin-root-contract.md,
copilot-hooks-observations.md, root-cause-governance-enforcement.md,
root-cause-late-feedback.md, root-cause-scope-creep-tools.md,
ci-infrastructure-observations.md (PR #1361 reproduce-on-main rule);
.claude/rules/generated-artifacts.md, canonical-source-mirror.md,
retros.md. Update this skill when FAILURE-MODES.md gains a pattern or a new
incident retro of #2205-class severity lands.
1---2name: ai-agents-failure-archaeology3description: The chronicle of this repo's settled battles. Maps each major incident to symptom, root cause, evidence path, and the artifact that fixed it, so nobody re-fights a decided question. Use when you say `has this failed before`, `why does this rule exist`, `failure archaeology`, `what happened with issue 2205`. Do NOT use for triaging a live failure (use `ai-agents-debugging-playbook`) or for extracting learnings from the current session (use `retrospective`).4license: MIT5---67# ai-agents Failure Archaeology89<!-- vendor-portability: contributor-facing knowledge pack for the rjmurillo/ai-agents repo itself; intentionally references upstream paths (.agents/, .claude/, scripts/, build/) because its audience is repo contributors, not plugin consumers (issue #2050) -->10This repo's rules are fossils of incidents. Before you challenge a gate, weaken11a guard, or propose a "simpler" approach, check whether that battle was already12fought and what it cost. The canon lives in `.agents/retrospective/` and13`.serena/memories/`; the provenance table below carries the commands that count14both. Full local history is15present (`git rev-list --count HEAD` = ~1471 as of 2026-07-03), but retro-cited16short SHAs (e.g. `ddb76e0`, `01e76615a`) may not resolve locally, so archaeology17still routes through the retros and memories as primary sources, not `git log`.1819Depth per incident lives in `references/incidents.md` (read it when a table row20below is not enough). This file is the index and the verdict list.2122## Triggers2324- `has this failed before`25- `why does this rule exist`26- `failure archaeology`27- `what happened with issue 2205`28- `are we re-litigating a settled decision`2930## Process3132### Phase 1: Check the Major Incidents Table3334If your question touches hooks, generators, drift, review iteration, escape35hatches, or silent defaults, it is probably one of these eight. Read the36matching subsection in `references/incidents.md` before doing anything else.3738| Incident | One-line summary | Primary retro (in `.agents/retrospective/`) | Status |39|----------|------------------|--------------------------------------------|--------|40| #2205 customer wedge | Bare `./hooks/...` paths + Copilot CLI running hooks from the USER's cwd wedged every plugin customer for 33 days (v0.3.0 to v0.5.6); recovery was uninstall. First fix added 3 new defects; session 1873 fixed it with an empirical probe of Copilot CLI 1.0.57 | `2026-06-02-pr-2205-customer-wedge-incident.md` | Settled; gated by `scripts/validation/validate_hook_anchoring.py` + `tests/build_scripts/test_generate_hooks_runtime_contract.py` |41| #2290 payload casing | Copilot CLI payload field names depend on event-key casing: camelCase sends `toolName`/`toolArgs` (toolArgs is a JSON string); PascalCase sends `tool_name`/`tool_input`. FM-11 second occurrence | `2026-06-02-issue-2290-copilot-hook-payload-format.md` | Settled (PascalCase + dual-format shim); exit-143 timeout flagged P0, unresolved in that retro |42| #1887 iteration paradox | Guard framework built to cut review iteration took 69 commits / 254 conversations; Phase-6 audit showed the guards would have prevented 0 of its own 35 fix commits | `2026-05-05-pr-1887-iteration-paradox.md` | Settled diagnosis: bot concurrency (60-70% overlap) + `reviewThreads(first: 100)` pagination cliff drove the cost |43| #1989 recursive failure | The mitigation PR reproduced the failure modes it mitigated; M1 built on a false premise (pagination already existed), M4 threshold 6 vs repo max 4 could never fire, M5 guard never run on its own branch | `2026-05-10-pr-1989-recursive-failure.md` | Settled: 3 process rules (self-application, memory contradiction check, threshold calibration) |44| Session 1187 trust incident | `SKIP_PREPUSH` abused 3x within hours of creation; `git checkout --ours` corrupted main's session log; user: "You can't be trusted in the least bit." | `2026-02-08-session-1187-skip-prepush-abuse.md` | Settled: SKIP_PREPUSH removed; session-merge rule binding |45| PR #908 scope explosion | 59 commits / 95 files; unscoped `markdownlint --fix` reformatted 53 unrelated memory files into the diff | `2026-01-15-pr-908-comprehensive-retrospective.md` | Settled: birthed `scripts/validation/pre_pr.py`, commit caps, scoped lint |46| 2025-12-15 drift inversion | Agent edited the SOURCE (Claude agents) to match the GENERATED (templates); commit `ddb76e0` reverted | `2025-12-15-drift-detection-disaster.md` | Settled: drift shows difference, never direction; always ask which side is canonical |47| PR #1965 silent defaults | Verdict parser defaulted a missing `VERDICT:` line to non-blocking; 3 fix rounds because parser, exit-code translator, and workflow gate each had their own silent default | `2026-05-10-pr-1965-review-axes-convergence.md` + FM-10 in `FAILURE-MODES.md` | Settled: "there is no neutral default for a missing signal" |4849Also settled, no dedicated retro: PR #1942 stale plugin cache. It deleted the50deprecated `workflow` skill from `.claude/skills/` but left `plugin.json` at510.3.0; installed plugin caches key off the version, so installs kept shipping52the dead `/workflow` until the gap was hand-caught in PR #2114. Record and gate:53`build/scripts/validate_plugin_version_bump.py` docstring (lines 1-20).5455### Phase 2: Check the Settled Battles List5657These are decided. Re-opening one requires new evidence plus the change-control58path in `ai-agents-change-control`, not a fresh opinion.5960| Settled position | Verdict | Evidence |61|------------------|---------|----------|62| Launcher-level fail-open wrapper | REJECTED. Exiting 0 on a broken launcher silently disables the hook; prevent the bad launcher at generation time, fail closed and loud if one escapes. Issue #2230 closed addressed-by-prevention | `2026-06-02-pr-2205-customer-wedge-incident.md:289-297`, `:411` |63| Self-referential tests | BANNED for runtime contracts. A test asserting the generator's own output passes when the generator is consistently wrong; it shipped 2 of the 3 session-1872 defects | `.claude/rules/canonical-source-mirror.md`; `2026-06-02-pr-2205-customer-wedge-incident.md:143` |64| Copilot CLI plugin-root env contract | SETTLED EMPIRICALLY (CLI 1.0.57, probe + env dump): `COPILOT_PLUGIN_ROOT`, `CLAUDE_PLUGIN_ROOT`, and bare `PLUGIN_ROOT` are all set, though the public docs list none of them. Anchor form: `${COPILOT_PLUGIN_ROOT:-${CLAUDE_PLUGIN_ROOT}}` | `.serena/memories/decision-copilot-cli-hook-plugin-root-contract.md` |65| Session-file merge conflicts | Always `git checkout --theirs` (keep main's file), rename yours to the next number. Main's session files are immutable audit records | `2026-02-08-session-1187-skip-prepush-abuse.md:334`; merge-resolver agent |66| Threshold-based detectors | MUST ship with a calibration table replaying the last ~5 real merged PRs. A detector that cannot fire on real history is not calibrated | `2026-05-10-pr-1989-recursive-failure.md:149-157` |67| Drift-gate failures | The output shows a difference, not a direction. Identify the canonical side before editing anything | `2025-12-15-drift-detection-disaster.md:283-286` |68| Silent defaults | No neutral default for a missing signal: raise or block, never assume PASS | `.agents/governance/FAILURE-MODES.md:387` (FM-10) |69| Frictionless escape hatches | Get teeth (logging, guards, approval) or get abused within hours. SKIP_PREPUSH is the proof | `2026-02-08-session-1187-skip-prepush-abuse.md:706`; catalog in `ai-agents-config-catalog` |70| Guards shipped without self-application | A guard PR must show the guard's output run against its own branch | `2026-05-10-pr-1989-recursive-failure.md:129-137` |71| CLI subcommands proposed from analogy | BANNED. Run `--help` first; two hallucinated install commands cost user trust in the #2290 session | `2026-06-02-issue-2290-copilot-hook-payload-format.md:74-82` |7273### Phase 3: Map to the Failure-Mode Catalog7475`.agents/governance/FAILURE-MODES.md` is the canonical taxonomy (11 patterns as76of 2026-07-02). Classify any new incident against it BEFORE proposing a new77class; FM-CONTRACT was once invented in a retro and had to be corrected to78FM-11 (`2026-06-02-issue-2290-copilot-hook-payload-format.md:311-321`).7980| FM | Name | Anchor incident |81|----|------|-----------------|82| FM-1 | Context reading failure (95.8% session-start non-compliance in the anchor sample) | `2025-12-20-session-protocol-mass-failure.md` |83| FM-2 | Continuation reset after compaction | `2026-01-09-session-protocol-violation-analysis.md` |84| FM-3 | Ambiguous instruction inversion | `2025-12-17-protocol-compliance-failure.md` |85| FM-4 | False completion markers | `2026-01-13-pr894-test-coverage-failure.md` |86| FM-5 | Premature merge and deploy | `2025-12-22-pr-226-premature-merge-failure.md` |87| FM-6 | Multi-agent rubber-stamping | `2025-12-24-parallel-pr-review-session.md` |88| FM-7 | Self-contained agent delegation failure | `2025-12-19-self-contained-agents.md` |89| FM-8 | Security drift | `2026-01-04-pr760-security-suppression-failure.md` |90| FM-9 | Confident-incorrectness recurrence (claims of "matches/mirrors" without quoting the canonical source) | `2026-05-05-pr-1887-iteration-paradox.md`, `2026-05-08-pr-1897-confident-incorrectness-recurrence.md` |91| FM-10 | Silent defaults and guard-clause suppression | PR #1965 rounds 9-11; FM-10 section of `FAILURE-MODES.md` |92| FM-11 | Customer-facing generated artifact shipped without runtime verification | #2205 (first), #2290 (second) |9394FM-10 is a mechanism that produces FM-4 symptoms; fix at the FM-10 layer95(harden the parser), not by lecturing the agent about honesty96(`FAILURE-MODES.md:391`).9798### Phase 4: Do Your Own Archaeology99100When the tables above do not answer the question:1011021. **Search retros by keyword**, not the index:103 `grep -rli "<term>" .agents/retrospective/`. Do NOT trust104 `.agents/retrospective/INDEX.md`: it indexes a small fraction of the retro105 files. The provenance table's coverage command prints both numbers.1062. **Search memories**: `grep -rli "<term>" .serena/memories/` or the107 `memory-search` skill. Decision memories (`decision-*.md`) record settled108 contracts; `root-cause-*.md` record why gates exist;109 `*-observations.md` record per-domain corrections.1103. **Resolve ADR references by content, never by number.** Numbers collided111 historically (8 ADRs renamed in issue #474) and retros cite numbers that112 later moved: the #2205/#2290 retros say "ADR-063" for the runtime contract,113 but the shipped ADR is `ADR-071-plugin-hook-runtime-contract-verification.md`;114 the real ADR-063 is memory-skill decomposition. Use115 `grep -rl "<topic>" .agents/architecture/`.1164. **Do not lean on `git log` for incident history.** Full local history is117 present (~1471 commits as of 2026-07-03), but SHAs cited in retros (e.g.118 `01e76615a`, `ddb76e0`) do not resolve locally; look them up on GitHub and119 treat retros and memories as the primary record.1205. **Auto-retros are shallow.** Files named `*-auto-retro.md` are unfilled121 Stop-hook skeletons; a later hand-written incident retro may supersede them122 (the #2205 retro explicitly supersedes three of them, `:452-458`).1236. **Write what you find.** If you settle a new question, record it: a retro124 per `.claude/rules/retros.md` (classify against FAILURE-MODES.md, evidence125 links mandatory), a decision memory for contracts, or both. Use the126 `retrospective` skill for the structured workflow and `chestertons-fence`127 when the question is "why does this existing thing exist".128129## Anti-Patterns130131| Anti-pattern | Why it fails | Instead |132|--------------|-------------|---------|133| Proposing a fail-open wrapper "so hooks never break the user" | Re-litigates #2230; silently disabled protection is the worst outcome | Prevent at generation time; fail closed and loud (see `references/incidents.md`, Incident 1) |134| Testing a generator by asserting its own output string | Self-referential; passed while #2205 shipped broken for 33 days | Runtime-contract test with foreign cwd/env and a negative control (`ai-agents-empirical-probe-toolkit`) |135| Trusting vendor docs for runtime contracts | Docs were wrong by omission twice (#2205 env vars, #2290 payload) | Probe the pinned CLI version empirically (`agent-harness-reference` for the settled contract table) |136| "Fixing" a drift gate by editing whichever side is easier | 2025-12-15 inversion; edited the source of truth to match generated output | Identify the canonical side first (`ai-agents-architecture-contract`) |137| Citing an ADR by number from an old document | Numbers collided and moved (ADR-063 vs ADR-071) | Grep architecture dir by topic, verify the title |138| Shipping a detector with an intuition-chosen threshold | #1989 M4 threshold could never fire on real PRs | Calibrate against the last ~5 merged PRs, show the table |139| Adding a quick env-var bypass to reduce friction | Session 1187: abused 3x within hours; ended in a trust failure | Route new flags through `ai-agents-config-catalog` (define semantics, guard, telemetry) |140| Mining `git log` for incident history | Retro-cited SHAs do not resolve locally even with full history present | Retros + memories + GitHub |141142## Verification143144Before you rely on or extend this chronicle:145146- [ ] The claim you are about to act on cites a retro or memory path that you147 opened and read (not just this skill's summary of it).148- [ ] If you cited a line number, you re-verified it against the current file149 (retros are append-only but line numbers can shift when corrections are150 appended).151- [ ] If you are re-opening a settled battle, you have NEW evidence and you are152 going through `ai-agents-change-control`, not around it.153- [ ] If you found a new settled result, you recorded it (retro classified154 against FAILURE-MODES.md, or a decision memory) so the next reader finds155 a fossil instead of re-fighting the battle.156157## Provenance and Maintenance158159Compiled 2026-07-02 from primary sources; every claim was verified against the160working tree on that date. Volatile facts and their re-verification commands:161162| Fact | Source | Re-verify |163|------|--------|-----------|164| Retro file count and INDEX.md coverage | `.agents/retrospective/` and `.agents/retrospective/INDEX.md` | `python3 -c "import pathlib;d=pathlib.Path('.agents/retrospective');f={p.name for p in d.glob('*.md')}-{'INDEX.md'};t=(d/'INDEX.md').read_text();print(len(f),'retro files,',sum(n in t for n in f),'indexed')"` |165| Memory file count | `.serena/memories/` | `python3 -c "import pathlib;print(len(list(pathlib.Path('.serena/memories').rglob('*.md'))))"` |166| Full history present (~1471 commits) but retro-cited SHAs unresolvable | local clone | `git rev-list --count HEAD; git cat-file -t ddb76e0` (expect a count near 1471 and "Not a valid object name") |167| 11 failure modes | `.agents/governance/FAILURE-MODES.md:16-28` | `python3 -c "print(sum(1 for l in open('.agents/governance/FAILURE-MODES.md') if l[:2]=='\x7c ' and l[2].isdigit()))"` |168| Historical SKIP_PREPUSH removal | Session 1187 retrospective | Confirm current Git hook jobs in `lefthook.yml`; do not reintroduce a global bypass |169| Anchoring gate + runtime-contract test + e2e exist | repo tree | `ls scripts/validation/validate_hook_anchoring.py tests/build_scripts/test_generate_hooks_runtime_contract.py tests/e2e/test_cli_hook_e2e.py` |170| ADR-071 is the runtime-contract ADR; ADR-063 is memory decomposition | `.agents/architecture/` | `head -1 .agents/architecture/ADR-071*.md .agents/architecture/ADR-063*.md` |171| Plugin-root env contract (CLI 1.0.57) | `.serena/memories/decision-copilot-cli-hook-plugin-root-contract.md` | open the memory; re-probe per `ai-agents-empirical-probe-toolkit` if the CLI version moved |172| Exit-143 timeout status "unresolved" | `2026-06-02-issue-2290-copilot-hook-payload-format.md:59` | check open issues before assuming; `ai-agents-portability-campaign` Phase 3 owns verification |173174Primary sources: the eight retros named in Phase 1;175`.agents/governance/FAILURE-MODES.md`;176`.serena/memories/decision-copilot-cli-hook-plugin-root-contract.md`,177`copilot-hooks-observations.md`, `root-cause-governance-enforcement.md`,178`root-cause-late-feedback.md`, `root-cause-scope-creep-tools.md`,179`ci-infrastructure-observations.md` (PR #1361 reproduce-on-main rule);180`.claude/rules/generated-artifacts.md`, `canonical-source-mirror.md`,181`retros.md`. Update this skill when FAILURE-MODES.md gains a pattern or a new182incident retro of #2205-class severity lands.
rjmurillo