agent-context-audit — unhobble this repo's agent context
Goal: find where this repo's context (CLAUDE.md, docs, skills, tool designs)
hobbles a Claude 5-generation model — overconstrains it, contradicts itself,
repeats itself, or hides context the model actually needs — and leave behind a
findings report plus approved fixes.
Background: Anthropic removed over 80% of Claude Code's system prompt for
Claude 5 models with no measurable loss on coding evals. Older context was
written for models that needed rules; newer models need judgment, good
interfaces, and the facts they can't infer. This skill audits against that
shift, plus the "finding your unknowns" framework (the gap between the map —
your prompts/docs — and the territory — the actual codebase).
You are auditing first, fixing second. Do not edit anything until Step 4.
The six shifts (the audit rubric)
Every finding maps to one of these. Cite the shift number in the report.
- Rules → Judgment. Hard rules ("NEVER…", "ALWAYS…", "do not add
comments", "one-line docstrings max") that encode a preference, not a real
constraint, should become judgment framing ("write code that reads like the
surrounding code") — or be deleted if the model would infer it anyway.
Keep hard rules only where violation is genuinely costly (security, prod
data, irreversible actions, legal/billing).
- Examples → Interface design. Long tool-usage examples and few-shot
transcripts constrain exploration. Prefer expressive interfaces: good
parameter names, enums that hint at valid states, tight descriptions.
In tool/MCP definitions, an enum of
pending | in_progress | completed
teaches more than three worked examples.
- Upfront context → Progressive disclosure. Anything long that's only
sometimes needed (review checklists, deploy runbooks, style deep-dives)
should move out of CLAUDE.md into a skill or linked file loaded on demand.
CLAUDE.md is loaded every session — it should carry only what every
session needs.
- Repetition → Concise, single-home instructions. The same instruction
appearing in CLAUDE.md and a skill and a tool description is a bug:
copies drift and eventually conflict. Each instruction gets exactly one
home — tool-usage guidance lives in the tool description, repo gotchas in
CLAUDE.md, team opinions in skills.
- Manual memory → Automatic memory. Sections telling the agent to
hand-maintain notes/changelogs in CLAUDE.md, or accumulated session-specific
trivia, are obsolete where auto-memory exists. Flag CLAUDE.md content that
is really memory (per-user, per-incident, time-bound) rather than
repo truth.
- Simple specs → Rich references. Where docs describe behavior in loose
prose, prefer pointing at the real thing:
@-referenced source files, a
test suite, an HTML mockup, a rubric a verifier can score against. Code-based
specs beat prose paraphrases of code.
Cross-cutting failure modes to hunt alongside the shifts:
- Conflicts — instructions that clash across layers (e.g. "document
thoroughly" in one file, "DO NOT add comments" in another). Highest-value
findings; a conflict forces the model to deliberate or guess on every task.
- Staleness (map ≠ territory) — docs naming files, commands, flags, or
services that no longer exist, or missing ones that now do. Verify every
concrete claim you audit against the actual repo.
- Missing unknown-knowns — things obvious to the team but written nowhere:
the non-obvious build step, the directory you must never touch, the reason a
weird pattern exists. These are what CLAUDE.md is for ("repository
gotchas rather than obvious patterns").
Step 0 — Inventory the context surface
Collect everything that gets assembled into an agent's context here. Look for
the capability, not a specific filename:
- CLAUDE.md files — root, nested per-directory,
~/.claude/CLAUDE.md only
if the user asks for a global audit. Also AGENTS.md, .cursorrules,
.github/copilot-instructions.md if present (same disease, same cure).
- Skills —
.claude/skills/**/SKILL.md, skills/**/SKILL.md, plugin
skills committed to the repo.
- Tool designs — MCP server definitions the repo owns (tool names,
descriptions, parameter schemas), custom slash commands, hooks, and any
agent definitions (
.claude/agents/*.md).
- Codebase docs agents are pointed at — README, CONTRIBUTING, docs/
referenced from CLAUDE.md or skills.
Record rough sizes (lines/tokens) per artifact — total always-loaded weight is
itself a finding when large.
Step 1 — Audit each artifact against the rubric
For each artifact, walk the six shifts and cross-cutting modes. For every
finding record: file:line, quote, shift #, severity, proposed rewrite
(the actual replacement text — or "delete", with one line of why it's safe).
Severity:
- high — conflicts between layers; rules that block correct behavior;
stale facts an agent would act on.
- medium — overconstraint, redundancy, always-loaded bulk that belongs in
a skill.
- low — style, phrasing, minor bloat.
Verify before you flag: a claim of staleness must be checked against the repo
(does that script exist? does that command run?); a claim of redundancy must
cite both locations.
Step 2 — Probe for unknowns (the gaps docs don't show)
Auditing text only finds what's written. Now find what's missing:
- Blind-spot pass: skim the actual territory — build config, CI, scripts,
the weirdest-looking directories — and list load-bearing facts that appear in
no doc. Each is a candidate "unknown known" to add.
- Knowledge quiz: write 5–10 questions a fresh agent must answer to work
here safely ("how do I run one test?", "what must never be committed?",
"which service is the source of truth for X?"). Answer each using only the
audited docs. Unanswerable questions = gaps; wrong answers = stale docs.
- Git check:
git log --oneline -20 -- <doc> — a CLAUDE.md untouched for
months in an active repo is presumptively stale; recent churny areas of the
codebase with no doc coverage are presumptive gaps.
Step 3 — Report
Deliver a findings report (markdown in the repo, e.g.
docs/agent-context-audit-YYYY-MM-DD.md, or just in the reply if the user prefers):
- Scorecard — per artifact: size, finding counts by severity, one-line
verdict (keep / trim / restructure / delete).
- Findings table — file:line, quote, shift #, severity, proposed rewrite.
- Gaps — missing unknown-knowns from Step 2, each with proposed text and
the home it belongs in (CLAUDE.md vs skill vs tool description).
- Projected result — estimated always-loaded context before → after.
Lead with the top 3–5 highest-value changes; don't bury a layer conflict under
twenty style nits.
Step 4 — Apply (with approval)
Ask which findings to apply (all high, everything, or cherry-pick). Then:
- Make the edits exactly as proposed in the report.
- When moving content out of CLAUDE.md into a skill, create the skill and leave
a one-line pointer behind.
- Keep each change reviewable — don't reflow or rewrite text you didn't flag.
- If
/doctor is available in this Claude Code install, suggest the user also
run it as a second opinion on CLAUDE.md/skill sizing.
Anti-patterns for the auditor
- Deleting a hard rule that guards something genuinely irreversible — the
point is removing fake constraints, not real ones.
- Flagging brevity as a problem — short, dense CLAUDE.md files are the goal.
- Rewriting voice/style wholesale — preserve the team's phrasing where content
is sound.
- Reporting a finding without a concrete rewrite — every finding must be
actionable as written.
Sources: Anthropic, "The new rules of context engineering for Claude 5
generation models" (claude.com/blog); "A field guide to Claude Fable: finding
your unknowns" (claude.com/blog).
1---
2name: agent-context-audit
3description: >
4 Audit a repo's agent context — CLAUDE.md files, codebase docs, skills, and
5 tool/MCP designs — against Anthropic's Claude 5 context-engineering guidance
6 ("unhobbling": Anthropic cut ~80% of Claude Code's system prompt with no eval
7 loss). Finds overconstraint, conflicting instructions, redundancy, stale
8 facts, and missing "unknown knowns"; produces a scored findings report with
9 concrete rewrites, then applies approved fixes. Use when someone says "audit
10 my CLAUDE.md", "context audit", "unhobble this repo", "review our agent
11 docs/skills/tools", or after upgrading to Claude 5-generation models.
12user_invocable: true
13metadata:
14 source: https://skillmd.com/skills/ai-builder-club/agent-context-audit
15---
16
17# agent-context-audit — unhobble this repo's agent context
18
19Goal: find where this repo's context (CLAUDE.md, docs, skills, tool designs)
20**hobbles** a Claude 5-generation model — overconstrains it, contradicts itself,
21repeats itself, or hides context the model actually needs — and leave behind a
22findings report plus approved fixes.
23
24Background: Anthropic removed over 80% of Claude Code's system prompt for
25Claude 5 models with **no measurable loss** on coding evals. Older context was
26written for models that needed rules; newer models need judgment, good
27interfaces, and the facts they can't infer. This skill audits against that
28shift, plus the "finding your unknowns" framework (the gap between the *map* —
29your prompts/docs — and the *territory* — the actual codebase).
30
31You are **auditing first, fixing second**. Do not edit anything until Step 4.
32
33## The six shifts (the audit rubric)
34
35Every finding maps to one of these. Cite the shift number in the report.
36
371. **Rules → Judgment.** Hard rules ("NEVER…", "ALWAYS…", "do not add
38 comments", "one-line docstrings max") that encode a *preference*, not a real
39 constraint, should become judgment framing ("write code that reads like the
40 surrounding code") — or be deleted if the model would infer it anyway.
41 Keep hard rules only where violation is genuinely costly (security, prod
42 data, irreversible actions, legal/billing).
432. **Examples → Interface design.** Long tool-usage examples and few-shot
44 transcripts constrain exploration. Prefer expressive interfaces: good
45 parameter names, enums that hint at valid states, tight descriptions.
46 In tool/MCP definitions, an enum of `pending | in_progress | completed`
47 teaches more than three worked examples.
483. **Upfront context → Progressive disclosure.** Anything long that's only
49 sometimes needed (review checklists, deploy runbooks, style deep-dives)
50 should move out of CLAUDE.md into a skill or linked file loaded on demand.
51 CLAUDE.md is loaded *every* session — it should carry only what every
52 session needs.
534. **Repetition → Concise, single-home instructions.** The same instruction
54 appearing in CLAUDE.md *and* a skill *and* a tool description is a bug:
55 copies drift and eventually conflict. Each instruction gets exactly one
56 home — tool-usage guidance lives in the tool description, repo gotchas in
57 CLAUDE.md, team opinions in skills.
585. **Manual memory → Automatic memory.** Sections telling the agent to
59 hand-maintain notes/changelogs in CLAUDE.md, or accumulated session-specific
60 trivia, are obsolete where auto-memory exists. Flag CLAUDE.md content that
61 is really *memory* (per-user, per-incident, time-bound) rather than
62 *repo truth*.
636. **Simple specs → Rich references.** Where docs describe behavior in loose
64 prose, prefer pointing at the real thing: `@`-referenced source files, a
65 test suite, an HTML mockup, a rubric a verifier can score against. Code-based
66 specs beat prose paraphrases of code.
67
68Cross-cutting failure modes to hunt alongside the shifts:
69- **Conflicts** — instructions that clash across layers (e.g. "document
70 thoroughly" in one file, "DO NOT add comments" in another). Highest-value
71 findings; a conflict forces the model to deliberate or guess on every task.
72- **Staleness (map ≠ territory)** — docs naming files, commands, flags, or
73 services that no longer exist, or missing ones that now do. Verify every
74 concrete claim you audit against the actual repo.
75- **Missing unknown-knowns** — things obvious to the team but written nowhere:
76 the non-obvious build step, the directory you must never touch, the reason a
77 weird pattern exists. These are what CLAUDE.md is *for* ("repository
78 gotchas rather than obvious patterns").
79
80## Step 0 — Inventory the context surface
81
82Collect everything that gets assembled into an agent's context here. Look for
83the capability, not a specific filename:
84
85- **CLAUDE.md files** — root, nested per-directory, `~/.claude/CLAUDE.md` only
86 if the user asks for a global audit. Also `AGENTS.md`, `.cursorrules`,
87 `.github/copilot-instructions.md` if present (same disease, same cure).
88- **Skills** — `.claude/skills/**/SKILL.md`, `skills/**/SKILL.md`, plugin
89 skills committed to the repo.
90- **Tool designs** — MCP server definitions the repo owns (tool names,
91 descriptions, parameter schemas), custom slash commands, hooks, and any
92 agent definitions (`.claude/agents/*.md`).
93- **Codebase docs agents are pointed at** — README, CONTRIBUTING, docs/
94 referenced from CLAUDE.md or skills.
95
96Record rough sizes (lines/tokens) per artifact — total always-loaded weight is
97itself a finding when large.
98
99## Step 1 — Audit each artifact against the rubric
100
101For each artifact, walk the six shifts and cross-cutting modes. For every
102finding record: **file:line, quote, shift #, severity, proposed rewrite**
103(the actual replacement text — or "delete", with one line of why it's safe).
104
105Severity:
106- **high** — conflicts between layers; rules that block correct behavior;
107 stale facts an agent would act on.
108- **medium** — overconstraint, redundancy, always-loaded bulk that belongs in
109 a skill.
110- **low** — style, phrasing, minor bloat.
111
112Verify before you flag: a claim of staleness must be checked against the repo
113(does that script exist? does that command run?); a claim of redundancy must
114cite both locations.
115
116## Step 2 — Probe for unknowns (the gaps docs don't show)
117
118Auditing text only finds what's written. Now find what's missing:
119
120- **Blind-spot pass:** skim the actual territory — build config, CI, scripts,
121 the weirdest-looking directories — and list load-bearing facts that appear in
122 no doc. Each is a candidate "unknown known" to add.
123- **Knowledge quiz:** write 5–10 questions a fresh agent must answer to work
124 here safely ("how do I run one test?", "what must never be committed?",
125 "which service is the source of truth for X?"). Answer each *using only the
126 audited docs*. Unanswerable questions = gaps; wrong answers = stale docs.
127- **Git check:** `git log --oneline -20 -- <doc>` — a CLAUDE.md untouched for
128 months in an active repo is presumptively stale; recent churny areas of the
129 codebase with no doc coverage are presumptive gaps.
130
131## Step 3 — Report
132
133Deliver a findings report (markdown in the repo, e.g.
134`docs/agent-context-audit-YYYY-MM-DD.md`, or just in the reply if the user prefers):
135
1361. **Scorecard** — per artifact: size, finding counts by severity, one-line
137 verdict (keep / trim / restructure / delete).
1382. **Findings table** — file:line, quote, shift #, severity, proposed rewrite.
1393. **Gaps** — missing unknown-knowns from Step 2, each with proposed text and
140 the home it belongs in (CLAUDE.md vs skill vs tool description).
1414. **Projected result** — estimated always-loaded context before → after.
142
143Lead with the top 3–5 highest-value changes; don't bury a layer conflict under
144twenty style nits.
145
146## Step 4 — Apply (with approval)
147
148Ask which findings to apply (all high, everything, or cherry-pick). Then:
149- Make the edits exactly as proposed in the report.
150- When moving content out of CLAUDE.md into a skill, create the skill and leave
151 a one-line pointer behind.
152- Keep each change reviewable — don't reflow or rewrite text you didn't flag.
153- If `/doctor` is available in this Claude Code install, suggest the user also
154 run it as a second opinion on CLAUDE.md/skill sizing.
155
156## Anti-patterns for the auditor
157
158- Deleting a hard rule that guards something genuinely irreversible — the
159 point is removing *fake* constraints, not real ones.
160- Flagging brevity as a problem — short, dense CLAUDE.md files are the goal.
161- Rewriting voice/style wholesale — preserve the team's phrasing where content
162 is sound.
163- Reporting a finding without a concrete rewrite — every finding must be
164 actionable as written.
165
166Sources: Anthropic, "The new rules of context engineering for Claude 5
167generation models" (claude.com/blog); "A field guide to Claude Fable: finding
168your unknowns" (claude.com/blog).