Harness Audit

Do not load shared snapshots for every audit. Read the plugin-root STATE.md when an installed capability, limit, or cross-tool convention is in question; read MODELS.md only when the audit includes model policy, routing economics, or model-release convergence. Resolve both from this skill's source. They orient the check but never replace inspection of the installed harness or live canonical docs.

Start here: why is this audit running?

Steps 5 and 6 always apply: report first, apply only on the owner's go-ahead.

The attention this skill redirects

From "read the config files and comment" to "measure what every session pays before any work starts, then check the harness's claims against its own behavior."

The default failure is a vibes review: opening CLAUDE.md, calling it well-written, and missing that the setup burns five figures of tokens per session on descriptions, instructions, and rosters nobody chose deliberately. The second failure is trusting the harness's documentation of itself: files that say "this is not loaded" while being loaded, checklists that say "always run X" with nothing enforcing X.

The third failure is this skill's own trap: treating cost as the verdict. Tokens are what's easy to count; what a surface lets the owner produce is what decides, and an audit that only counts always lands on cut, because the cheapest harness is the empty one. A surface the owner reaches for is already earning its cost, and that she uses it is the evidence it clears the bar, not a number to override. Report what each surface costs so the owner spends deliberately; never strip toward zero.

Operating contract

Three rules frame every step, and no finding overrides them:

• The audit is read-only. Steps 1–4 inventory, measure, and check; none of them writes anything, anywhere. The deliverable is the findings report (Step 5). Edits happen only in Step 6, after the owner's explicit go-ahead on named findings — invoking the audit is consent to look, never to change.
• Writes stay inside the project directory, always. Even after a go-ahead, nothing outside this repo is edited: not ~/.claude/, not another repo, not user, enterprise, or managed settings. No approval unlocks machine scope.
• The subject is this project's harness, not the machine. Machine-scope surfaces are inventoried because they load into this project's sessions and this project pays their cost — not because the machine is under audit. The only remedy an audit proposes for a machine-level problem is a project-scoped counterweight (a plugin disabled for this repo via enabledPlugins in .claude/settings.local.json, a project rule that overrides a global instruction for this repo's files); the machine-level fix itself is reported for the owner to make.

Step 1: Inventory the standing surfaces

List everything that enters context at session start, with its scope and owner:

• Instruction files: CLAUDE.md chain (enterprise → user ~/.claude/CLAUDE.md → project → CLAUDE.local.md), auto-memory MEMORY.md. Where the repo also carries AGENTS.md, .cursorrules, or .github/copilot-instructions.md, read those too and establish which file actually holds the content. Claude Code reads CLAUDE.md and not AGENTS.md, so in a repo that serves more than one harness the two files can disagree, or one can be a stub that steers nobody: an AGENTS.md holding only @CLAUDE.md reaches Claude not at all and reaches the harnesses that read AGENTS.md as a literal ten-byte string, because the AGENTS.md spec has no import syntax. Check which sessions each file is actually paid by before judging what it says.
• Skills: personal (~/.claude/skills/), project (.claude/skills/), and plugin-delivered (enabledPlugins across all settings files). Every visible skill's description + when_to_use loads every turn.
• MCP servers: .mcp.json (minus disabledMcpjsonServers, plus enabledMcpjsonServers allowlists), plugin-shipped servers, claude.ai connectors. Tool schemas defer behind tool search, but server instruction blocks load in full.
• Agent roster: every plugin-shipped subagent type adds its description to the Agent tool's listing.
• Hooks and settings: all levels (managed → user → project → local). Note which hooks exist and what they enforce (including DirectoryAdded when mid-session /add-dir behavior is claimed), managed model/version policy (availableModels + enforceAvailableModels, requiredMinimumVersion), the two subagent fan-out limits (CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH defaults to 3 as of v2.1.219 — =1 disables nesting; plus the 20-deep concurrency cap. The per-session total was removed in v2.1.224, so don't audit for one), sandbox credential/network settings (sandbox.credentials including mode: "mask", sandbox.network.strictAllowlist), and workflowSizeGuideline / Dynamic workflow size when ultracode or workflows are in play (default medium, aim <15 agents). On macOS and Linux from v2.1.224, also inspect cross-session messaging, which is on by default and needs no enabling: it binds an inbox socket other sessions deliver to, so it is an inbound channel into the session that a permissions review will otherwise miss entirely. Its controls are crossSessionInbound (accept/hold/refuse), isolatePeerMachines for anything leaving the machine, dialogExpiry for how long a held message waits, and permissions.deny entries on the bare tool names SendMessage and ListAgents — noting that denying SendMessage also cuts messaging to subagents and agent-team teammates, which share the tool. /list-agents reports whether the session has the feature at all.

The session you are in is a specimen: the listings injected into your own context are the ground truth of what loads.

Step 2: Quantify before judging

Numbers first; opinions after. The unit that matters is tokens-per-session-forever, and prefix size also costs prompt-cache build time on every cache miss.

• Skill descriptions: sum frontmatter description + when_to_use chars across each scope, divide by ~4 for tokens. One pass:

python3 - <<'EOF'
import os, re
root = os.path.expanduser('~/.claude/skills')   # repeat for project + plugin cache dirs
total = 0
for name in sorted(os.listdir(root)):
    p = os.path.join(root, name, 'SKILL.md')
    if not os.path.isfile(p): continue
    m = re.match(r'^---\n(.*?)\n---', open(p, encoding='utf-8', errors='replace').read(), re.S)
    fm = m.group(1) if m else ''
    total += sum(len(v) for _, v in re.findall(r'^(description|when_to_use):(.*?)(?=^\w|\Z)', fm, re.S | re.M))
print(total, 'chars ~', total // 4, 'tokens/session')
EOF

• Instruction files: wc -l on every always-loaded file. The repo's own cost model applies (shorter is better-read; loaded local files count too).
• Ask the user to run /context for the authoritative split. Your estimates bound it; the command confirms it.

Step 3: The setup checks

1. Self-consistency. Diff what the harness's documentation claims against what the harness does. Classic finds: a loaded file that says it is not loaded; a dateline pinned to a version several releases back; a "this is enforced" line with no enforcement; a skill or command body that invokes or assumes another skill (/X) — fragile by construction (the target can be absent, disabled via disableBundledSkills, or set to disable-model-invocation) and a dead cross-reference that reads as working capability when /X is not in the inventory at all. This flags the presence of the reference, not the quality of the body — authoring skills to stand alone belongs to whoever writes them. The instruments for this diff are live commands, never the harness's word for itself: /context is the authoritative record of what loaded and from where; /skills, /hooks, /mcp, /permissions, /status, and /doctor each resolve one surface; a fault that won't localize bisects with claude --safe-mode (all customizations off, v2.1.169+) or a clean CLAUDE_CONFIG_DIR, reintroducing surfaces one at a time. Every instrument here serves the diff between what the harness claims and what it does, on a harness that starts and runs; repairing one that will not start is debugging, not this audit. The canonical page for this battery, settings precedence, and the common-misconfiguration table (hook matcher syntax, ~/.claude.json vs ~/.claude/settings.json, skill folder layout, mcpServers in the wrong file) is code.claude.com/docs/en/debug-your-config — fetch it at audit time rather than trusting this paragraph's snapshot; code.claude.com/docs/llms.txt indexes every other page a claim can be verified against.

2. Duplication across scopes and install paths. The same artifact listed twice pays twice: a project symlink and a user-scope plugin copy of the same skill; two plugins shipping the same command under different names; a personal skill shadowing a project one (personal wins silently). Extend the sweep past one harness's scopes wherever a repo publishes to several: a skill mirrored into a global directory (~/.cursor/skills, ~/.agents/skills) and shipped in that harness's native plugin is registered twice, and the two registrations rarely share freshness semantics. Resolve every path to what it actually serves before judging — a mirror symlink serves the working tree, a plugin cache may hold real copies of origin/main, and a cache directory can itself be a symlink back into the repo, in which case there is duplication but no skew. The finding that matters is the skew: two registrations whose contents differ, so one session loads both versions of a skill at once. diff -rq each installed copy against the repo to establish it, and read the refresh command's actual contract rather than its reputation — the step that syncs a marketplace snapshot is not the step that rewrites an installed plugin cache, and a version-keyed cache directory can look current while serving an older revision. Where a scheduled updater (launchd, cron) is the only thing that closes the gap, its interval is the width of the stale window: a daily job means a day.

3. Enforcement parity. Every "always", "never", and "run X before Y" sentence in the instruction files should map to a hook (including DirectoryAdded for mid-session directory grants), a CI gate, a permission rule (permissions.deny, now matching tool parameters too, e.g. Bash(rm *) / Edit(*.env), v2.1.178), a managed or project setting (sandbox mask/strictAllowlist, workflowSizeGuideline, model allowlists — a "we standardize on model X" line is enforced only by availableModels/enforceAvailableModels, not prose), or a conscious decision to stay manual. A promise with no mechanism is a request, not a guarantee. Parity starts one step earlier: a guardrail has to be a fact of the repo before its mechanism matters. git status and git diff every instruction file and hook in the chain, because a guardrail that exists only in the working tree (git log -S"<its key phrase>" returns nothing) is one checkout away from not existing and has reached no teammate, other machine, or plugin install. The classic find: a prior session writes the fix for an incident into CLAUDE.md and never commits it, so the audit reads standing policy where git holds nothing. And tier what stays prose-only: an order the model demonstrably read and still skipped (the owner had to re-issue it in session) has already failed as prose, because prose is re-weighed against everything else in context every session; it leads the hook/gate candidates rather than settling as a conscious stay-manual.

   Parity also runs one step later than the mechanism's existence: a hook can be registered, syntactically valid, matched to the right tool, and still never do its work. The usual cause is a precondition guard that fails silently — a filename that doesn't exist (-f README in a repo whose file is README.md), a jq path that returns empty, an env var that is unset outside the authoring session — after which the script exits 0 and reads as a clean pass forever. No amount of reading the config finds this, because the config is correct. Execute each hook once against a synthetic input for the event it matches, and confirm it reaches its actual work rather than merely exiting 0: trace it (bash -x) or check for a value it should have computed. A hook that has never fired is a guardrail the owner believes she has.

4. Scope discipline. Each artifact should live at the narrowest scope that serves it. Personal-scope skills used in one domain belong in a toggleable plugin or a project; product plugins needed in some repos should be per-project enabledPlugins entries, not global; a fleet of client-work skills in ~/.claude/skills/ taxes every repo on the machine.

5. Routing effectiveness (provisional — one directional validation 2026-07-27; see CHANGELOG). Applies wherever the repo routes agents to other resources: a doc fleet (subsystem docs routed from an AGENTS.md table or equivalent), or instruction-file rows steering the model among installed skills and MCPs. Skip it for single-doc repos with no fleet routing. The question is whether the routing actually reaches the agent rather than living as good intentions:

   • Dead rows. Every routing row points to something that still exists: a doc at that path, a skill or MCP under that exact name. Plugin renames silently orphan old-prefix:skill references, so check the prefix too. A row naming a moved, renamed, or deleted target routes nowhere, and is worse than an absent row because it reads as coverage.
   • Invocation-style fit. Skill-routing rows assume the model picks among collisions autonomously. Where the owner invokes those skills manually, picker rows are dead weight; what earns its place there is the misfire guard — the row, or better the path-scoped rule, stopping an autonomously-triggered skill from acting on the wrong slice.
   • Two-level routing. Each system doc opens with a greppable header (Purpose / Read when / Key constraints / Relevant paths / Last verified) so an agent can confirm the doc applies before paying to read it. A fleet routed only by a top-level table forces whole-doc reads just to check relevance.
   • Mechanical backing. The highest-value rows are duplicated down into mechanical surfaces (path-scoped rules, skill descriptions, hooks). A routing table whose only enforcement is a prose "read the router first" instruction is a request, not a guarantee — the same failure as enforcement parity one level up, because the model won't reliably run a preflight.
   • Staleness. Each doc's Last verified sits within a sane window; a fleet of confidently-worded but months-stale docs is the doc-fleet form of the frozen-fact trap.

Step 4: Probe the assembled behavior when the harness changed

After a substantial harness change, or when individually sound artifacts produce incoherent sessions, test the system as installed. This is conditional because a fresh live run costs tokens: reuse a recent representative transcript already in the audit's scope when it exercises the changed seams; otherwise state the proposed job, tools, token cost, mutation boundary, and external side effects, then get the owner's approval before launching one or two fresh sessions.

Choose one or two high-traffic jobs that cross more than one surface — for example an instruction file plus a skill plus a tool or hook — and state the expected output and required human boundary before running them. Default to replay, a read-only job, a finished real job, or a disposable copy. A composition probe never mutates active work, another repo, machine scope, or an external system without explicit approval for those exact side effects; when no safe representative probe exists, report composition as unverified. Use the complete installed harness, not an isolated artifact fixture. Skills with context: fork background by default — don't treat non-blocking completion as a probe failure. Background agents from claude agents commit and push to preserve work, and open a draft PR only when the task calls for it (v2.1.221), so don't list "opens draft PR" as an automatic side effect. Inspect the trace and result together:

• Which instructions and context were loaded or supplied, and which artifacts and tools were invoked?
• Did two surfaces compete, duplicate ownership, or issue contradictory guidance?
• Was intent or state lost between a trigger, a tool call, and the next surface?
• Did the system pause, ask, or stop at the intended human decision point?
• Was the final work better against the job's objective, not merely compliant with each component?

Report the observable composition path and the first seam that failed. A transcript cannot prove which loaded instruction causally changed hidden reasoning; leave that unknown unless a controlled comparison establishes it. An artifact passing alone does not clear this check; the unit of quality is the installed system's behavior. Keep this at the interaction boundary: reviewing the expertise inside one skill body is a separate job from tracing how the assembled surfaces interact.

The CLAUDE.md chain, read as intent

CLAUDE.md is the always-on spine, the most-paid surface and the one that rots fastest, because it sits next to the code it describes. Give each file in the chain a closer read than the cost pass, against the job it alone can do: transmit what the code can't tell you. Flag, in the file:

• No code-authority clause. The file should state that when it and the code disagree, the code wins, and that a contradiction means this file is stale, flag it, not that the agent should obey the doc. This is the single highest-leverage line a CLAUDE.md can carry: without it, every sentence that has quietly gone stale is still read as current instruction. Two scopings keep the clause from overreaching. Code is authoritative about what is, never about what's good — read unscoped, "code wins" licenses conforming to entrenched drift instead of flagging it, on top of a harness already instructing the model to match surrounding code. And normative documents run the other way: when the contradicting file is an ADR or spec recording decided intent, the drift may indict the code, so the finding is investigate and ask, never a silent rewrite of the requirement to match the implementation.
• Restated current state. Any line restating stack, dependencies, scripts, file layout, or "we use X" where X already lives in a manifest or the source pays tokens to go stale on the next commit. What survives a refactor is intent, spirit, durable traps, and pointers. Grep the file's version numbers and dependency names against the actual manifests; each match is a line the code already owns and the doc should drop. /doctor automates part of this find from v2.1.206: it flags checked-in CLAUDE.md content Claude can derive from the codebase and proposes the trim.
• Frozen facts that should be discovered. A value that lives at runtime (a token file, a config, a schema, a vendored doc set) belongs behind a read-first instruction ("read globals.css for the theme before writing classNames"; "read the docs in node_modules/… before coding"), not pasted as today's snapshot. The discovery survives every project the snapshot dies in.
• Bare directives. A rule the model can't generalize from ("Y before Z", no reason, no condition) underperforms a condition-shaped, reasoned one ("when X genuinely needs Y, do Z, because …"). The model generalizes from a why where it can't from a bare MUST.
• Unverified mechanisms. An intent voice does not protect a wrong mechanism inside it: check each concrete claim (a path, a script name, a flag) against the filesystem it names.
• Bloat. Compliance dilutes as rules and examples accumulate. Count them and report the counts; size is judgment for the owner, not a threshold to enforce.

Finding and quantifying these gaps is this skill's job; authoring the fixes (voice, the Why pattern, the goes-elsewhere table) is the writing job that follows the audit.

Step 5: The findings report

The report is the audit's deliverable, and nothing in it is applied yet. Everything at machine scope is inventory-and-report only: anything under ~/.claude/ (personal skills, user CLAUDE.md/settings.json, skillOverrides, user-scope hooks), enterprise or managed settings, global enabledPlugins, and anything that changes another repo. Never edit it, never stage an edit, never ask "shall I apply this?"; the audit's job at machine scope ends at the observation the owner acts on herself, plus any project-scoped counterweight the contract allows. This holds even when the finding is obviously correct and the fix is one line, because the owner owns her machine. It is a hard boundary, not a default to weigh.

A cost is not yet a verdict. Before any finding recommends removing or disabling a capability (a skill, plugin, MCP server, or hook), weigh whether the owner uses it; a used surface is reported at its cost, never cut, and a disable becomes a recommendation only where she confirms she does not reach for it. An unrequested disable is the harm the audit exists to prevent, not a saving it delivers.

Every finding is quantified and lands in one bucket:

• Proposed is for current-repo edits: content moves (splitting a loaded file), this repo's .claude/ settings and hooks, project-scope plugin toggles — including the project-scoped counterweight to a machine-level problem (the global plugin is the owner's to remove; disabling it for this repo in .claude/settings.local.json is a project edit and belongs here). Carry the exact change (file, edit, token saving) so a go-ahead is all that's needed.
• Reported is for every machine-scope and cross-repo finding. State the exact edit the owner can make herself (file, change, token saving) and stop. No staged file, no apply prompt, no question that reads as an offer to apply — and no go-ahead changes this.
• Held is for what fails the bar: would not change what the user or agent actually does.

Shape the report so the owner can act on it, not just read it:

• The first line is the single highest-value item and what it buys — an approvable action, not a summary of the audit.
• Findings are numbered and ranked by impact. Each is one bounded item: what, where, the number that proves it, the exact remedy, the bucket.
• Cap the ranked list at five; everything else goes under a separate "later" heading. Five ranked beats fifteen unranked.
• End with exactly one question: which Proposed items get the go-ahead.
• On any later turn, restate where things stand (applied, pending, reported) rather than assuming the report is remembered.

Step 6: Apply on the go-ahead

Apply only what the owner approved, only inside the project directory, exactly as proposed — a go-ahead on the report is not consent to improvise adjacent fixes, and a partial go-ahead ("just 1 and 3") leaves the rest untouched. Note the reversal where each change lives ("flip this back by..."). The auto-mode classifier gates harness self-modification (this repo's hooks, settings.local.json) even after a yes; surface the change and let the prompt through, never route around a denial. Close by restating what applied and what remains.

Anti-patterns

• Vibes without counts. "Your CLAUDE.md looks clean" is not an audit.
• Editing during the audit. The tell is a "quick fix while I'm here" mid-inventory. Invocation is consent to look, never to change; every edit waits for the report and the go-ahead, however small and however obviously right.
• Recommending a cut you never use-tested. Flagging a skill, plugin, or server for removal on token cost alone, without weighing whether the owner reaches for it. The description budget is real and the unchosen marginal artifact taxes every turn, but a used one out-earns its cost by being used; report its cost, do not charge it as waste.
• Bulk-disabling without a reversal note. Every disable gets a one-line "flip this back by..." where the change lives.
• Auditing only the visible skill list. MCP instruction blocks and agent rosters are quieter and often bigger.
• Declaring composition from isolated passes. Individually valid skills, hooks, and rules can still compete or lose context when assembled. Trace at least one representative cross-surface job after a substantial change before calling the system coherent.
• Drifting into config debugging. The tell is an audit that opens by reading an error message, a stack trace, or a startup warning, and starts localizing why a command fails or a hook will not load. That is a debugging job on a broken harness; this audit measures and checks one that already runs. Say so and hand the session back rather than working the error.
• Drifting into skill-body content review. Whether a skill's guidance is expert-grade is a different audit; this skill audits the setup (including the always-on CLAUDE.md chain that frames it), not the quality of individual artifact bodies.
• Offering to apply machine-scope fixes. The tell is a question like "which of these should I prepare exact changes for?" aimed at ~/.claude/, user settings, or global plugins. Report them; do not stage or offer them. The only edit an audit may propose against a machine-scope problem is the project-scoped counterweight in this repo's .claude/.