Session Audit

Session Audit

Fix the system so the same failures don't repeat. The output is edits to rules, docs, hooks, and memories — not a report card.

Important

• The audit exists to produce CHANGES, not observations. If the audit doesn't edit a file, propose a hook, or save a memory, it failed.
• A clean session is a valid outcome for the failure-finding pass. "Nothing to fix" is not a failure of the audit — it means the system worked. Do not manufacture findings to justify the audit's existence. This applies to Steps 1–3 only; the doc-sync phase still runs.
• Project doc sync is mandatory. ARCHITECTURE.md, CLAUDE.md (project root), and TASKS.md must be checked on every audit, regardless of whether systemic failures were found. A clean session still updates the manifest, rules, and task list to reflect what actually changed in the session. Skip a doc only if the file does not exist in the project root.
• Permission sweep is mandatory. Every audit checks for Bash approval events and proposes adding safe, narrow-scoped patterns to ~/.claude/settings.json permissions.allow. Dangerous commands stay in ask/deny untouched. Runs on every audit regardless of whether systemic failures were found.
• Scope is THIS session only. Only audit failures that occurred in the current conversation. Do not pull in unrelated config issues, old incidents from memory, or problems spotted while reading files for other purposes. If you noticed something unrelated, mention it as an aside after the audit, clearly separated — never as a finding.
• Never score sessions. Scores are vanity metrics. The user wants fixes, not grades.
• Never persist YAML audit logs. Nobody reads them. The fixes ARE the persistence.
• Be brutally honest about what went wrong, but spend 20% of effort on diagnosis and 80% on fixing the system.
• Every systemic failure gets exactly one of: a rule edit, a hook proposal, a doc fix, a memory save, or a reviewer SKILL edit. Not several. Pick the one that actually prevents recurrence.

Instructions

Step 1: Scan

Walk the conversation. Identify every failure — user corrections, abandoned approaches, wasted cycles, cascading errors. For each, note:

• What happened (1 sentence)
• Why (root cause — not symptoms)
• Whether it's systemic (will repeat) or situational (one-off)

Ignore situational failures. They happened, they're over, nothing to fix. Focus exclusively on systemic failures — patterns that will recur unless the system changes.

GATE: If zero systemic failures found, output "No systemic failures found. Nothing to fix." and skip Steps 2 and 3 (diagnose + apply fixes). Do not go looking for unrelated problems to fill the void. A clean session means the failure-finding pass worked — that is the best possible outcome. Then proceed directly to Step 4 (Permission Sweep) and Step 5 (Sync Project Docs), which both run on every audit regardless of whether failures were found.

Present findings as a brief numbered list. No categories, no severity scores, no taxonomy lookups. Just: what broke and why, in plain language.

Reviewer provenance sub-pass

For each systemic failure already identified above — and ONLY for those, never as a standalone check — ask whether plan-review or code-review was supposed to catch it. Tag the failure with exactly one of these states, or leave it untagged if none apply:

1. Missed catch — a reviewer fired on the surface area, returned CLEAN (plan-review) or no surfaced findings (code-review), and the failure is the kind of issue that reviewer's own checklist is supposed to catch. Root cause sits in the reviewer's rules, not the main agent's behavior.
2. Ignored verdict — a reviewer fired, returned FIX / blockers (plan-review) or surfaced findings (code-review), and the main agent implemented anyway without addressing them. Root cause is a workflow gap between reviewer and main agent, not the reviewer's content.
3. Never fired — a substantive plan or non-trivial diff touched the failure's surface area and neither review skill was invoked at all. Root cause is a missing trigger (rule or hook), not the reviewer's content.
4. Noisy false positive — a reviewer fired, flagged something that was correctly dismissed, and the noise contributed to the failure (wasted cycles, or desensitized the agent to a real finding nearby). Root cause is the reviewer's scoring rubric.

Do not force a tag. Most failures won't have one — the main agent simply made a mistake no reviewer was positioned to catch, and the fix belongs in normal CLAUDE.md / hook / memory territory. The provenance tag only matters when a reviewer was in the loop and either dropped the ball or was never consulted.

Remember plan-review's own posture — "CLEAN Until Proven Otherwise" — and code-review's 80+ confidence threshold. A reviewer that correctly said CLEAN on something outside its checklist is not a missed catch. Only tag when the reviewer's own rules say it should have caught this exact class of issue.

Step 2: Diagnose Fixes

For each systemic failure, determine the RIGHT fix. Only one per failure — pick the highest leverage option.

CLAUDE.md is load-bearing and space-constrained. Every line in CLAUDE.md is paid for on every single turn of every session across every project. Rules compete for attention — the more rules there are, the less each one lands. A bloated CLAUDE.md is a failed CLAUDE.md, even if every individual line was justified when added. Default to memory for per-session slips. Promote to a rule only when the criteria below are met.

Memory save (DEFAULT for per-session failures) — the failure is real but specific enough that a pattern-matched memory is sufficient.

• Feedback memories for behavioral corrections the user gave you this session.
• Project memories for decisions/context that only matter in this repo.
• Use this for: one-off tool gotchas, specific API quirks, domain-specific corrections, anything that only applies in narrow contexts.
• If a compliance gap already has memory coverage and the agent still failed, do NOT add a second memory — escalate to a rule rewrite or hook. More memories don't fix compliance.

Hook proposal — when a step keeps being forgotten and a programmatic check could catch it deterministically.

• Read ~/.claude/settings.json to understand existing hooks.
• Propose the exact hook entry (matcher, type, command) and the shell script content.
• Hooks beat rules when the failure is a missed step (not a judgment error).
• Better than a rule whenever enforcement can be mechanized — hooks don't compete for attention the way CLAUDE.md lines do.

Doc edit/delete — when project docs are stale, contradictory, or misleading agents.

• If a doc led an agent astray, fix or delete the misleading content.
• If two docs contradict each other, resolve the contradiction.
• If a doc describes something that no longer exists, delete the stale section.

Rule edit (CLAUDE.md) — HIGH BAR, use sparingly. Promote to a rule only when ALL of these hold:

1. The failure pattern is cross-project and cross-session. Not "this tool returned unexpected output once" — "this failure mode will recur across unrelated codebases."
2. A memory is insufficient. Either the failure mode is so frequent and silent that the agent needs the reminder loaded every turn, OR a past memory already tried to fix this exact thing and didn't stick.
3. The rule generalizes. It must express a principle that applies to a broad class of situations, not a specific incident. "Anchor short-token replace_all" is narrow — the hundredth rule like it drowns out the ten rules that actually matter.
4. You checked for an existing rule first. If one already covers the territory, rewrite or strengthen it — do not add a parallel rule. If the existing rule isn't working, the fix is to sharpen it, not pile on.

Before writing a new CLAUDE.md rule, state in one line: "Why a memory won't catch this:" and answer it honestly. If the answer is "it probably would," write the memory instead. The session-audit's job is to keep the system healthy — adding a rule per audit is how CLAUDE.md becomes unreadable and stops working.

Global ~/.claude/CLAUDE.md for cross-project patterns. Project CLAUDE.md for project-specific rules — project rules have a lower bar because they're only loaded for that one repo, but the "does it generalize within this project?" test still applies.

Deletion — when a rule, doc section, or memory is actively harmful.

• Confusing instructions that agents misapply are worse than no instructions.
• Overly specific rules that don't generalize cause agents to cargo-cult.
• Delete with a brief note about why.
• Audit CLAUDE.md for bloat as part of every session-audit. If a rule hasn't fired in recent sessions, contradicts a newer rule, or expresses a one-time incident that never recurred, it's a candidate for deletion. A shorter CLAUDE.md that's fully absorbed beats a long one that's half-skimmed.

Reviewer SKILL edit — when a failure was tagged "missed catch" or "noisy false positive" in Step 1's provenance sub-pass. The reviewer's own rules or scoring rubric are the root cause, not the main agent. Edit the reviewer skill directly:

• ~/.claude/skills/plan-review/SKILL.md
• ~/.claude/skills/code-review/SKILL.md

Sub-cases:

• Missed catch → add or strengthen the specific check in the reviewer's "Find Real Problems" section (plan-review Step 3) or the relevant agent's guidance (code-review Step 3, Agents 1–5) that should have caught this class of issue. Include a one-line example of the pattern so future reviewers recognize it. Do not bloat the skill with generic "consider X" reminders — the addition must name the specific failure mode.
• Noisy false positive → tighten the scoring rubric or add an explicit exclusion. For plan-review: extend the "Things that are never findings" list. For code-review: extend the Step 4 "These are false positives — do NOT report them" list or sharpen the confidence rubric. One-line exclusions only; never rewrite the whole rubric.

Ignored-verdict and never-fired tags do NOT map here — those are workflow gaps, so the fix is a rule edit or hook proposal, not a reviewer SKILL edit. Keep the routing strict: only reviewer-content problems get reviewer-content fixes.

Step 3: Apply Fixes

Execute every fix. Do not ask permission for each one individually — the user invoked the audit because they want the system fixed.

For each fix, state what you're doing in one line, then do it:

• RULE: Adding to [file] — [what and why] → Edit the file
• HOOK: Proposing [hook name] — [what it enforces] → Write the script, show the settings.json entry
• DOC: Editing [file] — [what's wrong] → Edit the file
• DOC: Deleting [section] from [file] — [why it's harmful] → Edit the file
• MEMORY: Saving [type] — [what] → Write the memory file, update MEMORY.md
• DELETE: Removing [file/rule/memory] — [why it's harmful] → Delete or edit out
• REVIEWER: Editing [plan-review|code-review] SKILL.md — [what's being tightened] → Edit the reviewer skill file
• PERMISSION: Adding [pattern] to ~/.claude/settings.json allow — [command summary, why safe] → Edit settings.json (only when the approval itself was the systemic failure; otherwise belongs in Step 4)

Hook proposals are the exception — present the exact script and settings.json entry but ask before writing, since hooks run on every tool call and bad hooks break the workflow.

Step 4: Permission Sweep

This step runs on every audit, including clean sessions where Steps 2–3 were skipped by the GATE. Its purpose is to reduce approval friction: find every Bash command the user had to manually approve this session and, for the safe ones, add them to ~/.claude/settings.json permissions.allow so the user isn't asked again. Dangerous commands stay in ask/deny untouched. Approval friction is orthogonal to systemic failures — skipping this wastes the easiest win of the whole audit.

Scope: Bash tool calls only. Edit/WebFetch/MCP approvals are out of scope — the friction-to-reward ratio is best for Bash, and broader tools risk overbroad globs.

Procedure:

1. Scan the conversation for approval events. Signals:

   • A Bash tool result containing a denial/confirmation string (e.g. "user did not approve", "requires approval", "permission denied by user"), followed by the same command being retried and succeeding.
   • User messages like "yes", "approve", "allow that", "go ahead" immediately following a tool-permission prompt that references a Bash command.
   • Any Bash tool call you remember being asked to confirm before running.
   • If no approval events occurred this session, state PERMISSION: no approvals this session and move on.
   • When in doubt, skip. The detection is heuristic because the agent doesn't see a structured "approval" event — you reconstruct from transcript text. If you can't point to a specific transcript signal, do not classify the command as "approved." Better to leave a real approval unpromoted than to fabricate one and expand the allow surface on a guess.

2. Read ~/.claude/settings.json once. Extract the existing permissions.allow, permissions.deny, and permissions.ask lists so you can check for conflicts and duplicates.

3. Classify each approved command into exactly one bucket:

   SAFE → propose for allow (all must hold):

   • Read-only or status-only: ls, cat, head, tail, wc, grep, rg, find, git status, git log, git diff, git branch --show-current, git show, gh pr view, gh run view, kubectl get, docker ps, docker logs, jq, read-only python3 -c snippets, and similar inspection commands.
   • No writes to disk outside /tmp, no network sends, no package installs, no credentials, no sudo.
   • Pattern does not match any existing deny or ask entry.
   • Pattern is scoped — Bash(tool subcommand*) not Bash(tool*). Example: Bash(gh pr view*), not Bash(gh*).
   • Not already present in allow.

   DANGEROUS → leave alone (any one disqualifies):

   • Mutates state: installs, upgrades, rm, mv, git push, git reset --hard, git clean, git checkout --, git restore, branch/repo deletion, gh pr merge, gh pr close.
   • Elevated: sudo, csrutil, spctl, nvram, firewall toggles.
   • Network-fetch-to-disk: curl -o, curl -O, wget.
   • Writes to shared config: shell rcs, .git/**, .env*, credentials paths.
   • Arbitrary code execution with side effects: python3 script.py (not -c snippet), node script.js, bash script.sh.
   • Anything already listed in ask or deny — the user wants to approve those each time.
   • Ambiguous — when unsure, leave it alone. Bias toward caution.

4. Safety floor. If a proposed pattern would shadow an ask or deny entry (e.g. Bash(git*) would swallow the Bash(git push*) ask rule), reject the proposal and state why. Prefer the narrowest glob that covers the observed invocation.

5. Apply each SAFE proposal by editing ~/.claude/settings.json to append the pattern to permissions.allow. Preserve JSON formatting; append, don't re-sort. Announce each one:

   PERMISSION: Adding Bash(gh pr view*) to ~/.claude/settings.json allow — read-only PR inspection
   

   For each DANGEROUS command that was approved, announce the skip so the user can see it was considered:

   PERMISSION: Skipping Bash(rm -rf build/) — destructive, belongs in ask
   

6. Never auto-remove from deny or ask, and never propose adding to deny. Removals and deny-list edits require an explicit user request outside this skill.

7. If the approval itself was also a systemic failure (e.g. the skill should have pre-allowed a read-only command the user has approved many times), the PERMISSION fix already fired in Step 3 — note already handled in Step 3 here and move on.

Step 5: Sync Project Docs

This step runs on every audit, including clean sessions where Steps 2–3 were skipped by the GATE. Walk all three living docs in this exact order. For each one: if the file does not exist in the project root, skip it silently (do not create it — the global rule is "no file = no action"). If the file exists but the session made zero relevant changes, state DOC: <file> — no changes needed and move on. That's a legitimate outcome, not a failure of the phase.

TASKS.md — the project's source of truth for what's done and what remains.

• Mark completed tasks as [x]
• Add new tasks discovered during the session (bugs found, deferred work, follow-ups)
• Remove or update tasks that are no longer accurate
• If the session introduced a new feature area, add a section header for it

CLAUDE.md (project root, NOT the global one) — the project's ruleset and agent brief.

• Add new project-specific rules that emerged from the session (conventions adopted, gotchas confirmed, build/test commands learned, native-platform workarounds)
• Update stale instructions the session proved wrong (e.g., a documented file path that moved, a command that no longer works)
• Delete rules that no longer apply
• Do NOT duplicate failure-driven rule edits already made in Step 3. If a rule was added in Step 3, just note CLAUDE.md already updated in Step 3 and move on.

ARCHITECTURE.md — the structural manifest of the project.

• Reflect structural changes the session introduced: new files/widgets/services, deleted components, renamed modules, new data flows, new dependencies
• Update "the X screen contains:" inventories and line-count estimates when sections were materially edited
• Remove sections describing code that was deleted in the session
• Do NOT rewrite the whole manifest — only touch sections the session actually changed

Step 6: Summary

After Steps 1–5 are complete, print a one-paragraph summary:

• How many systemic failures found
• What was fixed (list the files edited in Step 3)
• What hooks were proposed (if any)
• Permission sweep result — how many approvals were found, how many were added to ~/.claude/settings.json allow, and how many were deliberately skipped (dangerous). Must appear on every audit so the sweep's work is visible.
• Doc-sync results — which of TASKS.md / CLAUDE.md / ARCHITECTURE.md were edited, which were up-to-date (no changes needed), and which were absent from the project. All three must be accounted for on every audit so the doc-sync work is visible.
• What the user needs to do (approve hooks, verify doc changes, etc.)

No scores. No labels. No elaborate formatting. Just: here's what I fixed, which permissions I loosened, and which docs I synced.

Handling User Frustration

If the user is visibly frustrated (swearing, "this is useless", "what are you doing"), the audit should focus on:

1. What specific action caused the frustration
2. What system change prevents it
3. Apply the fix immediately

Do not apologize. Do not explain. Fix the system.

Error Handling

1. No conversation context: "Session context unavailable — run the audit before context compresses."
2. All failures are situational: "No systemic failures found. Nothing to fix."
3. Can't determine root cause: Flag it honestly — "Root cause unclear for [X]. Skipping rather than guessing."

Examples

Example 1: Worktree changes lost

Scan finds: Agent produced changes in worktree, changes weren't verified in main repo, worktrees were cleaned up, output lost.

Fix applied:

HOOK: Proposing worktree-verify — blocks worktree removal until changes exist in main repo
[writes script, shows settings.json entry, asks to apply]

Example 2: Wrong branch, wasted work

Scan finds: Started coding on wrong branch because session snapshot was stale.

Fix applied:

RULE: Adding to ~/.claude/CLAUDE.md — "Run git branch --show-current before any code
modification task. Do not trust session-start snapshot."

Example 3: Confusing doc misled agent

Scan finds: Agent read PLAN.md which contradicted IDENTITY.md, used the wrong one.

Fix applied:

DOC: Deleting PLAN.md — content is 90% duplicated in IDENTITY.md, contradiction on privacy
stance caused agent to use wrong guidance. Unique content merged into IDENTITY.md first.

Example 4: Clean session

Scan finds: No systemic failures — session executed cleanly against existing rules.

Output:

No systemic failures found. Nothing to fix. Running permission sweep and doc sync.

PERMISSION: no approvals this session
DOC: TASKS.md — marked "Add login form" and "Remove deprecated endpoint" as [x]
DOC: CLAUDE.md — no changes needed
DOC: ARCHITECTURE.md — added new AuthService module under section 3 (Backend)

Example 5: Permission sweep promotes safe approvals

Scan finds: User approved gh pr view 123, gh pr checks 456, docker logs app-1, and rm -rf build/ during the session. None of the gh pr view, gh pr checks, or docker logs patterns are in allow, ask, or deny. rm is already in ask — deliberate.

Output:

PERMISSION: Adding Bash(gh pr view*) to ~/.claude/settings.json allow — read-only PR inspection
PERMISSION: Adding Bash(gh pr checks*) to ~/.claude/settings.json allow — read-only CI status
PERMISSION: Adding Bash(docker logs*) to ~/.claude/settings.json allow — read-only container logs
PERMISSION: Skipping Bash(rm -rf build/) — destructive, belongs in ask