Doc Drift

doc-drift — Claude memory audit

Purpose

Scan the memory and documents Claude loads for this project, and surface what's stale, what contradicts something else, what's risky or ambiguous, and what leaks session-only context into skills/agents/commands docs — sorted by priority.

Memory verification: the CLAUDE.md / MEMORY.md / rules files this skill reads are past-point-in-time snapshots. Verify current state with Glob/Read before acting — never assume "it's in memory, so it must still be true" (per memory-discipline principles).

Dominant Variable

False-positive rate — if a finding turns out to be wrong, the person stops trusting the report and the tool itself gets abandoned. One false positive outweighs one missed real issue.

Trigger

• /doc-drift
• "문서 정합성"
• "docs 체크"

Discard If

• The project has neither CLAUDE.md nor MEMORY.md (nothing to audit) — hard discard, no override (there is nothing to read).
• Default: skip an immediate re-audit within 24h of the last one (drift needs time to accumulate — an immediate re-audit mostly reproduces the same result). Override: this is a recommendation, not a block — if the user just changed something and wants to confirm the fix landed (regression check), or invokes /doc-drift --force-after-change, run anyway and note Re-audit before 24h elapsed: forced in the report header.
• Default: skip projects with fewer than 10 files (audit cost exceeds the benefit for a large low-risk repo). Override: a small repo that is high-risk (handles secrets, is a shared/published skill, or the user explicitly asks for it despite the size) is worth auditing regardless — run it and note Small-repo override applied (N files) in the report header.

Key Assumptions

This skill runs on the following assumptions (per reasoning-standard principles):

1. Filesystem access is available — Read/Glob/Grep must work. If broken: fail-fast (report the missing tool).
2. The CLAUDE.md @import token convention is honored — @path syntax resolves per the standard spec. If broken: graceful degrade (warn + flag any import it couldn't resolve).
3. .drift-reports/ is writable — create it if missing. If broken: fall back to /tmp or stdout.

Workflow

Step 0 — Deterministic pre-filter

Before the LLM read, run two deterministic scripts over the same starting-point files Step 1 targets (CLAUDE.md, MEMORY.md, skills/agents/commands) — a scripted pass that doesn't need Step 1's full @import expansion to run first. Sorting "what can be counted mechanically" from "what needs a judgment call" keeps the model's read in Step 2 focused and cheap.

SLOP_SCRIPT=$(find ~/.claude -name "slop_detector.py" -path "*/doc-drift/scripts/*" -type f 2>/dev/null | head -1)
LINT_SCRIPT=$(find ~/.claude -name "claude_md_lint.py" -path "*/doc-drift/scripts/*" -type f 2>/dev/null | head -1)
PYBIN=$(command -v python3 || command -v python)
[ -n "$SLOP_SCRIPT" ] && "$PYBIN" "$SLOP_SCRIPT" <path> --json
[ -n "$LINT_SCRIPT" ] && "$PYBIN" "$LINT_SCRIPT" <path> --json   # CLAUDE.md-style files only

1. slop_detector.py — Ambiguous-wording / AI-tell density scan: a fixed additive-weight pattern list (hedging phrases, hype vocabulary, empty intensifiers, antithesis constructions, stray em-dashes) scored per file, returning verdict: HIGH/MEDIUM/LOW and the hit list. Files at HIGH become pre-flagged Risky/Ambiguous candidates.
2. claude_md_lint.py — CLAUDE.md-as-machine-prompt linter: CLAUDE.md is a machine prompt, not documentation prose. Rewards concrete anchors (code fences, file paths, shell commands, imperative-mood lines) and penalizes vague directives ("use your judgment", "as appropriate", "depending on the situation") and length over a 150-line budget, returning verdict: WEAK/OK/STRONG. A WEAK verdict is itself a Risky/Ambiguous signal — a paragraph with no actionable instruction a model could act on.

If neither script is found on this install, skip Step 0 and proceed straight to Step 1 — it's an optional accelerant, not a hard dependency (see Error Recovery).

Both scripts produce supporting evidence only for the Risky/Ambiguous row in Step 2 — a HIGH/WEAK verdict raises suspicion, it doesn't decide inclusion. The confidence ≥80% gate stays the actual gate (see Invariants); Step 0 output without a corroborating LLM read is not a finding on its own.

Step 1 — Gather what gets loaded

Collect every file Claude Code actually loads or can reference in this project's context. No scripts — use Read/Glob/Grep directly.

Starting points

• ~/.claude/CLAUDE.md (if present)
• <cwd>/CLAUDE.md + nested **/CLAUDE.md
• ~/.claude/projects/<encoded-cwd>/memory/MEMORY.md
  • Encoding: replace / in the cwd's absolute path with -. Example: /foo/bar → -foo-bar
  • Windows: also replace the drive colon and \ with -. Example: C:\project → C--project (one colon + one backslash = two hyphens). If unsure, Glob ~/.claude/projects/* to check the actual directory name.
• <cwd>/.claude/skills/**/SKILL.md
• <cwd>/.claude/agents/*.md
• <cwd>/.claude/commands/*.md
• ~/.claude/plugins/** (skills/agents/commands of installed plugins)

Expansion From each file, extract and recursively follow:

• @import tokens (resolve user-global ones relative to ~/.claude/, others relative to the file's directory or the project root)
• Relative markdown links [text](./path.md)
• File paths mentioned inside backticks (only if they actually exist)

Keep collecting until no new nodes appear. Verify current state — re-check with Glob that existing referenced paths still exist (per memory-discipline principles).

Step 2 — Detect four things

Read every audited file and look for only these four things:

Every finding needs evidence: where the claim was made (file:line) and the counter-evidence (file:line or a quote of the current code). No resolvable anchor → label it asserted_without_anchor and cut it (see Invariants).

When the quoted claim comes from personal/global memory (~/.claude/CLAUDE.md, ~/.claude/projects/*/memory/MEMORY.md), redact personal identifiers (usernames, home-directory paths, unrelated project/company names, private architecture detail) before the quote goes into the report — see Invariants #4.

Derivability signal: if a CLAUDE.md/rules line hardcodes a fact that could be mechanically reconstructed from the code (directory layout, dependency list, build command, etc.), that's a structural Outdated risk even when the current value happens to be correct — the two will drift independently over time. Tag such findings [derivable] as supporting evidence for priority. Not a new category — it's a sub-signal of Outdated, the four-kind taxonomy above is unchanged.

No resolvable anchor → drop it (asserted_without_anchor). An anchor exists but confidence is below 80% → keep it, labeled UNCERTAIN, in its own report section instead of dropping it — false positives are this tool's biggest enemy, but an evidence-backed lead you're not fully sure about is not a false positive, it's an unconfirmed one.

Step 3 — Prioritize and propose fixes

Sort the report by:

1. Blast radius — files loaded into every conversation (CLAUDE.md / MEMORY.md) rank highest
2. Severity — HIGH (following the wrong instruction breaks something), MED (confusing but easy to recover from), LOW (minor inconsistency)
3. Fix difficulty — clearest fixes first

Include a proposed fix for every finding, specific enough that a human can judge it with a single OK/NO.

Scope Boundary

Rationalization Table

Error Recovery

On failure: Stop → Classify → Apply Recovery → Report & Resume.

Safety Layers

• L1 (Invariants): only saves the drift report. Never directly edits CLAUDE.md, rules, or MEMORY.md (diagnostic only, read-only).

Invariants (never violate)

1. Evidence required: every finding cites both sides — file:line for the claim and file:line for the counter-evidence. No finding without evidence. A candidate that can't produce a resolvable anchor is labeled asserted_without_anchor and cut before the report is written — the label makes the cut auditable instead of a silent drop. Violation → impossible to trace what the tool actually based its judgment on, so a human can't decide whether to fix it.

2. Confidence < 80% → label UNCERTAIN, don't drop it (if it has an anchor): a finding that clears Invariant 1 (resolvable file:line anchor on both sides) but sits below 80% confidence is not a false positive — it's an unconfirmed lead. Put it in a separate "Uncertain" report section instead of discarding it, so evidence-backed suspicion isn't silently lost. Only findings that fail Invariant 1 (no resolvable anchor → asserted_without_anchor) are cut outright. Violation → dropping every sub-80% item throws away exactly the finding class this skill exists to surface (a real issue the model isn't fully sure about), trading false negatives to make the false-positive count look better on paper.

3. Auto-fix requires all 3 conditions AND: Outdated + a clear fix + explicit user approval. Auto-fixing Conflict/Risky items is never allowed. No file edits without user approval. Violation → a bad auto-fix can make the drift worse or break the document system.

4. Redact personal/global memory before quoting it: ~/.claude/CLAUDE.md and ~/.claude/projects/<encoded-cwd>/memory/MEMORY.md are the user's personal, cross-project files — they can hold personal file paths, real names, unrelated project/company names, or private architecture notes unrelated to this project's drift. Before a line from either file is quoted into .drift-reports/, strip personal identifiers and generalize unpublished architecture detail — keep only what's needed to demonstrate the drift. Violation → a report meant to be read or committed inside this project's repo leaks the user's personal information into it.

Output

Saved to .drift-reports/ (create if missing, never add it to .gitignore — its history should be visible in PRs):

• .drift-reports/<YYYY-MM-DD-HHMM>.md — timestamped report
• .drift-reports/latest.md — a copy of the latest one

Report template:

# Memory Audit — {timestamp}

**Scanned:** {n} files reachable from CLAUDE.md / MEMORY.md / skills / agents
**Findings:** HIGH {h} / MED {m} / LOW {l} / UNCERTAIN {u}

## Top priority
1. **[HIGH] `path:line`** — {one-line summary}
   - Claim: "..."
   - Reality: `other/path:line` — ...
   - Proposed fix: ...
2. ...

## Medium
...

## Low
...

## Uncertain (confidence < 80%, evidence-backed — not dropped)
- **`path:line`** — {one-line summary}
  - Claim: "..."
  - Anchor: `other/path:line` — ...
  - Why unconfirmed: {reason confidence is below 80%}

## Needs a human decision
- Conflicts where it's unclear which side is correct
- Ambiguous items where the intent is unclear

## Next step
Want an auto-fix PR? (Outdated items with a clear fix only)

Auto-fix (optional)

Only ask after printing the report summary:

"Found {h} HIGH items. What would you like to do?

1. Generate a PR for the ones with a clear fix
2. Report only"

If chosen, commit one atomic commit per finding on a docs/drift-fix-<timestamp> branch, then gh pr create.

Always excluded: Conflict (needs a human to say which side is right), Risky/Ambiguous (needs intent confirmation).

Arguments

Principles

• Minimize false positives — no anchor, drop it; an anchor but low confidence, label it UNCERTAIN rather than presenting it as confirmed. The report only survives if people trust it.
• Evidence required — every finding cites both sides (file:line).
• Redact before quoting personal/global memory — CLAUDE.md/MEMORY.md quotes in the report get personal identifiers stripped first.
• Respect the summary-and-link pattern — it's normal for CLAUDE.md to summarize/link to other documents. Only flag it when the meaning has actually drifted.

Truthful Reporting

When reporting completion, this skill:

1. no mock deception: confirm actual execution results before reporting. No completion claims based on guesswork.
2. no test façade: never hide failures behind skip/xfail. Mark skips as ⚠️ SKIPPED: reason.
3. no silent brokenness: final state must be labeled WORKING / PARTIAL / BROKEN. PARTIAL/BROKEN must list the concrete defects.

• Prioritize files loaded every conversation — drift in CLAUDE.md / MEMORY.md is more dangerous than drift in any other file.