Memory Factcheck

Memory Fact-Check

Origin: the skill of the same name in leeyudok/doksam-skills. Deliberately forked for template bundling — not auto-synced; good improvements are cherry-picked manually.

Memory decays. A fact that was true when written becomes false as code, schema and infrastructure move on. Stale memory is worse than no memory — an agent reads it and confidently does the wrong thing.

This is not a structural hygiene pass. Orphan files, duplicate entries, index bloat, broken internal links — those checks compare memory files to each other. This skill compares each memory to the world it describes: the code, the database, the issue tracker, the filesystem. A memory can be perfectly well-formed, correctly indexed, recently touched — and completely false.

Automatic deletion is forbidden. Only a source-of-truth comparison can tell what is dead, and losing an incident lesson is expensive. Hence semi-automatic: correct freely, archive only with approval, never delete.

1. Locate and inventory

Find the memory set. Common locations, in order:

• a path declared by the project's agent instructions (AGENTS.md / CLAUDE.md — e.g. "memory SSOT is .claude/memory/"). A declared path wins over every default.
• .claude/memory/ in the repo (team-shared, committed)
• the host's per-project memory dir (e.g. ~/.claude/projects/<slug>/memory/)

Collect for each file: frontmatter (name/description/type), and last modification date (git log -1 --format=%cs -- <file> for committed memory, stat otherwise). Note the index file (MEMORY.md) if one exists — it is audited too, but it is an index, not a memory.

Personal files are out of scope — anything the project marks personal (user_*.md or equivalent) belongs to its owner. Leave untouched.

Reading many files: dumping 50 memories into context at once blows the tool output cap and wastes the budget. Concatenate to one scratch file with ########## <filename> headers, then page through it. Do not skim — a stale claim is usually one clause inside an otherwise correct paragraph.

2. Extract load-bearing claims

Per file, pick the 1–3 claims that change what an agent would do. Ignore prose, rationale and background; a memory is only as stale as its actionable assertions.

Load-bearing looks like: "X lives at path P" · "table T has N rows" · "issue #N is still open" · "feature F does not exist yet" · "library L is not installed" · "the fix for this is still pending" · "run command C to verify".

3. Verify against the source of truth

Work cheapest-and-highest-yield first. In practice the ranking below holds: issue status is one API call and catches the largest share of stale claims, because memories are written mid-work and the work then finishes without anyone going back to edit the memory.

Parallelize independent verifications. If a source is unreachable, say so explicitly in the report — never silently downgrade "couldn't check" to "checked".

4. Classify

• fresh — every claim holds. Don't touch it.
• stale — some claim is outdated (moved path, changed number, closed issue, implemented gap). → Correct the body now, with the measured value and the date. Corrections are within autonomous scope; they are additive truth, not deletion.
• dead — the core premise is gone (library removed, feature retired, wholly superseded). → Mark as an archive candidate only.

5. Stale patterns worth hunting

Beyond "the number changed", these recur and are easy to miss:

• Fixed-gap drift — the memory documents a missing capability ("there is no startup reconcile", "no rate limiting yet") and it has since been built. This is the most dangerous class: the agent re-implements or re-reports work that already shipped. Check the linked issue and grep for the symbol.
• Cross-memory contradiction — two memories disagree (one says a script is the way to do X, another says that script was retired). At least one is stale by definition. Compare claims across files, not only file-by-file.
• Scale drift — "table T has ~8M rows" written months ago now off by 25%. Harmless as trivia, harmful when the memory derives advice from it (batch sizes, timeout budgets, "this query takes 19s").
• Recipe rot — the memory stores a command/query as a verified recipe, and the recipe no longer works at today's data volume or API version. Re-run stored recipes; a recipe you did not execute is unverified.
• Progress-state drift — a long-running job/backfill memory whose "current status" section is weeks behind, sometimes with two internally contradictory status sections stacked up. Date each section, keep the newest, mark superseded ones.
• Identity mismatch — name/description says one thing, the body says the opposite (e.g. a file named *-via-toolX whose body records that toolX was abandoned). Recall matches on description, so the file gets loaded for the wrong reason, or missed entirely.

6. Report, then apply

Report as a table before changing anything — file · class · one-line evidence · action:

Then:

1. Apply corrections to stale bodies — measured value + date, keeping the original observation where it still carries a lesson ("was 8M as of , 9.9M as of ").
2. Archive dead candidates only after explicit user approval: git mv into <memory>/archive/ and add archived: <date> <reason> to the frontmatter. Never rm.
3. Sync the index — reflect corrections, drop archived entries from MEMORY.md.
4. Commit through the project's normal workflow (issue → branch/worktree → PR/MR). Memory is team-shared content; it does not get direct-to-main commits.

Judgment rules — stay conservative

• Unverifiable ⇒ fresh. If the source of truth is unreachable, leave the memory alone and say the check was skipped. Unknown is not dead.
• Incident lessons stay fresh even when the code moves. A memory recording why something broke exists to prevent recurrence, not to snapshot a call site. Correct its stale path references; do not retire the lesson because the file was renamed.
• Report the drift, don't invent the cause. If a count inverted or a number moved inexplicably, record the measurement and flag it as unexplained. A plausible story written into memory becomes tomorrow's false fact.
• Prefer merge over create. Two memories on one topic → propose merging into the existing one.
• Non-obvious facts discovered by the audit itself become new memories — the audit is itself a source of ground truth.

Field notes

• Fresh timestamps prove nothing. A file committed this week can carry a claim that was already false when written; a file untouched for months can be perfectly true. Verify claims, never sort by date and trim the tail.
• count(*) on a large table may exceed the DB tool's statement timeout — that failure is a finding when the memory claims the query is fast.
• Quote globs when shelling out from zsh (grep --include="*.java"), or the shell eats them and the check silently returns nothing — a false "fresh".
• An issue being closed does not by itself prove the described work shipped. For fixed-gap claims, confirm with a grep for the symbol as well.