Auto Learn
Inputs
$run: a checkpoint path or run id to harvest; defaults to the most recent .ulpi/runs/*.json.
Overview
The difference between an autonomous machine and a self-improving one is whether run N+1 knows what
run N paid to find out. Every blocked task, guard block, thrash loop, and rejected finding is evidence
about how THIS repo and THIS process actually behave. This skill turns that evidence into verified,
deduplicated learnings and writes them into the memory Claude Code already loads every session —
CLAUDE.md, .claude/rules, auto memory — so the next run starts with the lessons in context
automatically, without any phase having to remember to read a file. That is what closes the loop.
Phase 0: Harvest the run's evidence — it is CODE
node <skill-dir>/scripts/harvest-run.mjs <checkpoint.json> --json
The harvester extracts candidates mechanically from the checkpoint: blocked/dep_blocked units with
their notes, gate failures from the register, delegation degradations, units that needed many fix
iterations, and openItems by phase. Supplement with what the checkpoint can't hold: guard-block
messages you observed, verifier rejection patterns (repeated false-positive shapes), and anything the
user corrected mid-run. Every candidate keeps its evidence pointer.
Success criteria: a candidate list where each entry cites its evidence; nothing from memory alone.
Phase 1: Verify and generalize (the anti-poison gate)
For each candidate, adversarial-verify with three questions — TRUE (does the evidence actually
support it — reread the artifact, don't trust the summary)? GENERAL (would it recur — or was it a
one-off flake/typo)? ACTIONABLE (what future decision changes)? Then rewrite survivors into the
canonical learning shape:
- <the rule> — WHY: <one line, from evidence> — APPLY: <what to do differently> (evidence: <ref>, <date>)
Success criteria: ≤5 survivors, each true, general, actionable, and evidence-linked.
Phase 2: Route each learning to native, auto-loaded memory
Claude Code loads these every session (or on matching-file access) with no prompting. Put each learning
in the one that will actually reach the next run:
| Learning is about… |
Route to (Claude Code auto-loads it) |
How |
| A project-wide convention, invariant, or plan-shape rule the run kept violating (tasks too fat here, a validate form that lies, a boundary that bit) |
CLAUDE.md — loaded in full every session |
append to a stamped ## Learnings section, or hand it to auto-map (which owns CLAUDE.md) |
| A lesson that only applies to one area (an API rule, a DB/migration gotcha, a test-suite quirk) |
.claude/rules/<area>.md with paths: frontmatter — loads when the agent touches those files |
create or append the rule; scope its paths: glob |
| An environment or tooling quirk the run discovered (build flake, slow suite, a service dependency, a CI oddity) |
auto memory (~/.claude/projects/<project>/memory/) — MEMORY.md loaded every session |
a topic file plus one MEMORY.md index line |
| A defect in the machine itself (a skill gap, a guard bypass, a template bug) |
the user — reported, never self-patched |
surface it in the Output Contract |
| A cross-agent lesson both Codex AND Claude Code must load (the shared native context file) |
the nearest AGENTS.md — read by Codex and Claude Code alike |
route it through scripts/route-learnings.mjs (below) — DRY-RUN first, then --apply |
The AGENTS.md router (DRY-RUN-FIRST, fail-closed) — Codex + Claude Code
AGENTS.md is the ONE native memory file both Codex and Claude Code read, so a lesson that must reach
either agent belongs there. Because that file is shared, auto-loaded, and cross-agent, writing to it by
hand is how a bad lesson silently poisons every future run. Route it through code instead:
# DEFAULT is a dry run: emits a JSON patch manifest, writes NOTHING.
node <skill-dir>/scripts/route-learnings.mjs learnings.json --root <project-root>
# Only after you've read the manifest, actually apply it:
node <skill-dir>/scripts/route-learnings.mjs learnings.json --root <project-root> --apply
Each learning in the input ({"learnings":[…]} or a bare array) MUST carry five fields — id (stable),
rule (actionable), evidence (a real artifact ref), verification (an affirmative marker), and
scope (a repo directory the lesson applies to). The router's fail-closed contract:
- Dry run is the default. No
--apply, no writes — you get a patch manifest (per-target before/after
SHAs, the survivors, and every rejection with its reason) to inspect first.
--apply edits ONLY the stamped, auto-learn-owned block of the target AGENTS.md
(<!-- BEGIN auto-learn:learnings … --> … <!-- END … -->). Every other byte of that file is
preserved; prior generated learnings are merged forward (dup ids merge evidence), never dropped.- It targets the nearest
AGENTS.md at or above the learning's scope, falling back to the root
AGENTS.md. It writes ONLY files named AGENTS.md inside the project root — it NEVER edits CLAUDE.md
or private Codex memory.
- ≤5 survivors per run; duplicate ids merge evidence. More than five distinct additions ⇒ the whole
run refuses to mutate anything (
refused: "too_many_additions").
- No mutation on any of: missing/blank required field, placeholder/absent evidence, a non-affirmative
(
unverified/…) verification, a secret-shaped value anywhere in the record, a scope that path-
traverses (../absolute/escapes root), a nonexistent scope directory, or a machine/environment
defect (kind:"machine_defect", or defect-shaped rule/evidence text). Defects are reported for the
user, never self-patched into shared memory.
Use this router for AGENTS.md; use auto-map for CLAUDE.md/.claude/rules and auto-memory for the
Claude-only layers. Dedupe on write: if an existing entry covers it, strengthen that entry (add the new evidence ref)
instead of appending a twin; prune superseded/disproved entries while you are there. Keep CLAUDE.md and
each rules file within budget — they load into every session, so bloat taxes all future work.
Success criteria: every survivor landed in exactly one native-loaded location; nothing written to a
file Claude Code does not load; memory deduped and within budget.
Phase 3: Close the loop — the feed-forward is now AUTOMATIC
Because the learnings live in native, auto-loaded memory, the next run picks them up with no extra step —
with one nuance to state honestly: CLAUDE.md and auto-memory MEMORY.md load in full EVERY session, so
lessons routed there close the loop unconditionally; a .claude/rules/<area>.md lesson loads only when a
future session touches files matching its paths: glob (path-scoped by design — that is the point of an
area-specific rule). So route a lesson the next run must ALWAYS see to CLAUDE.md / auto-memory, and an
area-specific one to .claude/rules. Confirm and report that each learning actually landed in a loaded
location (/memory lists what is loaded; check the CLAUDE.md ## Learnings section and any new
.claude/rules file). In pipeline composition this skill runs BEFORE auto-map, which then verifies the
CLAUDE.md/rules the learnings touched against the real repo.
Success criteria: every lesson is in a location Claude Code confirms it loads; none stranded in a
file nothing reads.
Common Rationalizations
| Rationalization |
Reality |
| "Obvious lesson, no need to verify." |
Obvious-feeling lessons from one bad run are how false laws get written. Verify against the evidence or drop it. |
| "Keep every learning — more is better." |
CLAUDE.md loads into every session. Bloat taxes every run; 5 curated beats 50 hoarded. |
"Drop it in a .ulpi/learnings.md and have the skills read it." |
Claude Code doesn't load that file. It's a lesson written into the void. Use CLAUDE.md, .claude/rules, or auto memory — what actually loads. |
| "Just append; dedup later." |
Later never comes. Twins drift apart and contradict; merge on write. |
| "That failure was the skill's fault — tweak my own instructions quietly." |
Machine defects go to the user. Self-patching hides systemic bugs from the person who owns the system. |
| "Record it in the transcript summary." |
Transcripts die with the session. Only routed, durable layers change the next run. |
| "One flake = a rule." |
One occurrence is an anecdote. Two with the same evidence shape is a pattern. Generalize only what recurs or is structurally certain. |
Red Flags
- A learning with no evidence pointer, or one contradicting the artifact it cites.
- A learning written to any file Claude Code does not auto-load (a
.ulpi/ note, a random doc): it will
never reach the next run.
- CLAUDE.md or a rules file growing past its budget or accumulating near-duplicate entries.
- A machine defect recorded as a repo note instead of surfaced to the user.
- The same mistake appearing in two consecutive runs' registers — the loop is NOT closing; escalate.
Guardrails
- Never write an unverified or evidence-free learning; never learn from memory alone.
- Never write a learning to a file Claude Code does not load (CLAUDE.md /
.claude/rules / auto memory
only); never invent a .ulpi/learnings.md.
- Never exceed 5 new learnings per run; keep CLAUDE.md and rules files within budget.
- Never duplicate — merge and strengthen; prune superseded entries on every pass.
- Never store secrets; never touch human-written memory outside stamped sections.
- Never self-patch machine defects silently — they belong to the user.
When To Load References
scripts/harvest-run.mjs — mechanical candidate extraction from a checkpoint (Phase 0). CI-tested by
scripts/test-harvest.sh (every signal class extracted with evidence citations; unreadable → exit 2).scripts/route-learnings.mjs — the DRY-RUN-FIRST, fail-closed router that writes verified learnings to
the shared cross-agent AGENTS.md (Phase 2, Codex + Claude Code). CI-tested by
scripts/test-learn-route.sh (dry-run default, --apply byte-preservation, the ≤5 cap + dup-merge,
and every no-mutation refusal). Use it for AGENTS.md; use auto-map for CLAUDE.md/.claude/rules.adversarial-verify (skill) — the TRUE/GENERAL/ACTIONABLE gate (Phase 1).auto-map (skill) — the routing target for repo-fact learnings (Phase 2).checkpoint-resume (skill) — the artifact format this skill harvests.
Output Contract
Report:
- run harvested (checkpoint path) + candidates found by the harvester vs. added manually
- verification: survivors vs. rejected (with the rejection reason)
- routing: which learning landed where (CLAUDE.md /
.claude/rules/<area>.md / auto memory / user-report)
- machine defects surfaced to the user (never self-patched)
- confirmation each lesson is in a location Claude Code auto-loads (entries merged/pruned, within budget)
1---2name: auto-learn3description: <EXTREMELY-IMPORTANT>4---56<EXTREMELY-IMPORTANT>7A wrong or bloated learning poisons every future run — this skill's failure mode is worse than8learning nothing. Non-negotiable:91. LEARN FROM ARTIFACTS, NOT VIBES. Candidates come from the run's structured evidence — the10 checkpoint's register/openItems, blocked units with their notes, guard-block reasons, fix-loop11 iteration counts, delegation degradations, verifier rejection ledgers — each candidate carries its12 evidence reference. No evidence, no learning.132. VERIFY BEFORE WRITING. Every candidate passes `adversarial-verify`: is it TRUE (matches the14 evidence), GENERAL (a pattern, not a one-off flake), and ACTIONABLE (changes a future decision)?15 One flaky run does not make a law. Rejected candidates are dropped, not hoarded.163. WRITE TO WHERE CLAUDE CODE ACTUALLY LOADS CONTEXT — never a bespoke file nothing reads. A learning17 only changes behavior if it lands in native, auto-loaded memory: **CLAUDE.md** (loaded in full every18 session), a path-scoped **`.claude/rules/<area>.md`** (loaded when the agent touches that area), or19 **auto memory** (`~/.claude/projects/<project>/memory/`, loaded every session). Machine defects (a20 skill/guard/template gap) → surfaced to the user, never self-suppressed. NEVER invent a `.ulpi/`21 learnings file — nothing loads it, so it is a lesson written into the void.224. DEDUPE AND CURATE — memory is a working set, not an archive: merge with an existing entry instead of23 appending a twin, prune entries a later run superseded or disproved, convert relative dates to24 absolute, and keep CLAUDE.md within its line budget. Cap: at most 5 new learnings per run.255. NEVER learn secrets/tokens/URLs-with-credentials, and never rewrite human-written memory — the26 same stamps-and-preservation rules as auto-map.27</EXTREMELY-IMPORTANT>2829# Auto Learn3031## Inputs3233- `$run`: a checkpoint path or run id to harvest; defaults to the most recent `.ulpi/runs/*.json`.3435## Overview3637The difference between an autonomous machine and a self-improving one is whether run N+1 knows what38run N paid to find out. Every blocked task, guard block, thrash loop, and rejected finding is evidence39about how THIS repo and THIS process actually behave. This skill turns that evidence into verified,40deduplicated learnings and writes them into the memory Claude Code already loads every session —41CLAUDE.md, `.claude/rules`, auto memory — so the next run starts with the lessons in context42automatically, without any phase having to remember to read a file. That is what closes the loop.4344## Phase 0: Harvest the run's evidence — it is CODE4546```bash47node <skill-dir>/scripts/harvest-run.mjs <checkpoint.json> --json48```4950The harvester extracts candidates mechanically from the checkpoint: blocked/dep_blocked units with51their notes, gate failures from the register, delegation degradations, units that needed many fix52iterations, and openItems by phase. Supplement with what the checkpoint can't hold: guard-block53messages you observed, verifier rejection patterns (repeated false-positive shapes), and anything the54user corrected mid-run. Every candidate keeps its evidence pointer.5556**Success criteria:** a candidate list where each entry cites its evidence; nothing from memory alone.5758## Phase 1: Verify and generalize (the anti-poison gate)5960For each candidate, `adversarial-verify` with three questions — TRUE (does the evidence actually61support it — reread the artifact, don't trust the summary)? GENERAL (would it recur — or was it a62one-off flake/typo)? ACTIONABLE (what future decision changes)? Then rewrite survivors into the63canonical learning shape:6465```66- <the rule> — WHY: <one line, from evidence> — APPLY: <what to do differently> (evidence: <ref>, <date>)67```6869**Success criteria:** ≤5 survivors, each true, general, actionable, and evidence-linked.7071## Phase 2: Route each learning to native, auto-loaded memory7273Claude Code loads these every session (or on matching-file access) with no prompting. Put each learning74in the one that will actually reach the next run:7576| Learning is about… | Route to (Claude Code auto-loads it) | How |77|---|---|---|78| A project-wide convention, invariant, or plan-shape rule the run kept violating (tasks too fat here, a validate form that lies, a boundary that bit) | **CLAUDE.md** — loaded in full every session | append to a stamped `## Learnings` section, or hand it to `auto-map` (which owns CLAUDE.md) |79| A lesson that only applies to one area (an API rule, a DB/migration gotcha, a test-suite quirk) | **`.claude/rules/<area>.md`** with `paths:` frontmatter — loads when the agent touches those files | create or append the rule; scope its `paths:` glob |80| An environment or tooling quirk the run discovered (build flake, slow suite, a service dependency, a CI oddity) | **auto memory** (`~/.claude/projects/<project>/memory/`) — MEMORY.md loaded every session | a topic file plus one MEMORY.md index line |81| A defect in the machine itself (a skill gap, a guard bypass, a template bug) | **the user** — reported, never self-patched | surface it in the Output Contract |82| A cross-agent lesson both Codex AND Claude Code must load (the shared native context file) | **the nearest `AGENTS.md`** — read by Codex and Claude Code alike | route it through `scripts/route-learnings.mjs` (below) — DRY-RUN first, then `--apply` |8384### The AGENTS.md router (DRY-RUN-FIRST, fail-closed) — Codex + Claude Code8586`AGENTS.md` is the ONE native memory file both Codex and Claude Code read, so a lesson that must reach87either agent belongs there. Because that file is shared, auto-loaded, and cross-agent, writing to it by88hand is how a bad lesson silently poisons every future run. Route it through code instead:8990```bash91# DEFAULT is a dry run: emits a JSON patch manifest, writes NOTHING.92node <skill-dir>/scripts/route-learnings.mjs learnings.json --root <project-root>93# Only after you've read the manifest, actually apply it:94node <skill-dir>/scripts/route-learnings.mjs learnings.json --root <project-root> --apply95```9697Each learning in the input (`{"learnings":[…]}` or a bare array) MUST carry five fields — `id` (stable),98`rule` (actionable), `evidence` (a real artifact ref), `verification` (an affirmative marker), and99`scope` (a repo directory the lesson applies to). The router's fail-closed contract:100101- **Dry run is the default.** No `--apply`, no writes — you get a patch manifest (per-target before/after102 SHAs, the survivors, and every rejection with its reason) to inspect first.103- **`--apply` edits ONLY the stamped, auto-learn-owned block** of the target `AGENTS.md`104 (`<!-- BEGIN auto-learn:learnings … -->` … `<!-- END … -->`). Every other byte of that file is105 preserved; prior generated learnings are merged forward (dup ids merge evidence), never dropped.106- **It targets the nearest `AGENTS.md`** at or above the learning's `scope`, falling back to the root107 `AGENTS.md`. It writes ONLY files named `AGENTS.md` inside the project root — it NEVER edits `CLAUDE.md`108 or private Codex memory.109- **≤5 survivors per run; duplicate ids merge evidence.** More than five distinct additions ⇒ the whole110 run refuses to mutate anything (`refused: "too_many_additions"`).111- **No mutation on any of:** missing/blank required field, placeholder/absent evidence, a non-affirmative112 (`unverified`/…) verification, a secret-shaped value anywhere in the record, a `scope` that path-113 traverses (`..`/absolute/escapes root), a nonexistent `scope` directory, or a **machine/environment114 defect** (`kind:"machine_defect"`, or defect-shaped rule/evidence text). Defects are reported for the115 user, never self-patched into shared memory.116117Use this router for `AGENTS.md`; use `auto-map` for `CLAUDE.md`/`.claude/rules` and auto-memory for the118Claude-only layers. Dedupe on write: if an existing entry covers it, strengthen that entry (add the new evidence ref)119instead of appending a twin; prune superseded/disproved entries while you are there. Keep CLAUDE.md and120each rules file within budget — they load into every session, so bloat taxes all future work.121122**Success criteria:** every survivor landed in exactly one native-loaded location; nothing written to a123file Claude Code does not load; memory deduped and within budget.124125## Phase 3: Close the loop — the feed-forward is now AUTOMATIC126127Because the learnings live in native, auto-loaded memory, the next run picks them up with no extra step —128with one nuance to state honestly: **CLAUDE.md and auto-memory `MEMORY.md` load in full EVERY session**, so129lessons routed there close the loop unconditionally; a **`.claude/rules/<area>.md` lesson loads only when a130future session touches files matching its `paths:` glob** (path-scoped by design — that is the point of an131area-specific rule). So route a lesson the next run must ALWAYS see to CLAUDE.md / auto-memory, and an132area-specific one to `.claude/rules`. Confirm and report that each learning actually landed in a loaded133location (`/memory` lists what is loaded; check the CLAUDE.md `## Learnings` section and any new134`.claude/rules` file). In pipeline composition this skill runs BEFORE `auto-map`, which then verifies the135CLAUDE.md/rules the learnings touched against the real repo.136137**Success criteria:** every lesson is in a location Claude Code confirms it loads; none stranded in a138file nothing reads.139140## Common Rationalizations141142| Rationalization | Reality |143|---|---|144| "Obvious lesson, no need to verify." | Obvious-feeling lessons from one bad run are how false laws get written. Verify against the evidence or drop it. |145| "Keep every learning — more is better." | CLAUDE.md loads into every session. Bloat taxes every run; 5 curated beats 50 hoarded. |146| "Drop it in a `.ulpi/learnings.md` and have the skills read it." | Claude Code doesn't load that file. It's a lesson written into the void. Use CLAUDE.md, `.claude/rules`, or auto memory — what actually loads. |147| "Just append; dedup later." | Later never comes. Twins drift apart and contradict; merge on write. |148| "That failure was the skill's fault — tweak my own instructions quietly." | Machine defects go to the user. Self-patching hides systemic bugs from the person who owns the system. |149| "Record it in the transcript summary." | Transcripts die with the session. Only routed, durable layers change the next run. |150| "One flake = a rule." | One occurrence is an anecdote. Two with the same evidence shape is a pattern. Generalize only what recurs or is structurally certain. |151152## Red Flags153154- A learning with no evidence pointer, or one contradicting the artifact it cites.155- A learning written to any file Claude Code does not auto-load (a `.ulpi/` note, a random doc): it will156 never reach the next run.157- CLAUDE.md or a rules file growing past its budget or accumulating near-duplicate entries.158- A machine defect recorded as a repo note instead of surfaced to the user.159- The same mistake appearing in two consecutive runs' registers — the loop is NOT closing; escalate.160161## Guardrails162163- Never write an unverified or evidence-free learning; never learn from memory alone.164- Never write a learning to a file Claude Code does not load (CLAUDE.md / `.claude/rules` / auto memory165 only); never invent a `.ulpi/learnings.md`.166- Never exceed 5 new learnings per run; keep CLAUDE.md and rules files within budget.167- Never duplicate — merge and strengthen; prune superseded entries on every pass.168- Never store secrets; never touch human-written memory outside stamped sections.169- Never self-patch machine defects silently — they belong to the user.170171## When To Load References172173- `scripts/harvest-run.mjs` — mechanical candidate extraction from a checkpoint (Phase 0). CI-tested by174 `scripts/test-harvest.sh` (every signal class extracted with evidence citations; unreadable → exit 2).175- `scripts/route-learnings.mjs` — the DRY-RUN-FIRST, fail-closed router that writes verified learnings to176 the shared cross-agent `AGENTS.md` (Phase 2, Codex + Claude Code). CI-tested by177 `scripts/test-learn-route.sh` (dry-run default, `--apply` byte-preservation, the ≤5 cap + dup-merge,178 and every no-mutation refusal). Use it for `AGENTS.md`; use `auto-map` for `CLAUDE.md`/`.claude/rules`.179- `adversarial-verify` (skill) — the TRUE/GENERAL/ACTIONABLE gate (Phase 1).180- `auto-map` (skill) — the routing target for repo-fact learnings (Phase 2).181- `checkpoint-resume` (skill) — the artifact format this skill harvests.182183## Output Contract184185Report:1861871. run harvested (checkpoint path) + candidates found by the harvester vs. added manually1882. verification: survivors vs. rejected (with the rejection reason)1893. routing: which learning landed where (CLAUDE.md / `.claude/rules/<area>.md` / auto memory / user-report)1904. machine defects surfaced to the user (never self-patched)1915. confirmation each lesson is in a location Claude Code auto-loads (entries merged/pruned, within budget)
ulpi-io