Improve Skill Learnings
Apply captured learnings from unified-learnings.jsonl to their target files.
Headless mode (--headless <target>)
Invoked as /improve-skill-learnings --headless <target>. This is a mode of this
skill, not a separate skill: the Step 4 validator prompt and the Step 6 drafter
prompt are read from this file by the driver, never copied. Without --headless,
ignore this section and follow Steps 1 to 9 unchanged. Not used by the seed bank
pickup flow.
Fixed answers, no questions asked:
- The target is the argument. No Step 2 picker, no run-all prompt.
- Every root-cause cluster of the target runs (clusterer, then
cluster_flags.py). - The per-diff
Apply?prompt is auto-yes: the PR is the review. - Contract drift (the Step 6 type-drift check firing) marks the cluster
needs-grilland makes no commit for it. Clusters flagged bycluster_flags.pyareneeds-grilltoo.
Run the driver and stop; do not run Steps 1 to 9:
python3 ~/.dotfiles/claude-code-shared/scripts/learnings-lane/headless_run.py \
--target <target> --run-file <run-file.json>
Rules the driver enforces:
- It never calls
update-learning.pyand never touchesunified-learnings.jsonl. Stale, invalid, retarget, and needs-grill verdicts go to the run file, whose shape isscripts/learnings-lane/run-file-contract.json(check it withvalidate_run_file.py). - One commit per root-cause cluster, with a trailer listing the member learning
ids that were applied:
Learning-Ids: id1,id2. - Models are reached through
$CLAUDE_BIN -p(Haiku for validation), so tests swap in a stub binary.
Step 1: Load and filter learnings
Run the following to load all status == "captured" entries. Entries with no
status field are treated as captured (legacy entries predate the field).
python3 - <<'PYEOF'
import json, os
from collections import defaultdict
from datetime import datetime, timezone
path = os.path.expanduser(
"~/.dotfiles/claude-code-shared/learnings/unified-learnings.jsonl"
)
entries = []
try:
with open(path) as f:
for line in f:
line = line.strip()
if line:
try:
e = json.loads(line)
if e.get("status", "captured") == "captured":
entries.append(e)
except json.JSONDecodeError:
pass
except FileNotFoundError:
pass
groups = defaultdict(list)
for e in entries:
key = e.get("improves") or "__unassigned__"
groups[key].append(e)
now = datetime.now(timezone.utc)
def age_str(items):
oldest = None
for e in items:
ts = e.get("timestamp")
if ts:
try:
dt = datetime.fromisoformat(ts.replace("Z", "+00:00"))
if oldest is None or dt < oldest:
oldest = dt
except (ValueError, TypeError):
pass
if oldest is None:
return "?"
days = (now - oldest).days
if days < 1:
return "<1d"
if days < 14:
return f"{days}d"
weeks = days // 7
if weeks < 9:
return f"{weeks}w"
return f"{days // 30}mo"
sorted_groups = sorted(groups.items(), key=lambda x: -len(x[1]))
print(f"TOTAL={len(entries)}")
for rank, (target, items) in enumerate(sorted_groups, 1):
display = target if target != "__unassigned__" else "(unassigned)"
itype = items[0].get("improves_type", "—")
ids = ",".join(e["id"] for e in items)
print(f"RANK={rank}|TARGET={target}|TYPE={itype}|COUNT={len(items)}|OLDEST={age_str(items)}|IDS={ids}|DISPLAY={display}")
PYEOF
Parse the output into a ranked table:
Rank | Target | Type | Learnings | Oldest
-----|---------------------|----------|-----------|-------
1 | debug | skill | 4 | 3mo
2 | to-seed | skill | 3 | 2w
...
N | (unassigned) | — | 2 | 5d
If TOTAL=0, print:
No captured learnings found. Nothing to do.
And stop.
Step 2: User selects target (or "run all")
Ask the user which target they want to improve. Accept a rank number or a slug name. Wait for the response.
Run-all option: When TOTAL ≤ 10, include a "Run all (N learnings)" option.
When selected, set run_all = true and skip to Run-all mode below — no
further target selection prompts appear.
If you use AskUserQuestion, pass at most 4 options (the tool maximum). When run-all is offered, it takes one slot (pass it first). Pass the top 3 targets in the remaining slots. Lower-ranked targets stay reachable via the built-in "Other" free-text option.
Collect all captured entries for the selected target (by their id values from
step 1).
Run-all mode
When run_all = true:
- Work through targets in rank order. Skip
__unassigned__targets unless that is the only one. - For each target, print a progress header:
--- Target <X>/<Y>: <slug> (<N> learnings) --- - Iterate through the target's learnings using the pick-one loop (Steps 5–7). Lazy per-learning validation (Step 4) still applies — each learning is validated when it comes up in the loop, not upfront.
- After all learnings for a target are exhausted (applied, skipped, or discarded), advance to the next target automatically. No prompt between targets. A learning retargeted mid-pass to a target outside the original rank list is deferred to the next run-all pass. Include it now only if the user explicitly asks.
- After all targets are done, go to Step 8 (summary).
Step 3: Fetch entry fields (no upfront validation)
Skip batch validation. Each learning is validated lazily in Step 4 when the user selects it. This keeps parallel agent output out of the session context.
Step 1 prints only IDs. Fetch the full fields for the target's captured
entries. Set TARGET to the selected slug (__unassigned__ for unassigned).
In run-all mode, rerun this for each target.
TARGET='<slug>' python3 - <<'PYEOF'
import json, os
target = os.environ["TARGET"]
path = os.path.expanduser(
"~/.dotfiles/claude-code-shared/learnings/unified-learnings.jsonl"
)
keys = ("id", "improves", "improves_type", "reported_by", "problem", "lesson", "fix")
try:
with open(path) as f:
for line in f:
line = line.strip()
if not line:
continue
try:
e = json.loads(line)
except json.JSONDecodeError:
continue
if e.get("status", "captured") != "captured":
continue
if (e.get("improves") or "__unassigned__") != target:
continue
print(json.dumps({k: e.get(k) for k in keys}))
except FileNotFoundError:
pass
PYEOF
Each output line is one entry. This output is the only source of entry fields.
Step 5 shows problem from it. Step 4 and Step 6 fill {id}, {improves},
{improves_type}, {problem}, {lesson} and {fix} verbatim from it. Never
paraphrase them. Never compose them from memory. If an entry is missing from
the output, rerun this step before dispatching any agent.
Then go to Step 5.
Step 4: Validate one learning (lazy — called per pick from Step 5)
Spawn ONE Haiku validation agent for the selected learning. Build path_candidates
based on improves_type:
| improves_type | Paths to check (in order) |
|---|---|
skill |
~/.dotfiles/claude-code-shared/skills/<improves>/SKILL.md |
agent |
First hit of find ~/.dotfiles/claude-code-shared/agents -name '<improves>.md', else ~/.dotfiles/claude-code-shared/agents/<improves>.md |
process |
~/.dotfiles/claude-code-shared/resources/<improves>.md, then ~/.dotfiles/claude-code-shared/resources/<improves>.json |
contract |
~/.dotfiles/claude-code-shared/contracts/<improves>.json, then ~/.dotfiles/claude-code-shared/contracts/<improves>.md |
After the declared type's paths, append the other three types' paths (same slug) as fallback candidates, in table order.
Resolve the agent row with the recursive find every time it is used. Agent
files can be nested (e.g. agents/seed-review/<improves>.md). The flat path alone
misses them.
Unassigned entries (improves is null): Use reported_by as the slug.
List all four types' paths for it, in table order. If reported_by is also
null, pass an empty list. These paths are a starting point only. The
validator greps for the real owner. Do not add custom rules to the prompt.
Pass this prompt to the Haiku agent (model: haiku). Fill the learning entry fields verbatim from the Step 3 fetch output:
You are a learning validation agent. Assess one learning entry and return a JSON verdict.
Learning entry:
id: {id}
improves: {improves}
improves_type: {improves_type}
problem: {problem}
lesson: {lesson}
fix: {fix}
Target file paths to check (in order):
{path_candidates}
Steps:
1. Check whether each path exists (use Bash: test -f <path> && echo exists).
If {improves} is null, skip the rest of this step. The paths come from the
reporter, not an owner. If none exist, go to step 5.
If none exist → return {"id": "{id}", "verdict": "invalid", "reason": "file_not_found", "file_path": null}
If the first existing path belongs to a different type than {improves_type} →
return {"id": "{id}", "verdict": "misrouted", "reason": "type_mismatch", "file_path": "<path>", "suggested_improves": "{improves}", "suggested_improves_type": "<type of that path>"}
2. Read the first file that exists (Read tool).
3. Does the file ALREADY contain the substance of the fix or lesson?
If yes → return {"id": "{id}", "verdict": "stale", "reason": "already_applied", "file_path": "<path>"}
Also check for partial coverage. The file may hold the core lesson but omit
one dimension (a trigger, scenario, or criterion). If so → return
{"id": "{id}", "verdict": "partial", "reason": "partial_coverage", "file_path": "<path>",
"existing_line": <line number of the existing coverage>, "missing": "<the omitted dimension>"}
4. Does this file itself perform the action the fix changes (e.g. edit code, spawn
the agent, run the command)? If it only delegates that action to another skill
or agent (e.g. an orchestrator skill that hands code edits to a runner agent) →
return {"id": "{id}", "verdict": "misrouted", "reason": "<delegate file path>",
"file_path": "<path>", "suggested_improves": "<slug>", "suggested_improves_type": "<skill|agent|process|contract>"}
5. Does the step or mechanism the fix changes live in a DIFFERENT file? (e.g. the
fix edits an evidence-pack step, but this file has none.) Grep
~/.dotfiles/claude-code-shared/{skills,agents,resources,contracts} for the
mechanism's distinctive terms. If exactly one other file clearly owns it →
return {"id": "{id}", "verdict": "misrouted", "reason": "<owner file path>",
"file_path": "<path>", "suggested_improves": "<slug>", "suggested_improves_type": "<skill|agent|process|contract>"}
Before returning misrouted in step 4 or 5, read the suggested owner's file and
confirm it performs the action itself. A name in the fix text is not proof of
ownership. If the owner only delegates, do not return misrouted; go to step 6.
6. Has the file changed so significantly that the lesson is no longer relevant?
If yes → return {"id": "{id}", "verdict": "stale", "reason": "no_longer_relevant", "file_path": "<path>"}
7. Does the fix contradict a shared managed resource the target is meant to
follow? Such resources include files the target pulls in by a managed marker
block (e.g. `<!-- learning-capture:start -->`) and files it references by
path under ~/.dotfiles/claude-code-shared/resources/. Grep each one for the
fix's distinctive terms and read the matching sections. A fix that changes
the resource itself is not a contradiction. Only a fix that makes the target
diverge from the resource counts. If it does →
return {"id": "{id}", "verdict": "invalid", "reason": "contradicts_shared_resource", "file_path": "<path>", "resource_path": "<resource path>"}
If the conflict is unclear, do not return invalid. Continue to step 8.
8. If {improves} is null and no file owns the mechanism →
return {"id": "{id}", "verdict": "no_applicable_target", "reason": "no_owner", "file_path": null}
9. Otherwise → return {"id": "{id}", "verdict": "valid", "reason": "ok", "file_path": "<path>"}
Return ONLY the JSON object. No prose.
Apply the verdict
invalidwith reasonfile_not_found: Confirm before marking. Runfind ~/.dotfiles/claude-code-shared -name '<improves>.*'. If it hits, use the first hit asfile_path. If its type (derived below) matchesimproves_type, treat the verdict asvalid. Otherwise treat it asmisroutedwith the derived slug and type. Only markinvalidif find returns nothing.staleorinvalid: Run update-learning.py to mark it. Print a one-line note. Remove from the picker list. Return to Step 5.python3 ~/.dotfiles/claude-code-shared/scripts/learning/update-learning.py \ --id <id> --status <stale|invalid>misrouted: Retarget via update-learning.py (status stayscaptured). Print a one-line note. Remove from the picker list. Return to Step 5.python3 ~/.dotfiles/claude-code-shared/scripts/learning/update-learning.py \ --id <id> \ --improves <suggested_improves> \ --improves-type <suggested_improves_type>no_applicable_target: No file owns the fix. Mark itinvalid. Print a one-line note. Remove from the picker list. Return to Step 5.python3 ~/.dotfiles/claude-code-shared/scripts/learning/update-learning.py \ --id <id> --status invalid- Malformed verdict: Read its prose. If it names another owner, handle as
misrouted; otherwise asstale. validwithfile_pathoutside path candidates (compare after expanding~): treat asmisrouted. Derive slug and type from the path:skills/<slug>/SKILL.md→ skill,agents/<slug>.md→ agent, nestedagents/<subdir>/<slug>.md→ agent (slug = file basename),resources/<slug>.md|json→ process,contracts/<slug>.json|md→ contract.valid: Setfile_pathas the canonical target path. Proceed to Step 6.partial: Handle likevalid. Setfile_pathand proceed to Step 6. Carryexisting_lineandmissinginto the drafter prompt.
No validation summary block is printed. Validation is silent except for a brief inline note when a learning is discarded or retargeted.
Step 5: Pick-one loop — show learnings and let user choose
Display ALL captured learnings for the target as a numbered sub-list (no
pre-filtering). Show only the problem field per entry (one line each),
taken from the Step 3 fetch output.
These have not been validated yet — validation runs when one is selected.
Learnings for <target> (<N> remaining):
1. filename versioning double-encoded when internal schema_version exists
2. cleanup step uses shell rm, blocked by destructive-fs hook
3. accuracy persona silently returns empty on large transcripts
Pick a learning to apply (number), or "done" to stop:
Target run-all option: When the target has ≤ 10 remaining learnings and
global run-all is not active, also offer "Run all for ". This
applies even when the global TOTAL > 10 hid the Step 2 run-all option. If you
use AskUserQuestion, pass this option first and fill the remaining slots with
the top learnings. When selected, set target_run_all = true. Then work
through every remaining learning in list order: Step 4, then Step 6, then Step
7. Do not show the picker between learnings. Each diff still gets its own y/n.
When the list is empty, go to Step 8.
Wait for user response. If "done" or equivalent, jump to Step 8 (summary).
When the user picks a number: run Step 4 (validate that one learning). Proceed based on the verdict — Step 4 either returns to this list (stale / invalid / misrouted) or proceeds to Step 6 (valid or partial).
Step 6: Draft diff for selected learning
Read the target file (resolved file_path from validation). Spawn a
session-model agent (subagent_type: general-purpose) with this prompt:
Always spawn the drafting agent. Drafting the diff inline in your own turn is a process violation, because the drafter works from the file and the learning alone, without this session's context and assumptions. You may add extra constraints to its prompt (style rules, which section to touch), not the diff.
Fill {id}, {problem}, {lesson} and {fix} verbatim from the Step 3
fetch output. Never paraphrase them.
For a partial verdict, append this constraint to the drafter prompt:
Existing coverage is at line {existing_line}. Extend that text. Add only: {missing}. Do not duplicate what is already there.
You are drafting a unified diff to apply ONE learning to a target file.
Target file: {file_path}
Current file content:
<file>
{full content of file_path}
</file>
Learning to apply:
id: {id}
problem: {problem}
lesson: {lesson}
fix: {fix}
Instructions:
1. Read the learning's `fix` field carefully.
2. Produce a standard unified diff (diff -u format) suitable for `patch -p0`.
3. Be minimal and surgical. Do not rewrite sections unrelated to this learning.
4. Include 3 context lines before and after each change hunk.
5. If the lesson or fix uses absolute language (must, always, never), check
for edge cases where it does not hold. If any exist, use softer framing.
Flag the case as a risk or a question instead of a hard rule.
6. Return ONLY the diff, starting with "--- " and ending after the last hunk.
No prose, no explanation.
Before presenting, check contract edits for type drift. If the target is a
contract .md and the diff changes a field's type or shape, grep that field
name in the contract's JSON schema counterpart under contracts/ (for example
refutation-schema.json). Also grep scripts/ for consumers that read it.
If the schema or a consumer still expects the old shape, flag the mismatch
with the diff. Ask whether to extend the change to those files. Skip this
check for wording-only contract edits.
Present the diff to the user. Show the actual diff, not a prose walkthrough of it. Format rules:
- One numbered section per file. A diff touching several files gets several sections.
- Each section header names the file and the learning it addresses. Write the
learning as a one-line summary of
problem, so the user knows why this file is changing without scrolling back. - Put each file's hunks in a fence tagged
diffso added lines render green and removed lines red. An untagged fence renders every line the same color. - Trim displayed context to 1 line per hunk. Strip
@@headers. Elide long unchanged lines with.... Break long added prose lines at about 100 chars so nothing wraps mid-word. This is display only. Apply from the full drafter diff in Step 7. - After the last section, add a short My take list (2 to 4 bullets): soft spots, a hunk worth dropping, risks. Offer matching choices in the prompt.
**1. `<file_path>`**
Learning: <one-line summary of problem>
```diff
<1 line of context>
-<removed line>
+<added line>
```
**2. `<next file_path>`**
Learning: <one-line summary of problem>
```diff
...
```
**My take**
- <soft spot or risk>
- <hunk worth dropping, if any>
Apply? [y/n]
Wait for response.
nor rejection: Print "Skipped." Do NOT mark the learning as applied. Jump back to Step 5 (show remaining list without the skipped one).yor acceptance: Proceed to Step 7.
Step 7: Apply, commit, and loop
Apply each hunk of the approved diff with the Edit tool. Do NOT pipe the diff
to patch. Agent-generated diffs pass through model text output and may
contain HTML-escaped entities (e.g. <, >, &) that corrupt a
patch. Use the literal file text as old_string/new_string. Match the raw
text first (files may legitimately contain &, <, >); unescape
entities only as a fallback when the raw old_string is not found.
If an Edit fails (old_string not found), the diff's context drifted from the
real file. That is a drafting failure, not a problem with the learning. Re-read
the file and re-anchor the hunk against the current text once. If it still
fails, report the error and skip. Leave the learning captured (it stays in
the backlog for a later run). Jump back to Step 5 with the remaining list.
Mark the learning as applied:
python3 ~/.dotfiles/claude-code-shared/scripts/learning/update-learning.py \
--id <id> --status applied
Commit:
git add <file_path>
git commit -m "improve(<target>): apply learning <short problem summary>"
Keep the commit message subject under 50 chars. Use a truncated version of the
problem field if needed.
Auto-loop: Remove the applied learning from the list. If learnings remain
for the current target, jump back to Step 5 and show the updated list. When
target_run_all = true, skip the list and validate the next learning. If
none remain, print "All learnings for done." then:
- In single-target mode: continue to Step 8.
- In run-all mode: advance to the next target (Step 2 run-all logic).
Step 8: Summary report
Re-read unified-learnings.jsonl to tally the remaining backlog:
python3 - <<'PYEOF'
import json, os
from collections import defaultdict
path = os.path.expanduser(
"~/.dotfiles/claude-code-shared/learnings/unified-learnings.jsonl"
)
groups = defaultdict(int)
try:
with open(path) as f:
for line in f:
line = line.strip()
if line:
try:
e = json.loads(line)
if e.get("status", "captured") == "captured":
key = e.get("improves") or "(unassigned)"
groups[key] += 1
except json.JSONDecodeError:
pass
except FileNotFoundError:
pass
for target, cnt in sorted(groups.items(), key=lambda x: -x[1]):
print(f" {target}: {cnt}")
PYEOF
Display:
Summary for <target>:
Applied: N (one commit each)
Skipped: N (user declined)
Retargeted: N (moved to correct owner, still captured)
Stale: N (marked stale, filtered before picker)
Invalid: N (marked invalid, filtered before picker)
Remaining backlog (<M> total across all targets):
<slug1>: N learnings
<slug2>: N learnings
...
(unassigned): N learnings
If the backlog is empty, print "Backlog clear. No remaining captured learnings."
Read and execute ~/.dotfiles/claude-code-shared/resources/learning-capture.md.
This skill's slug is improve-skill-learnings.
Step 9: Final outcome line
After the learning-capture agent finishes (or while it runs in background), print a single concise outcome line so the user sees the result without scrolling back. Format:
<target>: <N> applied, <N> skipped, <N> retargeted, <N> stale, <N> invalid. Backlog: <M> remaining.
This is the LAST thing printed. It must appear after the learning-capture tail block, not before it.