Adam Self Improvement

adam-self-improvement

When to invoke

• User types /reflect
• User types /reflect --explain (same flow, but the analyst's clustering trace is shown to the user — see §2b below)
• User asks: "what has adam learned", "any proposals", "review the queue"
• SessionStart nudge said proposals are pending and user wants to act on it

Protocol

0. Parse flags

Check the slash-command argument string for the literal token --explain. Set explain=true when present; otherwise explain=false. Unknown flags: print one-line warning, continue with explain=false. This single flag is the only argument /reflect currently accepts.

1. Pre-filter the journal (window + exclusion) + score

Before dispatching the analyst, run the windowed-journal filter:

node ~/.claude/adam/scripts/adam-window.mjs --home ~/.claude > /tmp/adam-windowed-journal.jsonl 2> /tmp/adam-windowed-journal.log

The script reads the active journal plus all rotated journal files (new journal/YYYY-Www.jsonl weekly format AND legacy journal/YYYY-MM-DD-<ts>.jsonl size-rotated format are both supported), applies per-signal-type sliding windows (see SIGNAL_WINDOWS_DAYS in adam-window.mjs), and drops entries already actioned via applied/*.md / rejected/*.md frontmatter source_entries.

If adam-window.mjs exits non-zero: log the stderr file to the user, fall through to passing the raw ~/.claude/adam/journal.jsonl path to the agent (graceful degradation — the agent's manual excluded-timestamps logic still filters actioned entries; only the freshness window is lost).

Then run the scoring pre-step on the same windowed journal:

node ~/.claude/adam/scripts/adam-score.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-scores.json 2> /tmp/adam-scores.log

This produces a per-session dampener (0.5 / 0.75 / 1.0 based on task_completed_count) and a reinforcement_candidates list (skills cited by ≥3 clean task_completed events). The analyst uses both — see agents/adam.md §"Scoring: task_completed dampener". If the score step fails, log stderr to the user and pass an empty {"sessions":[],"reinforcement_candidates":[]} to the analyst (dampener defaults to 1.0).

Finally, run the A/B measurement pre-step on any previously auto-applied proposals (see §3 ab-tracking write):

node ~/.claude/adam/scripts/adam-ab-measure.mjs --home ~/.claude --format json > /tmp/adam-ab-regressions.json 2> /tmp/adam-ab-regressions.log

The JSON output is an array of A/B delta objects (pre_count, post_count, delta_pct, status ∈ {improved,neutral,regressed,no_baseline,pending}). Filter to status == "regressed" before passing to the analyst as ab_regressions. The analyst is required (see agents/adam.md §"A/B effectiveness") to surface a ## Regressions section at the top of its output when this list is non-empty. If the script fails: log stderr, pass [].

Auto-rollback (MOSS §3.5): if any entries have status == "regressed", run the rollback script to auto-revert them before analyst dispatch:

node ~/.claude/adam/scripts/adam-rollback.mjs --auto --home ~/.claude > /tmp/adam-rollback-results.json 2> /tmp/adam-rollback.log

For each rolled-back proposal, print to user: adam: rolled back "<proposal_id>" — regression detected (delta: <delta_pct>%). The rollback script moves the proposal from applied/ back to proposals/ with rolled_back: true and creates a regression nudge. If the script fails: log stderr, continue (rollback is best-effort).

Evidence batching (MOSS §3.1): pre-cluster the windowed journal into coherent failure batches:

node ~/.claude/adam/scripts/adam-batch.mjs --input /tmp/adam-windowed-journal.jsonl > /tmp/adam-batches.json 2> /tmp/adam-batch.log

This groups entries by (signal_type, cluster_key) and reports per-batch metadata including has_context_window (whether transcript evidence is attached). If the script fails: log stderr, pass null to the analyst (graceful degradation — analyst falls back to raw journal clustering).

Skill utility (execution-grounded selection signal, in the spirit of SkillsInjector arXiv 2605.29794 — utility Δ(s), not surface match): compute per-skill good:bad outcome ratios over the windowed journal:

node ~/.claude/adam/scripts/adam-skill-utility.mjs --input /tmp/adam-windowed-journal.jsonl --json > /tmp/adam-skill-utility.json 2> /tmp/adam-skill-utility.log

This ranks skills by how often they co-occur with positive (task_completed, clean_recovery, correction_free_streak) vs negative outcome events, surfacing skills below the baseline positive rate (with sufficient sample) — advisory candidates for description disambiguation or archival. CO-OCCURRENCE, NOT CAUSATION: display the worst 3 below-baseline skills (lift < 0, not low-sample) to the user as a one-line advisory before listing proposals (e.g. skill-utility: chezmoi 9% pos n=85, ghostty-config 14% pos n=50, …). Do NOT feed this into the analyst's proposal machinery or auto-draft skill-archival from it — the human decides. If the script fails: log stderr, skip (best-effort).

2. Dispatch the analyst (two-stage pipeline)

MOSS §3.3: "A single prompt asked to diagnose, plan, implement, verify, and decide overloads context and produces lower-quality output than a sequenced flow." The analyst is dispatched in two stages with a validation gate between them.

Stage 1 — Diagnose + Plan: Use the Agent tool with subagent_type: "adam" and prompt:

stage=diagnose

Read the batched journal entries, cluster by signal type, diagnose root causes,
plan fix types, and score the keypoint matrix. Write diagnoses to /tmp/adam-diagnoses.json.
Do NOT draft proposal files.

Inputs:
- windowed_journal_path: /tmp/adam-windowed-journal.jsonl
- batches_path: /tmp/adam-batches.json                     # pre-clustered evidence batches
- scores_path: /tmp/adam-scores.json
- ab_regressions_path: /tmp/adam-ab-regressions.json
- journal_path: ~/.claude/adam/journal.jsonl               # raw — fallback only
- state_path: ~/.claude/adam/state.json
- usage_path: ~/.claude/adam/usage.json
- applied_dir: ~/.claude/adam/applied/
- rejected_dir: ~/.claude/adam/rejected/
- transcripts_root: ~/.claude/projects/
- skills_root: ~/.claude/skills/

Use batches_path for pre-clustered evidence when available. Prefer context_window
fields in journal entries over transcript file lookups. Write /tmp/adam-diagnoses.json
per the "Diagnose-stage output format" in your system prompt.

Wait for return.

Inter-stage validation (§2a): after stage 1 returns, read /tmp/adam-diagnoses.json and validate each diagnosis:

1. Every source_entries timestamp exists in the windowed journal (read /tmp/adam-windowed-journal.jsonl, check timestamps match).
2. Every diagnosis has all four fields (trigger, action, mismatch, outcome).
3. The planned type is a valid proposal type.
4. Remove diagnoses that fail validation — log a one-line warning per removal.

If all diagnoses are removed or the file is missing/empty, print "adam: no valid diagnoses — nothing to implement" and skip to §6.

Stage 2 — Implement: Use the Agent tool with subagent_type: "adam" and prompt:

stage=implement

Read the validated diagnoses and draft full proposal files.

Inputs:
- diagnoses_path: /tmp/adam-diagnoses.json                 # validated stage-1 output
- windowed_journal_path: /tmp/adam-windowed-journal.jsonl
- scores_path: /tmp/adam-scores.json
- ab_regressions_path: /tmp/adam-ab-regressions.json
- state_path: ~/.claude/adam/state.json
- usage_path: ~/.claude/adam/usage.json
- proposals_dir: ~/.claude/adam/proposals/
- applied_dir: ~/.claude/adam/applied/
- rejected_dir: ~/.claude/adam/rejected/
- transcripts_root: ~/.claude/projects/
- skills_root: ~/.claude/skills/

Draft proposal files to proposals_dir/ for each diagnosis. Score against the
confidence rubric. Emit the clustering trace and punch list as your final message.

Wait for return.

2b. Persist and render the clustering trace

The analyst's final message always contains a fenced ```trace block (per agents/adam.md §"Clustering trace (always emit)") immediately before its punch-list JSON line.

1. Extract the trace block. If it is missing, print a one-line warning to the user (adam: trace block missing from agent output — proceeding without observability) and continue; do not block on this.
2. ALWAYS write the trace verbatim (without the surrounding fences) to ~/.claude/adam/last-trace.txt (overwrite each run). This persists for retrospection via node ~/.claude/adam/scripts/adam-explain.mjs.
3. Extract the SUMMARY: line from the trace. ALWAYS display it as a one-line status to the user BEFORE the proposals are listed, e.g. clustering: <SUMMARY line>. This single-line status is shown in both --explain and default modes.
4. If explain=true (from §0): ALSO render the full trace block back to the user as a fenced code block (```text … ```) under a header Clustering trace:. If explain=false: SUPPRESS the cluster-line body from the user-visible output (the SUMMARY line is already shown in step 3).

The user can re-render any past trace at any time via:

node ~/.claude/adam/scripts/adam-explain.mjs --mode summary    # SUMMARY + per-decision counts
node ~/.claude/adam/scripts/adam-explain.mjs --mode full       # verbatim trace + rejection histogram
node ~/.claude/adam/scripts/adam-explain.mjs --mode json       # machine-readable

3. Pre-apply verification gate (MOSS §3.4)

MOSS §3.4: "Verification must therefore be runtime, on a production-equivalent environment, and against the same prompts that produced the failure evidence." Before auto-applying, verify each proposal deterministically:

For each id in high_confidence:

• Read the proposal file from ~/.claude/adam/proposals/<id>-*.md.
• Verification checks (all must pass for auto-apply to proceed):
  1. Source entries exist: every timestamp in source_entries frontmatter must appear in /tmp/adam-windowed-journal.jsonl. If any are missing, the evidence is stale or was already actioned — demote to queued.
  2. Diagnosis grounded: the # Diagnosis section must have all four fields (Trigger, Action, Mismatch, Outcome) with ≥1 backtick-wrapped quote. If malformed, demote to queued.
  3. Type-evidence match: the proposal type must match what the evidence supports:
     • correction signals → memory, skill_new, skill_edit (not nudge)
     • dead_end signals → nudge, skill_new, skill_edit (not memory)
     • tool_error_loop signals → memory, skill_new, skill_edit
     • harness_edit → must cite harness-level evidence (false negative, scoring bias, window miscalibration) If mismatch, demote to queued.
  4. No conflicting applied proposal: grep ~/.claude/adam/applied/ for any proposal with the same target applied in the last 7 days. If found, demote to queued (prevents stacking rapid edits).
• Print verification result: verified: <id> (4/4 checks passed) or demoted: <id> (failed: <check_name>).
• Demoted proposals are moved from high_confidence to queued for manual review.

3a. Apply verified high-confidence items

For each id that passed verification:

• Print id, target, confidence, blast_radius, cross_session_evidence, auto_apply_eligible.

• Apply the change:

  • For skill_new: mkdir -p ~/.claude/skills/<slug>/, then Write the proposal's # Proposed change body to ~/.claude/skills/<slug>/SKILL.md. After write, print: "skill <slug> written to ~/.claude/skills/<slug>/SKILL.md — activates immediately — Claude Code v2.1.0+ auto-hot-reloads user-level skills, no restart needed."

  • For memory: Write the proposal's # Proposed change body (which MUST include the auto-memory frontmatter — see "Memory drafting protocol" in agents/adam.md) to the path in target. Then update MEMORY.md index with a one-line pointer.

  • For nudge: low-blast auto-apply path. Single-session evidence is sufficient — skip the cross-session gate. Append a new entry to ~/.claude/adam/active-nudges.json (create the file with [] if absent) with shape {kind, message, created_at: <now_ms>, expires_at_ts: <now_ms + 7*86400000>, max_displays: 3, displays_used: 0, source_session: <session_id from proposal>}. Do NOT modify any skill, memory, agent, or CLAUDE.md. Tell user: "nudge queued — surfaces on next SessionStart in a different session (expires in 7 days)."

  • For reinforcement: gated by confidence >= 4 AND blast_radius == low (same as memory). Apply by invoking the helper:

    node ~/.claude/adam/scripts/adam-apply-reinforcement.mjs ~/.claude/adam/proposals/<id>-*.md --home ~/.claude
    

    The helper reads the proposal frontmatter (skill_slug, count, source_session) and appends one JSON line to ~/.claude/adam/reinforcements.jsonl. No code/memory/skill modifications. Output: {"status":"applied"|"gated", ...} — on gated leave proposal in proposals/ (helper failed its own re-check), on applied continue to the archive step. Tell user: "reinforcement logged for <skill_slug> (count=) — appended to reinforcements.jsonl."

  • For skill_edit: enforce the apply-time gate before writing.

    1. Verify proposal frontmatter has auto_apply_eligible: true. If not, abort and queue for review.
    2. Read target SKILL.md, capture current_bytes from a fresh stat — do NOT trust frontmatter bytes_before.
    3. Verify diff in # Proposed change:
       • Unified-diff format.
       • Zero - lines on existing SKILL.md content (additions only).
       • Total + lines ≤ 30. If any check fails, print one-line refusal reason, leave proposal in proposals/, continue.
    4. Cooldown re-check: run node ~/.claude/adam/scripts/adam-cooldown.mjs --skill <target_skill> --fingerprint <proposal_fingerprint> (both fields come from proposal frontmatter; missing fingerprint → "legacy"). Refuse if the script returns status: cooldown OR status: blacklisted. This per-(skill, fingerprint) gate replaces the previous coarse per-skill scan — proposals for the same skill with a different fingerprint are NOT blocked by an older entry.
    5. (covered by step 4 — blacklisted status is returned by adam-cooldown.mjs when auto_apply_blacklist: true is found in rejected/ within 30 days for the same (skill, fingerprint))
    6. Apply via Edit tool (append the new section per the diff). Never use Write on existing SKILL.md.
    7. Re-stat target. If new size exceeds 2 * current_bytes (captured in step 2), revert via Edit (remove the just-appended section) and refuse — print refusal reason.
    8. Add last_auto_edit: <iso8601 utc now> to the proposal frontmatter before moving it.
    9. Tell user: "skill <slug> extended (added lines) — auto-applied via win-evidence gate."

• Move proposal to ~/.claude/adam/applied/<UTC-ts>-<id>.md.

• A/B tracking append (skip for reinforcement — positive-only ledger, intentionally not A/B-tracked per agents/adam.md §"reinforcement proposals"): as a separate atomic step right after the move, append one JSON line to ~/.claude/adam/ab-tracking.jsonl (create with empty contents if absent). Read fields from the proposal's frontmatter (proposal_fingerprint, originating_signals — both populated per agents/adam.md; originating_signals is a list of {type, count, session_ids} objects). Schema:

  {
    "applied_at": <unix_ms now>,
    "proposal_id": "<id>",
    "proposal_type": "skill_edit|skill_new|memory|nudge",
    "target_skill": "<slug or target basename>",
    "proposal_fingerprint": "<hash>",
    "originating_signals": [{"type":"<signal>","count":<N>,"session_ids":[...]}],
    "pre_window_days": 7
  }
  

  This entry is consumed by adam-ab-measure.mjs on subsequent /reflect runs to compute pre/post signal-count deltas. See agents/adam.md §"A/B effectiveness". If the append fails (disk-full etc.) log a warning but do NOT abort the apply path — A/B is observability, not a gate.

• Archive consumed journal entries: node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<UTC-ts>-<id>.md — moves entries listed in proposal's source_entries from journal.jsonl to journal/actioned-<id>.jsonl so subsequent /reflect runs do not re-cluster them.

Print: auto-applied N proposals: [ids].

4. Walk the queue

For each id in queued:

a. Read and display the proposal in full (frontmatter + body). b. Ask the user: approve / reject / edit. c. On approve:

• For claude_md_edit: backup cp ~/.claude/CLAUDE.md ~/.claude/adam/applied/<ts>-claude-md-backup.md first.
• For deletion: mkdir -p ~/.claude/adam/trash/<ts> then mv the artifact into it. Print restoration command.
• For skill_new: mkdir -p ~/.claude/skills/<slug>/, then write # Proposed change body to <slug>/SKILL.md. Tell user: "skill <slug> written — activates immediately (CC v2.1.0+ auto-hot-reload)."
• For skill_edit: apply the unified diff in # Proposed change to the existing SKILL.md at target (append-only — never replace existing content).
• For memory: write # Proposed change body (must include auto-memory frontmatter) to target and update MEMORY.md index with a one-line pointer.
• For harness_edit (MOSS §1): apply the unified diff to the target harness file. Before applying:
  1. Run bash ~/.claude/adam/tests/run-tests.sh — capture pass count.
  2. Apply the diff via Edit.
  3. Run bash ~/.claude/adam/tests/run-tests.sh again — verify pass count is equal or higher and 0 failures.
  4. If test regression: revert the edit, print "harness_edit reverted — test regression detected", leave proposal in proposals/.
  5. If tests pass: tell user "harness edit applied to <target> — tests pass ( passed)."
• For all others: apply via Write/Edit per the proposal's # Proposed change.
• Move proposal to ~/.claude/adam/applied/<ts>-<id>.md.
• Archive: node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/applied/<ts>-<id>.md. d. On reject: ask for reason in one line. Append # Reason\n<reason> to proposal body. If the proposal type is skill_edit, ALSO add auto_apply_blacklist: true to its frontmatter (so future reflects skip auto-apply on this target for 30 days). Move to ~/.claude/adam/rejected/<id>.md. Archive: node ~/.claude/adam/scripts/adam-archive.mjs ~/.claude/adam/rejected/<id>.md. e. On edit: ask the user for the change, edit the proposal in place, then loop back to step 4a for that same id.

5. Handle failures

If apply fails (file write error, target missing): leave proposal in proposals/, append # Apply error\n<error> to its body. Tell the user. Do not move it.

6. Summary

End with one block:

adam reflect summary:
  observations processed: <new>
  batches formed: <N>
  diagnoses validated: <N>/<total>
  rolled back (regression): <N>
  verification passed: <N>/<total high_confidence>
  auto-applied: <N>
  approved: <N>
  rejected: <N>
  edited+approved: <N>
  failed: <N>

Keypoint history: after all proposals are processed, append one JSON line to ~/.claude/adam/keypoint-history.jsonl with the aggregate keypoint scores from the diagnose stage:

{"ts":"<iso>","session":"<session_id>","keypoints":{"tool_selection":N,"scope_discipline":N,"error_recovery":N,"first_attempt":N,"build_reliability":N},"proposals_emitted":N,"proposals_applied":N}

This builds a longitudinal record of which capabilities are improving across /reflect runs.

Karpathy constraints (you must enforce on each apply)

Before writing any proposal:

• Confirm # Assumptions section is non-empty.
• Confirm # Diagnosis section exists and contains all four labelled lines (Trigger:, Action:, Mismatch:, Outcome:) AND at least one backtick-wrapped quote ≤80 chars in the Outcome line. Refuse if missing or malformed — agent must redraft per the "Diagnosis drafting protocol" in agents/adam.md.
• Confirm # Success criterion section is non-empty and runnable.
• Confirm change is ≤50 LOC for non-skill_new, or ≤80 LOC for skill_new body. If larger, ask the user once: "this proposal is N LOC — proceed?"
• For claude_md_edit: confirm 3+ distinct cwds in the # Why section.
• For deletion: confirm both criteria (a) and (b) from the agent's special handling are documented in the proposal.
• For skill_new: confirm the slug doesn't collide with any existing skill in ~/.claude/skills/. If it does, refuse and ask user to rename.
• For skill_edit: confirm the diff is append-only (no - lines that remove existing content) and that target SKILL.md exists. When auto-applying, ALSO re-verify the eligibility gate steps in §3 (cooldown, blacklist, byte cap) before any Edit call — never trust frontmatter alone.
• For skill_edit with auto_apply_eligible: true: confirm contradiction_flag is absent or null in frontmatter. Refuse auto-apply if contradiction_flag is set with any non-empty value (treat the agent's flag as a hard veto on auto-apply; user can still manually approve in walk-the-queue if they disagree with the heuristic).
• For memory: confirm # Proposed change body starts with --- frontmatter matching the live auto-memory schema — top-level name (the slug) + description, plus a metadata: block with node_type: memory, type, and originSessionId. Cross-check the shape against an existing file in the target memory dir. Refuse if frontmatter is flat (type:/originSessionId: at top level) or missing the metadata: block — agent must redraft per the Memory drafting protocol.
• For harness_edit: confirm auto_apply_eligible: false (never auto-apply). Confirm confidence ≥ 5. Confirm # Test verification section names the test command. Confirm diff is ≤30 LOC and targets a single allowed harness file (see agents/adam.md §"Harness self-modification"). Run test suite before AND after applying — revert on any regression.
• Confirm source_entries is present in proposal frontmatter as a non-empty list (used for archive). Warn (do not refuse) if missing — legacy proposals from before v0.2.0 won't have it.

If any check fails, refuse to apply and ask the user how to proceed.

Things you MUST NOT do

• Do not auto-apply anything not in high_confidence.
• Do not invoke other skills during a /reflect run.
• Do not modify settings.json without explicit user yes.
• Do not hard-delete anything. Use mv to ~/.claude/adam/trash/<ts>/.
• Do not bypass the rubric (auto_apply_eligible: false means queue, full stop).