Auto-Evolver
Overview
Auto-Evolver turns usage into capability. It mines agent session history, failure receipts, and repeated fix patterns, then promotes them into new skills or upgrades existing ones. The loop:
- Observe: Track which skills are invoked, which queries miss, which fixes repeat across sessions.
- Extract: Identify recurring patterns (the same bash recovery sequence, the same refactor applied 3+ times, the same diagnostic flow).
- Promote: Draft a new SKILL.md (or extend an existing one) with the extracted pattern, triggers, and verification gates.
- Verify: Run the full validation ladder before registering.
Paired with auto-consolidator: evolution adds skills, consolidation keeps the registry dense. Evolution without consolidation causes bloat; consolidation without evolution causes stagnation.
When to Use
Trigger phrases:
- "auto evolve", "evolve skills", "learn from usage"
- "promote pattern", "skill evolution", "mine patterns"
Use directly when:
- The same manual fix/recovery sequence appears in 3+ sessions.
- A query repeatedly scores < 30 in skill-router and no import exists (gap signal).
- A skill's verification section accumulates recurring new failure modes (extend its Anti-Rationalization Table).
- User explicitly asks to capture a lesson as a reusable skill.
When NOT to Use
- One-off failures with no recurrence evidence — log via
learn, not a new skill.
- Patterns better served as a managed-skill update (small procedural tweak) than a full catalog skill.
- Speculative features with zero usage evidence (violates YAGNI).
Anti-Rationalization Table
| Rationalization |
Reality |
| "This fix was quick, no need to capture it." |
The third time you do it manually, it costs more than the 10 minutes to write the skill. |
| "I remember the pattern, that's enough." |
Agent context does not survive compaction. Only skills and memory files persist. |
| "More skills means a stronger library." |
Unverified, un-routed skills add noise. Evolution requires validation ladder + trigger registration or the new skill is invisible. |
| "Evolution can run fully automatic." |
Skill promotion is a write to shared infrastructure. Require diff review before registering. |
Workflow
1. Signal Sources (ranked by evidence strength)
| Signal |
Source |
Strength |
| Repeated fix in ≥3 sessions |
history:// transcripts, agent receipts |
Strong — promote now |
| Query misses (score < 30) recurring |
skill_router.py logs |
Strong — gap exists, find or create |
| New failure mode on existing skill |
Skill verification failures, user reports |
Medium — extend skill |
| Single user "remember this" request |
learn tool capture |
Medium — managed skill first |
| Theoretical improvement |
Brainstorm |
Weak — backlog, do not implement |
2. Pattern Extraction
For each candidate pattern, capture:
- Trigger: what the user actually typed/said (verbatim phrases →
triggers: frontmatter).
- Sequence: exact tool calls that resolved it (the workflow body).
- Guardrails: what failed when done wrong (→ Anti-Rationalization rows).
- Proof: the receipt that shows it worked (→ Verification checklist).
3. Promote
# create skill skeleton in the correct category
mkdir -p <category>/<new-skill>
# write SKILL.md with required sections:
# frontmatter (name, description "Use when...", domain, tags, triggers, version)
# ## When to Use, ## When NOT to Use, ## Anti-Rationalization Table,
# ## Workflow/Process, ## Verification
4. Validate & Register
python3 scripts/lint-skills.py --write # indexes into SKILLS.json
python3 scripts/sync-counts.py
python3 scripts/check-broken-links.py
python3 scripts/validate-skills.py
python3 scripts/test-skills.py --quick
bash scripts/sync-skills.sh # runtime mirror
5. Measure Evolution Impact
Track per skill: invocation count, success rate, post-promotion query score. Skills scoring consistently < 10 after 30 days are candidates for auto-consolidator demotion.
Loop
sessions → signals → extract → promote → validate → register
↑ |
└──────── auto-consolidator prunes bloat ←──────────┘
Verification
1---2name: auto-evolver3description: Use when evolving the skill library from repeated usage patterns — mining successful fixes, recurring failures, and refactor patterns from agent sessions, then promoting them into new or upgraded skills so the library becomes more efficient the more it is used.4domain: core5subdomain: meta-skills6category: core7version: 1.0.08author: oyi779license: Apache-2.010tags:11- auto-evolve12- continuous-learning13- skill-evolution14- pattern-extraction15- self-improvement16triggers:17- auto evolve18- evolve skills19- learn from usage20- promote pattern21- skill evolution22- mine patterns23argument-hint: "[session-log | skill-name]"24metadata:25 source: https://skillmd.com/skills/oyi77/auto-evolver26---2728# Auto-Evolver2930## Overview3132Auto-Evolver turns usage into capability. It mines agent session history, failure receipts, and repeated fix patterns, then promotes them into new skills or upgrades existing ones. The loop:33341. **Observe**: Track which skills are invoked, which queries miss, which fixes repeat across sessions.352. **Extract**: Identify recurring patterns (the same bash recovery sequence, the same refactor applied 3+ times, the same diagnostic flow).363. **Promote**: Draft a new SKILL.md (or extend an existing one) with the extracted pattern, triggers, and verification gates.374. **Verify**: Run the full validation ladder before registering.3839Paired with `auto-consolidator`: evolution adds skills, consolidation keeps the registry dense. Evolution without consolidation causes bloat; consolidation without evolution causes stagnation.4041## When to Use4243**Trigger phrases:**44- "auto evolve", "evolve skills", "learn from usage"45- "promote pattern", "skill evolution", "mine patterns"4647Use directly when:48- The same manual fix/recovery sequence appears in 3+ sessions.49- A query repeatedly scores < 30 in skill-router and no import exists (gap signal).50- A skill's verification section accumulates recurring new failure modes (extend its Anti-Rationalization Table).51- User explicitly asks to capture a lesson as a reusable skill.5253## When NOT to Use5455- One-off failures with no recurrence evidence — log via `learn`, not a new skill.56- Patterns better served as a managed-skill update (small procedural tweak) than a full catalog skill.57- Speculative features with zero usage evidence (violates YAGNI).5859## Anti-Rationalization Table6061| Rationalization | Reality |62|---|---|63| "This fix was quick, no need to capture it." | The third time you do it manually, it costs more than the 10 minutes to write the skill. |64| "I remember the pattern, that's enough." | Agent context does not survive compaction. Only skills and memory files persist. |65| "More skills means a stronger library." | Unverified, un-routed skills add noise. Evolution requires validation ladder + trigger registration or the new skill is invisible. |66| "Evolution can run fully automatic." | Skill promotion is a write to shared infrastructure. Require diff review before registering. |6768## Workflow6970### 1. Signal Sources (ranked by evidence strength)7172| Signal | Source | Strength |73|---|---|---|74| Repeated fix in ≥3 sessions | `history://` transcripts, agent receipts | Strong — promote now |75| Query misses (score < 30) recurring | `skill_router.py` logs | Strong — gap exists, find or create |76| New failure mode on existing skill | Skill verification failures, user reports | Medium — extend skill |77| Single user "remember this" request | `learn` tool capture | Medium — managed skill first |78| Theoretical improvement | Brainstorm | Weak — backlog, do not implement |7980### 2. Pattern Extraction8182For each candidate pattern, capture:83- **Trigger**: what the user actually typed/said (verbatim phrases → `triggers:` frontmatter).84- **Sequence**: exact tool calls that resolved it (the workflow body).85- **Guardrails**: what failed when done wrong (→ Anti-Rationalization rows).86- **Proof**: the receipt that shows it worked (→ Verification checklist).8788### 3. Promote8990```bash91# create skill skeleton in the correct category92mkdir -p <category>/<new-skill>93# write SKILL.md with required sections:94# frontmatter (name, description "Use when...", domain, tags, triggers, version)95# ## When to Use, ## When NOT to Use, ## Anti-Rationalization Table,96# ## Workflow/Process, ## Verification97```9899### 4. Validate & Register100101```bash102python3 scripts/lint-skills.py --write # indexes into SKILLS.json103python3 scripts/sync-counts.py104python3 scripts/check-broken-links.py105python3 scripts/validate-skills.py106python3 scripts/test-skills.py --quick107bash scripts/sync-skills.sh # runtime mirror108```109110### 5. Measure Evolution Impact111112Track per skill: invocation count, success rate, post-promotion query score. Skills scoring consistently < 10 after 30 days are candidates for `auto-consolidator` demotion.113114## Loop115116```text117sessions → signals → extract → promote → validate → register118 ↑ |119 └──────── auto-consolidator prunes bloat ←──────────┘120```121122## Verification123124- [ ] Pattern has recurrence evidence (≥3 sessions or explicit user request)125- [ ] Triggers copied from verbatim user phrases, not invented126- [ ] New skill passes full validation ladder before registration127- [ ] Existing skill extensions add to Anti-Rationalization Table + Verification, never rewrite wholesale128- [ ] No promotion from single-session evidence129- [ ] Post-registration router test: query scores ≥ 30 for the new skill