# Agentic Self Improvement > Run behavioral benchmarks against the agent, detect failure patterns, generate guidance patches, apply them with auto-revert safety net. Implements the closed-loop self-improvement cycle from Hermes PR - Skill: `tomevault-io/agentic-self-improvement` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add tomevault-io/agentic-self-improvement` - Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/agentic-self-improvement/raw - Safety review: pending (external: skill-scanner PASS, skillspector CAUTION) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: tomevault-io (https://skillmd.com/u/tomevault-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tomevault-io/agentic-self-improvement --- # Agentic Self-Improvement Loop Behavioral benchmarking + evolutionary self-improvement for Hermes Agent. Combines closed-loop guidance patching (from Hermes PR #6120) with the GEPA optimization framework from `hermes-agent-self-evolution` (NousResearch, MIT). --- ## Core Loop ``` BENCHMARK → DIAGNOSE → PATCH → VERIFY → (auto-revert if regression) ↓ GEPA OPTIMIZE → EVAL DATASET → EVOLVE → DEPLOY ↓ SESSION MINE → LLM-AS-JUDGE → NEW BENCHMARKS ``` **Two modes operate together:** - **Guidance Patch Loop** — fast, surgical, targets specific behavioral failures via guidance blocks - **Evolutionary Optimization Loop** — principled, uses GEPA + DSPy to optimize skill text end-to-end --- ## Part 1: Guidance Patch Loop ### Core Loop ``` BENCHMARK → DIAGNOSE → PATCH → VERIFY → (auto-revert if regression) ``` ### Usage ```bash # Run all benchmarks, show suggested patches (no changes made) /self-improve # Run all benchmarks, auto-apply if improvement, auto-revert if regression /self-improve --mode=apply # Run specific category only /self-improve --benchmarks=mandatory_tool.yaml # Run subset of categories /self-improve --categories=mandatory_tool,act_dont_ask,no_hallucination # Test a specific model /self-improve --model=claude-sonnet-4 # Control parallelism (default: 4) /self-improve --parallel=8 # View past results /self-improve --view-results=2026-04-12_1400 # Revert last applied patch /self-improve --revert ``` ### Options | Flag | Default | Description | |------|---------|-------------| | `--mode` | `suggest` | `suggest` (show diff only) or `apply` (apply + verify) | | `--benchmarks` | all | Benchmark YAML file(s) to run | | `--categories` | all | Run only specific categories (by filename without .yaml) | | `--model` | current | Model to benchmark | | `--parallel` | 4 | Number of parallel prompt executions | | `--view-results` | none | Show a past run's results | | `--revert` | false | Revert the last applied patch | ### Benchmark Categories | Category | File | What it tests | |----------|------|---------------| | `mandatory_tool` | `BENCHMARKS/mandatory_tool.yaml` | Must use tools for math/hashes/time/files | | `act_dont_ask` | `BENCHMARKS/act_dont_ask.yaml` | Act on obvious interpretation, don't ask | | `no_hallucination` | `BENCHMARKS/no_hallucination.yaml` | Check system, don't answer from profile/memory | | `verification` | `BENCHMARKS/verification.yaml` | Verify outputs before finalizing | | `prerequisite` | `BENCHMARKS/prerequisite.yaml` | Discover before acting | | `path_accuracy` | `BENCHMARKS/path_accuracy.yaml` | Verify paths before using | | `context_grounding` | `BENCHMARKS/context_grounding.yaml` | Check actual context, don't assume | | `auth_state` | `BENCHMARKS/auth_state.yaml` | Verify credentials before using them | | `remember_to_obsidian` | `BENCHMARKS/remember_to_obsidian.yaml` | Write durable facts to Obsidian, not just memory tool | ### How the Guidance Patch Loop Works **1. Benchmark Runner** Each prompt runs in isolation via `hermes chat --model -q QUERY -Q` as a subprocess. Captures: - Tool calls made (detected as "preparing X..." text previews in quiet mode) - Tool outputs - Final response text - Exit code **2. Failure Analyzer** Groups failures by category, computes per-category pass rates, extracts failure examples. **3. Patch Generator** Generates XML guidance blocks for the relevant model type: - OpenAI/GPT/Codex → `OPENAI_MODEL_EXECUTION_GUIDANCE` - Anthropic/Claude → `ANTHROPIC_MODEL_EXECUTION_GUIDANCE` - Google/Gemini → `GEMINI_MODEL_EXECUTION_GUIDANCE` - MiniMax → `MINIMAX_MODEL_EXECUTION_GUIDANCE` **4. Apply + Verify + Auto-Revert** **Suggest mode:** Shows the diff, writes to `~/.hermes/self-improvement/proposed_patches/.diff` **Apply mode:** 1. Run baseline benchmarks → capture pass rates 2. Backup `prompt_builder.py` to `~/.hermes/self-improvement/backups//` 3. Apply patch to relevant GUIDANCE constant 4. Re-run benchmarks → compare pass rates 5. If ANY category regresses → auto-revert to backup 6. If all improve → keep patch, report delta ### Output Locations | What | Where | |------|-------| | Benchmark results | `~/.hermes/self-improvement/results//` | | Proposed patches | `~/.hermes/self-improvement/proposed_patches/.diff` | | Backups | `~/.hermes/self-improvement/backups//` | | Logs | `~/.hermes/logs/` | --- ## Part 2: Session Pattern Mining Extract behavioral failure patterns from conversation history and convert them into concrete, actionable test cases. ### When to Use - Designing benchmarks for a self-improvement loop - Finding recurring mistakes to address in skills or system prompts - Building a behavioral test suite for an agent's capabilities - Reviewing past sessions for systematic failures vs one-off errors ### The Process **1. Search Broadly, Then Narrow** ``` # Good starting queries: "failure pattern mistake error" "wrong incorrect mistake" "assume guess hallucinate" "context switch forgot lost" "should I do ask clarification" "file not found path error" "not sure unknown do not know" "directory path file location" "behavior bias habit pattern" ``` **2. Extract the Failure Narrative** For each relevant session: - **What was the task?** (the user's request) - **What went wrong?** (the failure) - **What was the root cause?** (why it failed) - **What category does it belong to?** (see Categories below) ```markdown ## Session: — ### Task ### Failure ### Root Cause ### Category ``` **3. Categorize Failures** | Category | What it means | Signs in session | |----------|---------------|------------------| | `mandatory_tool` | Model computed/guessed instead of using a tool | Mental math in response, hashes from training data | | `act_dont_ask` | Model asked for clarification on obvious cases | "Should I do X or Y?" when X was clearly correct | | `no_hallucination` | Model answered from memory/profile instead of checking | User profile substituted for system reality | | `verification` | Model finalized without checking output | Responded to empty pages, wrong files, failed commands | | `prerequisite` | Model started task without discovery steps | Tried to use a file/path without checking if it existed | | `path_accuracy` | Model assumed paths without verification | FileNotFoundError in logs, "does ~/.X exist?" when it didn't | | `context_grounding` | Model answered from wrong context | Wrong directory, wrong git branch, wrong session state | | `auth_state` | Model tried to use credentials/files without checking validity | 403/401 errors from unverified tokens | **4. Convert Failures to Benchmark Prompts** ```yaml id: - # e.g. mt-001, ada-001 prompt: validator: type: detail: ground_truth: failure_example: what_happened: session_id: ``` **Validator types:** - `tool_used` — model must call a specific tool - `output_contains` — tool output must contain expected string - `refused` — model must refuse to answer (not enough context) - `exit_code` — command must exit with specific code - `error_in_output` — model must report an error it detected - `command_contains` — response or tool outputs contain specific strings - `non_empty_or_explicit_not_found` — non-empty response or explicit not-found - `two_step` / `sequence` — multi-step (tool use demonstrates it) **5. Group Into Benchmark Sets** Cluster prompts by category. A complete benchmark set should have: - 4-10 prompts per category - Varied phrasings (don't make the expected action obvious) - At least one negative example (task that should NOT use a tool) ### Session Search Tips 1. **Search for the fix, not the failure** — sessions that end with "fixed" often document what broke first 2. **Check session summaries first** — `session_search` returns LLM-generated summaries that surface failures efficiently 3. **Look at truncated sessions** — sessions that were "cut off" mid-investigation often reveal root causes 4. **Cross-reference with logs** — `~/.hermes/logs/errors.log` often has the raw error that sessions later explained ### Output Location Save mined patterns to `/03-Notes/session-patterns/.md` so they're preserved in long-term memory alongside other session research. --- ## Part 3: Evolutionary Optimization (from hermes-agent-self-evolution) The full GEPA + DSPy optimization framework from [NousResearch/hermes-agent-self-evolution](https://github.com/NousResearch/hermes-agent-self-evolution). ### What Can Be Optimized — The Tier System | Tier | Target | Risk | Status | |------|--------|------|--------| | **Tier 1** | Skill files (SKILL.md) | Low | Highest value, lowest risk | | **Tier 2** | Tool descriptions | Low-Medium | Medium value, schema must stay frozen | | **Tier 3** | System prompt sections | High | High value, must stay within 20% size budget | | **Tier 4** | Tool implementation code | Highest | Darwinian Evolver, strictest guardrails | **Tier 1 is the primary target for self-improvement** — skills are pure text, easily mutated, directly measurable. ### The Optimization Loop (GEPA Framework) ``` 1. SELECT TARGET - Pick a skill, prompt section, or tool - Load current version as baseline 2. BUILD EVALUATION DATASET - Mine SessionDB for real usage examples - Or use session-pattern-mining to extract from transcripts - Or generate synthetic test cases - Split: train / validation / test 3. RUN OPTIMIZER - Primary: GEPA (Genetic-Pareto Prompt Evolution) - Fallback: DSPy MIPROv2 (Bayesian optimization) - Generates mutations, evaluates, selects best 4. EVALUATE & COMPARE - Run optimized version on held-out test set - Compare: accuracy, cost, latency - Statistical significance check 5. DEPLOY (with approval) - Git commit the improved version - Rollback mechanism via git revert ``` ### Evaluation Dataset Sources (in priority order) **Source A: SessionDB mining (real usage, LLM-as-judge scored)** - Query SessionDB for sessions where the skill was loaded - Extract the task the user gave and the agent's full response - Use LLM-as-judge to score each (task, response) pair on a rubric - High-scoring pairs → "good" examples; low-scoring → failure cases for GEPA **Source B: Synthetic generation (bootstrapping)** - Use a strong model (Claude Opus) to generate 15-30 test cases - Read the skill file → understand what it does - Generate (task_input, expected_behavior) pairs - GEPA works with as few as 3 examples, so start small **Source C: Hand-curated golden sets** - Manually written test cases with expected outputs - Highest quality signal but requires manual effort - Reserve for critical skills ### LLM-as-Judge Scoring (Multi-Dimensional Rubrics) For most skills, there's no binary right/wrong — quality is subjective. Use an LLM judge with a rubric: - Did the agent follow the skill's procedure? (0-1) - Was the output correct/useful? (0-1) - Was it concise (within token budget)? (0-1) - Did it avoid the known failure modes? (0-1) Aggregate into a composite score. ### Constraint Gates (must pass before any change is valid) | Constraint | Rule | Why | |------------|------|-----| | **Full test suite** | pytest tests/ must pass 100% | Hard floor — nothing ships that breaks existing functionality | | **Size limits** | Skills ≤15KB, tool descriptions ≤500 chars, system prompts ≤120% of current | Prevents evolutionary bloat | | **Caching compatibility** | No mid-conversation changes — all changes take effect on new sessions | Preserves prompt caching | | **Semantic preservation** | Evolved text must not drift from original purpose | Fitness function includes similarity check | | **Deployment via PR** | All changes go through PR, never direct commit | Human review is mandatory | ### Benchmark Gate Hierarchy | Benchmark | What It Tests | Speed | Role | |-----------|---------------|-------|------| | **TBLite fast subset** | Coding/sysadmin (20 tasks) | ~20 min | Quick regression gate | | **Full TBLite** | Coding/sysadmin (100 tasks) | ~1-2 hrs | Thorough regression gate | | **YC-Bench fast_test** | Long-horizon coherence (50 turns) | ~30 min | Coherence check | | **Session-specific** | Skill-targeted eval dataset | Varies | Primary fitness signal | **Key principle:** Benchmarks are GATES, not fitness functions. A variant that improves skill quality by 20% but drops TBLite by 5% is REJECTED. ### Practical Considerations **Cost:** - GEPA optimization: ~$2-10 per run (API calls only, no GPU) - Synthetic eval data generation: ~$1-3 per skill - Start with small eval sets (10-20 examples), scale up for important skills **Safety:** - All changes via PR, never direct commit - Full test suite gate (zero tolerance) - Character/token budgets enforced - Caching compatibility checked - Git-tracked lineage, trivial rollback **No GPU required** — everything operates via API calls. GEPA mutates text and evaluates via LLM calls, not weight training. ### References - hermes-agent-self-evolution: https://github.com/NousResearch/hermes-agent-self-evolution - GEPA paper: ICLR 2026 Oral - DSPy: https://github.com/stanfordnlp/dspy --- ## Part 4: Tool Description Improvement (Anthropic Pattern) Anthropic found: "We created a tool-testing agent — when given a flawed MCP tool, it attempts to use the tool and then rewrites the tool description to avoid failures. This process resulted in a 40% decrease in task completion time for future agents using the new description." This is a separate loop from behavioral benchmarking. Use it when: - An MCP tool or skill consistently causes failures across multiple agents - A tool description is vague, misleading, or causes agents to take wrong paths - You notice agents consistently misusing the same tool ### Tool-Testing Loop ``` IDENTIFY → TEST → DIAGNOSE → PATCH → VERIFY ``` 1. **Identify**: Log which tool/skill caused failures in session logs 2. **Test**: Run the tool with various inputs — document failures 3. **Diagnose**: Why does the tool description cause agents to fail? 4. **Patch**: Rewrite the tool description to make failure modes explicit 5. **Verify**: Re-run benchmark tasks using the tool — measure improvement ### Tool Improvement Prompt ``` TASK: Improve the description for [tool/skill name] at [path] FAILURE OBSERVED: [what agents did wrong when using this tool] TOOL DESCRIPTION CURRENTLY: [copy the current description] WHY IT FAILS: [analysis — what in the description caused the wrong behavior] PROPOSED IMPROVEMENT: [new description that makes the failure mode explicit] VERIFICATION: After patching, run [task type that previously failed] and confirm improvement ``` --- ## Critical Rules - Each lens must RETHINK the question, not just add more information. Technical and contrarian should feel like two researchers who disagree. - The tension between lenses IS the insight. Don't resolve it away. - Never present a single-lens finding as a conclusion. - Separate "what the data shows" from "what I interpret." - [[open-questions]] is as important as [[executive-summary]]. --- ## Folder Structure ``` agentic-self-improvement/ ├── SKILL.md # This file ├── BENCHMARKS/ # Behavioral benchmark YAMLs │ ├── mandatory_tool.yaml │ ├── act_dont_ask.yaml │ ├── no_hallucination.yaml │ ├── verification.yaml │ ├── prerequisite.yaml │ ├── path_accuracy.yaml │ ├── context_grounding.yaml │ ├── auth_state.yaml │ └── remember_to_obsidian.yaml ├── REFERENCES/ │ └── guidance_anatomy.md # Guidance block reference └── src/ # Python tooling ├── benchmark_runner.py # Run benchmarks via hermes chat ├── failure_analyzer.py # Analyze results, group failures ├── patch_generator.py # Generate guidance patches └── apply_and_verify.py # Apply patches with auto-revert ``` --- > Source: [ChuckSRQ/awesome-hermes-skills](https://github.com/ChuckSRQ/awesome-hermes-skills) — distributed by [TomeVault](https://tomevault.io).