# Tune Agent

> Propose a Reflexion-style patch to a specific agent's prompt based on evidence of recurring failures. Use when the same agent makes the same class of mistake twice, when /mine-transcripts surfaces an agent in agent-tuning.md, or when /eval flags a regression on an agent. Triggers on /tune-agent, "tune the X agent", "fix the agent prompt".

- Skill: `sethdford/tune-agent-2` (Agent Skill)
- Install (CLI): `npx skillmds@latest add sethdford/tune-agent-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sethdford/tune-agent-2/raw
- Safety review: pending (external: skill-scanner PASS, skillspector CAUTION)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: sethdford (https://skillmd.com/u/sethdford)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/sethdford/tune-agent-2

---


# /tune-agent — Reflexion patch for an agent's prompt

`/tune-agent <name>` spawns the `agent-tuner` agent with collected evidence and proposes a focused patch to that agent's `.md` file.

## When to invoke

- `/mine-transcripts` produced `agent-tuning.md` listing agents with ≥2 failures
- `/eval` shows a regression for a specific agent vs prior runs
- Verifier or critic flagged the same failure pattern from the same agent twice in this session
- The user says "this agent keeps doing X — fix it"

## What you do

1. **Resolve the agent.** `~/.claude/agents/<name>.md` must exist. If not, stop.

2. **Collect evidence.** Pull from:
   - Most recent `~/.claude/telemetry/mining-runs/*/agent-tuning.md` (if applicable)
   - Verifier/critic outputs in this session that mention the agent
   - Any context the user just provided
   
   Need ≥2 pieces. If you have only 1, tell the user — wait for more before tuning.

3. **Spawn agent-tuner.** Use the `Agent` tool with `subagent_type: agent-tuner`. Pass the evidence as a structured prompt:
   ```
   Agent to tune: <name>
   Evidence:
   1. <task summary> — <what went wrong> — session=<id>
   2. <task summary> — <what went wrong> — session=<id>
   3. <task summary> — <what went wrong> — session=<id>
   
   Desired behavior: <if user specified, otherwise infer>
   ```

4. **Read the tuner's output.** It ends with one of:
   - `RESULT_agent-tuner=PATCHED` — proceed to step 5 (A/B gate). Do NOT leave the patch live.
   - `RESULT_agent-tuner=INSUFFICIENT_EVIDENCE` — tell user, suggest waiting
   - `RESULT_agent-tuner=AMBIGUOUS` — surface options to user
   - `RESULT_agent-tuner=NO_PATCH_NEEDED` — evidence didn't add up; explain

5. **Stage as candidate — NEVER apply directly.** Reflexion-style prompt patching has no published validation (2026-07 deep-research pass found zero peer-reviewed evidence it works); the A/B gate IS the validation. Immediately after PATCHED:
   ```bash
   mkdir -p ~/.claude/rl/policy/<name>/candidates
   cp ~/.claude/agents/<name>.md ~/.claude/rl/policy/<name>/candidates/tuner-$(date +%Y-%m-%d).md
   cp "$(ls -t ~/.claude/agents-history/<name>-*.md | head -1)" ~/.claude/agents/<name>.md   # restore live prompt
   ```
   Surface the diff and rationale to the user, then log the staging to `~/.claude/agents-history/CHANGELOG.md`.

6. **A/B gate (mandatory before promotion).** Run `/ab-test <name>`. Promote ONLY if the decision meets the promotion rule (candidate mean > current mean + max(stderr, 0.05), n ≥ 10):
   ```bash
   python3 ~/.claude/rl/ab_test.py <name> --runs 3
   # if decision.json recommends the candidate:
   python3 ~/.claude/rl/ab_test.py <name> --promote tuner-<date>
   ```
   If scenarios for the agent don't exist yet, run `/eval-author <name>` first — no scenarios means no gate means no promotion. An inconclusive A/B leaves the candidate staged; re-run after more scenarios accumulate rather than promoting on vibes.

7. **Record a prediction (mandatory on promotion).** Every promoted patch carries a falsifiable prediction the ledger scores later (AHE decision observability, arXiv 2604.25850):
   ```bash
   python3 ~/.claude/rl/prediction_ledger.py add \
     --change "tune-agent: <name> <one-line patch summary>" \
     --metric mean_reward --agent <name> --direction up --horizon-days 30
   ```
   Pick the metric the evidence actually implicates (`mean_reward`, `correction_rate` down, `verifier_fail_rate` down). `/rl-status` scores it when due — a wrong prediction is evidence for reverting.

8. **Post-promotion re-eval.** After a promotion, run `/eval <name>` to confirm no regression on behaviors the A/B scenarios didn't cover.

## Cost note

The tuner runs on Opus (precise prompt engineering matters). One invocation is typically <$0.50. If you find yourself tuning the same agent more than weekly, the agent's design is wrong — propose a redesign, not another patch.

## Anti-patterns to refuse

- Tuning on a single occurrence (refuse — collect more evidence first)
- Tuning based on user vibes rather than logged failures
- Tuning multiple agents in one shot — each `/tune-agent` is one agent
- Skipping the post-patch `/eval` — you must verify the patch doesn't regress