System Prompt Doctor
Do not optimize a system prompt by taste. Optimize it against behavior.
Scope
This skill works on prompts/instruction files the user owns or can provide, including:
- system/developer prompts
CLAUDE.mdAGENTS.md- reusable agent instruction files
- agent persona/policy prompts
Do not claim access to hidden platform system prompts that the runtime does not expose.
Workflow
- Capture the contract. Record the target model, prompt surface, task family, tools, hard constraints, output shape, failure cases, latency/cost limits, and what must not change.
- Build the eval before editing. Prefer real historical failures and corrections. Otherwise create 8 to 20 representative cases plus hard negatives and at least a 20% holdout split.
- Baseline the current prompt. Preserve prompt hash, model/version, outputs, scores, token usage, tool behavior, and failures.
- Audit instruction architecture. Find contradictions, duplicate rules, vague defaults, missing completion criteria, tool-policy gaps, bad ask-vs-act calibration, stale examples, and context that should live outside the system prompt.
- Generate a small candidate beam. Always include:
- minimal-diff repair
- structure-first rewrite
- compact/progressive-disclosure variant
Add a provider-specific adapter only when eval failures justify it.
- Optimize on train/validation only. If Promptfoo is available, use
promptfoo optimize with a validation split. Never tune on the final holdout set.
- Red-team important boundaries. Test conflicting instructions, ambiguous tasks, tool misuse, prompt injection exposure, refusal/escalation boundaries, and over/under-triggering where applicable.
- Battle the winner against the current prompt. Same model, same task manifest, same tool permissions. Report quality lift and token/cost/latency change.
- Prefer the smallest prompt that preserves measured behavior. A larger prompt is not a better prompt.
- Return evidence, not vibes. Include the optimized prompt, diff, eval set, before/after scores, regressions, token delta, residual risks, and a shareable result card.
Recommended references
The closest existing public primitive is Sentry's Apache-2.0 prompt-optimizer skill. Promptfoo's optimizer supplies the baseline → candidate → eval → held-out-selection loop. Anthropic's agent-development guidance provides a useful system-prompt structure template. See references/SOURCES.md for the source record.
Output
Produce:
Current prompt auditSuccess criteriaEval manifestOptimized promptBehavioral diffBattle result (only after real evals)Token/cost deltaRegressions / residual risksShare card
Never publish synthetic/demo scores as real benchmark results.
1---2name: system-prompt-doctor3description: Audit, optimize, and benchmark a user-owned system prompt, developer prompt, CLAUDE.md, AGENTS.md, or agent instruction file against representative tasks. Use when the user wants to improve an agent's core instructions, make an agent more reliable, reduce instruction bloat, fix tool-use or ask-vs-act behavior, or prove whether a revised system prompt is actually better.4---56# System Prompt Doctor78Do not optimize a system prompt by taste. Optimize it against behavior.910## Scope1112This skill works on prompts/instruction files the user owns or can provide, including:1314- system/developer prompts15- `CLAUDE.md`16- `AGENTS.md`17- reusable agent instruction files18- agent persona/policy prompts1920Do not claim access to hidden platform system prompts that the runtime does not expose.2122## Workflow23241. **Capture the contract.** Record the target model, prompt surface, task family, tools, hard constraints, output shape, failure cases, latency/cost limits, and what must not change.252. **Build the eval before editing.** Prefer real historical failures and corrections. Otherwise create 8 to 20 representative cases plus hard negatives and at least a 20% holdout split.263. **Baseline the current prompt.** Preserve prompt hash, model/version, outputs, scores, token usage, tool behavior, and failures.274. **Audit instruction architecture.** Find contradictions, duplicate rules, vague defaults, missing completion criteria, tool-policy gaps, bad ask-vs-act calibration, stale examples, and context that should live outside the system prompt.285. **Generate a small candidate beam.** Always include:29 - minimal-diff repair30 - structure-first rewrite31 - compact/progressive-disclosure variant32 Add a provider-specific adapter only when eval failures justify it.336. **Optimize on train/validation only.** If Promptfoo is available, use `promptfoo optimize` with a validation split. Never tune on the final holdout set.347. **Red-team important boundaries.** Test conflicting instructions, ambiguous tasks, tool misuse, prompt injection exposure, refusal/escalation boundaries, and over/under-triggering where applicable.358. **Battle the winner against the current prompt.** Same model, same task manifest, same tool permissions. Report quality lift and token/cost/latency change.369. **Prefer the smallest prompt that preserves measured behavior.** A larger prompt is not a better prompt.3710. **Return evidence, not vibes.** Include the optimized prompt, diff, eval set, before/after scores, regressions, token delta, residual risks, and a shareable result card.3839## Recommended references4041The closest existing public primitive is Sentry's Apache-2.0 `prompt-optimizer` skill. Promptfoo's optimizer supplies the baseline → candidate → eval → held-out-selection loop. Anthropic's agent-development guidance provides a useful system-prompt structure template. See `references/SOURCES.md` for the source record.4243## Output4445Produce:46471. `Current prompt audit`482. `Success criteria`493. `Eval manifest`504. `Optimized prompt`515. `Behavioral diff`526. `Battle result` (only after real evals)537. `Token/cost delta`548. `Regressions / residual risks`559. `Share card`5657Never publish synthetic/demo scores as real benchmark results.
getedgehq