Improve Skill

Improve Skill

Analyze and improve an existing Claude Code skill using findings from the SkillsBench research paper (arXiv:2602.12670v1), which evaluated 7,308 agent trajectories and identified what makes skills effective.

Prerequisites

• Path to an existing skill directory containing a SKILL.md file
• If no path is provided, ask the user which skill to improve

Improvement Workflow

Step 1: Load and Audit the Skill

Read the target skill's SKILL.md and all files in its references/, examples/, and scripts/ directories. Produce a structured audit against the five evaluation dimensions below.

Step 2: Score the Five Dimensions

Rate each dimension 1-5 and note specific issues:

Dimension 1 — Procedural Density Measures whether content teaches how to do things versus what things are.

• 5: Nearly all content is step-by-step procedures with decision points
• 3: Mix of procedural and reference/factual content
• 1: Reads like API documentation or a reference manual

Count the ratio of procedural sentences ("To X, do Y", "Run Z", "Configure W by...") to factual/descriptive sentences ("X is a...", "The Y property contains...", "There are three types of..."). Target: >70% procedural in SKILL.md body.

Dimension 2 — Conciseness Measures whether SKILL.md body stays in the optimal token range.

• 5: Body is 845-1,165 tokens (~1,200-1,800 words), detailed content in references/
• 3: Body is 1,500-2,500 tokens, some content could move to references/
• 1: Body exceeds 2,500 tokens, comprehensive documentation hurting effectiveness

Estimate token count of the SKILL.md body (excluding frontmatter). Comprehensive skills (-2.9pp) perform worse than no skill at all. Detailed (1,165 tokens, +18.8pp) and compact (845 tokens, +17.1pp) are optimal.

Dimension 3 — Working Examples Measures whether the skill provides concrete, copy-pasteable code for its procedures.

• 5: Every major procedure has a complete working example
• 3: Some procedures have examples, others are abstract
• 1: No working examples, or examples are fragments/pseudocode

The research found "stepwise guidance with at least one working example" is essential. Code templates and reference implementations matter more than documentation volume.

Dimension 4 — Signal-to-Noise Ratio Measures whether content focuses on non-obvious, domain-specific knowledge versus standard practices models already know.

• 5: All content addresses specialized knowledge or non-obvious procedures
• 3: Mix of specialized and commonly-known content
• 1: Mostly standard practices (basic patterns, common conventions)

Software engineering skills showed only +4.5pp improvement in SkillsBench because models already know standard programming. Content teaching common practices adds noise without value.

Dimension 5 — Trigger Quality Measures whether the frontmatter description reliably activates the skill for intended use cases.

• 5: Third-person, 5+ specific trigger phrases covering all use cases
• 3: Some trigger phrases but missing scenarios
• 1: Vague description, wrong person, or missing trigger phrases

Step 3: Identify Improvements

Based on the audit scores, identify specific improvements. Prioritize dimensions scoring 1-3. Apply these research-backed principles:

Principle A — Extract to References (Conciseness) Move factual content, detailed schemas, comprehensive lists, and API documentation from SKILL.md to references/ files. Keep only procedural steps and essential context in SKILL.md.

Before: One large SKILL.md with everything. After: Lean SKILL.md pointing to references/detailed-guide.md, references/api-reference.md.

Principle B — Rewrite Factual as Procedural (Procedural Density) Transform "X is a Y that does Z" into "To accomplish Z, use X by doing..."

Before: "The $() function returns a collection object with filter, add, and each methods." After: "To query model elements, call $('element-type'). Chain .filter() to narrow results and .each() to iterate."

Principle C — Add Working Examples (Working Examples) For each major procedure that lacks a concrete example, add a complete, copy-pasteable code block showing the procedure in context.

Principle D — Remove Common Knowledge (Signal-to-Noise) Delete content teaching standard practices. Ask: "Would a competent developer using Claude already know this without the skill?" If yes, remove it.

Principle E — Strengthen Triggers (Trigger Quality) Add specific phrases users would say. Use third person. Cover edge-case phrasings.

Principle F — Eliminate Conflicting Guidance When multiple approaches exist for the same task, designate one as the recommended default. Mention alternatives only with explicit decision criteria for when to deviate.

Principle G — Verify No Negative Value Content The research found 16/84 tasks where skills hurt performance. Review for content that:

• Contradicts correct model pretraining knowledge
• Over-specifies solutions, preventing flexible problem-solving
• Adds complexity without procedural value

Remove or restructure such content.

Step 4: Present Findings and Propose Changes

Present the audit results to the user as a summary table:

| Dimension           | Score | Key Issues                    |
|---------------------|-------|-------------------------------|
| Procedural Density  | X/5   | ...                           |
| Conciseness         | X/5   | ...                           |
| Working Examples    | X/5   | ...                           |
| Signal-to-Noise     | X/5   | ...                           |
| Trigger Quality     | X/5   | ...                           |

Then list proposed changes grouped by principle (A-G), showing what will change and why. Wait for user approval before making changes.

Step 5: Apply Approved Changes

After user approval, apply the changes:

1. Edit SKILL.md — restructure body, improve frontmatter
2. Create or update references/ files for extracted content
3. Add working examples where needed
4. Remove low-value content
5. Verify all referenced files exist

Step 6: Post-Improvement Validation

After applying changes, verify:

• SKILL.md body is under 1,200 tokens (target: 845-1,165)
• Frontmatter uses third person with 5+ trigger phrases
• Body uses imperative/infinitive form throughout
• Every major procedure has a working example
• No factual-only sections remain in SKILL.md (moved to references/)
• All references/ files mentioned in SKILL.md actually exist
• No content teaches standard practices models already know
• No conflicting guidance without decision criteria

Key Research Numbers

Quick reference for the most actionable SkillsBench findings:

Reference Files

For detailed research findings, anti-patterns, and evaluation criteria:

• references/skillsbench-findings.md — Complete SkillsBench research summary including anti-patterns, evaluation dimensions, and leakage audit criteria