Skill Review

Skill Review

One skill, two modes. Use --scope=all for an ecosystem-wide bulk scan; use --scope=<skill-name> for a thorough single-skill deep dive. Both modes produce a structured markdown report plus a JSON sidecar that skill-update can consume to generate fix plans.

When to Use

• Periodic health check across the skill ecosystem (--scope=all)
• After bulk edits or a new batch of skills lands (--scope=all)
• Reviewing a single skill before publishing (--scope=<name>)
• Investigating why one skill won't trigger or produces poor output (--scope=<name>)
• Quality-gating a skill before it enters the ecosystem (--scope=<name>)

Inputs

The skill takes a single positional argument:

• all — bulk scan across skills/**/SKILL.md
• <skill-name> — deep dive on skills/**/<skill-name>/SKILL.md

Optional follow-on context:

• Focus — narrow the run to specific checks ("just ownership", "just descriptions")
• Why-now — what prompted the review (e.g., "it never triggers", "outputs are wrong")

Process

Phase 0: Parse the Argument

1. Read the argument. If it equals all (case-insensitive), enter Mode A.
2. Otherwise treat the argument as a skill name. Glob for skills/**/<arg>/SKILL.md. If exactly one match, enter Mode B. If zero matches, return a list of fuzzy candidates and stop. If multiple matches, ask the user to disambiguate.

Mode A — Bulk Scan (--scope=all)

Optimize for speed. Score quickly, flag issues, move on. Use subagents to parallelize check categories where available.

A1. Discovery

1. Glob skills/**/SKILL.md
2. Parse frontmatter from each
3. Build a skill inventory table with columns: Skill, Category, Version, Description Length, Body Lines, Refs Count

A2. Bulk Checks

Run these check categories. Full checklist lives in references/audit-checklist.md.

• Frontmatter consistency — required fields present, name matches directory, version is valid semver, description starts with action verb and is ≤200 chars
• Spec compliance (FAIL) — no < or > in frontmatter field values (security rule); description ≤1024 chars (hard ceiling)
• Spec compliance (WARN) — reserved-prefix name (claude-* / anthropic-*). Acceptable as a documented exception when the skill targets the corresponding Anthropic product. Verify the documented exception exists in the skill body.
• Tool field naming — allowed-tools (hyphen) is canonical; allowed_tools (underscore) accepted as deprecated alias (warn but don't fail)
• Ownership conflict detection — collect every owns.directories and owns.patterns across agent roles. Flag overlaps. Validate against the v1.1 resolved conflicts table in frontmatter-spec.md
• Description quality scoring — has action verb, ≥3 trigger contexts, keyword variants, states exclusions if ambiguous, estimated "pushiness" (low/medium/high)
• Progressive disclosure — SKILL.md body line + word count, references linked from body, body >5,000 words OR >500 lines flagged (soft warning), references >300 lines without TOC flagged
• Cross-skill consistency — composes_with and spawned_by point to real skills, no circular composes_with chains, no orphan reference files
• Coverage gaps — compare inventory against docs/architecture.md, CLAUDE.md, and the orchestrator's File Ownership Map; flag roles or workflows referenced but not implemented

A3. Triage

Highlight the top 3–5 most impactful issues. Suggest which skills warrant a single-skill follow-up (re-run with --scope=<name>). Note which issues skill-update can address directly.

Mode B — Deep Dive (--scope=<skill-name>)

Take time. Read every file in the skill's directory tree. Score each rubric dimension 1–5 with specific evidence.

B1. Structural Analysis

Read SKILL.md and every file in references/. Score these dimensions against references/deep-review-rubric.md:

1. Frontmatter compliance — required fields, types, semver, optional fields used appropriately
2. Description quality — action verb, trigger contexts, keyword variants, length, "pushiness"
3. Progressive disclosure — body ≤5,000 words (soft warning past 500 lines), references used appropriately, clear pointers
4. Instruction clarity — imperative voice, logical flow, no ambiguity, explains "why" not just "what"
5. Coordination — ownership declarations, composes_with accuracy, no overlaps
6. Completeness — referenced files exist, no dead links, validation checklists where needed
7. Anti-patterns — see the anti-pattern checklist in references/audit-checklist.md, including the model-adaptation checks (reasoning-extraction refusal risk, prior-model over-prescription) that flag a skill written for a superseded model, and the writing-craft checks (leading-word anchor, no-op-sentence deletion, positive-prompting lint). When a skill scores poorly there or "worked before and got worse," hand off to the model-adaptation skill's migration audit.

B2. Live Trigger Testing

First, check the skill carries a triggering test — at least one example phrasing that must activate it (ideally one near-miss that must not). Flag its absence as an issue. Then verify it actually holds.

If /skill-creator is available, use its eval infrastructure to test whether the skill actually triggers:

1. Generate 3–5 realistic prompts that should trigger this skill
2. Generate 2–3 near-miss prompts that should not trigger it
3. Run trigger evaluation via skill-creator's description optimization tooling
4. Report should-trigger hit rate and false-positive rate. List any problem triggers.

If /skill-creator is unavailable, skip this phase and note it in the report.

B3. Output Quality Sampling

For skills that produce structured output (reports, files, configs):

1. Pick 2 representative test prompts
2. Run them through the skill (or skill-creator's test harness)
3. Compare actual output against the skill's stated format
4. Note gaps between promised and actual output

Optionally (recommended for non-trivial skills), run one prompt with and without the skill and compare — if the skill doesn't measurably improve the output, flag that it may not earn its context cost.

Phase 4: Report

Both modes write the report following references/report-format.md. Output two artifacts:

• skill-review-report.md — structured markdown for humans
• skill-review-report.json — sidecar consumable by skill-update

In Mode A, save to the repo root or a user-specified path. In Mode B, save to {skill-path}/skill-review-report.{md,json}.

Output Handoff

The report is designed to feed directly into skill-update, which consumes the JSON sidecar and produces a prioritized edit plan. After writing the report, tell the user:

"Review complete. [X] issues found. Feed the JSON sidecar into /skill-update to generate a fix plan, or re-run me with --scope=<skill-name> for a deeper look at [specific skills]."

Guidelines

• Be constructive — every issue gets a concrete suggestion
• Score honestly but explain reasoning, especially for low scores. Don't nitpick style if the skill is functionally sound
• Weight trigger testing heavily — a skill that doesn't trigger is useless regardless of how well-written it is
• When recommending fixes, expect them applied one focused change at a time with re-validation between — don't bundle many unrelated edits into one pass
• In Mode A, don't read reference file contents unless checking for orphans or broken links. Frontmatter + body line count is enough for most checks
• In Mode B, if the user supplied context (e.g., "it never triggers"), lead with that complaint
• If scope is filtered in Mode A, still validate cross-skill references against the full inventory
• Use subagents to parallelize independent check categories when available

Data source

Before recommending changes, consult real usage signal: run scripts/skill-health.sh report --json (P2-C skill-health telemetry). It returns per-skill total invocations, 7-day/30-day success rates, a declining flag, and version drift (recorded vs current SKILL.md frontmatter version). The math is deterministic in code, never a prompt. Treat the signal as coarse and best-effort: the emitter can only attribute a skill on a Skill-tool call and records outcome:unknown, so a skill reported as no-data means "unobserved", not "broken". Use it to prioritize which skills to deep-review and to back up declining-quality verdicts — never as the sole basis for a low score. See contracts/installer/skill-health.md.

Reference Files

• references/audit-checklist.md — checklist of every per-skill and ecosystem-level check, plus the anti-pattern list
• references/deep-review-rubric.md — 1–5 scoring criteria for each of the seven deep-review dimensions, with verdict thresholds (SHIP / NEEDS WORK / MAJOR REWORK)
• references/report-format.md — exact shape of the markdown report and JSON sidecar