# Evaluate Plugin Or Skill

> Evaluate Plugin or Skill

- Skill: `viktorbezdek/evaluate-plugin-or-skill` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add viktorbezdek/evaluate-plugin-or-skill`
- Raw SKILL.md: https://api.skillmd.com/api/skills/viktorbezdek/evaluate-plugin-or-skill/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: viktorbezdek (https://skillmd.com/u/viktorbezdek)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/viktorbezdek/evaluate-plugin-or-skill

---


# Evaluate Plugin or Skill

> **A plugin is ready to share when its structure is correct, its activation is reliable, its content is disciplined, its documentation answers the three install-time questions, and its anti-patterns are absent.** This workflow checks all five dimensions against the same standards skill-foundry and plugin-dev enforce — and returns a single verdict instead of a wall of unsorted feedback.

## When to use this workflow

- You finished a plugin or skill and want to know if it is shippable
- You are reviewing a third-party plugin before installing or recommending it
- You drafted a SKILL.md and want to know whether the description will actually trigger
- You want a structured second opinion before publishing to a marketplace
- You need feedback that is ranked by priority — not a flat list of nitpicks

## When NOT to use this workflow

- **Building a new plugin from scratch** → use `build-a-plugin`
- **Extending or fixing an existing plugin** → use `update-a-plugin`
- **Authoring a single new SKILL.md** → use `write-your-own-skill`
- **Measuring runtime agent quality (eval suites for production agents)** → use `evaluate-and-improve-agent`
- **Fixing a single broken validation check** → run `plugin-validation` directly

## Prerequisites

Install the underlying plugins so the audit has real depth:

```
/plugin install plugin-validation@skillstack
/plugin install plugin-evaluation@skillstack
/plugin install skill-foundry@skillstack
/plugin install plugin-documenter@skillstack
```

Without these installed, the workflow still runs as a checklist — but the depth and the live eval harness disappear.

## Core principle

**A verdict beats a list.** Ten unranked findings overwhelm; one verdict with three ranked actions ships. The audit sorts every finding by severity (blocker, gap, polish) and folds them into one of three outcomes: SHIP, IMPROVE, or REWORK. Authors get a clear next step instead of a homework pile.

Secondary principle: **measure activation, do not assume it.** Most plugins fail in the wild because their description does not trigger reliably — not because their content is bad. Trigger evals are non-negotiable; if none exist, this workflow generates seed evals and runs them before issuing a verdict.

Third principle: **eat the dog food.** This skill audits other skills against skill-foundry's anti-patterns. It must itself avoid those anti-patterns — lean SKILL.md, references on demand, decision tables over prose.

## The phases

### Phase 1 — Intake and classification

Detect what the user provided. Inputs come in three forms: a GitHub URL, a local plugin directory, or a single SKILL.md path.

Classify the artifact:

- **Plugin** — has `.claude-plugin/plugin.json` and a `skills/` (or `hooks/`, `mcp-servers/`, `agents/`, `commands/`) directory
- **Single skill** — has a single `SKILL.md` with frontmatter, no plugin manifest
- **Ambiguous** — neither — ask the user before proceeding

Inventory components without deep-reading: count skills, hooks, MCP servers, subagents, commands. Note size of each SKILL.md in lines. This inventory drives which phases are relevant.

Output: a one-paragraph summary of what is being evaluated and which phases will run.

### Phase 2 — Structural validation (plugin-validation)

Load the `plugin-validation` skill.

Check, for plugins:

- `plugin.json` has `name`, `version`, `description`, `author`, `license`
- Each `skills/<name>/SKILL.md` exists and has YAML frontmatter
- `name` in frontmatter equals the directory name
- All referenced files (`references/*.md`, `scripts/*.py`, etc.) exist on disk
- Plugin name and version follow conventions of the target marketplace
- No SKILL.md exceeds 500 lines (skill-foundry hard limit)

For single skills, run a subset: frontmatter validity, name field present, referenced files exist.

If `plugin-dev/scripts/validate_plugins.py` is available, run it for objective output. Otherwise, run the checklist by hand.

Severity classification:

- **Blocker** — manifest missing, name mismatch, frontmatter invalid, broken reference
- **Gap** — missing optional but expected fields (keywords, repository URL)
- **Polish** — convention drift (kebab-case violations, unusual naming)

Output: structural score (blocker count, gap count, polish count).

### Phase 3 — Skill content audit (skill-foundry)

Load the `skill-foundry` skill.

For each SKILL.md, audit against the five canonical anti-patterns:

| Anti-pattern | Symptom | Fix |
|---|---|---|
| Reference Illusion | SKILL.md cites references that do not exist or are never loaded | Inline the content or remove the citation |
| Description Soup | Frontmatter description is generic prose without trigger keywords or NOT clauses | Rewrite with concrete trigger phrases + explicit NOT clause |
| Template Theater | Sections present because the template said so, not because the skill needs them | Delete unused sections |
| Everything Skill | Tries to handle multiple unrelated capabilities | Split into multiple skills, one per capability |
| Orphaned Sections | Sections never referenced from anywhere — readers cannot find them | Either link to them or delete them |

Also check the structural quality bar:

- Description has trigger phrases AND a NOT clause (target: ≥3 trigger keywords, ≥2 NOT cases)
- SKILL.md ≤500 lines (≤200 preferred for activation skills)
- Decision tables, not paragraphs, for routing logic
- Progressive disclosure: heavy content lives in `references/`, not inline

Severity classification:

- **Blocker** — Everything Skill, Reference Illusion, description with no trigger keywords
- **Gap** — Description Soup, no NOT clause, missing decision tables
- **Polish** — Template Theater, mild Orphaned Sections

Output: per-skill anti-pattern findings with severity.

### Phase 4 — Activation evaluation (plugin-evaluation)

Load the `plugin-evaluation` skill.

If `skills/<skill>/evals/trigger-evals.json` exists, run the harness:

```bash
python3 plugin-dev/scripts/run_eval.py \
  --plugin-dir <path> --skill <name> --mode trigger
```

If no trigger evals exist, this is itself a finding — but do not stop. Generate seed evals: 5 positive cases (varied phrasings of when the skill should activate) and 5 near-miss negatives (queries that look related but should NOT trigger). Run those.

Targets:

- Positive trigger rate ≥ 90%
- Negative no-trigger rate ≥ 95%

Severity classification:

- **Blocker** — positive trigger rate < 60%, or negative no-trigger rate < 80% (skill collides with other skills)
- **Gap** — positive 60–89%, or negative 80–94%
- **Polish** — positive 90–94%, or negative 95–98%
- **No finding** — positive ≥ 95% AND negative ≥ 99%

Output: trigger-rate table per skill.

### Phase 5 — Output evaluation (plugin-evaluation, optional)

If `skills/<skill>/evals/evals.json` exists, run output evals to check whether the skill produces correct results when activated. Otherwise, mark as a gap and proceed — output evals are nice-to-have for shipping, mandatory for production-critical plugins.

Severity classification:

- **Blocker** — output pass rate < 70% on existing evals
- **Gap** — no output evals exist for a plugin marketed as production-ready
- **Polish** — output pass rate 70–89%

Output: output-eval pass rate (or absence note).

### Phase 6 — Documentation and shareability

The best plugin documentation answers three questions in install order: **what problem does this solve, how do I install it, what can I do with it.** Check whether the README answers these in that order.

Checklist:

- README opens with a problem statement, not "this plugin provides…"
- Installation block exists and works as written
- At least three concrete example prompts, scenarios, or invocations
- License and attribution footer present
- CHANGELOG.md exists with at least one version entry
- Plugin manifest keywords are searchable (matches actual use cases, not generic words like "tools" or "ai")

Severity classification:

- **Blocker** — no README, broken install instructions, no license
- **Gap** — README leads with capabilities instead of the problem; no example prompts; no CHANGELOG
- **Polish** — keyword stuffing, missing screenshots for visual plugins

Output: documentation gap list.

### Phase 7 — Verdict synthesis

Score each dimension 0–3:

| Score | Meaning |
|---|---|
| 3 | Zero blockers, zero gaps |
| 2 | Zero blockers, ≤2 gaps |
| 1 | Zero blockers, 3+ gaps OR 1 blocker |
| 0 | 2+ blockers OR critical activation failure |

Aggregate the five scores (structure, content, activation, output, documentation) into one verdict:

- **SHIP** — every dimension ≥ 2 AND total ≥ 12. Worth sharing as-is or with cosmetic polish.
- **IMPROVE** — one or two dimensions at 1, none at 0. Concrete, addressable fixes. List the top three ranked actions; the author can ship after addressing them.
- **REWORK** — any dimension at 0, OR three+ dimensions at 1, OR positive trigger rate < 60%. The plugin needs a structural revisit; suggest restarting from `build-a-plugin` (for plugins) or `write-your-own-skill` (for single skills).

Output: a one-screen verdict report with:

1. Verdict (SHIP / IMPROVE / REWORK) — single word, top of the report
2. Dimension scores table (5 rows, score + one-sentence reason each)
3. Top three ranked actions (only the ones that move the verdict — skip polish-only items unless the verdict is already SHIP)
4. If REWORK: which workflow to restart from and why

## Gates and failure modes

| Gate | Failure mode | Recovery |
|---|---|---|
| Phase 1 classification ambiguous | Cannot tell what is being audited | Ask the user before running phases 2–7 |
| Phase 2 blocker found | Plugin will not load | Stop — return a REWORK verdict with structural fixes only |
| Phase 4 trigger evals missing AND skill description is the only signal | Cannot measure activation | Generate seed evals before issuing verdict; do not skip Phase 4 |
| Phase 7 produces SHIP without trigger eval data | Verdict is unfounded | Downgrade to IMPROVE pending eval evidence |
| Author asks "just tell me if it is good" | Tempting to skip phases | Run all phases — verdict is only credible if every dimension was checked |

The most common failure mode is **issuing SHIP without measuring activation.** If you cannot point to a trigger pass rate, you have not evaluated the plugin — you have read it.

## Output artifacts

- A verdict report (SHIP / IMPROVE / REWORK) with dimension scores and ranked actions
- Optional: generated `trigger-evals.json` seed file if none existed
- Optional: a one-page diff between the plugin's current state and shippable state, suitable for handing back to the author

## Related workflows and skills

- `build-a-plugin` — when REWORK is the verdict for a plugin, restart here
- `write-your-own-skill` — when REWORK is the verdict for a single skill
- `update-a-plugin` — when IMPROVE is the verdict and the author wants a guided fix loop
- `plugin-validation`, `plugin-evaluation`, `skill-foundry`, `plugin-documenter` — the underlying skills this workflow composes

---

> *Workflow part of [skillstack-workflows](../../README.md) by [Viktor Bezdek](https://github.com/viktorbezdek) — licensed under MIT.*