/evaluate-agent — Static linter for agents and skills
Phase 1 of the agents-team evaluator. Deterministic. Cheap. CI-safe. Every rule cites a source.
Quick start
# Lint one file; writes JSON + Markdown to .claude/agent-quality/<name>.{json,md}
/evaluate-agent .claude/agents/code-reviewer.md
# Print Markdown report to stdout (don't write files)
/evaluate-agent --stdout .claude/agents/code-reviewer.md
# CI mode: exit nonzero on `revise` or `reject` verdicts
/evaluate-agent --ci .claude/agents/code-reviewer.md
# Strict mode: promote suggestion-severity findings to warnings
/evaluate-agent --strict path/to/SKILL.md
# Deep mode: run the LLM-as-judge after the static linter (Phase 2).
# Requires `pip install anthropic` and ANTHROPIC_API_KEY.
# Without those, judge dimensions emit a `judge.skipped_no_sdk` suggestion.
/evaluate-agent --deep .claude/agents/code-reviewer.md
What it checks
Six dimensions, with role-aware weights (agents weight description higher than tool_hygiene; skills the opposite):
| Dimension |
Sample rules |
frontmatter |
name valid + present, description present, name not reserved, model not retired |
description |
length 80–400 (agents), no first/second person, has "use when"/"use proactively" trigger |
tool_hygiene |
tools set explicitly (not inherited), no write tools on reviewer roles, Bash safeguards |
model_fit |
Opus on read-only roles → warn; Haiku on architecture/orchestrator roles → warn |
body_structure |
Agents: "When invoked" + "Constraints" sections. Skills: body <500 lines, TOC if >100 lines |
anti_patterns |
Prompt-injection smell, hardcoded absolute paths, body/tool contradictions |
Every rule has a stable ID (e.g. tool_hygiene.write_on_review_role) and a citation (Anthropic spec or published prior art).
Output
- JSON: conforms to
lib/eval/schema/v1.json. Fields: agent, path, kind, overall.{score,grade,verdict}, dimensions.{frontmatter,description,…}.{score,weight,findings}, findings[], produced_by, produced_at, schema_version.
- Markdown: rendered via
lib/eval/render.sh. Verdict line, dimension table, findings grouped by severity (critical / warning / suggestion).
- Default paths:
.claude/agent-quality/<agent>.json + sibling .md. The skill writes these unless --stdout is passed.
Exit codes
| Code |
Verdict |
Meaning |
0 |
ship |
No critical findings; score ≥ 75 |
1 |
revise |
No critical findings; score 50–74 |
2 |
reject |
At least one critical finding, OR score < 50 |
64 |
— |
Bad CLI args |
66 |
— |
Target file not found |
--ci carries verdict-based exit codes through; without it, the skill exits 0 (so a one-shot /evaluate-agent doesn't kill an interactive session).
When to use this skill
- Before merging any change to
.claude/agents/ or any SKILL.md.
- In CI on every PR that touches
.claude/.
- When the orchestrator under-delegates to a specialist — usually a description-clarity issue.
- As a sweep — run across an entire team's roster to spot weak links.
Anti-patterns
- Treating the linter's score as authoritative without reading the evidence.
- Ignoring
warning-severity findings — they are leading indicators of misbehavior.
- Auto-applying suggested fixes without reviewing the diff.
- Adding rules that aren't backed by Anthropic spec or published prior art (those go in
experimental/ only).
Implementation
lib/eval/lint.py — Python 3, stdlib only. Parses frontmatter without pyyaml.lib/eval/render.sh — jq-based Markdown renderer.lib/eval/lint.sh — entry point combining both.lib/eval/schema/v1.json — JSON Schema for the report shape.
References
Source: fadymondy/agents-team — distributed by TomeVault.
1---2name: evaluate-agent3description: Statically lint a Claude Code agent or skill file against a citation-backed rule set (frontmatter, description, tool hygiene, model fit, body structure, anti-patterns). Use proactively before merging any change to .claude/agents/ or any SKILL.md. Use when an agent is misbehaving or the team adds a new specialist. Use when this capability is needed.4---56# /evaluate-agent — Static linter for agents and skills78Phase 1 of the agents-team evaluator. Deterministic. Cheap. CI-safe. Every rule cites a source.910## Quick start1112```bash13# Lint one file; writes JSON + Markdown to .claude/agent-quality/<name>.{json,md}14/evaluate-agent .claude/agents/code-reviewer.md1516# Print Markdown report to stdout (don't write files)17/evaluate-agent --stdout .claude/agents/code-reviewer.md1819# CI mode: exit nonzero on `revise` or `reject` verdicts20/evaluate-agent --ci .claude/agents/code-reviewer.md2122# Strict mode: promote suggestion-severity findings to warnings23/evaluate-agent --strict path/to/SKILL.md2425# Deep mode: run the LLM-as-judge after the static linter (Phase 2).26# Requires `pip install anthropic` and ANTHROPIC_API_KEY.27# Without those, judge dimensions emit a `judge.skipped_no_sdk` suggestion.28/evaluate-agent --deep .claude/agents/code-reviewer.md29```3031## What it checks3233Six dimensions, with role-aware weights (agents weight description higher than `tool_hygiene`; skills the opposite):3435| Dimension | Sample rules |36|-----------------|-------------------------------------------------------------------------------------------------------|37| `frontmatter` | `name` valid + present, `description` present, `name` not reserved, model not retired |38| `description` | length 80–400 (agents), no first/second person, has "use when"/"use proactively" trigger |39| `tool_hygiene` | `tools` set explicitly (not inherited), no write tools on reviewer roles, Bash safeguards |40| `model_fit` | Opus on read-only roles → warn; Haiku on architecture/orchestrator roles → warn |41| `body_structure`| Agents: "When invoked" + "Constraints" sections. Skills: body <500 lines, TOC if >100 lines |42| `anti_patterns` | Prompt-injection smell, hardcoded absolute paths, body/tool contradictions |4344Every rule has a stable ID (e.g. `tool_hygiene.write_on_review_role`) and a citation (Anthropic spec or published prior art).4546## Output4748- **JSON**: conforms to `lib/eval/schema/v1.json`. Fields: `agent`, `path`, `kind`, `overall.{score,grade,verdict}`, `dimensions.{frontmatter,description,…}.{score,weight,findings}`, `findings[]`, `produced_by`, `produced_at`, `schema_version`.49- **Markdown**: rendered via `lib/eval/render.sh`. Verdict line, dimension table, findings grouped by severity (critical / warning / suggestion).50- **Default paths**: `.claude/agent-quality/<agent>.json` + sibling `.md`. The skill writes these unless `--stdout` is passed.5152## Exit codes5354| Code | Verdict | Meaning |55|------|-----------|--------------------------------------------------------|56| `0` | `ship` | No critical findings; score ≥ 75 |57| `1` | `revise` | No critical findings; score 50–74 |58| `2` | `reject` | At least one critical finding, OR score < 50 |59| `64` | — | Bad CLI args |60| `66` | — | Target file not found |6162`--ci` carries verdict-based exit codes through; without it, the skill exits 0 (so a one-shot `/evaluate-agent` doesn't kill an interactive session).6364## When to use this skill6566- **Before merging** any change to `.claude/agents/` or any `SKILL.md`.67- **In CI** on every PR that touches `.claude/`.68- **When the orchestrator under-delegates** to a specialist — usually a description-clarity issue.69- **As a sweep** — run across an entire team's roster to spot weak links.7071## Anti-patterns7273- Treating the linter's score as authoritative without reading the evidence.74- Ignoring `warning`-severity findings — they are leading indicators of misbehavior.75- Auto-applying suggested fixes without reviewing the diff.76- Adding rules that aren't backed by Anthropic spec or published prior art (those go in `experimental/` only).7778## Implementation7980- `lib/eval/lint.py` — Python 3, stdlib only. Parses frontmatter without pyyaml.81- `lib/eval/render.sh` — jq-based Markdown renderer.82- `lib/eval/lint.sh` — entry point combining both.83- `lib/eval/schema/v1.json` — JSON Schema for the report shape.8485## References8687- Anthropic Sub-agents — https://code.claude.com/docs/en/sub-agents88- Anthropic Skill best practices — https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices89- Anthropic *Demystifying evals for AI agents* — https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents9091---92> Source: [fadymondy/agents-team](https://github.com/fadymondy/agents-team) — distributed by [TomeVault](https://tomevault.io).93<!-- tomevault:4.0:skill_md:2026-05-23 -->
tomevault-io