How do I install the Evals skill?

Run npx skillmds@latest add adriancooney/evals in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Evals safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Evals?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Evals free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

adriancooney (@adriancooney) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Evals by adriancooney — Agent Skill

Name: Evals
Author: adriancooney

File contents

Eval Skill

Run and create evals for testing agent behavior.

Discovering Evals

Evals are markdown files matching *.eval.md. Use glob to find them:

**/*.eval.md

A common pattern is to collect evals in an evals/ directory.

Eval Structure

An eval file contains a prompt and an expectation:

# Eval Title

<prompt>
Instructions for the agent to execute.
</prompt>

<expectation>
Success criteria - describe what must be true for the eval to pass.
</expectation>

Running an Eval

Read the eval file
Extract the <prompt> content
Spawn a subagent with the prompt (runs in current working directory with shared state)
The subagent evaluates its own result against the <expectation> using LLM judgment
Subagent outputs SUCCESS or FAIL with reasoning

When running multiple evals, spawn all subagents in parallel. Report aggregate results at the end.

Always end output with exactly one of these lines for CI parsing:

eval result: pass — all evals passed
eval result: fail — one or more evals failed

Subagent Instructions

IMPORTANT: The subagent must only test and observe. It must NOT attempt to fix, modify, or change anything to make the expectation pass. The subagent executes the prompt, observes the outcome, and reports whether the expectation was met. If the expectation fails, report FAIL — do not try to make it pass.

Commands

Run a single eval:

/eval run <path-to-eval.eval.md>

Run all evals:

/eval run-all

Creating an Eval

Gather from the user:

Context - The process or flow to evaluate
Expectation - Success criteria in natural language

/eval create <name>

Write the eval to <name>.eval.md in the current directory.

Isolation

When creating an eval, try to make it self-contained and reproducible. This isn't critical, but helps:

Try to avoid hardcoded paths — prefer relative paths or have the prompt create its own working directory rather than encoding specific temp directories or absolute paths.
Try to avoid external state — if the process relied on existing files or services, consider whether the eval should create that state itself.
Parameterize where possible — replace specific values (ports, filenames, IDs) with generic ones the eval can generate.

If you see an opportunity to improve isolation but need clarification, ask the user.

---name: evalsdescription: Run and create evals for testing agent behavior. Use when the user wants to create or run an eval.---# Eval SkillRun and create evals for testing agent behavior.## Discovering EvalsEvals are markdown files matching `*.eval.md`. Use glob to find them:```**/*.eval.md```A common pattern is to collect evals in an `evals/` directory.## Eval StructureAn eval file contains a prompt and an expectation:```markdown# Eval Title<prompt>Instructions for the agent to execute.</prompt><expectation>Success criteria - describe what must be true for the eval to pass.</expectation>```## Running an Eval1. Read the eval file2. Extract the `<prompt>` content3. Spawn a subagent with the prompt (runs in current working directory with shared state)4. The subagent evaluates its own result against the `<expectation>` using LLM judgment5. Subagent outputs `SUCCESS` or `FAIL` with reasoningWhen running multiple evals, spawn all subagents in parallel. Report aggregate results at the end.**Always end output with exactly one of these lines for CI parsing:**- `eval result: pass` — all evals passed- `eval result: fail` — one or more evals failed### Subagent Instructions**IMPORTANT:** The subagent must only test and observe. It must NOT attempt to fix, modify, or change anything to make the expectation pass. The subagent executes the prompt, observes the outcome, and reports whether the expectation was met. If the expectation fails, report `FAIL` — do not try to make it pass.### CommandsRun a single eval:```/eval run <path-to-eval.eval.md>```Run all evals:```/eval run-all```## Creating an EvalGather from the user:1. **Context** - The process or flow to evaluate2. **Expectation** - Success criteria in natural language```/eval create <name>```Write the eval to `<name>.eval.md` in the current directory.### IsolationWhen creating an eval, try to make it self-contained and reproducible. This isn't critical, but helps:- **Try to avoid hardcoded paths** — prefer relative paths or have the prompt create its own working directory rather than encoding specific temp directories or absolute paths.- **Try to avoid external state** — if the process relied on existing files or services, consider whether the eval should create that state itself.- **Parameterize where possible** — replace specific values (ports, filenames, IDs) with generic ones the eval can generate.If you see an opportunity to improve isolation but need clarification, ask the user.

Frequently asked questions

Evals