How do I install the Evals skill?

Run npx skillmds@latest add carlheath/evals in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Evals safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Evals?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Evals free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

carlheath (@carlheath) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Evals by carlheath — Agent Skill

Name: Evals
Author: carlheath

Evals

Evaluation skill for AI output: apply measurable criteria to text/code/ research/policy. Supports four eval types: rubric-based (1-5 scale), pairwise (A vs B), checklist (binary), and red-team pass (uses thinking-redteam). Output: score + reasoning + improvement suggestions. [WHAT] Structured way to answer "is this good enough?". Uses domain rubrics: chronicle quality, memo rigor, academic precision, OSINT credibility. Eval is a feedback mechanism, not a gate. [WHEN] Use when: evaluate, score, judge, "is this good enough?", "rate this", quality check, compare A vs B. NOT for: fact-checking (use fact-check), academic peer review (use academic-opponent). [LANGUAGE] English and other languages; matches input.

carlheath 25a0f95 3.0 KB Updated Sep 17, 2026

File contents

Evals

Role: structured evaluation of AI output.

Four eval types

1. Rubric-based (1-5 scale)

Define 3-7 dimensions relevant to the task, score per dimension, justify per score.

2. Pairwise (A vs B)

Two versions of the same output. Winner per dimension. Total winner.

3. Checklist (binary)

List of must-haves. Yes/no per item. Threshold (e.g. 8/10) = pass.

4. Red-team pass

Delegate to the thinking-redteam skill. Pass if <3 weaknesses at Medium or above.

Domain rubrics

Chronicle (chronicle skill output)

Dimension	1	3	5
Voice fidelity	Generic AI	Approximates	Distinct author voice
Anti-AI patterns	Em-dashes + "not X. It is Y"	Few	None
Narrative arc	Linear summary	Attempts arc	Opening-build-landing
Concreteness	Abstract	Some examples	Specific numbers, names, dates
Closing	"In summary..."	OK	Action call, sharp

Memo (policy-brief output)

Dimension	1-5
Administrative tone	...
Evidence basis	...
Structural clarity	...
Recommendation sharpness	...
Recipient adaptation	...

Academic (academic output)

Dimension	1-5
IMRAD discipline	...
Source criticism	...
Argument chain	...
Terminological precision	...
Original contribution	...

OSINT (person-osint, osint-network output)

Dimension	1-5
Source credibility	...
Verification trail	...
Threat-actor perspective	...
Network depth	...
Security-risk handling	...

Output format

## Eval: [output name]

### Rubric: [chronicle / memo / academic / OSINT / custom]

| Dimension | Score | Reasoning |
|-----------|-------|-----------|
| [Dim 1] | 4/5 | [Specific observation] |
| [Dim 2] | 2/5 | [What's missing] |
| ... | ... | ... |

**Total: X / Y**

### Top 3 improvement points
1. [Concrete action]
2. [...]
3. [...]

### Pass/Fail
[With given threshold, or pass if >70%]

Auto-eval in ALGORITHM mode

In the ALGORITHM VERIFY phase: run the eval rubric automatically on produced output. If below threshold, iterate. The user sees the eval result, not just the final product.

Version history

v3.0 (2026-05-02): initial public release.

---name: evalsdescription: Evaluation skill for AI output: apply measurable criteria to text/code/ research/policy. Supports four eval types: rubric-based (1-5 scale), pairwise (A vs B), checklist (binary), and red-team pass (uses thinking-redteam). Output: score + reasoning + improvement suggestions. [WHAT] Structured way to answer "is this good enough?". Uses domain rubrics: chronicle quality, memo rigor, academic precision, OSINT credibility. Eval is a feedback mechanism, not a gate. [WHEN] Use when: evaluate, score, judge, "is this good enough?", "rate this", quality check, compare A vs B. NOT for: fact-checking (use fact-check), academic peer review (use academic-opponent). [LANGUAGE] English and other languages; matches input.---# Evals**Role:** structured evaluation of AI output.## Four eval types### 1. Rubric-based (1-5 scale)Define 3-7 dimensions relevant to the task, score per dimension, justify per score.### 2. Pairwise (A vs B)Two versions of the same output. Winner per dimension. Total winner.### 3. Checklist (binary)List of must-haves. Yes/no per item. Threshold (e.g. 8/10) = pass.### 4. Red-team passDelegate to the thinking-redteam skill. Pass if <3 weaknesses at Medium or above.## Domain rubrics### Chronicle (chronicle skill output)| Dimension | 1 | 3 | 5 ||-----------|---|---|---|| Voice fidelity | Generic AI | Approximates | Distinct author voice || Anti-AI patterns | Em-dashes + "not X. It is Y" | Few | None || Narrative arc | Linear summary | Attempts arc | Opening-build-landing || Concreteness | Abstract | Some examples | Specific numbers, names, dates || Closing | "In summary..." | OK | Action call, sharp |### Memo (policy-brief output)| Dimension | 1-5 ||-----------|-----|| Administrative tone | ... || Evidence basis | ... || Structural clarity | ... || Recommendation sharpness | ... || Recipient adaptation | ... |### Academic (academic output)| Dimension | 1-5 ||-----------|-----|| IMRAD discipline | ... || Source criticism | ... || Argument chain | ... || Terminological precision | ... || Original contribution | ... |### OSINT (person-osint, osint-network output)| Dimension | 1-5 ||-----------|-----|| Source credibility | ... || Verification trail | ... || Threat-actor perspective | ... || Network depth | ... || Security-risk handling | ... |## Output format```markdown## Eval: [output name]### Rubric: [chronicle / memo / academic / OSINT / custom]| Dimension | Score | Reasoning ||-----------|-------|-----------|| [Dim 1] | 4/5 | [Specific observation] || [Dim 2] | 2/5 | [What's missing] || ... | ... | ... |**Total: X / Y**### Top 3 improvement points1. [Concrete action]2. [...]3. [...]### Pass/Fail[With given threshold, or pass if >70%]```## Auto-eval in ALGORITHM modeIn the ALGORITHM VERIFY phase: run the eval rubric automatically on produced output. If below threshold, iterate. The user sees the eval result, not just the final product.## Version history- v3.0 (2026-05-02): initial public release.

Frequently asked questions

Evals

File contents

Evals

Four eval types

1. Rubric-based (1-5 scale)

2. Pairwise (A vs B)

3. Checklist (binary)

4. Red-team pass

Domain rubrics

Chronicle (chronicle skill output)

Memo (policy-brief output)

Academic (academic output)

OSINT (person-osint, osint-network output)

Output format

Auto-eval in ALGORITHM mode

Version history

Related skills

Frequently asked questions

How do I install the Evals skill?

What does the Evals skill do?

Is Evals safe to use?

Which AI agents work with Evals?

Is Evals free to use?

Who published Evals?