How do I install the Openkata Eval Runner skill?

Run npx skillmds@latest add tovrleaf/openkata-eval-runner in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Openkata Eval Runner safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Openkata Eval Runner?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Openkata Eval Runner free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Openkata Eval Runner?

tovrleaf (@tovrleaf) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Openkata Eval Runner by tovrleaf — Agent Skill

Name: Openkata Eval Runner
Author: tovrleaf

File contents

Eval Runner

Run eval scenarios locally and iterate skill fixes until the 95% threshold is met.

Tool Split

Tool	Use for
`./bin/openkata-eval`	Scoring (runs locally, same model as users)
`tessl skill review --optimize`	Instruction polish after passing
`tessl eval run`	NOT used — model mismatch makes results unreliable

Workflow

Identify the skill — Determine which skill from the user's request. Resolve path (skills//).
Check evals exist — Verify skills/<name>/evals/ has scenario directories. If missing, activate the create-evals skill first.
Build the runner — make eval-local builds the binary automatically if missing.
Run evals — Execute:
```
./bin/openkata-eval skills/<name>
```
The overall average must be 95% or above.
Report — If passing (95%+), report the score. If failing, report which scenarios/criteria failed with reasons, and suggest specific SKILL.md fixes.
Fix and retry — If the user confirms fixes, apply them to SKILL.md, then re-run. Iterate until 95%+.
Optimize — Once passing locally, run:
```
tessl skill review --optimize skills/<name>
```
Apply any worthwhile suggestions, re-run local evals to confirm no regression.
Commit — Stage changes and commit.

Single Scenario Debugging

To iterate on one failing scenario without running all:

./bin/openkata-eval skills/<name>/evals/scenario-X

Single scenario mode always exits 0 (debug tool).

Boundaries

DOES run evals, suggest and apply SKILL.md fixes
DOES run tessl optimize for polish after passing
Does NOT publish, tag, or release
Does NOT use tessl eval run for scoring

Gotchas

Only distributable skills (skills/) have evals
Conversational skills need "sandbox": false in scenario.json (no Docker required)
Skills that use tools need "sandbox": true (default) and Docker running

---name: openkata-eval-runnerdescription: Run eval scenarios for an OpenKata skill locally and iterate fixes until 95% threshold. Use when the user says "run evals", "check evals", "eval this skill", or when validating skill quality.---# Eval RunnerRun eval scenarios locally and iterate skill fixes untilthe 95% threshold is met.## Tool Split| Tool | Use for ||------|---------|| `./bin/openkata-eval` | Scoring (runs locally, same model as users) || `tessl skill review --optimize` | Instruction polish after passing || `tessl eval run` | NOT used — model mismatch makes results unreliable |## Workflow1. **Identify the skill** — Determine which skill from the user's request. Resolve path (skills/<name>/).2. **Check evals exist** — Verify `skills/<name>/evals/` has scenario directories. If missing, activate the `create-evals` skill first.3. **Build the runner** — `make eval-local` builds the binary automatically if missing.4. **Run evals** — Execute: ```bash ./bin/openkata-eval skills/<name> ``` The overall average must be 95% or above.5. **Report** — If passing (95%+), report the score. If failing, report which scenarios/criteria failed with reasons, and suggest specific SKILL.md fixes.6. **Fix and retry** — If the user confirms fixes, apply them to SKILL.md, then re-run. Iterate until 95%+.7. **Optimize** — Once passing locally, run: ```bash tessl skill review --optimize skills/<name> ``` Apply any worthwhile suggestions, re-run local evals to confirm no regression.8. **Commit** — Stage changes and commit.## Single Scenario DebuggingTo iterate on one failing scenario without running all:```bash./bin/openkata-eval skills/<name>/evals/scenario-X```Single scenario mode always exits 0 (debug tool).## Boundaries- DOES run evals, suggest and apply SKILL.md fixes- DOES run tessl optimize for polish after passing- Does NOT publish, tag, or release- Does NOT use tessl eval run for scoring## Gotchas- Only distributable skills (`skills/`) have evals- Conversational skills need `"sandbox": false` in scenario.json (no Docker required)- Skills that use tools need `"sandbox": true` (default) and Docker running

Frequently asked questions

Openkata Eval Runner