How do I install the Eval Backfill skill?

Run npx skillmds@latest add hoangsonww/eval-backfill in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Eval Backfill skill do?

Run the eval harness against every card under example-cards/ in parallel and persist scores to eval/store.sqlite. Use when the user asks to "backfill", "score everything", or "score all cards" for quality eval. It is listed under Coding & Dev Tools on SkillMD.

Is Eval Backfill safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Eval Backfill?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Eval Backfill free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Eval Backfill?

hoangsonww (@hoangsonww) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Eval Backfill by hoangsonww — Agent Skill

Name: Eval Backfill
Author: hoangsonww

File contents

Eval — Backfill All Cards

Score every card in example-cards/*-card.json and write rows to eval/store.sqlite.

How to invoke

make eval-backfill                       # stub judge, offline
make eval-backfill JUDGE=claude          # real Claude Haiku judge, ~$0.04 for 18 cards

Direct invocation supports --workers and --max-calls:

python3 eval/runner.py backfill --judge claude --workers 4 --max-calls 50

Behavior

Stub runs serially (instant).
Real backends parallelize via a ThreadPoolExecutor (default 4 workers). 18 cards usually finish in ~3-5 minutes.
A pre-call status line prints per card ([HH:MM:SS] YYYY-MM-DD: judging...). Full subprocess traces append to logs/eval-judge-YYYY-MM-DD.log so the user can tail -f while it runs.
Failures on individual cards are logged but do not abort the whole backfill.
--max-calls 50 caps accidental sweeps; bump it deliberately if the cap is hit.

What to tell the user

Show the trailing summary: number of cards judged, total elapsed time, per-card amortized cost. If any card failed, surface the failing date and the error. Mention tail -f logs/eval-judge-$(date +%F).log so the user can monitor progress without re-running.

Frequently asked questions

Eval Backfill