How do I install the Eval Regression skill?

Run npx skillmds@latest add hoangsonww/eval-regression in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Eval Regression safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Eval Regression?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Eval Regression free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Eval Regression?

hoangsonww (@hoangsonww) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Eval Regression by hoangsonww — Agent Skill

Name: Eval Regression
Author: hoangsonww

Eval Regression

Re-judge every pinned golden card in eval/golden/ and fail if any composite has dropped more than 0.5 points vs its baseline. Use when the user asks to "run regression", "check golden set", or wants CI-style quality gating after a prompt or judge change.

hoangsonww a20b148 1.7 KB Updated Sep 17, 2026

File contents

Eval — Regression Gate

Guard against silent quality regressions when the daily prompt, judge model, or judge prompt changes.

How to invoke

make eval-regression                        # stub judge, fast smoke test
make eval-regression JUDGE=claude           # real Claude Haiku judge — production gate

Direct invocation supports a tighter (or looser) drop tolerance:

python3 eval/runner.py regression --judge claude --regression-drop 0.5

Behavior

Read every eval/golden/*.json baseline (currently 18 cards, real-judge composites 2.9–4.2).
Re-judge each card against the configured backend.
Compute delta = new_composite - baseline_composite.
Fail (exit 2) if any card's delta is below -regression-drop (default -0.5).
Print per-card OK / REGRESSED with explicit deltas.

When to run

After bumping PROMPT_VERSION in eval/judge.py or editing eval/judge_prompt.md.
After switching judge models (EVAL_JUDGE_MODEL).
In CI before merging changes to the daily prompt.md.
After a re-baselining workflow to confirm the new goldens hold.

What to tell the user

If regressions are flagged, list each offending card with its delta and the judge's notes — they explain why the score dropped. If no regressions, report total cards passed and the worst-case delta so the user knows how much headroom remains.

Frequently asked questions