# Monitor Evals

> Review evaluation history, distinguish system, data, suite, and evaluator changes, and apply an operating response. Use for regressions, eval monitoring, release gates, or trend questions.

- Skill: `ai-analyst-lab/monitor-evals` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ai-analyst-lab/monitor-evals`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ai-analyst-lab/monitor-evals/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: ai-analyst-lab (https://skillmd.com/u/ai-analyst-lab)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/ai-analyst-lab/monitor-evals

---


# Monitor evaluation history

Read versioned run manifests with `helpers.evals.monitoring.load_history`. Use `classify_changes` before comparing scores.

Separate:

- system behavior changes;
- data changes;
- task-mix or suite-version changes;
- evaluator changes; and
- operational failures such as blocked tools or expired connections.

Compare only compatible runs. Show per-case and slice movement, not only the aggregate. Run frozen sentinel examples for model graders so evaluator drift does not look like system drift.

Apply the named operating rule:

- continue when the intended change improved the target slice without a blocking regression;
- investigate when the cause is unclear or several inputs changed;
- rollback when a blocking regression follows a controlled system change; or
- escalate when the evaluator, data, or authorization boundary may be invalid.

Record the owner and next action. Monitoring is an operating practice, not a dashboard someone passively observes.