Eval Audit

Audit an LLM eval pipeline and surface problems: missing error analysis, unvalidated judges, vanity metrics, etc. Use when inheriting an eval system, when unsure whether evals are trustworthy, or as a starting point when no eval infrastructure exists. Do NOT use when the goal is to build a new evaluator from scratch (use error-discovery, write-judge-prompt, or validate-evaluator instead).

ai-evals-course Updated
Error Discovery Skill Workflow

File contents

ai-evals-course/error-discovery-skill commit 4414298e40

Frequently asked questions

npx get-skill add ai-evals-course/eval-audit