Eval Dataset Design

Design eval datasets that actually measure model quality — coverage, difficulty distribution, labeling consistency, and avoiding contamination. Covers sourcing, stratification, label quality, and when to generate vs curate. Use this skill when building a new eval set, realizing your current evals don't catch regressions, or labeling is inconsistent. Activate when: eval dataset, benchmark, test set, eval coverage, label quality, synthetic eval, dataset design.

latestaiagents Updated 6 repo stars

File contents

latestaiagents/agent-skills commit 11165a52d6

Frequently asked questions

npx get-skill add latestaiagents/eval-dataset-design