Agentsop Domain Eval Set

Build and govern a 50-200 example domain-specific held-out benchmark sampled from real traffic. Distinct from public benchmarks (MMLU/HumanEval/GSM8K via lm-evaluation-harness) which measure GENERAL capability. Only a held-out domain set predicts whether THIS system works on YOUR data. Collect real examples, label, hold out (never train/prompt on it), size 50-200, version it, refresh on drift.

agentsope Updated 483 repo stars

File contents

agentsope/SkillAlchemy commit da421fcf2a

Frequently asked questions

npx get-skill add agentsope/agentsop-domain-eval-set