LLM Eval Harness

Design an evaluation harness for an LLM-powered feature — a versioned golden set (representative + adversarial + regression cases), the cheapest adequate grading method per case, a metric with a pre-set pass bar and regression gate, and a failure taxonomy that targets iteration. Use when the user is building or tuning an LLM feature (prompt, RAG, agent, classifier) and needs evals, a way to test prompt/model changes, or to stop shipping quality regressions on vibes.

sananthanarayan Updated 2 repo stars

File contents

sananthanarayan/skilldrop commit 7f5bf5b59a

Frequently asked questions

npx get-skill add sananthanarayan/llm-eval-harness