Lm Evaluation Harness

LLM evaluation framework (EleutherAI lm-evaluation-harness). Unified benchmark evaluation for language models with 200+ tasks, support for HuggingFace transformers, vLLM, SGLang, OpenAI API, GGUF, and custom models. Used by HuggingFace Open LLM Leaderboard. Covers MMLU, HellaSwag, ARC, GSM8K, HumanEval, BBH, TruthfulQA, and more.

mkurman Updated 327 repo stars
Picture from the README

File contents

mkurman/zorai commit 208029653f

Frequently asked questions

npx get-skill add mkurman/lm-evaluation-harness