Evaluation

Evaluates accuracy of quantized or unquantized LLMs using NeMo Evaluator Launcher (NEL). Triggers on "evaluate model", "benchmark accuracy", "run MMLU", "evaluate quantized model", "run nel". Handles deployment, config generation, and evaluation execution. Not for quantizing models (use ptq), deploying/serving models (use deployment), or comparing completed baseline-vs-quantized results (use compare-results).

NVIDIA Updated 23k repo stars
Banner image

File contents

nvidia/model-optimizer commit 96aa0d9f09 Runs: nvidia/common

Frequently asked questions

npx get-skill add nvidia/evaluation