Modelopt Evaluation

Evaluates accuracy of quantized or unquantized LLMs using NeMo Evaluator Launcher (NEL). Triggers on "evaluate model", "benchmark accuracy", "run MMLU", "evaluate quantized model", "run nel". Handles deployment, config generation, and evaluation execution. Always load when validating completed runs or reporting evaluation scores, including standalone modelopt-launching-evals analysis. Not for quantizing models (use modelopt-ptq), standalone serving (use modelopt-deployment), or comparing completed baseline-vs-quantized results (use modelopt-compare-results).

NVIDIA Updated 23k repo stars
Banner image

File contents

NVIDIA/Model-Optimizer commit ca53591193 Runs: nvidia/common

Frequently asked questions

npx get-skill add nvidia/modelopt-evaluation