Huggingface Community Evals

Runs benchmark evaluations of Hugging Face Hub models on local GPUs with lighteval or inspect-ai (GSM8K, HellaSwag, MMLU and similar) and picks the backend (vLLM, Transformers or accelerate) with exact commands. Use when the user wants to evaluate or benchmark an HF model locally, mentions lighteval or inspect-ai, a local GPU such as an RTX 4090, or asks which eval backend to use.

The-Utopia-Studio Updated 2 repo stars

File contents

the-utopia-studio/skills commit 5238275624

Frequently asked questions

npx get-skill add the-utopia-studio/huggingface-community-evals