Results for “eval-harness”

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware, with backend selection between vLLM, Transformers, and accelerate.

Validates the Emotional Attitude Score (EAS) metric by measuring its consistency with human judgment on word-level sentiment polarity, using the AmbGIMT dataset and pairwise score comparisons.

Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution on local Docker, Slurm HPC, or cloud platforms.