Results for “benchmarking”
11 skills
nvidia Nemotron Customize
Plan, configure, and chain Nemotron model customization steps into single-step or multi-step pipelines for curation, translation, fine-tuning, RL alignment, benchmarking, checkpoint conversion, optimization, and evaluation.
2.2k · bundle
orchestra-research Evaluating Code Models
Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality.
10.4k · bundle
qhjqhj00 Runtime
Benchmarks inference latency and computational runtime of transformer models and MLX operations across Apple Silicon and NVIDIA GPU backends, with configurable input lengths and batch sizes.
3
nvidia Jetson Speculative Decoding
Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.
2.2k · bundle
orchestra-research Evaluating Llms Harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) using standardized prompts and metrics. Supports HuggingFace, vLLM, and API backends.
10.4k · bundle
More results
samyakjhaveri Cuda Omp Translator
Reference guide for evaluating LLM-generated translations between CUDA and OpenMP, covering memory model mapping, kernel launch patterns, shared memory, atomics, and common failure modes.
0
orchestra-research Nemo Evaluator Sdk
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution on local Docker, Slurm HPC, or cloud platforms.
10.4k · bundle
qhjqhj00 Pymoo
Solve single- and multi-objective optimization problems with NSGA-II/III, MOEA/D, and other evolutionary algorithms, including Pareto front analysis, constraint handling, and benchmarking on standard test problems.
3 · bundle
qhjqhj00 Latency
Measures inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations.
3
qhjqhj00 Adp Eval
Benchmarks LLM agents fine-tuned with the Agent Data Protocol across software engineering, web browsing, OS/database tool use, and reasoning tasks, reporting unit test pass rates and task success rates.
3
qhjqhj00 Dior
Quantifies how sensitive a language model benchmark's reliability and ranking stability are to specific design choices, such as the selection of scenarios, subscenarios, examples, and few-shot prompts. Use when the user has predictions and gold and needs to compute DIoR.
3