Model Speed Benchmark

Use when running throughput/latency (serving speed) benchmarks against an already-running model server. Uses sgl_jax.bench_serving to sweep batch-size/concurrency and measure throughput and latency, producing summary.csv and per-point bs_<N>/result.jsonl. Speed only — accuracy/eval belongs to model-accuracy-benchmark. Triggers on throughput, latency, bench_serving, batch-size sweep, tok/s, TTFT, TPOT — not accuracy/eval.

sgl-project Updated 38.9k repo stars
SGLang-JAX Architecture

File contents

sgl-project/sglang-jax commit 0f486094f1

Frequently asked questions

npx get-skill add sgl-project/model-speed-benchmark