Inference Perf Hillclimb

Profiler-driven hill-climbing to close the inference throughput gap between TorchTitan's unified model (running inside vLLM) and vLLM's native model. Benchmark with generate.py --benchmark, climb optimization rungs (compile / cudagraph / fused kernels), profile torchtitan vs the native target, then patch the single biggest gap at a time and re-measure. Use when the user wants to benchmark or optimize RL inference generation speed, reproduce previous hill climbing study, or invokes /inference_perf_hillclimb.

pytorch Updated
torchtitan

File contents

pytorch/torchtitan commit d39e2e91a9

Frequently asked questions

npx get-skill add pytorch/inference-perf-hillclimb