Jetson Speculative Decoding

Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.

NVIDIA Updated 23k repo stars

File contents

NVIDIA/skills commit 63c02e122e

Frequently asked questions

npx get-skill add nvidia/jetson-speculative-decoding