Trtllm Modeling V2 Features

Load when working on, testing or debugging a TensorRT-LLM engine feature around a ModelingV2 target -- KV-cache block (prefix) reuse, disaggregated serving, chunked prefill, KV-cache dtype (fp8/nvfp4 and their scales), CUDA graphs and the overlap scheduler, multimodal inputs, speculative decoding, or attention-DP/TP/EP where it changes one of those -- including a check that fails on a feature field, or a number that moves only with a feature on. Says which engine stage owns the feature and what the target must honour, where its code and tests are in the tree, and how to prove it from a CPU check up to the flow's accuracy, disagg or acceptance gate. Not needed to write or certify one catalog entry (trtllm-modeling-v2-entry) or to debug core accuracy with every feature at its default.

NVIDIA Updated 23k repo stars

File contents

nvidia/tensorrt-llm commit 51381619ad

Frequently asked questions

npx get-skill add nvidia/trtllm-modeling-v2-features