Edge LLM Plugin Extension

Add or change a TensorRT Edge-LLM operation (TensorRT IPluginV3 plugin, native TensorRT layer composition, or runtime-only operation) so one tensor, dtype, shape, layout, serialization, and weight-layout contract holds across the ONNX export route, the experimental checkpoint-direct builder, the C++ engine builders, libNvInfer_edgellm_plugin, and the runtime. Use when adding a custom op, plugin creator field, plugin input or output, dtype, weight repacking, or external-weight recipe; when a plugin is not found or rejects a configuration at engine build; or when choosing between a plugin and native TensorRT layers. Do not use for writing CuTe DSL kernels (use edge-llm-cutedsl-kernel), C++ test discipline (edge-llm-cpp-test), model CI cases (edge-llm-e2e-test), or diagnosing wrong model output (edge-llm-few-layer-test, then edge-llm-accuracy-debugging).

NVIDIA Updated 23k repo stars

File contents

nvidia/tensorrt-edge-llm commit 2533e9aa1a

Frequently asked questions

npx get-skill add nvidia/edge-llm-plugin-extension