# Fp16 Trt Network

> Implement or review FP16 and BF16 behavior in one family-owned, strongly typed TensorRT network without changing unrelated families.

- Skill: `nvidia/fp16-trt-network` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add nvidia/fp16-trt-network`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nvidia/fp16-trt-network/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: NVIDIA (https://skillmd.com/u/nvidia)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/nvidia/fp16-trt-network

---


# FP16 and BF16 TensorRT Networks

Work in the owning `families/<family>/` implementation. Read its `model.py`,
local graph helpers, manifests, and tests before changing dtype behavior.

## Preserve the typed graph

- Create the network with
  `trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED`.
- Resolve `request.precision` explicitly and reject unsupported values.
- Track checkpoint storage dtype, TensorRT tensor dtype, shape, and layout at
  every cast-sensitive boundary.
- Create constants in an intentional storage dtype and use
  `network.add_cast(...)` when the runtime dtype differs.
- Keep all inputs to an elementwise operation dtype-compatible, including
  epsilon, masks, scales, and affine parameters.
- Preserve compact K/V projection and cache shapes for grouped or multi-query
  attention.

Use FP32 only where the family implementation or measured reference evidence
requires it, commonly reductions, normalization, probability arithmetic, or
comparison-sensitive outputs. Cast back explicitly. Do not apply a blanket
FP32 layer list or remove an established family boundary without evidence.

Precision and quantization are separate request values. Change one at a time,
and do not add a shared model helper merely because another family has similar
code. Copy a small proven pattern into the owner when needed.

## Prove the change

Build through the public Python build command with the exact checkpoint and
precision. Inspect the resulting bundle, run focused family Python/C++ tests,
then run the declared family E2E testcase against its existing oracle.

Bundle size is only a signal. Qualification requires the requested graph path
to execute and the unchanged family correctness contract to pass. Report any
hardware or precision variant not run.