# Nvfp4

> Generate a working NVFP4 (W4A4) quantization example script and save a compressed-tensors checkpoint. Triggers on: "nvfp4", "NVFP4", "fp4", "nvfp4 example", "quantize to nvfp4", "w4a4".

- Skill: `vllm-project-llm-compressor/nvfp4` (Agent Skill)
- Install (CLI): `npx skillmds add vllm-project-llm-compressor/nvfp4`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vllm-project-llm-compressor/nvfp4/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: vllm-project (https://skillmd.com/u/vllm-project-llm-compressor)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/vllm-project-llm-compressor/nvfp4

---


# Write NVFP4 Example

Generate a working Python example script that quantizes a model to an NVFP4 scheme and saves a compressed-tensors checkpoint.

## Shared Documentation

Read `.claude/skills/shared_quantization.md` for common steps on gathering model information, applying model-type adjustments (dense, MoE, multimodal), GPTQ, transforms, and calibration dataset configuration.

## Step 1 — Gather information

Follow the shared documentation for gathering model information, GPTQ, transforms, and calibration dataset configuration.

**IMPORTANT:** NVFP4 is a W4A4 quantization scheme with:
- Weights: fp4 with per-group-16 scaling
- Activations: fp4 with calibrated global scale
- **Requires calibration dataset** for both weight and activation quantization — always use the shared `oneshot_with_data.py` template
- `model_free_ptq` is **NOT supported** — NVFP4 uses the `oneshot` path only

If the user specifically requests `model_free_ptq`, inform them it's not available for NVFP4 and proceed with the `oneshot` approach.

## Step 2 — Use the oneshot template (only path for NVFP4)

Read the shared template at `.claude/skills/templates/oneshot_with_data.py` and use it as the starting point. Set `scheme="NVFP4"`.

Follow the shared documentation to apply GPTQ and/or transform modifications to the recipe if requested.

Apply the model-type adjustments from the shared documentation before writing the final file.

## Step 3 — Apply model-type adjustments

Apply the model-type adjustments documented in `.claude/skills/shared_quantization.md`.

**Note:** For MoE models, the pipeline automatically handles expert calibration via `CalibrationAfmoeMoE` module — no manual intervention needed.

## Step 4 — Write the file

Place the file in `examples/quantization_w4a4_fp4/`.

Name the file `{model_name_slug}_nvfp4.py` (e.g. `llama3_nvfp4.py`, `gemma4_nvfp4.py`).

Run `make style` after writing the file.

## Notes
- NVFP4 **requires** calibration data — unlike FP8 schemes, you cannot use `oneshot(model=model, recipe=recipe)` without a dataset.
- `model_free_ptq` is **not supported** for NVFP4 — always use the `oneshot` path with calibration data.
- NVFP4 targets NVIDIA hardware for W4A4 quantization with per-group weight scaling and calibrated activation scaling.
- The `oneshot` call must include `dataset`, `max_seq_length`, and `num_calibration_samples` parameters.
- `save_compressed=True` is optional — the checkpoint saves in compressed-tensors format either way. Omit unless explicitly requested.
- For MoE models, expert calibration is handled automatically by the `CalibrationAfmoeMoE` module during the calibration phase.