Add a FastGen method
Apply fastgen-develop
and fastgen-runtime. Read the
method index and the nearest maintained implementation,
the FastGenModel docstrings in model.py, then the
method contract.
- Plan. Resolve the algorithm version and its authoritative paper or code. Write down
the state and time convention, target construction, detach boundaries, loss weighting
and reduction, update order, required teacher or auxiliary networks and the inference
rollout. Separate the method from the reference's architecture, dataset and
engineering defaults, and record unresolved paper/code discrepancies.
- Reuse. If existing options express the algorithm, hand off to
fastgen-configure.
Otherwise choose the narrowest compatible method extension. Share mechanisms only
where semantics match; do not inherit losses or networks because they are defaults.
Move objective math into
common_loss.py only when another caller shares it.
- Implement. Extend the selected method, creating a
FastGenModel subclass in the
matching category only when needed. Follow the contract's model, optimizer, scheduler,
batch and checkpoint interfaces; for a new method add a typed config under
fastgen/configs/methods/ and its export in
methods/init.py. Keep model and dataset names out
of the method; experiment configs own them.
- Dependencies. Trace the network operations and batch fields the method needs
through the actual callers; use
fastgen-add-network for missing network
capabilities and fastgen-dataset for supervision.
Declare the supported prediction types, samplers, modalities and causal modes.
- Configs. Use fastgen-configure for a small
supported experiment and the target recipe, including the method's student grid,
logging and staged initialisation.
- Verify the math. Compare each objective term's value and gradients with the reference on
fixed tensors, times and noise before any expensive run; cover boundary times,
disabled terms and detach boundaries. Add and run unit tests following the
tests/test_*model.py pattern and use fastgen-validate
for sampling parity and checkpoints.
- Run and document. Use fastgen-train for a
bounded run covering every optimizer phase and the requested distributed mode; check
that intended modules update, frozen targets stay fixed, validation preserves training
state and resume restores the new state. Evaluate with
fastgen-inference at the target NFE, add the
method to the README tables (they feed the documentation site).
Report method and config paths, the correspondence to the reference math, tests, runs
and supported combinations. For finite but ineffective training inspect gradient paths
and optimizer phases before sweeping hyperparameters, and separate unverified reference
claims from demonstrated results.
1---2name: fastgen-add-method3description: >-4 Implement and validate a new NVIDIA FastGen training method or objective (distillation, consistency, adversarial or fine-tuning variants) from a paper or reference code: FastGenModel subclass, method config, gradients, student sampling and tests. Use it whenever an algorithm is not expressible through the existing method configs.5license: Apache-2.06metadata:7 source: https://skillmd.com/skills/nvidia/fastgen-add-method8 author: "Julius Berner <jberner@nvidia.com>"9 domain: ml10---1112# Add a FastGen method1314Apply [fastgen-develop](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-develop/SKILL.md)15and [fastgen-runtime](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-runtime/SKILL.md). Read the16[method index](https://github.com/NVlabs/FastGen/blob/main/fastgen/methods/README.md) and the nearest maintained implementation,17the `FastGenModel` docstrings in [model.py](https://github.com/NVlabs/FastGen/blob/main/fastgen/methods/model.py), then the18[method contract](references/method-contract.md).19201. **Plan.** Resolve the algorithm version and its authoritative paper or code. Write down21 the state and time convention, target construction, detach boundaries, loss weighting22 and reduction, update order, required teacher or auxiliary networks and the inference23 rollout. Separate the method from the reference's architecture, dataset and24 engineering defaults, and record unresolved paper/code discrepancies.252. **Reuse.** If existing options express the algorithm, hand off to26 [fastgen-configure](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-configure/SKILL.md).27 Otherwise choose the narrowest compatible method extension. Share mechanisms only28 where semantics match; do not inherit losses or networks because they are defaults.29 Move objective math into30 [common_loss.py](https://github.com/NVlabs/FastGen/blob/main/fastgen/methods/common_loss.py) only when another caller shares it.313. **Implement.** Extend the selected method, creating a `FastGenModel` subclass in the32 matching category only when needed. Follow the contract's model, optimizer, scheduler,33 batch and checkpoint interfaces; for a new method add a typed config under34 `fastgen/configs/methods/` and its export in35 [methods/__init__.py](https://github.com/NVlabs/FastGen/blob/main/fastgen/methods/__init__.py). Keep model and dataset names out36 of the method; experiment configs own them.374. **Dependencies.** Trace the network operations and batch fields the method needs38 through the actual callers; use39 [fastgen-add-network](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-add-network/SKILL.md) for missing network40 capabilities and [fastgen-dataset](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-dataset/SKILL.md) for supervision.41 Declare the supported prediction types, samplers, modalities and causal modes.425. **Configs.** Use [fastgen-configure](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-configure/SKILL.md) for a small43 supported experiment and the target recipe, including the method's student grid,44 logging and staged initialisation.456. **Verify the math.** Compare each objective term's value **and gradients** with the reference on46 fixed tensors, times and noise before any expensive run; cover boundary times,47 disabled terms and detach boundaries. Add and run unit tests following the48 `tests/test_*model.py` pattern and use [fastgen-validate](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-validate/SKILL.md)49 for sampling parity and checkpoints.507. **Run and document.** Use [fastgen-train](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-train/SKILL.md) for a51 bounded run covering every optimizer phase and the requested distributed mode; check52 that intended modules update, frozen targets stay fixed, validation preserves training53 state and resume restores the new state. Evaluate with54 [fastgen-inference](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-inference/SKILL.md) at the target NFE, add the55 method to the README tables (they feed the documentation site).5657Report method and config paths, the correspondence to the reference math, tests, runs58and supported combinations. For finite but ineffective training inspect gradient paths59and optimizer phases before sweeping hyperparameters, and separate unverified reference60claims from demonstrated results.