Validate FastGen behavior
Apply fastgen-runtime. This skill owns comparison
and acceptance evidence; fastgen-inference
generates outputs and fastgen-train executes training.
Follow the new-network checklist
for adapters, and the
run contracts
when callback logs are the evidence.
- Manifest. Freeze source and checkpoint revisions, environment, inputs, seeds and
noise tensors, preprocessing, dtype, schedule, guidance, grid/NFE and output geometry;
keep validation inputs held out from training. Fix tolerances and quality criteria
before looking at results.
- Parity. Compare upstream and FastGen at widening scope: normalised inputs and
encoded conditioning and latents, the same noisy state and time through the denoiser,
converted predictions and sampler steps, decoded outputs. Match the actual noise and
time grids rather than seeds, since equal seeds do not imply equal RNG consumption.
Prefer independently constructed upstream components over shared prepared ones, check
that shared weights and buffers survive preparation, and treat precision differences
as a separate experiment. Find the first differing intermediate before loosening
tolerances; a loosened tolerance is a separately documented comparison that leaves
the failure against the manifest's criteria on record.
- Media. Generate matched seeded examples through both implementations across
prompts and conditioning modes; present side-by-side galleries for human judgment and
record judgments separately from metrics. Small galleries are not benchmarks.
- Trainability. Check finite losses and gradients before clipping, intended
parameter updates and frozen modules; run generation followed by backward under the
real distributed wrapper, since inference can leave lazy state behind. Inspect
per-iteration media and fixed-input outputs around a short SFT run, exercise
auxiliary phases, save, reload and compare outputs, and verify
all saved training state
when resume is claimed.
- Distillation quality. Compare held-out teacher and student outputs at the target
NFE with the actual checkpoint and sample count; use task metrics (for images the
FID script), perceptual inspection and
timing/memory measurements. Count network evaluations, not sampling intervals, when
methods use guidance or multiple heads.
- Tests. Reuse the patterns in tests/ (shared helpers, markers such as
large_model and integration, tiny CPU models) and run make lint and the relevant
tests as described in CONTRIBUTING.md
before claiming a code change is validated.
Keep a compact report with commands, manifests, metrics, media, tracker links, hardware
and explicit pass/fail/not-run states; keep large artifacts outside the source tree and
name repeated evaluations distinctly so they cannot replace the reference. Measure and
report reference nondeterminism separately.
1---2name: fastgen-validate3description: >-4 Validate NVIDIA FastGen integrations and training quality: numerical parity against reference implementations, seeded side-by-side media, gradient and trainability checks, checkpoint round trips and repository tests. Use it whenever a claim about correctness, parity or quality needs evidence.5license: Apache-2.06metadata:7 source: https://skillmd.com/skills/nvidia/fastgen-validate8 author: "Julius Berner <jberner@nvidia.com>"9 domain: ml10---1112# Validate FastGen behavior1314Apply [fastgen-runtime](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-runtime/SKILL.md). This skill owns comparison15and acceptance evidence; [fastgen-inference](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-inference/SKILL.md)16generates outputs and [fastgen-train](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-train/SKILL.md) executes training.17Follow the [new-network checklist](https://github.com/NVlabs/FastGen/blob/main/fastgen/networks/README.md#validate-a-new-network)18for adapters, and the19[run contracts](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-train/references/run-contract.md#interpret-media-and-scalars-correctly)20when callback logs are the evidence.21221. **Manifest.** Freeze source and checkpoint revisions, environment, inputs, seeds and23 noise tensors, preprocessing, dtype, schedule, guidance, grid/NFE and output geometry;24 keep validation inputs held out from training. Fix tolerances and quality criteria25 before looking at results.262. **Parity.** Compare upstream and FastGen at widening scope: normalised inputs and27 encoded conditioning and latents, the same noisy state and time through the denoiser,28 converted predictions and sampler steps, decoded outputs. Match the actual noise and29 time grids rather than seeds, since equal seeds do not imply equal RNG consumption.30 Prefer independently constructed upstream components over shared prepared ones, check31 that shared weights and buffers survive preparation, and treat precision differences32 as a separate experiment. Find the first differing intermediate before loosening33 tolerances; a loosened tolerance is a separately documented comparison that leaves34 the failure against the manifest's criteria on record.353. **Media.** Generate matched seeded examples through both implementations across36 prompts and conditioning modes; present side-by-side galleries for human judgment and37 record judgments separately from metrics. Small galleries are not benchmarks.384. **Trainability.** Check finite losses and gradients before clipping, intended39 parameter updates and frozen modules; run generation followed by backward under the40 real distributed wrapper, since inference can leave lazy state behind. Inspect41 per-iteration media and fixed-input outputs around a short SFT run, exercise42 auxiliary phases, save, reload and compare outputs, and verify43 [all saved training state](https://github.com/NVlabs/FastGen/blob/main/skills/fastgen-train/references/run-contract.md#resume-restores-state-not-the-exact-trajectory)44 when resume is claimed.455. **Distillation quality.** Compare held-out teacher and student outputs at the target46 NFE with the actual checkpoint and sample count; use task metrics (for images the47 [FID script](https://github.com/NVlabs/FastGen/blob/main/scripts/README.md#fid-evaluation)), perceptual inspection and48 timing/memory measurements. Count network evaluations, not sampling intervals, when49 methods use guidance or multiple heads.506. **Tests.** Reuse the patterns in [tests/](https://github.com/NVlabs/FastGen/tree/main/tests) (shared helpers, markers such as51 `large_model` and `integration`, tiny CPU models) and run `make lint` and the relevant52 tests as described in [CONTRIBUTING.md](https://github.com/NVlabs/FastGen/blob/main/CONTRIBUTING.md#continuous-integration)53 before claiming a code change is validated.5455Keep a compact report with commands, manifests, metrics, media, tracker links, hardware56and explicit pass/fail/not-run states; keep large artifacts outside the source tree and57name repeated evaluations distinctly so they cannot replace the reference. Measure and58report reference nondeterminism separately.