adu2021 Summary
Adversarial Flow Models unifies adversarial training with flow-based generative models by adding an optimal transport regularization term that constrains generators to learn transport-optimal mappings. This enables native one-step generation without distillation, stabilizes adversarial training on transformers, and achieves FID 2.38 on ImageNet-256 with single-step generation.
Core Technique
The key innovation adds an optimal transport regularization to the adversarial objective. While adversarial training learns distribution matching through min-max optimization, it doesn't guarantee learning a deterministic, Wasserstein-2 optimal transport plan like flow matching does.
Optimal Transport Loss Term: ℒ_ot^G = E_z[1/n ||G(z) - z||²_2] constrains generator outputs to match the transport plan induced by random Gaussian noise. This ensures G(z) satisfies optimal transport properties—straightness and determinism—even under adversarial training.
Combined Objective: Minimize: ℒ_adv^G + λ * ℒ_ot^G where ℒ_adv^G is the standard adversarial generator loss and λ balances both objectives.
Implementation
Regularization weight selection: Start with λ = 1.0 and adjust based on FID convergence. Too high prioritizes transport optimality over distribution matching; too low loses benefits of the constraint.
Noise handling: Sample z ~ N(0, I) and pass through generator G(z). Compute both the adversarial loss (discriminator feedback) and the L2 distance ||G(z) - z||², averaging over the batch.
One-step generation: Train for native one-step inference without teacher-student distillation. Sample z once and evaluate G(z) directly—no iterative denoising.
Transformer stability: The optimal transport constraint stabilizes transformer-based generators compared to pure adversarial training, reducing training divergence and gradient scaling issues.
When to Use
- Real-time image generation where latency is critical
- Applications needing diverse generation without distillation overhead
- Scenarios where straightforward adversarial training diverges on transformer architectures
- Tasks requiring deterministic, principled generation mappings
When NOT to Use
- Scenarios where teacher-distilled multi-step models are acceptable
- Applications where discriminator training overhead is prohibitive
- Tasks needing iterative refinement or coarse-to-fine generation
- Models where likelihood-based training is preferred over adversarial objectives
Key References
- Optimal transport theory and Wasserstein distances in generative modeling
- Flow matching and matching networks for generative tasks
- Adversarial training stabilization techniques for transformers