adu2021 Overview
DiRL introduces RL infrastructure tailored for diffusion language models.
Core Technique
DiPO Algorithm: First unbiased Group Relative Policy Optimization for dLLMs.
def dipo_training(model, dataset):
# Blockwise attention for efficient computation
# Unbiased logit computation (fixes prior biases)
# GRPO with dLLM-specific optimizations
Performance
- State-of-the-art dLLM math performance
- Outperforms larger autoregressive models
References
- DiPO: unbiased GRPO for dLLMs
- Blockwise training optimization