Llada2 Diffusion Scaling

Convert pre-trained autoregressive models into large-scale diffusion language models via Warmup-Stable-Decay training strategy. Progressively increase block size during warmup, perform stable diffusion training, then decay to smaller blocks for inference. Achieve 535 tokens-per-second with document-level masking and confidence-aware training.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 3e855ba69f

Frequently asked questions

npx get-skill add adu2021/llada2-diffusion-scaling