T3d Trajectory Distillation

Accelerate diffusion language models from many to few generation steps using trajectory self-distillation. Collect clean-noisy state pairs along teacher trajectories, apply reverse-KL mode-seeking distillation, and weight losses by decoding order to prioritize early predictions where cascading errors compound.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 2b641fd5d4

Frequently asked questions

npx get-skill add adu2021/t3d-trajectory-distillation