Memdlm Parametric Memory

Enhance diffusion language model performance on long-context tasks by embedding simulated denoising into training via bi-level optimization. Fast weights capture local trajectory experience; base model optimized with accumulated parametric memory. Achieves +17.0% on RULER Variable Tracking (8K) and +9.6% on BABILong with gains primarily from training-stage improvements.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit cf8551271d

Frequently asked questions

npx get-skill add adu2021/memdlm-parametric-memory