Dope Denoising Rotary Embeddings

Improve long-context length extrapolation by denoising instabilities in Rotary Position Embeddings (RoPE) through spectral analysis and selective head rewriting—training-free post-hoc intervention for longer context windows.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 5c2c3b8cee

Frequently asked questions

npx get-skill add adu2021/dope-denoising-rotary-embeddings