Repo Context Repositioning

Enable language models to dynamically assign continuous position values to tokens based on contextual relevance instead of fixed linear positions. Implement learnable SwiGLU module extracting position information, integrate with RoPE for end-to-end optimization. Improves performance on noisy contexts and long-sequence tasks.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit e200ba734e

Frequently asked questions

npx get-skill add adu2021/repo-context-repositioning