Stapo Spurious Token Silencing

Stabilize RL training in LLMs by detecting and masking gradient contributions from spurious tokens that comprise 0.01% of output but cause disproportionate instability. Identifies tokens with low probability, low entropy, and positive advantage, then suppresses their gradients during optimization to maintain stable policy entropy and improve reasoning performance by 7%+ across model scales.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit c235a5e5dc

Frequently asked questions

npx get-skill add adu2021/stapo-spurious-token-silencing