Msign Stable Rank Restoration

Prevent unrecoverable gradient explosions in LLM training by periodically restoring weight matrix stable rank through SVD-based matrix sign operations, eliminating sudden training failures without computational burden.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit db52b2042f

Frequently asked questions

npx get-skill add adu2021/msign-stable-rank-restoration