Poet Orthogonal LLM Training

Improve LLM training stability and generalization by reparameterizing weight matrices as orthogonal transformations, achieving better perplexity than AdamW with fewer trainable parameters.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit ade2c68db9

Frequently asked questions

npx get-skill add adu2021/poet-orthogonal-llm-training