Derf Normalization Free Transformers

Replace LayerNorm with Derf(x) = erf(αx + s) for improved generalization in transformers. Derf outperforms LayerNorm across vision, speech, and DNA modeling—ideal when normalization-free training provides benefits without architectural complexity.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit e96544b324

Frequently asked questions

npx get-skill add adu2021/derf-normalization-free-transformers