Learned 4bit Quantization

Compress LLM weights to 4-bit precision using learned per-row lookup tables that minimize output activation error, achieving better accuracy than fixed formats (int4, fp4, nf4) while maintaining inference speed.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 3119c5b36a

Frequently asked questions

npx get-skill add adu2021/learned-4bit-quantization