Awq Quantization

Quantize large language models to 4-bit using activation-aware weight quantization, achieving ~3x speedup with minimal accuracy loss for deployment on limited GPU memory.

Orchestra Research Updated 14.5k repo stars
AI Research Skills Demo

File contents

Orchestra-Research/AI-Research-SKILLs commit 773a52944b

Frequently asked questions

npx get-skill add orchestra-research/awq-quantization