Awq Activation Quantization

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.

robertoatila Updated 9 repo stars
J.A.R.V.I.S. — Autonomous Cognitive Runtime

File contents

robertoatila/jarvis-skill-registry commit 3a9b99c4ba

Frequently asked questions

npx get-skill add robertoatila/awq-activation-quantization-2