Gptq

Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.

Orchestra Research Updated 14.5k repo stars
AI Research Skills Demo

File contents

Orchestra-Research/AI-Research-SKILLs commit 773a52944b

Frequently asked questions

npx get-skill add orchestra-research/gptq