Xquant Kv Cache Rematerialization

Reduce KV cache memory by 7.7-10x through quantization and rematerialization of input activations instead of caching Keys and Values, trading recomputation for memory efficiency.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 67f9d9203e

Frequently asked questions

npx get-skill add adu2021/xquant-kv-cache-rematerialization