Quantized Evolution Strategies

Fine-tune quantized LLMs directly in low-precision discrete parameter space using evolution strategies with accumulated error feedback. Overcome gradient stagnation in quantized models by accumulating fractional updates using Delta-Sigma modulation, achieving significant improvements in INT4 quantized models without full-precision gradients.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 07657a6af4

Frequently asked questions

npx get-skill add adu2021/quantized-evolution-strategies