Pop Prefill Only Pruning

Remove deep layers during context encoding (prefill) while keeping them for token generation (decode). Identifies layer importance asymmetry via virtual gates; achieves 1.37x prefill speedup without retraining on any pre-trained model.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit d8899081e0

Frequently asked questions

npx get-skill add adu2021/pop-prefill-only-pruning