Zip2zip Adaptive Vocab

Reduce token count by 15-40% at inference through context-adaptive compression, merging frequent token sequences into hypertokens using online Lempel-Ziv-Welch compression without retraining entire models.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 1c5ae2c1a3

Frequently asked questions

npx get-skill add adu2021/zip2zip-adaptive-vocab