Adaptive Lora Personalized Ranks

Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 63de40b7b9

Frequently asked questions

npx get-skill add adu2021/adaptive-lora-personalized-ranks