Weak Driven Learning Logit Mixing

Break through supervised fine-tuning saturation by mixing logits from weaker model checkpoints into strong model training targets. Amplifies hard negatives that strong models have already suppressed, enabling continued learning after standard training plateaus.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 26d93ee9ba

Frequently asked questions

npx get-skill add adu2021/weak-driven-learning-logit-mixing