Knowledge Distillation

Compress large language models using knowledge distillation from teacher to student models, covering temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

Orchestra Research Updated 14.5k repo stars
AI Research Skills Demo

File contents

Orchestra-Research/AI-Research-SKILLs commit 773a52944b

Frequently asked questions

npx get-skill add orchestra-research/knowledge-distillation