Knowledge Distillation Compression

Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

robertoatila Updated 9 repo stars
J.A.R.V.I.S. — Autonomous Cognitive Runtime

File contents

robertoatila/jarvis-skill-registry commit 059951e822

Frequently asked questions

npx get-skill add robertoatila/knowledge-distillation-compression-2