Knowledge Distillation

Knowledge distillation techniques for model compression: logit-level, feature-level, and relation-based distillation. KD-Lib library and practical workflows for training student models.

mkurman Updated 327 repo stars
Picture from the README

File contents

mkurman/zorai commit f1a7f53fb1

Frequently asked questions

npx get-skill add mkurman/knowledge-distillation