research-guide-model-distillation-techniques-for-deep-learning-4a100801c0eb 好料推荐 关于知识蒸馏的工作远不止这七篇论文,为了方便大家学习与应用,给大家推荐一个Github项目:《Awesome Knowledge-Distillation
可以减轻GPU显存的Overhead: def loss_fn_kd(outputs, labels, teacher_outputs, params): """ Compute the knowledge-distillation
英文原文:https://wandb.ai/authors/knowledge-distillation/reports/Distilling-Knowledge-in-Neural-Networks-