Preprint finds energy-aware distillation can cut code-model inference energy

A preprint on software-engineering language models reports that energy-surrogate-guided knowledge distillation reduced inference energy use by as much as 90% and memory use by 86% in its experiments, with modest accuracy trade-offs. The researchers tested clone detection, vulnerability prediction and code summarization, and found FLOPs did not consistently indicate actual energy consumption. The work uses direct CPU and GPU energy estimates during distillation rather than optimizing only for operation counts.