# Muon-RT preprint reports lower training memory and higher accuracy in self-distillation tests

_Published Monday, November 2, 2026 at 12:05 PM EST · AI, Science · Latest · Tier 2 — Notable_

Researchers propose Muon-RT, a language-model training method that reuses information already retained by a teacher model to reduce persistent memory. Their arXiv preprint reports that, in self-distillation fine-tuning of Qwen2.5-7B-Instruct, it improved average accuracy over AdamW by 2.6 points with 42% less persistent memory, and over momentum Muon by 5.4 points with 27% less persistent memory.

The method eliminates separate momentum buffers for hidden weight matrices by using the displacement between teacher and student models. Its theoretical convergence result depends on explicit regularity and alignment assumptions.

## Sources

- [cs.LG updates on arXiv.org](https://arxiv.org/abs/2610.41104)

---
Canonical: https://techandbusiness.org/newswire/lyt_m4Yb7OM4LK-tzrXcRL
Published: 2026-11-02T17:05:58.379Z
Story chronology: 2026-11-02T05:00:00.000Z
Retrieved: 2026-11-02T19:30:47.257Z
Publisher: Tech & Business (techandbusiness.org)