MUON-RT Reusing the Teacher's Trajectory for Momentum-Free Muon for LLM Self-Distillation.

arXiv:2610.41104v1 Announce Type: cross Abstract: Self-distillation methods for large language models with an exponential moving average (EMA) teacher already retain a history of student updates. Muon nevertheless stores a separate momentum buffer, increasing