EnergyCal: Taming Loss Spikes in Muon-Family Optimizers for Sparse MoE Training.

arXiv:2610.37062v1 Announce Type: cross Abstract: Sparse mixture-of-experts (MoE) language-model training can exhibit large loss spikes with Muon-family optimizers. We introduce EnergyCal, a lightweight rule that attenuates routed-expert updates according to t