MuonGS: Pretraining LLMs with Relaxed Weight-Norm Control.

arXiv:2610.37647v1 Announce Type: cross Abstract: Weight decay and explicit weight-norm constraints improve optimizer performance in language model pretraining, yet a unified understanding of why such control helps and how much is needed remains incomplete. We