Beyond the Matrix Sign: Quadratic Spectral Descent.

arXiv:2609.07597v2 Announce Type: replace-cross Abstract: Muon emerges as a strong competitor of the AdamW for LLM pretraining, because the matrix-wise update it employs can potentially incur smaller second-order penalty than the once dominating AdamW, which p