Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon.

arXiv:2610.19844v1 Announce Type: cross Abstract: Memory-efficient subspace optimization has emerged as a practical approach to reducing the memory cost of LLM training while preserving optimization performance. Previous algorithms typically rely on the leadin