Taming Softmax at the Limits of Muon: A Unified Approach through Hilbert Geometry.

arXiv:2610.17588v1 Announce Type: cross Abstract: Muon controls weight updates through the RMS movement of a linear layer's output. This implicitly treats the linear output as a proxy for the full module, including the nonlinearity that follows. This proxy can