Equivariance Breaks the Learning Rate.

arXiv:2609.08381v2 Announce Type: replace-cross Abstract: Equivariant networks are commonly trained with Adam, yet recent work reports that matrix structured optimizers such as Muon can perform better, with the reasons for these gains only partly understood. W