Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers.

arXiv:2608.07436v2 Announce Type: replace Abstract: Muon-trained modular-arithmetic transformers can lose accuracy while retaining linearly decodable task information. Adjacent swaps localize five captured unnormalized failures to AdamW readout updates. Multip