A Spectral Theory of Grokking: Weight Decay induces Feature Learning.

arXiv:2609.26679v1 Announce Type: cross Abstract: In grokking an early fit to the training data separates from a much later improvement in generalization. During this delay, training can move from a fixed neural tangent kernel (NTK) regime to one in which task