Weight Decay and Neuron Condensation: A Three-Stage Analysis of Two-Layer ReLU Networks.

arXiv:2610.04533v1 Announce Type: cross Abstract: Weight decay is widely used as a regularization technique in neural network training, yet its role in neuron condensation (parameter direction alignment) remains unclear. Starting from a parameter initializatio