Let the Neurons Die: Exploiting ReLU-Induced Model Degradation.

arXiv:2609.35528v1 Announce Type: cross Abstract: Rectified linear unit (ReLU) networks can suffer from dying neurons, where units with persistently negative pre-activations produce zero outputs, blocking gradients through their activations. To exploit this fa