arXiv:2610.14500v1 Announce Type: cross Abstract: Many widely used activation functions, including ReLU, GELU, Swish, and Sigmoid, produce predominantly positive outputs, resulting in a nonzero post-activation mean that we term activation bias.
← Back to all developments · Signals are a development reported by more than one source. See the methodology.