DepthBench: Measuring How Residual Connections Enable More Computational Depth.

arXiv:2609.32534v1 Announce Type: cross Abstract: Depth is a natural way to increase the computational capacity in Transformers, yet the contribution of deeper layers can diminish as depth grows larger. Recent approaches enhance normalization (\text{e.g.}, Lay