When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers.

arXiv:2606.16694v2 Announce Type: replace-cross Abstract: Transformers perform computations through many successive attention and feedforward blocks, allowing them to learn complex correlations between a large collection of coupled variables. When does stackin