Intrinsic Interaction Geometry Controls the Low-Rank Complexity of Softmax Attention.

arXiv:2608.28150v2 Announce Type: replace-cross Abstract: How much matrix rank is required to preserve every bounded value output of normalized softmax attention? We study the unrestricted maximum-row-\(\ell_1\) approximation rank \(r_\varepsilon(A)\), exactly