Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels.

arXiv:2610.13674v1 Announce Type: cross Abstract: Linearizing softmax attention offers a route to efficient Transformer inference, but preserving pretrained attention remains challenging. We identify an important source of approximation error overlooked by sta