Poly-attention: a general scheme for higher-order self-attention.

arXiv:2602.02422v2 Announce Type: replace-cross Abstract: The self-attention mechanism, at the heart of the Transformer model, is able to effectively model pairwise interactions between tokens. However, numerous recent works have shown that it is unable to per