Kernel-based Interpolatory Methods for Context Compression of Attention Mechanisms.

arXiv:2610.40943v1 Announce Type: cross Abstract: The bottleneck of attention mechanisms, which are the fundamental building block of modern language models, is their quadratic complexity in terms of context size. There has thus been an increasing interest in