Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers.

arXiv:2610.13735v1 Announce Type: cross Abstract: Efficient attention mechanisms, such as linear and sliding window attention, aim to reduce the computational complexity of Transformers from quadratic to linear. However, their degraded in-context recall often