Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity.

arXiv:2610.28441v1 Announce Type: new Abstract: Large Language Model inference over long contexts is increasingly bottlenecked by the cost of attending to a growing Key-Value (KV) cache. Sparse attention reduces this cost by accessing only a fraction of the ca