Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference.

arXiv:2610.22985v1 Announce Type: cross Abstract: Long-context decoding in large language models (LLMs) is increasingly bottlenecked by the memory and computation required to attend over an ever-growing Key-Value (KV) cache at each autoregressive step. Althoug