LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding.

arXiv:2607.24555v3 Announce Type: replace-cross Abstract: Serving large language models at long context is bottlenecked by the key-value (KV) cache, which is read at every decode step. We find that attention keys are approximately low-rank within pages. A sing