End-to-End Context Compression at Scale.

arXiv:2606.09659v2 Announce Type: replace-cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality subst