KVFetch: Temporal Prefetching for the Missing Half of KV Cache Compression.

arXiv:2610.08811v1 Announce Type: cross Abstract: As context windows scale to tens or hundreds of thousands of tokens, KV cache compression has become essential for efficient LLM inference. Existing methods fall into three families: score-based eviction, summa