BELT : BEhavior-Aligned Low-Rank KV Cache Compression with Learned Top-k Rank Allocation.

arXiv:2610.17966v1 Announce Type: cross Abstract: Long-context inference makes the key-value (KV) cache a major memory bottleneck for large language models. We propose BELT, a low-rank KV cache compression framework. Its key-side objective targets attention di