The KV-Surgeon: Compensation-Aware Offline KV-Cache Compaction.

arXiv:2610.25933v1 Announce Type: cross Abstract: Offline compaction of the key--value cache (KV-cache) reduces the cost of repeatedly querying a fixed context by constructing a smaller cache once and reusing it subsequently. The central challenge of compactio