E8 Lattice Key Quantization for Low-Bit KV-Cache Compression.

arXiv:2610.15384v1 Announce Type: cross Abstract: Compressing the key-value (KV) cache is essential for serving long-context large language models (LLMs) under memory constraints. While current scalar quantization methods achieve significant compression, they