GAAC: Lossless KV Compression for Bandwidth-Efficient LLM Inference on Personal Devices.

arXiv:2610.38210v1 Announce Type: new Abstract: Weight-only, lossy quantization substantially reduces the storage and memory requirements for serving large language models (LLMs), but the growing key-value (KV) cache remains a bottleneck for long-context infer