Output-Aware Rotation for INT2 KV-Cache Quantization.

arXiv:2608.02691v4 Announce Type: replace-cross Abstract: The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important. However, existing ro