RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies.

arXiv:2610.28385v1 Announce Type: new Abstract: Head-dimension compression can reduce the memory required by the key-value (KV) cache, but existing methods often reconstruct full-dimensional representations during inference, adding significant decoding FLOPs o