Virtual Head Attention.

arXiv:2610.17959v1 Announce Type: cross Abstract: The design of attention mechanisms in long-context large language models faces an ``impossible triangle'' among KV cache size, training and prefill FLOPs, and model quality. GQA achieves strong quality but incu