Tail-Replay: Escaping the Curse of Linear Attention in Prefix Caching for Hybrid LLMs.

arXiv:2608.30310v2 Announce Type: replace-cross Abstract: Hybrid large language models interleave full-attention layers with linear-attention layers to reduce the cost of long-context inference. This structure complicates prefix caching: full-attention key-val