Just Let Linear States Forget the Distant Past: Prefix Caching via Suffix Replay for Hybrid LLMs.

arXiv:2609.33477v1 Announce Type: new Abstract: Hybrid LLMs interleave full-attention layers with linear-attention layers to reduce long-context inference cost, but this structure complicates prefix caching. Full-attention KV caches are token-addressable, wher