Adaptive DHiSparse: Speculative Decoding over a Load-Adaptive KV Hierarchy.

arXiv:2610.13708v1 Announce Type: new Abstract: Long-context serving runs out of KV memory before the hardware reaches its useful throughput. HiSparse stores history in host memory and fetches only the selected entries, but it uses one cache budget at every lo