RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM Serving.

arXiv:2610.18143v1 Announce Type: new Abstract: Prefix reuse improves LLM inference by sharing the KV cache of common prompt prefixes. This helps only if the scheduler can execute a matched branch in the current worker round. Schedulers assume that a matched p