LLM Serving Optimization with Variable Prefill and Decode Lengths.

arXiv:2508.06133v5 Announce Type: replace-cross Abstract: We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) lengths. Given a backlog