Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice.

arXiv:2606.22327v3 Announce Type: replace Abstract: The rapid growth of interactive Large Language Model serving has made efficient management of dynamic Key-Value cache footprints increasingly important for inference performance. Modern inference systems over