Stable Token-level Scheduling for Serverless LLM Inference.

arXiv:2610.17689v1 Announce Type: new Abstract: Serverless inference is emerging as an efficient deployment model for Large Language Models (LLMs) due to its ability to elastically scale resources, improve GPU utilization through resource sharing, and reduce i