Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs.

arXiv:2605.04357v2 Announce Type: replace-cross Abstract: The landscape of large language models (LLMs) has become increasingly fragmented, with no single model dominating the field. Meanwhile, cloud providers offer a diverse array of mid-tier and older-genera