Scaling LLM Inference via Parallelism: Throughput-Latency Trade-off.

arXiv:2610.19405v1 Announce Type: cross Abstract: Large Language Model (LLM) serving systems must balance latency and throughput while efficiently utilizing distributed GPU resources. This challenge arises because inference consists of two fundamentally differ