When Entropy Is Not Enough: Reclaiming Lost Semantics in LLM Output Length Prediction.

arXiv:2608.15592v3 Announce Type: replace Abstract: Efficient LLM serving is often bottlenecked by the need to pad sequences to a fixed maximum length, and this wastes compute and degrades throughput. Output length prediction enables length-aware scheduling an