Only Accepted Tokens Count: An Energy Accounting of Speculative Decoding Under Latency Targets.

arXiv:2610.40287v1 Announce Type: cross Abstract: Large language model serving is limited by facility power. Energy per token determines how many tokens a datacenter can serve, while a latency target bounds how quickly each user receives them. During decode, e