ValuSpec: Plug-and-Play Candidate Valuation before Target Verification for Tree-Based Speculative Decoding.

arXiv:2610.31338v1 Announce Type: new Abstract: Tree-based speculative decoding accelerates large language model inference by verifying multiple candidate branches in parallel. However, in practical serving scenarios, similar requests repeatedly induce overlap