Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference.

arXiv:2609.15504v2 Announce Type: replace-cross Abstract: Lossless speculative decoding is typically defined at the algorithmic level: a speculative procedure proposes multiple tokens and a verification procedure is designed to preserve the output trajectory o