Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models.

arXiv:2610.38981v1 Announce Type: cross Abstract: Speculative decoding (SD) accelerates autoregressive inference by leveraging the memory bound nature of decoding, verifying multiple draft tokens in a single forward pass. For mixture-of-experts (MoE) models, p