Adaptive Multi-Token Prediction: How far should we go?

arXiv:2610.28619v1 Announce Type: new Abstract: Multi-token prediction (MTP) can accelerate language-model decoding by drafting several tokens and verifying them in a single target-model pass. However, the best speculation horizon changes with context: longer