LightMTP: Lightweight Latent Multi-Token Prediction.

arXiv:2610.06031v1 Announce Type: cross Abstract: Next-token prediction (NTP) is the standard pretraining objective for large language models, yet it provides an explicit training signal only for the immediate next token, which can lead models to exploit local