Training LLMs Beyond Next Token Prediction - Filling the Mutual Information Gap.

arXiv:2511.00198v2 Announce Type: replace-cross Abstract: Autoregressive language models are usually trained with next-token prediction under a fixed left-to-right target order. This paper studies target-token organization as a training-time design choice acro