Revisiting the 16x Pretraining Compute Gap between Diffusion and Autoregressive Language Models.

arXiv:2610.17590v1 Announce Type: cross Abstract: Diffusion language models (DLMs) have attracted wide attention for their fast, parallel decoding, but they are believed to be far less efficient to train than autoregressive language models (ARMs): a widely cit