Rethinking Pre-pretraining.

arXiv:2610.40051v1 Announce Type: cross Abstract: We report that allocating compute to synthetic pre-pretraining (PPT) never outperforms pure natural-language training in our compute-matched experiments. This holds across four PPT curricula (k-Dyck, neural cel