PRECOCIAL: Pretraining with Code Curriculums via Iterative Search for Accelerated Learning.

arXiv:2610.24970v1 Announce Type: cross Abstract: Fully synthetic pretraining data generated by symbolic programs can bootstrap downstream training on naturalistic tasks, such as language modeling, but the generating programs have so far been designed by hand.