How Does Local Landscape Geometry Evolve in Language Model Pre-Training?

arXiv:2609.39767v1 Announce Type: cross Abstract: The scale and expense of pre-training language models make efficient hyperparameter tuning essential, yet a principled guidance is still missing. In this work, we analyze language model pre-training dynamics fr