Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search.

arXiv:2609.01431v3 Announce Type: replace-cross Abstract: Optimal hyperparameter scaling laws describe how the best hyperparameters for large language model (LLM) training change with model and data scale, enabling practitioners to predict optimal configuratio