MoLS: SNR-Guided Module-Wise Learning-Rate Calibration for Large Language Models.

arXiv:2605.05794v2 Announce Type: replace-cross Abstract: Large language models (LLMs) contain functionally distinct modules with heterogeneous gradient statistics. Although Adam(W) adapts updates at the parameter level, selecting relative learning rates acros