Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?

arXiv:2603.02069v2 Announce Type: replace-cross Abstract: We study scaling laws of signSGD under a power-law random features (PLRF) model that accounts for both feature and target decay. We analyze the population risk of a linear model trained with one-pass si