Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods.

arXiv:2602.03802v2 Announce Type: replace-cross Abstract: Modern distributed optimization methods mostly rely on traditional synchronous approaches, despite substantial recent progress in asynchronous optimization. We revisit Synchronous SGD and its robust var