Accelerating Sharded Data Parallelism at Scale with Federated Learning.

arXiv:2609.20359v1 Announce Type: cross Abstract: The symbiotic scaling of artificial intelligence models and high-performance computing systems continually creates algorithmic challenges in their convergence. Foundation models (FMs) are a crucial example, req