AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization.

arXiv:2609.36662v1 Announce Type: cross Abstract: The pre-training of Large Language Models (LLMs) is increasingly conducted across multiple data centers. As training scales to a larger number of accelerators, the fraction of time spent on computation decrease