Source · Tier B · Preprint
DiLoCo: Distributed Low-Communication Training of Language Models
A. Douillard, Q. Feng, A. A. Rusu, R. Chhaparia, Y. Donchev, A. Kuncoro, M. Ranzato, A. Szlam, J. Shen. 2024. ICML 2024 Workshop on Advancing Neural Network Training (WANT).
| Link | https://arxiv.org/abs/2311.08105 |
|---|---|
| arXiv | 2311.08105 |
| Version | v1 submitted 14 November 2023; v3 revised 23 September 2024. Poster at the 2nd Workshop on Advancing Neural Network Training (WANT@ICML 2024), per the ICML 2024 page (https://icml.cc/virtual/2024/37144), checked 2026-09-24. |
| Accessed | 2026-09-23 |