Chronos: Prescheduled circuit switching for LLM training

作者：Sundararajan Renganathan, Nick McKeown · 发表于：NAIC · 年份：2025 · DOI：10.1145/3748273.3749210 · 被引用次数：10 · 研究领域：Computer Science

Hundreds of thousands of accelerators are used to train LLMs, with accelerators connected by packet-switched AI fabrics. In this paper, we ask if the fabric can be built entirely from time-synchronized circuit switches instead. The goal would be to reduce power, increase switching capacity, or reduce the number of network tiers. It appears to be possible, because traffic patterns are largely known a priori. We describe a tool that analyzes the training code and deduces a sequence of permutations that will correctly schedule a crossbar throughout the training run. Expert parallelism (used with mixture-of-experts models) is the only form of parallelism that cannot be pre-scheduled. For MoE traffic, we show how Birkhoffvon Neumann decomposition can be used to schedule the crossbar on demand.