Halley: A Predictive Control Plane for Optical AI Data Center Networks
cs.NI, cs.DC
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 24 pages, 18 figures
Project page: https://steveicarus.github.io/iverilog
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: A frontier AI model is trained in an AI data center, on tens of thousands of accelerators joined by a dedicated network, and every training step ends with all of them exchanging data at once.
Terminology
Abstract
A frontier AI model is trained in an AI data center, on tens of thousands of accelerators joined by a dedicated network, and every training step ends with all of them exchanging data at once. Between racks, that network is optical, and each of its four layers, from lasers to switches, runs a fixed or after-the-fact policy. Lasers sharing an enclosure, cooling, and power can fail together. At a fixed spare ratio, correlated failure provably caps a pool's availability however large it grows. In this paper, we present a comprehensive predictive control plane framework, called Halley, that reads each resource's next state and acts before the failure occurs. It has four components, each carrying a formal guarantee: LightReserve that sizes the laser pool to provide a coverage bound, FoldFit that schedules the switch within 11/9 of the optimal, ThermaLead that holds a microring with one-step-ahead control, and TriageGuard that protects the highest-risk modules first. Halley gives each layer a predictive controller that defers to the deployed policy when unsure. Composed from simulations at that point, effective goodput, the share of ideal throughput kept, rises from 65.2% to 76.5% if a window that exhausts its spares is lost. If only the dark channels' bandwidth is lost, it reaches 67.4%. Per megawatt, that is up to 1.03--1.17 times the training; for the same training, up to 3.3--14.7% less data center capacity. 's fallback circuits are formally verified and are shown to achieve close timing on a commercial FPGA, and pass a 130 nm signoff.
Sources
- Scalable Training of Mixture-of-Experts Models with Megatron Core
- The Llama 3 Herd of Models
- DeepSeek-V3 Technical Report
- Training LLMs with Fault Tolerant HSDP on 100,000 GPUs
- Scheduling Parallel Optical Circuit Switches for AI Training
- Flux: Optimal Scheduling of Optical Circuit Switches for LLM Training
- Beyond Traffic Matrix: DELTA -- A DAG-Aware OCS Logical Topology Optimization for AIDCs
- Non-Splitting Coflow Scheduling with Provable Guarantees in Heterogeneous Parallel Networks
- Scheduling Coflows in Multi-Core OCS Networks with Performance Guarantee
- Mission Apollo: Landing Optical Circuit Switching at Datacenter Scale
- RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training
- DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training
- APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generation
- Laminar: A Scalable Asynchronous RL Post-Training Framework
- Role-Based Fault Tolerance System for LLM RL Post-Training
Related papers
- Predicting Networks Before They Happen: Experimentation on a Real-Time V2X Digital Twin
- MultiMoQ: Multi-Access Media-Over-QUIC for Robust Immersive Video Streaming
- HiFiNet: Hierarchical Fault Identification in Wireless Sensor Networks via Edge-Based Classification and Graph Aggregation
- Flux: Optimal Scheduling of Optical Circuit Switches for LLM Training
- Tutorial on Closed-Loop and Application-Aware Digital Twins for Future Wireless Networks
- Claude Code Complete User Handbook