Dissecting the Carbon-SLO Tension in Geo-Distributed LLM Serving

Name: Dissecting the Carbon-SLO Tension in Geo-Distributed LLM Serving
Uploaded: 2026-10-18T13:34:25.756827+00:00
Duration: 283 s
Description: The paper designed a latency-centric simulator and schedulers to reduce carbon emissions from geo-distributed LLM serving. It models continuous batching, queueing delays, and energy consumption across regions. The resulting MILP scheduler found a balance, achieving up to 60% carbon reduction while maintaining an average SLO violation of only 3%.

The paper designed a latency-centric simulator and schedulers to reduce carbon emissions from geo-distributed LLM serving. It models continuous batching, queueing delays, and energy consumption across regions. The resulting MILP scheduler found a balance, achieving up to 60% carbon reduction while maintaining an average SLO violation of only 3%.

Open the full episode