Deft Scheduling of Dynamic Cloud Workflows with Varying Deadlines via Mixture-of-Experts
cs.AI
Submitted: 2026-05-31
Updated: 2026-06-02
Comments: This paper has been accepted by the Fourteenth International Conference on Learning Representations (ICLR 2026)
Journal ref: The Fourteenth International Conference on Learning Representations (ICLR 2026), 2026
Code: https://github.com/yashenCS/DEFT
License: http://creativecommons.org/licenses/by/4.0/
The gist: Workflow scheduling in cloud computing demands the intelligent allocation of dynamically arriving, graph-structured workflows with varying deadlines onto ever-changing virtual machine resources.
Terminology
Abstract
Workflow scheduling in cloud computing demands the intelligent allocation of dynamically arriving, graph-structured workflows with varying deadlines onto ever-changing virtual machine resources. However, existing deep reinforcement learning (DRL) schedulers remain limited by rigid, single-path inference architectures that struggle to handle diverse scheduling scenarios. We introduce DEFT (D eadline-p E rceptive Mixture-o F-Exper t s), an innovative DRL policy architecture that leverages a specialized mixture of experts, each trained to manage different levels of deadline tightness. To our knowledge, DEFT is the first to introduce and validate a Mixture-of-Experts architecture for dynamic cloud workflow scheduling. By adaptively routing decisions through the most appropriate experts, DEFT is capable of meeting a broad spectrum of deadline requirements that no single expert can achieve. Central to DEFT is a graph-adaptive gating mechanism that encodes workflow DAGs, task states, and VM conditions, using cross-attention to guide expert activation in a fine-grained, deadline-sensitive manner. Experiments on dynamic cloud workflow benchmarks demonstrate that DEFT significantly reduces execution cost and deadline violations, outperforming multiple state-of-the-art DRL baselines.
Sources
- SHIELD: Multi-task Multi-distribution Vehicle Routing Solver with Sparsity and Hierarchy
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Graph Attention Networks
- MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems