Addressing Exogenous Variability in Cooperative Multi-Agent Reinforcement Learning
cs.MA, cs.LG
Submitted: 2026-11-17
Updated: 2026-11-17
Comments: 28 pages. Includes supplementary material. Accepted at NeurIPS 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Cooperative multi-agent reinforcement learning (MARL) often fails under exogenous variability such as shifts in opponents or environment regimes that cannot be controlled by the team but alter the…
Terminology
Abstract
Cooperative multi-agent reinforcement learning (MARL) often fails under exogenous variability such as shifts in opponents or environment regimes that cannot be controlled by the team but alter the transition dynamics. We formalize this challenge as Exogenous Dec-POMDP (ED-POMDP), in which an episode-level exogenous regime, selected independently of the team, indexes the dynamics, and the global state is partitioned into components that are directly responsive to the current team action and components that are not directly action-responsive at the current step. This formulation exposes mediated influence: while team actions do not directly update the latter at the current step, they can shape their later evolution through induced changes in the former. Based on this, we propose LEICA, a CTDE-compatible algorithm that learns history-conditioned context representations for both components and shapes policy updates using agent-wise, influence-weighted counterfactual intrinsic rewards. Across SMAX benchmarks with opponent strategy shifts, LEICA consistently improves both training performance and generalization to unseen opponents' strategies over existing baselines, supporting the usefulness of influence-weighted shaping under exogenous train-test regime shifts.
Related papers
- MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
- Iterative Negotiation and Oversight: A Case Study in Decentralized Air Traffic Management
- Towards using Large Language Models for Risk Negotiation and Consensus-building in One Health contexts
- A-MapReduce: Executing Wide Search via Agentic MapReduce
- AgentCue: Statistical Summaries for Guiding Decentralized Multi-Agent Self-Organization
- Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning