G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies
cs.AI
Submitted: 2026-09-25
Updated: 2026-09-25
Comments: 24 pages, including appendices. Project page: https://g2maf.github.io/
Code: https://github.com/g2maf/G2MAF
Project page: https://g2maf.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: Offline multi-agent reinforcement learning (MARL) learns cooperative policies from fixed datasets without further environment interaction and a learned policy is frozen at deployment.
Terminology
Abstract
Offline multi-agent reinforcement learning (MARL) learns cooperative policies from fixed datasets without further environment interaction and a learned policy is frozen at deployment. Such a frozen policy typically proposes a single joint action and executes it directly at deployment time. However, this one-shot deployment often commits to a suboptimal proposal, even when better nearby alternatives remain consistent with the behavior data. To address this issue, we propose Gradient Guided Multi Agent Flow (G2MAF), a refinement framework for optimizing joint policies at test-time. G2MAF applies one globally normalized, projected critic gradient to guide and coordinate all agents' corrections while keeping the action both feasible and close to the frozen policy proposal. Across 24 MPE and SMAC settings, its canonical variant improves 20 frozen settings, with mean relative gains of 9.2% on MPE and 8.9% on SMAC, with model inference latency increased by about 6% only.
Sources
- FlowQ: Energy-Guided Flow Policies for Offline Reinforcement Learning
- Test-time Offline Reinforcement Learning on Goal-related Experience
- One-Step Flow Policy Mirror Descent
- Scaling Offline RL via Efficient and Expressive Shortcut Models
- Diffusion Guidance Is a Controllable Policy Improvement Operator
- OM2P: Offline Multi-Agent Mean-Flow Policy
- Value-Guidance MeanFlow for Offline Multi-Agent Reinforcement Learning
- Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
- Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning
- CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems