MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving
summary
The gist
Vision-language-action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings due to being trained under traditional imitation learning…
In short
MAPLE addresses brittleness in vision-language-action (VLA) models used for driving by training them in a closed-loop setting with reactive agents. It simulates ego vehicles and surrounding agents entirely within the VLA model's latent space using multi-agent rollouts. This framework improves safety and realism through supervised fine-tuning and a diversity reward, achieving state-of-the-art performance on Bench2Drive.
Key concepts
- VLA Models
- Vision-language models that take visual input (like camera feeds) and generate actions (like steering or braking). They are effective for motion planning but struggle when trained in real, interactive driving scenarios where other agents react dynamically.
- MAPLE Framework
- A novel framework simulating ego vehicles and multiple reactive agents inside the VLA model's latent space. It uses two stages: supervised fine-tuning on ground-truth data and reinforcement learning with rewards to encourage safe, realistic interactions between agents.
- Multi-agent Rollout
- The mechanism that jointly evolves the ego vehicle and surrounding reactive agents autoregressively within the latent space of the VLA model. This allows for closed-loop, multi-step agent interactions during training without needing pixel-level simulation, significantly reducing computational cost.
- Diversity Reward
- A reward signal used during reinforcement learning that encourages surrounding agents to exhibit diverse behaviors at training time. It measures the average pairwise L1 distance between behavior descriptors across different planner selections, promoting robust policies capable of handling varied agent interactions.
Terminology used across episodes
This episode discusses
- MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving · Paper Radio
- Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
- Qwen2.5-VL Technical Report
- ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation
- EMMA: End-to-End Multimodal Model for Autonomous Driving
- CaRL: Learning Scalable Planning Policies with Simple Rewards
- Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving
- AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning
- A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
- Learning Personalized Driving Styles via Reinforcement Learning from Human Feedback
- Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Driving
- GPT-Driver: Learning to Drive with GPT
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Motion Transformer with Global Intention Localization and Local Movement Refinement
- DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models
- Generative Scenario Rollouts for End-to-End Autonomous Driving · Paper Radio
- Diffusion-Based Planning for Autonomous Driving with Flexible Guidance
- AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
The paper
MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving · Read on arXiv
Qualcomm AI Research
Vision-language-action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings due to being trained under traditional imitation learning framework. Existing closed-loop supervision approaches lack scalability and fail to completely model a reactive environment. We propose MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model. The ego vehicle and nearby traffic agents are independently controlled over multi-step horizons, while being reactive to other agents in the scene, enabling closed-loop training. MAPLE consists of two training stages: (1) supervised fine-tuning on the latent rollouts based on ground-truth trajectories, followed by (2) reinforcement learning with global and agent-specific rewards that encourage safety, progress, and interaction realism. We further propose diversity rewards that encourage the model to generate planning behaviors that may not be present in logged driving data. Notably, our closed-loop training framework is scalable and does not require external simulators, which can be computationally expensive to run and have limited visual fidelity to the real-world. MAPLE achieves state-of-the-art driving performance on Bench2Drive and demonstrates scalable, closed-loop multi-agent play for robust E2E autonomous driving systems.
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving".
Dev: Here's the gist of the paper. Vision-language-action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings due to being trained under traditional imitation learning framework MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model.
Rosa: Pruning redundant reasoning tokens MAPLE proposes a framework that simulates the behavior of the ego-vehicle and multiple reactive agents in the latent space of a VLA model.
Dev: This addresses the challenge of simulating sensory data while also training an E2E planner in a closed-loop environment in the presence of reactive agents.
Rosa: The framework consists of two training stages: (1) supervised fine-tuning on the latent rollouts based on ground-truth trajectories, followed by (2) reinforcement learning with global and agent-specific rewards that encourage safety, progress, and interaction realism.
Title and authors: Rosa: So we're looking at this paper called MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving. It's tackling the problem of these vision language action models, or VLA models, which are supposed to be great for planning motion end-to-end. But the authors point out that they get brittle when you put them in a real closed loop setting because they're trained using traditional imitation learning methods <ref:2605.14201#pg1>.
Dev: Yeah, and those existing closed-loop supervision approaches just don't scale well and they don't actually model a reactive environment properly <ref:2605.14201#pg2>. They struggle with covariate shift where small changes in the driving behavior cause the planning to fail over time <ref:2605.14201#pg2>.
Taro: It’s that sparse ground-truth trajectory labels that make it really hard to learn a robust policy just from those alone <ref:2605.14201#pg2>. We need something that supports closed-loop, multi-agent interaction without needing costly simulators <ref:2605.14201#pg3>.
Rosa: Exactly, and MAPLE proposes a novel framework for reactive, multi-agent rollout of a dynamic driving scenario entirely in the latent space of the VLA model <ref:2605.14201#pg1>. They want to let both the ego vehicle and nearby traffic agents control themselves over multi-step horizons while reacting to each other in real time <ref:2605.14201#pg1>.
Dev: The framework sets up two main training stages then, first supervised fine-tuning on those latent rollouts using ground truth trajectories, and then reinforcement learning with both global and agent specific rewards focused on safety and progress <ref:2605.14201#pg1>.
Taro: So how do they actually build that training signal for the agents in this closed loop? Is it just feeding them things in?
Rosa: It involves a multi-agent rollout mechanism where both the ego vehicle and the surrounding reactive agents evolve together through autoregressive rollouts inside that latent space <ref:2605.14201#pg1>. This lets them have those closed-loop, multi-step agent interactions happening entirely in the latent space, which cuts down on computational cost compared to training at the pixel level <ref:2605.14201#pg3>.
Dev: And that rollout mechanism is what allows them to define their supervised fine-tuning objective loss as the sum of per-step planning and motion prediction losses across the entire rollout horizon <ref:2605.14201#pg1>. It’s a lot of data generation for the supervised part, I imagine.
Taro: That sounds like they are simulating the future state transitions between those different agents in this latent space, which stabilizes the token space over a certain time horizon T <ref:2605.14201#pg3>. It’s an attempt to make that simulation more stable so the planning works better when it gets to RL <ref:2605.14201#pg3>.
Rosa: And then they add this diversity reward, which encourages out-of-distribution behavior from the surrounding agents right at training time <ref:2605.14201#pg3>. They define that as the average pairwise l one distance between behavior descriptors across different planner selections <ref:2605.14201#pg3>.
Title and authors: Dev: That diversity reward is important because it gives them a richer feedback signal during training than just following ground truth trajectories <ref:2605.14201#pg3>. It's designed to prevent the policy from collapsing by promoting distinct agent behaviors across different planner selections <ref:2605.14201#pg3>.
Taro: So, if we look at the results on Bench2Drive, they say this multi-agent rollout mechanism combined with the diversity reward actually improves driving performance by over twenty five percent <ref:2605.14201#pg3>. That's a significant boost in complex scenarios like merging and emergency braking <ref:2605.14201#pg3>.
Rosa: And to put that number in context, the ablation study on the number of reactive agents shows that increasing that number consistently improves both the driving score and the success rate <ref:2605.14201#pg3>. They saw gains of up to two point six driving score and four point eight success rate when they used eight reactive agents <ref:2605.14201#pg3>.
Dev: Including that diversity reward seems to push the performance even further, getting them to the best overall results on Bench2Drive with a driving score of eighty five point two and a success rate of sixty seven point five <ref:2605.14201#pg3>. That's impressive interaction realism they’re achieving in that setting.
Taro: From an engineering standpoint, I'm interested in the inference computation analysis <ref:2605.14201#pg3>. They say MAPLE requires twelve thousand nine hundred eighty-eight gflops per inference and has a mean latency of four hundred eighty seven point seven milliseconds with a P ninety percent latency of five hundred twenty nine point one milliseconds <ref:2605.14201#pg3>.
Rosa: That latency figure is something to consider for real-time deployment, but they also mention that the rollouts are disabled at inference time without adding any extra overhead <ref:2605.14201#pg3>. That makes it seem like it’s a feasible path for closed-loop training systems <ref:2605.14201#pg3>.
Dev: So, what does this mean for the practical use of these VLA models in dynamic driving scenarios? It seems they're making them much more robust to those tricky, unexpected interactions we see on the road <ref:2605.14201#pg2>.
Taro: What it means is that instead of relying only on sparse labels from logged data, we can train these models to handle continuous adaptation to the evolving behavior of other agents <ref:2605.14201#pg2>. This moves us closer to systems that can handle long-tail events not present in the original training logs <ref:2605.14201#pg3>.
Rosa: And that scalability is key because it lets us co-plan the ego vehicle and multiple reactive agents entirely within the latent space, solving the simulation problem without needing pixel level inputs for future observations <ref:2605.14201#pg3>. It addresses how we simulate sensory data while training an end-to-end planner in a closed loop <ref:2605.14201#pg1>.
Title and authors: Dev: So the core idea is using the latent space to handle the interaction dynamics, which is computationally cheaper than simulating everything at pixel resolution <ref:2605.14201#pg3>. It’s about making the interaction learning scalable for end-to-end autonomous driving systems that need to adapt constantly <ref:2605.14201#pg2>.
Taro: I wonder how this framework handles scenarios where the environment misbehaves in a completely novel way, not just deviating from known trajectories <ref:2605.14201#pg3>. How gracefully does it recover when the world throws something totally new at it?
Rosa: The diversity reward is specifically designed to encourage that kind of out-of-distribution behavior during training, giving them richer feedback signals to handle those novel situations <ref:2605.14201#pg3>. It’s about making sure the agent learns more than just what it saw in the logs <ref:2605.14201#pg2>.
Dev: If we look at the overall picture, MAPLE is showing how to move beyond simple imitation learning by using interaction and diversity to create a training signal that supports closed-loop adaptation <ref:2605.14201#pg1>. It's about making the planning robust in an environment where agents are constantly changing their plans <ref:2605.14201#pg3>.
Taro: So, for someone just listening to the show, what does this paper actually change for them? It means we might see autonomous systems that can truly react and co-plan with other vehicles in complex traffic situations, not just follow a pre-programmed path <ref:2605.14201#pg3>.
Rosa: It changes how we build these planners by moving them into a latent space where they can learn interaction through play rather than just watching examples <ref:2605.14201#pg3>. It’s about making the training environment more representative of real-world complexity <ref:2605.14201#pg1>.
Dev: And from a control perspective, it suggests that we can train these complex planners in a way that respects the constraints of real-time loops and latency requirements <ref:2605.14201#pg3>. The inference speed of four hundred eighty seven point seven milliseconds shows it's actually viable for deployment <ref:2605.14201#pg3>.
Taro: I think the main thing here is that it gives us a way to train these large models in an environment that actively challenges them with diverse behaviors <ref:2605.14201#pg3>. It’s about building resilience through active, multi-agent interaction <ref:2605.14201#pg3>.
Rosa: So, to wrap up on MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving, it proposes a way to train VLA models by letting the ego vehicle and its neighbors interact in a latent space during rollout <ref:2605.14201#pg1>. This involves supervised fine tuning followed by reinforcement learning with rewards that encourage diversity among agents <ref:2605.14201#pg3>.
Title and authors: Dev: The results show this approach significantly boosts driving performance on Bench2Drive, achieving a driving score of eighty five point two and a success rate of sixty seven point five <ref:2605.14201#pg3>. It's about making the learning process more robust to those tricky, changing agent behaviors <ref:2605.14201#pg3>.
Taro: It moves us toward systems that can handle continuous adaptation by simulating interactive planning entirely in the latent space <ref:2605.14201#pg3>. We're getting closer to something that doesn't just follow a script but actually plays around with the environment <ref:2605.14201#pg3>.
Rosa: So, MAPLE shows how we can use multi-agent simulation and self play, even when extended to end-to-end planners, by simulating future observations in the latent space <ref:2605.14201#pg3>. It’s a new way to tackle the difficulty of learning robust policies without relying solely on sparse ground truth labels <ref:2605.14201#pg2>.
Dev: The paper also flags that it still relies on symbolic planners when it comes to adversarial scenario generation, meaning it doesn't completely escape the limitations of those symbolic representations <ref:2605.14201#pg3>. That’s a limitation we have to keep in mind <ref:2605.14201#pg3>.
Taro: Right, so the paper is strong on closed-loop interaction and diversity for improving performance, but the reliance on symbolic simulators for generating adversarial scenarios is a point they bring up <ref:2605.14201#pg3>. That's where future work might need to focus <ref:2605.14201#pg3>.
Rosa: For now, MAPLE gives us a concrete framework for how to train these models in a way that explicitly supports closed-loop, multi-agent interaction without needing external simulators <ref:2605.14201#pg1>. It’s about making the training process itself more realistic and robust <ref:2605.14201#pg3>.
Dev: That means we are getting a better tool for training autonomous systems that need to handle the dynamic, unpredictable nature of real-world driving environments <ref:2605.14201#pg2>. We’re getting closer to policies that can handle those long-tail events <ref:2605.14201#pg3>.
Taro: It’s about creating a training signal that encourages distinct agent behaviors across different planner selections to build better resilience <ref:2605.14201#pg3>. That interaction between the planning and the reward structure is key <ref:2605.14201#pg3>.
Rosa: So, we're leaving MAPLE as a framework that tackles the closed-loop training problem by rolling out agents in latent space and using diversity rewards to push them toward more realistic, robust behaviors <ref:2605.14201#pg3>. It’s a solid step forward in making these VLA models truly reactive <ref:2605.14201#pg3>.
Dev: A solid step, yeah, but we're always watching how those inference computations scale as they deploy this on the road <ref:2605.14201#pg3>. That latency figure of four hundred eighty seven point seven milliseconds is what matters for real-time operation <ref:2605.14201#pg3>.
Taro: We'll keep watching how they tackle those limitations, especially when it comes to truly novel, unexpected agent interactions <ref:2605.14201#pg3>. That’s the next big test for these systems <ref:2605.14201#pg3>.
The paper's summary: Rosa: So, MAPLE is basically taking those big vision language action models, right, and instead of just training them by showing them examples of perfect driving paths, they’re making the ego vehicle and all those other reactive agents play together inside a hidden space called the latent space.
Dev: Exactly. It’s about simulating everything in that latent space—the ego car and the surrounding agents—to get a better training signal than just looking at pictures of what happened before.
Rosa: They set up two main training steps there, first they use supervised fine-tuning on those simulated rollouts, which is like teaching the model how to follow ground truth paths in that latent space.
Dev: Then they add reinforcement learning with these rewards that push for safety and progress, not just following the path exactly.
Rosa: And this is where it gets interesting, because they introduce a diversity reward. This reward encourages the surrounding agents to behave differently at training time, even when they’re supposed to be reacting in a certain way.
Dev: That diversity reward is meant to give them richer feedback signals during training and stop the policy from collapsing into just one predictable behavior.
Rosa: So, by having these agents interact and play around with each other in that latent space while being rewarded for being diverse, they get this much better driving performance on Bench2Drive—a twenty-five percent boost or more.
Dev: That’s a big jump when you think about how those models usually learn from sparse labels from logged driving data. This suggests we can train these planners to handle long-tail events that weren't even in the original training logs.
Rosa: And on the engineering side, they show that this whole process is done entirely in the latent space, which saves a ton of computation compared to training at the pixel level where you’d be dealing with raw images.
Dev: But we gotta talk about deployment. They found it takes twelve thousand nine hundred eighty-eight gflops for one inference and a latency of four hundred eighty seven point seven milliseconds when you use eight reactive agents, which is pretty decent for real-time operation on an NVIDIA H100 GPU.
Rosa: That’s the practical side, though. It means we’re looking at a way to train these models that actually supports continuous adaptation to changing agent behavior in real-world scenarios without needing massive simulators for every single test case.
Dev: So the paper gives us a concrete framework for training end-to-end planners by making the environment itself dynamic and interactive through latent space rollout.
Rosa: It’s about moving away from just imitation learning, where you just watch what's on the road, to a system that learns how to play and adapt in complex multi-agent situations.
Dev: It tackles the simulation problem by doing it in the latent space, which is way more efficient than simulating everything at pixel resolution.
Rosa: The main implication is that we can train these models to be much more resilient when other agents don't just follow known trajectories but start behaving in unpredictable ways because of that diversity reward.
Dev: That interaction between the planning and the reward structure is key, building resilience by forcing the model to handle those tricky, changing agent behaviors.
Rosa: It’s a solid step toward autonomous systems that can truly co-plan with other vehicles in complex traffic situations rather than just following a pre-programmed path.
Dev: We'll keep watching how they tackle those limitations, especially when it comes to truly novel, unexpected agent interactions and making sure that inference speed holds up in the field.
The paper's improvements: Rosa: So, if we look at what MAPLE suggests as improvements, they’re really focusing on how they get better feedback when things go sideways in the real world.
Dev: Right, they’re talking about how that diversity reward helps them handle those long-tail events—the stuff you don't see in your training logs—by pushing the agents to try different behaviors during training.
Rosa: That means the resulting policies are supposed to be more robust because they aren't just following the expected script; they’re learning how to react when things get weird or unexpected.
Dev: And another big thing is that this entire framework lets them do reactive co-planning for the ego vehicle and multiple agents entirely inside that latent space, which solves the problem of simulating sensory data without needing external simulators.
Rosa: So instead of relying on a perfect simulator to generate every possible driving situation, they use the latent space itself to handle those complex interactions during training.
Dev: It’s about making the training process more realistic by letting it actively challenge the model with diverse behaviors right there in the simulation.
Rosa: This makes it feasible for building end-to-end autonomous driving systems that need continuous adaptation to evolving agent behavior on the road, not just one static set of rules.
Dev: And on the computation side, they show that even with eight reactive agents, they can still get a mean latency of four hundred eighty seven point seven milliseconds per inference when running the system.
Rosa: That means this approach isn't just theoretically sound; it’s showing it’s practical enough to be deployed in a real-time loop on hardware like an NVIDIA H100 GPU.
Dev: So, what this changes for us as engineers is that we can train these complex planners in a way that respects the constraints of real-time loops and latency requirements, which is a huge part of building a safety case.
Rosa: It moves the focus from just learning to follow examples to learning how to play around with the environment and adapt dynamically when things go wrong.
Dev: The paper also flags that it still relies on symbolic planners when it comes to generating adversarial scenarios, which means they don't completely escape the limitations of those symbolic representations.
Rosa: That’s a limitation we have to keep in mind, so while MAPLE is great for closed-loop training, there are still areas where the system might struggle with truly novel or unexpected agent interactions.
Dev: Exactly. The framework is strong on interaction and diversity, but the reliance on those symbolic tools for testing extreme scenarios means future work needs to focus on making those adversarial tests even more robust.
Conclusion: Tom: So we're wrapping up on MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving, which proposes training models by letting the ego vehicle and its neighbors interact in a latent space during rollout.
Rosa: Basically, it’s a new way to teach these vision language action models by having them play together in that hidden space instead of just learning from static examples.
Dev: The main implication is that we can train these models to handle continuous adaptation to changing agent behavior, moving us closer to systems that deal with long-tail events on the road.
Taro: I still think the way it uses diversity rewards is really clever for building resilience because it actively pushes the agents toward more realistic, varied behaviors during training.
Rosa: Right, and even though they flagged that they still use symbolic planners for generating adversarial scenarios, this framework gives us a concrete tool for training these models in a way that explicitly supports closed-loop, multi-agent interaction without needing external simulators.
Dev: The inference speed is decent too with four hundred eighty seven point seven milliseconds latency and twelve thousand nine hundred eighty eight gflops per inference on an NVIDIA H100 GPU, which shows it’s actually viable for deployment.
Taro: So the paper gives us a framework for training these models in a way that makes them more robust to unpredictable agent interactions through active, multi-agent interaction.
Rosa: It’s about building resilience by simulating future observations in the latent space, which is a new way to tackle the difficulty of learning policies without relying solely on sparse ground truth labels.
Dev: We're getting closer to policies that can handle those long-tail events, but we still have to be careful about how those symbolic planners generate truly novel agent scenarios.
Taro: That’s the next big test for these systems, figuring out how gracefully they recover when the world throws something totally new at them.
Rosa: We'll keep watching how they tackle those limitations, especially when it comes to truly novel or unexpected agent interactions in the real world.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies