Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features
summary
The gist
Large video diffusion models provide rich spatiotemporal priors for autonomous driving, but existing world-action models often inherit the cost of iterative future-video generation even though…
In short
Adaptive-WAM is a quality-aware world-action model that uses intermediate features from video diffusion models to make driving decisions efficiently. Instead of generating full future videos, it selectively executes parts of the model based on trajectory quality scores, achieving high planning performance with significantly reduced latency.
Key concepts
- Adaptive-WAM
- This is a quality-aware multi-exit planner built on a Wan2.2 backbone. It dynamically selects the appropriate depth of video diffusion features to use for planning, terminating inference early once a trajectory meets a predefined quality threshold.
- Video Diffusion Models
- These large models provide rich spatiotemporal information useful for autonomous driving. The paper uses intermediate layers from these models as robust representations for planning, suggesting that planning can be done effectively before the entire video generation process is complete.
- Quality-Guided Routing
- This mechanism involves using a lightweight scorer to predict planning sub-scores for different decoded trajectories. This score guides the system to select only the most promising trajectory, avoiding unnecessary computation by skipping deeper or lower-quality planning steps.
Terminology used across episodes
This episode discusses
- Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features · Paper Radio
- CLOVER: Closed-Loop Value Estimation and Ranking for End-to-End Autonomous Driving Planning
- VaViM and VaVAM: Autonomous Driving through Video Generative Modeling
- VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning
- GAIA-1: A Generative World Model for Autonomous Driving
- DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT
- OmniNWM: Omniscient Driving Navigation World Models
- DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving
- ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- DriveVA: Video Action Models are Zero-Shot Drivers
- DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Denoising Diffusion Implicit Models
- BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks
- Wan: Open and Advanced Large-Scale Video Generative Models
- MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Driving
- DriveLaW:Unifying Planning and Video Generation in a Latent Driving World
- Fast-WAM: Do World Action Models Need Test-time Future Imagination?
- Doe-1: Closed-Loop Autonomous Driving with Large World Model
The paper
Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features · Read on arXiv
Sining Ang, Yuguang Yang, Yan Wang
Institute for AI Industry Research (AIR) at Tsinghua University · Department of Automation, University of Science and Technology of China · School of Electronic Information Engineering, Beihang University
Large video diffusion models provide rich spatiotemporal priors for autonomous driving, but existing world-action models often inherit the cost of iterative future-video generation even though deployment only requires an ego trajectory. We ask a more basic question: how much of a video diffusion model must be executed to make a reliable driving decision? Through a controlled study of video denoising timesteps and Diffusion Transformer (DiT) depth, we find that planning performance is largely insensitive to the tested video-noise levels, whereas strong trajectories can already be decoded from intermediate layers. Based on this observation, we introduce Adaptive-WAM, a quality-aware multi-exit planner built on a Wan2.2-5B backbone. Trajectory diffusion heads are attached to selected DiT blocks, and a lightweight trajectory-quality scorer terminates inference once the best trajectory decoded so far satisfies a quality threshold; otherwise, computation continues from the cached hidden state to a deeper exit. The deployed planner therefore avoids the iterative classifier-free denoising loop and VAE decoding required for future-video synthesis, while dynamically allocating backbone depth according to trajectory quality. On NAVSIM, the adaptive single-trajectory planner achieves 90.8 PDMS; a separate fixed-exit variant reaches 92.6 PDMS with 64 proposals. It further obtains 89.9 EPDMS on NAVSIM v2, yielding the best reported results among the compared front-view video world-model planners. Without target-domain fine-tuning, Adaptive-WAM transfers to nuScenes with 0.88 m average L2 error and a 0.08% collision rate. On an A100, adaptive routing improves PDMS from 90.62 to 90.79 while averaging 170 ms end-to-end planning latency, approximately 10% below the 190 ms fixed block-15 planner and 47% below the 320 ms fixed full-depth planner. Code will be released.
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features".
Dev: Here's the gist of the paper. Large video diffusion models provide rich spatiotemporal priors for autonomous driving, but existing world-action models often inherit the cost of iterative future-video generation even though deployment only requires an ego trajectory.
Rosa: The single most informative sentence about the paper's subject and finding is: Adaptive-WAM, a quality-aware multi-exit planner built on a Wan2.2-5B backbone.
Dev: Pruning redundant reasoning tokens The paper introduces Adaptive-WAM to address the question of how much of a video diffusion model must be executed to make a reliable driving decision.
Rosa: This is motivated by the observation that planning performance is largely insensitive to tested video-noise levels, whereas strong trajectories can already be decoded from intermediate layers.
Paper summary: Rosa: So we're looking at this paper called "Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features". It deals with using those big video diffusion models that give us rich information about where things are and how they move in time for autonomous driving. But the existing world action models still often have to pay the cost of generating future videos, even though what we actually need when we're driving is just a simple path or trajectory.
Dev: Exactly, Rosa. The core question this paper asks is how much of that massive video diffusion model do you really need to run to make a safe driving decision? It’s motivated by the idea that planning performance doesn't change much if you test it with different levels of video noise, and strong trajectories can actually be decoded from the middle layers of the model.
Taro: So, they propose Adaptive-WAM as this quality aware world action model that changes how much of the video backbone gets executed based on how good the plan looks so far. It keeps a Wan two point two five billion parameter backbone and attaches a five step trajectory diffusion transformer to six intermediate blocks.
Rosa: And they use a small scorer, called DINOv2 Small, to predict sub scores for each trajectory that gets decoded. If the best trajectory found so far passes a quality threshold, they stop running things immediately. It’s a quality aware approach that routes the computation based on the plan's current quality.
Dev: That sounds like it cuts out all that iterative classifier free denoising loop and VAE decoding that you usually need for future video synthesis. They claim this method avoids generating any future video during deployment, which is a big deal for real time systems.
Taro: The training process involves adapting the Wan backbone using lo ra while optimizing the trajectory projections and heads fully. The actor objective they use includes terms related to both the visual and action losses, plus terms related to the trajectory itself. After imitation learning, they freeze the Wan and that quality scorer part of the model.
Rosa: Then they refine those trajectory heads using a diff grpo formulation, which is similar to what you see in other systems like ReCogDrive. They also fine tune that quality scorer on generated trajectories using evaluator provided component targets with equal weight soft label bce.
Dev: The performance gains they show come from this quality guided routing mechanism and the selective execution of backbone depth. They say the adaptive policy hits ninety point seven nine percent PDMS when using a threshold of ninety, which is a significant improvement over the fixed b15 planner by zero point one seven PDMS while cutting latency down to one hundred seventy milliseconds.
Taro: They also mention that this system transfers well across different datasets, reaching an average two ten percent eight eight m average l squared error and a zero point zero eight percent collision rate on nu scenes without needing any target domain fine tuning. That's interesting for deployment in the wild.
Rosa: But they also did some analysis showing that while block fifteen is the strongest fixed exit, later blocks still hold scene specific advantages because exits solve overlapping but not identical scene sets. The scorer diagnostic shows that only fifty one scenes, or zero point four two percent, select a trajectory at least fifty percent worse than an available near perfect trajectory.
Dev: That suggests the quality guided fallback is practical because passing that quality threshold isn't a formal safety certificate. They report that Adaptive-WAM gets ninety point eight PDMS on navsim one and eighty nine point nine epdms on navsim two.
Taro: The efficiency aspect is also significant because the adaptive planner averages an end to end planning latency of one hundred seventy milliseconds at a threshold of ninety, which is about ten percent lower than that fixed block fifteen planner and much less than the three hundred twenty milliseconds that a fixed full depth planner takes.
Rosa: They keep the full video generation runtime separate, saying it takes about thirteen point two two seconds when using cached text context. This comparison only holds for the planning path because Adaptive-WAM only executes one conditional pass up to the selected depth and never decodes a future video during that planning step.
Dev: The scorer backbone evaluation showed that Wan features don't offer any meaningful advantage over dino small while substantially increasing online cost. They found that the best wan exit only improves the diagnostic score by zero point zero three points over DINO small, which is why they use DINO small as the quality verifier because a wan based scorer would add a large world model forward at every attempted exit.
Taro: The conclusion they draw is that intermediate video diT features are robust planning representations well before full image synthesis completes. They conclude that Adaptive-WAM allocates world model computation according to the quality of the current plan.
Rosa: So what this means for us is that we can use these large generative world models for planning without having to run them all the way through to generate a full video or even synthesize a future frame, just by intelligently selecting which parts of the model are necessary based on how confident we are in our current plan.
Dev: It's about making that computational cost manageable. The paper's final contribution is that they systematically diagnose how video noise level and diT depth affect driving decisions. They also introduce a multi exit world action architecture where a learned trajectory quality controller dynamically selects the required diT depth without completing video generation.
Taro: And they evaluated planning, transfer, trajectory scoring, and efficiency on navsim one and two obtaining state of the art planning performance among compared world model methods while improving over every fixed singletrajectory exit at lower average cost than the strongest one.
Rosa: So basically, we get better performance with less computation by being smart about when to use the heavy video models. The paper's final conclusion is that intermediate video diT features are robust planning representations well before full image synthesis completes, and Adaptive-WAM allocates world model computation according to the quality of the current plan.
Dev: And they show strong zero shot transfer results on nu scenes too, with an average two ten point three five m average l squared error and a zero point zero eight percent average collision rate. It seems like a very practical path from large generative world models to efficient driving policies.
Conclusion: Rosa: So we've been talking about Adaptive-WAM, which is this new way to use those huge video diffusion models for planning by only running what you need based on how good the plan is so far.
Dev: Yeah, that's the main idea. It’s about making sure you don't waste all that massive computation just trying to predict a future video when you only need a short path right now.
Taro: The authors, they built this quality aware world action model using a Wan two point two five billion parameter backbone and some trajectory diffusion transformer attached to six intermediate blocks.
Rosa: And they use this small scorer called DINOv2 Small to check the quality of each possible plan at each step, and if the plan passes a certain threshold, they just stop. It’s a quality guided early-exit planner.
Dev: That makes sense for real time control because you don't want to wait for a full video synthesis loop when you're trying to make an immediate driving decision.
Taro: They show that this adaptive approach gets ninety point seven nine percent PDMS on NAVSIM and eighty nine point nine EPDMS on NAVSIM two, which is good performance.
Rosa: But the really interesting part for me is how it handles the world misbehaving. It adapts its computation based on the plan's quality, so if things get tricky, it can adjust what it runs next.
Dev: And that’s where I see the safety case. If you have a strong trajectory already decoded in those intermediate blocks, you don't need to run the whole massive model to find it.
Taro: It also handles transfer well across different datasets without needing any extra fine tuning, which is a big deal for real world deployment outside of just one test track.
Rosa: So, basically, this work shows we can use these complex video models for planning by being smart about when and how deep we need to look into the data.
Dev: It's about making that computational cost manageable in a way that fits real time constraints. And it opens up a new path from these big generative world models to policies that actually run on cars without needing full video generation every time.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies