Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Bridging Scene Generation and Planning".
Title and authors: Tom: So we're talking about this paper titled "Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation." It sounds a bit technical, but basically, it’s tackling that big problem where you need to plan a safe driving path while also making sure the future scene looks realistic.
Jane: Exactly. The core idea here is that we need one consistent way to describe the world—one representation—so that what's good for generating a realistic future scene is also good for deciding the actual motion you take right now.
Lu: What this paper proposes is WorldDrive, a holistic framework designed to link those two things by unifying vision and motion representation. The central philosophy they push is that the latent features used to generate the future scene should be the exact same features used to make decisions about future motion, which they call representation unification.
Meng: So instead of having one model for seeing and another for moving, WorldDrive tries to get both tasks using a single shared feature space. That sounds ambitious when you think about how different those two things usually are trained separately.
Lalam: I see what they mean by that unification; it means the latent features capable of generating the future scene should also be the features used to decide the future motion, which is what they call embedding c, which then conditions their diffusion model.
Tom: And they achieve this by first introducing something called a Trajectory-aware Driving World Model, or TA-DWM, which conditions on a trajectory vocabulary to enforce consistency between things. That's how they condition the scene generation part on motion intentions from the start.
Jane: That trajectory awareness is key because it builds a structured latent space where visual dynamics are intrinsically coupled with motion intentions, which is how they derive that final embedding c for the diffusion model. It’s not just about seeing; it’s about seeing *in relation* to where you want to go.
Lu: And after Phase one which is for scene generation, they move into Phase two where the TA-DWM becomes frozen, and then they introduce a lightweight Multi-modal Trajectory Planner for motion planning <ref:2603.14948#pg1>. This planner uses frozen encoders and a query-centric cross attention mechanism to fuse historical visual context with structured motion priors.
Title and authors: Meng: Wait, so they’re using those learned vision and motion encoders from the TA-DWM directly to initialize this planner? That sounds like they’re inheriting a really mature feature set instead of starting from scratch for the planning part.
Tom: Right. And for evaluating trajectories without needing to generate a whole new scene every time, they have something called the Future-aware Rewarder, or FAR. This mechanism uses a planning-oriented distillation process to directly distill future latents from that frozen world model to get rewards without high latency.
Jane: So, the naive question is, how does this actually translate into better driving? Because they’re not just making pretty pictures; they are trying to make the planner smarter by giving it a better sense of what's coming.
Lu: The improvement here is that FAR introduces learnable Future Scene Queries Qs as an information bottleneck, which lets them extract planning-relevant future features without having to explicitly generate the whole scene at inference time. This keeps things fast.
Meng: From a deployment standpoint, distillation sounds good for inference speed, but what about training? How do you handle the two-stage curriculum where Phase one optimizes the world model and Phase two optimizes the planner <ref:2603.14948#pg1>? That sounds like a lot of careful tuning to get that alignment right.
Tom: The training is decoupled into two stages: first, optimizing the TA-DWM to learn latent dynamics conditioned on historical context and expert motion. Then, they freeze that world model and optimize the multi-modal trajectory planner using only those frozen features. This separation helps stabilize both learning processes.
Jane: So what does this mean for a person just listening to the show, someone who isn't looking at a screen? It means the system can generate diverse and high-quality trajectory candidates because that unified representation allows the planner to select the best path from many options with high reward scores.
Title and authors: Lu: The experimental validation shows they achieved leading planning performance among vision-only methods on benchmarks like NAVSIM, even reaching a PDMS of eighty-eight point one on the NAVSIM navtest split. They also showed that inference latency is low at fifty-three milliseconds when using FAR, and performance gains are amplified by pre-training strategies <ref:2603.14948#pg2>.
Meng: Eighty-eight point one percent is respectable for planning, but what about the real world? Meng's concern is always about deployment; does this framework run well on noisy, real traffic data where the learned priors might break down?
Tom: The authors show that the synthesized scenes exhibit remarkable geometric consistency with the input motion commands. That’s a big deal because it means when you command a turn, the generated future scene actually looks like it follows that turn perfectly in terms of shape and dynamics.
Jane: So, to summarize, this paper confirms that unifying vision and motion representation creates a feature space where visual dynamics are coupled with motion intentions, allowing the planner to select optimal trajectories based on rich learned priors from the scene generation task.
Lu: The ultimate implication is bridging that gap between scene generation and planning by using representation inheritance to initialize a lightweight planner with features learned from the generative model. That’s what WorldDrive does in practice.
Meng: I’m still curious about that limitation they mention; if the world model sacrifices interpretable scene simulation for efficiency, how much safety margin are we losing in decision-making?
Lalam: The paper itself states that although this approach is more efficient, it sacrifices interpretable scene simulation and limits explicit visual verification that is valuable for safety-critical decision-making. That’s the trade-off they acknowledge.
Tom: So, the final word from this paper on "Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation" is that it successfully bridges the representational gap between scene generation and planning by coupling vision and motion representation to create a holistic framework. It shows that unifying these two domains can lead to strong planning performance alongside action-controllable scene synthesis.
Jane: It really confirms that for autonomous driving, having a single feature space where visual dynamics are coupled with motion intentions is the path forward. What’s next is seeing how robust this representation holds up when you throw it into unpredictable real-world driving scenarios.
The paper's summary: Tom: So, to wrap up what we just talked about, this paper basically introduces WorldDrive, which is this framework that tries to solve that gap between generating realistic future scenes and actually planning a safe path in real time by unifying vision and motion representation.
Jane: Yeah it's about getting the features used for planning to be the same ones used for making the scene look right.
Tom: Exactly. The paper proposes a Trajectory-aware Driving World Model, or TA-DWM, which builds this shared feature space where what you see visually is directly tied to your intended motion.
Jane: That’s the core idea then, that it’s not just looking at a picture but looking at the picture and knowing where you want to go simultaneously.
Tom: And they use this unified representation so that the planner gets initialized with features that are already mature and consistent because they learned those from the scene generation side first.
Jane: It sounds like they're essentially giving the planning part a head start by using knowledge gained from making things look good.
Tom: Right. Then, for making decisions, they use a lightweight Multi-modal Trajectory Planner that uses frozen encoders and some cross attention to fuse what’s happening now with those learned motion priors.
Jane: And they don't even need to run the full scene generation model when planning because they have this Future-aware Rewarder that distills future information directly from the frozen world model.
Tom: So, it’s a distillation process, which keeps inference fast, and they use learnable queries to pull out just the most relevant future features for evaluating different potential paths.
Jane: It's smart because it avoids generating a whole new scene for every single decision you make while still having that foresight into what might happen down the road.
Tom: The results are pretty strong. They’ve shown leading planning performance compared to other vision-only methods on benchmarks like NAVSIM, hitting a score of eighty-eight point one percent on the navtest split.
Jane: And they also showed that when you pre-train things correctly, those numbers get even better, which is pretty cool because it means the unified representation really pays off.
Tom: What this means for us is that we’re seeing a way to actually couple the "look" of the future with the "move" of the car in a single system.
Jane: It really suggests that for autonomous driving, having one consistent feature space where visual dynamics are coupled with motion intentions is a really strong direction to go.
Tom: But they do acknowledge that this unification means they're sacrificing some kind of explicit scene simulation, which is a trade-off for the efficiency gains and the planning performance.
Jane: That’s a fair point. It means we’re prioritizing better decision-making speed over perfect, pixel-perfect visualization of every possible future state.
Tom: So while it doesn't replace the full generative model for every single step, it gives the planner a much richer context to choose the best trajectory from a set of candidates with high reward scores.
Jane: It’s about finding that sweet spot where you get robust planning performance without getting bogged down in slow generation steps during operation.
Tom: This opens up a lot of questions about how this representation holds up when we move from controlled test environments to the messy, unpredictable real world.
The paper's improvements: Tom: So we’re talking about what this paper suggests improving moving forward, and it’s about how they take that unified representation and make the system even more efficient for real use.
Jane: Right, after showing the framework works on a benchmark, they look at how to actually make it run smoothly when you're driving or doing something else.
Tom: They suggest a two-stage training curriculum where you first train the Trajectory-aware Driving World Model to learn those dynamics conditioned on historical context and expert motion.
Jane: That’s the first stage, getting the world model to be really good at understanding how things move based on what it has seen before.
Tom: Then, in the second stage, they freeze that whole world model and focus solely on optimizing that lightweight Multi-modal Trajectory Planner using those mature features.
Jane: It’s like you teach a student everything about the world first, then you just have them practice making good decisions based on what they already know.
Tom: They also highlight the Future-aware Rewarder, or FAR, which is a distillation mechanism to extract future latents from that frozen model directly for evaluating trajectories.
Jane: So instead of running a heavy generation process every time you want to check a path, you distill the necessary information for evaluation in constant time.
Tom: That’s what they mean by maintaining real-time inference speeds even with that level of foresight built into the system.
Jane: It also mentions that performance gains are amplified when you use pre-training strategies, which means how you start training those models really matters for the final result.
Tom: They point out that while this approach is efficient, it doesn't offer the kind of interpretable scene simulation you might get from a full generative model for every single frame.
Jane: That’s their honesty; they’re trading off some level of perfect visualization to get that strong planning performance and real-time speed.
Tom: So the implication is that we can achieve high-quality trajectory selection with a unified system, even if the scene generation part isn't running in full detail during every move.
Jane: It changes things for someone just listening to the show because it means we can have a system that plans intelligently based on deep learned motion knowledge without needing massive computational power constantly.
Tom: What this points toward is integrating these kinds of unified representations into actual production systems where speed and reliability matter more than absolute visual fidelity in every single second.
Conclusion: Tom: So to wrap up, this paper on "Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation" shows how you can connect scene generation and planning by using one shared representation for both.
Jane: It’s about getting that visual information and the motion intention into the same space so the planner makes decisions based on what the future looks like.
Tom: Exactly. They establish a feature space where visual dynamics are coupled with motion intentions, which allows them to transfer learned features from scene generation directly to initialize a lightweight planner.
Jane: That’s a big deal because it means you don't have to train the planner from scratch; you can inherit knowledge that's already robust.
Tom: They also introduce the Future-aware Rewarder, or FAR, which uses distillation to get future latents without needing explicit scene generation during real-time inference.
Jane: That’s a smart way to handle prediction; you get the foresight needed for planning but keep it fast enough for actual driving.
Tom: The validation shows leading performance on benchmarks like NAVSIM, hitting an eighty-eight point one percent planning score on the navtest split, and they showed low inference latency at five thirty milliseconds with FAR.
Jane: It also confirmed that those pre-training strategies really boost the performance further, which tells us how important the initial setup is for this unified model to succeed.
Tom: The main thing this paper proves is that unifying vision and motion representation creates a holistic framework for autonomous driving where you get both high-quality trajectory selection and action-controllable scene synthesis.
Jane: It confirms that coupling those two domains into one system is a solid way to bridge the representational gap in this field.
Tom: The implication is that we can build systems that are both smart planners and visually consistent, even if it means trading some explicit scene detail for better decision-making speed.
Jane: It opens up a lot of questions about how this representation holds up when you move from controlled test environments to the messy, unpredictable real world.
Tom: That’s what we’ll be looking at next: whether this unified representation can actually handle the noise and variability of live traffic data at scale.
Jane: We'll see if that strong planning performance stays consistent when things get unpredictable on the road.
Lu: From a training standpoint, I think the idea of representation inheritance is really powerful because it simplifies the initialization problem for downstream tasks significantly.
Meng: For deployment, I’m still worried about the complexity of running both models at once; if this unification adds too much overhead to inference, it won't ship.
Lalam: As a model, I see this as a very logical step in improving autonomous driving systems because coupling what you see with where you want to go makes the entire decision process much more coherent.
Xingtai Gui, Meijie Zhang, Tianyi Yan, Wencheng Han, Jiahao Gong, Feiyang Tan, Cheng-zhong Xu, § Jianbing Shen
SKL-IOTSC, CIS, University of Macau
cs.CV
Submitted: 2026-03-16
Updated: 2026-11-17
Comments: NeurIPS2026. 30 pages, 11 figures. The code is available at https://github.com/TabGuigui/WorldDrive
Code: https://github.com/TabGuigui/WorldDrive
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 89/100
The gist: End-to-end autonomous driving aims to generate safe and plausible planning policies from raw sensor input, and constructing an effective scene representation is a critical challenge.
Key concepts
- Trajectory-aware Driving World Model (TA-DWM)
- This model is designed to create a latent representation where visual features are intrinsically linked with motion goals. It uses predefined trajectory anchors to structure this space, ensuring that the learned features are consistent and pre-aligned for both scene generation and future planning tasks.
- Representation Inheritance
- This mechanism transfers the robust vision and motion encoders learned during scene generation directly to initialize the downstream planner. This means the planner starts with high-quality, mature features already optimized for understanding visual dynamics and motion.
- Future-aware Rewarder (FAR)
- The FAR uses a planning-oriented distillation method to extract relevant future latents from the frozen world model. It introduces learnable 'Future Scene Queries' that act as an information bottleneck, allowing the system to efficiently evaluate trajectories against predicted future scenarios in real-time.
Terminology
Summary
End-to-end autonomous driving aims to generate safe and plausible planning policies from raw sensor input, and constructing an effective scene representation is a critical challenge. p. 0 Bridging Scene Generation and Planning: Driving with World Model
via Unifying Vision and Motion Representation
Xingtai Gui1
, Meijie Zhang2
, Tianyi Yan1
, Wencheng Han1
,
Jiahao Gong2
, Feiyang Tan2
, Cheng-zhong Xu1
, Jianbing Shen1∗
1SKL-IOTSC, CIS, University of Macau, 2Afari Intelligent Drive
Abstract
End-to-end autonomous driving aims to generate safe and
plausible planning policies from raw sensor input, and constructing an effective scene representation is a critical challenge. Driving world models have shown great potential
in learning rich representations by predicting the future
evolution of a driving scene. However, existing driving
world models primarily focus on visual scene representation, and motion representation is not explicitly designed
to be planner-shared and inheritable, leaving a schism between the optimization of visual scene generation and the
requirements of precise motion planning. We present WorldDrive, a holistic framework that couples scene generation
and real-time planning via unifying vision and motion representation.Where the paper says it
Where the paper says it
Core Framework Introduction
WorldDrive is designed to synergize end-to-end planning with scene generation through representation unification. The core philosophy posits that the latent features capable of generating the future (scene generation) should be the same features used to decide the future (planning). p. 1 To bridge this gap, we introduce WorldDrive, a holistic
framework designed to synergize end-to-end planning with
scene generation via unifying vision and motion representation. The core philosophy of WorldDrive is representation
unification: we posit that the latent features capable of generating the future (scene generation) should be the same features used to decide the future (planning). The core philosophy of WorldDrive is representation
unification: we posit that the latent features capable of generating the future (scene generation) should be the same features used to decide the future (planning). To this end, we
first propose a Trajectory-aware Driving World Model (TADWM). Unlike prior works that treat motion as a superficial condition, TA-DWM employs a multi-modal trajectory
encoding scheme built on predefined trajectory anchors to
construct a structured latent space where visual dynamics
are intrinsically coupled with motion intentions. This design enables representation inheritance: the robust vision and motion encoders learned by the TA-DWM
are directly transferred to initialize the downstream planner.Where the paper says it
Where the paper says it
Where the paper says it
Representation Unification Mechanism
The TA-DWM employs a multi-modal trajectory encoding scheme built on predefined trajectory anchors to construct a structured latent space where visual dynamics are intrinsically coupled with motion intentions. This results in the final motion embedding c being derived by fusing the anchor embeddings with the corresponding residual embeddings. p. 3 Given an expert trajectory Y , we
retrieve the top-K nearest anchors VK from the vocabulary
and compute the residuals to capture motion details. The
final motion embedding c is derived by fusing the anchor
embeddings with the corresponding residual embeddings:
c = Ea(VK) + Eo(Y − VK), (1)
where Ea and Eo denote the anchor encoder and offset encoder, respectively. The
final motion embedding c is derived by fusing the anchor
embeddings with the corresponding residual embeddings:
c = Ea(VK) + Eo(Y − VK), (1)
where Ea and Eo denote the anchor encoder and offset encoder, respectively. The resulting embedding c ∈ R
K×C
serves as the condition for the diffusion model.Where the paper says it
Where the paper says it
Planning and Reward Mechanism
A lightweight Multi-modal Trajectory Planner leverages frozen encoders to use a query-centric cross-attention mechanism to efficiently fuse historical visual context with structured motion priors. To harness predictive foresight without high latency, the Future-aware Rewarder (FAR) employs a planning-oriented distillation mechanism to directly distill the future latents from the frozen world model. p. 1 Although TA-DWM is capable of synthesizing future videos,
executing this generative process for every trajectory candidate is computationally prohibitive. Instead, the FAR employs a planning-oriented distillation mechanism to directly
distill the future latents from the frozen world model. 3, the future latent generator (TA-DiT) is used only
during training, and FAR is trained to distill the planningrelevant future latents from TA-DiT, thereby bypassing diffusion sampling at inference. FAR introduces a set of learnable Future Scene Queries
Qs ∈ RM×C as an information bottleneck for extracting
planning-relevant future features.Where the paper says it
Where the paper says it
Training and Inference Efficiency
The training process uses a decoupled two-stage curriculum where the first stage optimizes the TA-DWM to learn latent dynamics conditioned on historical context and expert motion. The second stage freezes the world model and optimizes the multi-modal trajectory planner. p. 4 The objective follows the standard latent diffusion formulation:
Lworld = Ez0,t,ϵ
∥ϵ − ϵθ(zt; f, t, c)∥
2
2
, (5)
where ϵ ∼ N (0, I) denotes the sampled Gaussian noise. In the second stage, we freeze the world model and optimize the multi-modal trajectory planner. DWM-based planners face a practical trade-off: incorporating future forecasts improves performance but increases inference latency. WorldDrive avoids this by distilling future
latents during training and eliminating explicit future scene
generation at inference.Where the paper says it
Where the paper says it
Experimental Validation
Extensive experiments on the NAVSIM, NAVSIM-v2, and nuScenes benchmarks demonstrate that WorldDrive achieves leading planning performance among vision-only methods while maintaining high-fidelity action-controlled video generation capabilities. WorldDrive achieves the best PDMS among vision-only methods on the NAVSIM navtest split, reaching a PDMS of 88. p. 5 Table 1
compares WorldDrive with state-of-the-art methods on the
NAVSIM navtest split. WorldDrive achieves the best PDMS
among vision-only methods. 0 53ms
Table 6. Inference latency comparison. Table 4 dissects the benefits of the pre-training curriculum in WorldDrive. Training
the planner without any pre-trained representation leads to
poor driving performance. Building upon this unified representation,
we design a lightweight Multi-modal Planner. Leveraging these frozen encoders, the planner uses a query-centric
cross-attention mechanism to efficiently fuse historical visual context with structured motion priors, generating diverse and high-quality trajectory candidates. The results show a tight coupling between the input motion and the generated scenes. 4 shows a clear inverse correlation where latent
similarity decreases as the geometric distance increases.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
WorldDrive presents a unified framework that bridges the representational gap between scene generation and planning. By introducing the Trajectory-aware Driving World Model, it establishes a feature space where visual dynamics are coupled with motion intentions. p. 8 Conclusion
We present WorldDrive, a unified framework that bridges
the representational gap between scene generation and planning. By introducing the Trajectory-aware Driving World
Model, we establish a feature space where visual dynamics are coupled with motion intentions. tance that initializes a lightweight planner with mature features learned from scene generation. We further propose
the Future-aware Rewarder, which leverages DWM foresight while maintaining real-time inference. tance that initializes a lightweight planner with mature features learned from scene generation. We further propose
the Future-aware Rewarder, which leverages DWM foresight while maintaining real-time inference. We further propose
the Future-aware Rewarder, which leverages DWM foresight while maintaining real-time inference. Extensive experiments show that WorldDrive achieves strong planning
performance and supports action-controllable scene synthesis. Furthermore, to exploit
the world model’s foresight, we propose a Future-aware Rewarder, which distills future latent representation from the
frozen world model to evaluate and select optimal trajectories in real-time. Extensive experiments on the NAVSIM,
NAVSIM-v2, and nuScenes benchmarks demonstrate that
WorldDrive achieves leading planning performance among
vision-only methods while maintaining high-fidelity actioncontrolled video generation capabilities, providing strong
evidence for the effectiveness of unifying vision and motion
representation for robust autonomous driving.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Unifying vision and motion representation enables a
holistic framework that bridges scene generation and planning.
This allows for generating high-fidelity, motion-consistent future scenes while simultaneously performing real-time trajectory optimization. -
Trajectory aware driving world model (TA-DWM) establishes a
feature space where visual dynamics are coupled with motion intentions,
ensuring that the planner operates onmature and consistent latent space pre-aligned by the future scene generation task.
-
The Future aware Rewarder (FAR) utilizes a
planning-oriented distillation mechanism to directly distill the future latents from the frozen world model
to evaluate trajectories, effectivelyaligning the planner’s selection with the world model’s learned dynamics while maintaining real-time inference speeds.
-
The system can perform end-to-end planning by generating
diverse and high-quality trajectory candidates
using a lightweight multi-modal planner that fuses historical visual context with structured motion priors. -
The improved AI system can achieve state of the art planning performance, as demonstrated by achieving a PDMS of 89.0 on the full navtrain split when trained on WorldDrive, and supporting
action-controllable scene synthesis.
Abstract
End-to-end autonomous driving aims to generate safe and plausible planning policies from raw sensor inputs, and constructing an effective scene representation is a critical challenge. Driving world models have shown great potential in learning rich representations by predicting the future evolution of a driving scene. However, existing driving world models primarily focus on visual scene representation, and motion representation is not explicitly designed to be planner-shared and inheritable, leaving a schism between the optimization of visual scene generation and the requirements of precise motion planning. We present WorldDrive, a holistic framework that couples scene generation and real-time planning via unifying vision and motion representation. We first introduce a Trajectory-aware Driving World Model, which conditions on a trajectory vocabulary to enforce consistency between visual dynamics and motion intentions, enabling the generation of diverse and plausible future scenes conditioned on a specific trajectory. We transfer the vision and motion encoders to a downstream Multi-modal Planner, ensuring the driving policy operates on mature representations pre-optimized by scene generation. A simple interaction between motion representation, visual representation, and ego status can generate high-quality, multi-modal trajectories. Furthermore, to exploit the world model's foresight, we propose a Future-aware Rewarder, which distills future latent representation from the frozen world model to evaluate and select optimal trajectories in real-time. Extensive experiments demonstrate that WorldDrive achieves leading planning performance among vision-only methods while maintaining high-fidelity action-controlled video generation capabilities.
Sources
- NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles
- Pseudo-Simulation for Autonomous Driving
- VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning
- DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers
- A Survey of World Models for Autonomous Driving
- MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control
- TrajDiff: End-to-end Autonomous Driving without Perception Annotation
- InfinityDrive: Breaking Time Limits in Driving World Models
- GAIA-1: A Generative World Model for Autonomous Driving
- Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
- ADriver-I: A General World Model for Autonomous Driving
- DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving
- 3D and 4D World Modeling: A Survey
- ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving
- DriVerse: Navigation World Model for Driving Simulation via Multimodal Trajectory Prompting and Motion Alignment
- Enhancing End-to-End Autonomous Driving with Latent World Model
- DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving
- End-to-End Driving with Online Trajectory Evaluation via BEV World Model
- Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation
- WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs