Grounding What Shapes the Plan: Rethinking Groundedness for Physical Intelligence in Autonomous Driving
cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: 20 pages; Project website: https://groundact.github.io/
Code: https://github.com/NVlabs/alpasim
Project page: https://groundact.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: Driving models increasingly ground reasoning in causal relations, spatial structure, perceptual evidence, and predicted futures.
Terminology
Abstract
Driving models increasingly ground reasoning in causal relations, spatial structure, perceptual evidence, and predicted futures. These advances make reasoning more faithful to the driving scene, but leave a fundamental question unresolved: what should groundedness mean when the model ultimately outputs an action? Correctly grounded reasoning does not, by itself, ensure desirable driving outcomes. We introduce GroundAct, which starts from a simple premise: driving unfolds through physical entities and their interactions. Entities therefore become the unit of grounding; a lightweight reference token keeps each selected entity's continuous state addressable through symbolic reasoning; and only the referenced entities' interactions with the evolving proposal correct the plan. The result is an explicit path from what reasoning grounds to what the plan does, which we call grounded planning. To assess its practical value, we evaluate GroundAct in both open- and closed-loop settings. GroundAct shows strong open-loop planning across normal, out-of-distribution, and safety-critical scenarios, with closed-loop results extending this evidence to driving in simulation.
Sources
- Qwen3-VL Technical Report
- nuScenes: A multimodal dataset for autonomous driving
- TC-LoRA: Temporally Modulated Conditional LoRA for Adaptive Diffusion Control
- v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning
- Action Emergence from Streaming Intent
- LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving
- Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
- Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
- Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations
- Get To The Point: Summarization with Pointer-Generator Networks
- FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
- Thinking with Visual Grounding
- SpanVLA: Learning from Negative-Recovery Samples with Fast Action Bridging for Vision-Language-Action Model
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems