NavSafe- infinity: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments
cs.RO
Submitted: 2026-09-22
Updated: 2026-10-04
Code: https://github.com/NVlabs/alpasim
Project page: https://navsafe-vail.github.io
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: End-to-end (E2E) driving policies have advanced rapidly on open-loop (OL) benchmarks, yet OL evaluation cannot reveal whether a policy can withstand compounding errors, recover from failures, or
Terminology
Abstract
End-to-end (E2E) driving policies have advanced rapidly on open-loop (OL) benchmarks, yet OL evaluation cannot reveal whether a policy can withstand compounding errors, recover from failures, or interact safely with surrounding actors. We introduce NavSafe- infinity, a photorealistic closed-loop (CL) benchmark comprising 280 scenarios spanning 28 event types, each with success and failure criteria defined within a structured traffic-safety taxonomy, yielding category-level capability scores for Traffic Crashes, Vulnerable Road User Crashes, Traffic Violations, and Traffic Incidents. After evaluating 20 E2E policies, we find that OL gains do not reliably transfer to CL safety. Analysis of two common remedies reveals that (1) passive demonstration perturbation helps mainly when CL rollouts stay near their perturbed training states, and (2) OL reinforcement-learning fine-tuning exhibits reward hacking by trading safety margin for ego progress, which CL feedback amplifies into compounding safety-critical errors. Together, these results demonstrate the blind spot of OL benchmarks indicating CL safety success. The benchmark and an extensible toolbox for customizable event curation and policy diagnosis will be open-sourced to facilitate future research.
Sources
- RAP: 3D Rasterization Augmented End-to-End Planning
- nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving
- WOMD-Reasoning: A Large-Scale Dataset for Interaction Reasoning in Driving
- DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving
- NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation
- SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Driving
- SimScale: Learning to Drive via Real-World Simulation at Scale
- Towards Accurate Generative Models of Video: A New Metric & Challenges
- Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negatives
- DriveLaW:Unifying Planning and Video Generation in a Latent Driving World
- DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer
- SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies