Beyond Waypoint Regression: Query-Based Cost Learning over Reachable Ego Futures for End-to-End Driving
cs.RO, cs.AI, cs.LG, cs.SY, eess.SY
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: Accepted in the 18th Asian Conference on Computer Vision (ACCV 2026)
License: http://creativecommons.org/licenses/by/4.0/
The gist: End-to-end planners based on waypoint regression achieve strong open-loop accuracy, but they primarily learn to mimic expert geometry and remain difficult to adapt to deployment-time safety
Abstract
End-to-end planners based on waypoint regression achieve strong open-loop accuracy, but they primarily learn to mimic expert geometry and remain difficult to adapt to deployment-time safety constraints. We propose a query-based cost-learning framework that estimates bounded costs for dynamically reachable ego trajectory queries, rather than dense BEV cells or a small regressed trajectory set. Compact joint scene tokens capture coherent multimodal agent futures, while contingency-aware cost aggregation and cost-guided intra-cluster MPPI mixing convert the learned cost topology into feasible ego plans. On nuScenes, our method improves over prior cost-estimation planners such as ST-P3 and NMP, outperforms most regression baselines in collision rate, while remaining competitive in L2, and retaining an interpretable cost interface. On real-world driving logs, the proposed planner reduces collision rates compared with SparseDrive and Alpamayo without fine-tuning, while maintaining a diverse set of candidate trajectories.
Sources
- Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
- DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving
- MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
- Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negatives
- Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
- Maximum Entropy Deep Inverse Reinforcement Learning
- SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Driving
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies