BEVTraj: Map-Free Multimodal Trajectory Prediction with Sensor-Derived BEV Context and Sparse Goal Reasoning

arXiv:2509.10080 · cs.CV · Submitted 2025-09-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "BEVTraj: Map-Free Multimodal Trajectory Prediction with Sensor-Derived BEV Context and Sparse Goal Reasoning".

Jane: Here's the gist of the paper. BEVTraj proposes a map-free framework that uses deformable attention and sparse goal proposals to predict trajectories in Bird’s-Eye View space without relying on pre-built HD maps.

Tom: This method is significant because it achieves performance comparable to HD map-based methods while offering greater robustness and flexibility by directly leveraging raw sensor data Core Framework Components The BEVTraj framework is structured around two main modules: the Scene Context Encoder and the Iterative Deformable Decoder.

Jane: The Scene Context Encoder builds scene-level context features by capturing complex interactions within the scene, utilizing historical agent states and BEV features from raw sensor data.

Tom: This encoder consists of three submodules: 1) Sensor Encoder uses BEVFusion to generate a BEV feature map B, aligning with the BEV paradigm.

Paper summary: Tom: Looking at the title "BEVTraj: Map-Free Multimodal Trajectory Prediction with Sensor-Derived BEV Context and Sparse Goal Reasoning," it really captures the core innovation of this work, which is moving away from map dependency entirely.

Jane: And I think what this means for us on the ground is that trajectory prediction doesn't have to be tethered to expensive, constantly updated HD maps if you can build a system that extracts context directly from raw sensor data using deformable attention.

Lu: The implication is that we can develop more flexible systems, especially for areas where maps aren't readily available or where the environment changes too quickly for static maps to keep up with.

Meng: From an engineering standpoint, if this works as claimed, it means we could potentially deploy prediction models onto a wider variety of hardware without needing custom map-matching software for every single deployment scenario.

Lalam: And the sparse goal reasoning part suggests that instead of trying to predict a long, complex path across the whole scene at once, you can break it down into predicting a few key destination points and then letting the system figure out the path between those points.

Tom: So it’s about trading map maintenance for computational efficiency and direct sensor utilization, achieving good performance without the structural constraints of a prebuilt environment.

Jane: It’s a significant step because it shows that raw, dense sensor data can actually be used effectively for agent-centric reasoning in Bird's-Eye View space.

Conclusion: Tom: So we’re wrapping up on BEVTraj, and I want to talk about that title: "Map-Free Multimodal Trajectory Prediction with Sensor-Derived BEV Context and Sparse Goal Reasoning."

Jane: That sounds a lot like it’s trying to solve the big problem of needing a map to predict where someone is going in Bird's Eye View space.

Lu: Exactly. The core idea is that you don't need those massive HD maps anymore, you can build the prediction system using just raw sensor data and context derived from that data itself.

Meng: From a deployment angle, if it truly works without a map, that cuts out a huge part of the pipeline complexity for getting models onto new hardware or in new locations.

Lalam: My read is that they’ve managed to do something similar to how we use human intuition—you don't need the whole city blueprint; you just need to see the immediate surroundings and what people are doing right now.

Tom: So, what does this mean for someone who only listens? It means a system that tries to guess movement based on what it sees in real time, instead of looking up a pre-drawn route on a map.

Jane: That’s the naive question—can it actually predict complex paths reliably without that map knowledge?

Lu: The results show it performs at levels comparable to those methods that rely on HD maps, but the authors emphasize that this is achieved by using deformable attention and sparse goal proposals instead of just brute-forcing the whole scene.

Meng: I’m interested in how they handle the scale. If I run this on a high-traffic scenario, does it stay fast? What breaks when you try to scale that context encoding up?

Lalam: The paper suggests the sparse goal reasoning is what keeps it manageable computationally, because you aren't processing every single pixel of the scene constantly.

Tom: It sounds like they’ve found a way to trade map dependency for computational efficiency by using these learned spatial contexts instead.

Jane: So, while it’s not a complete replacement for a detailed map system, it opens up possibilities where maps are too expensive or impossible to get in the first place.

Lu: The implication is that trajectory prediction can become much more flexible across different environments because the model learns what context matters directly from the sensors.

Meng: It changes how we think about training data too—instead of labeling paths against a map, maybe we just need good sensor recordings and agent behaviors.

Lalam: I'm pretty confident in this direction; the authors show that their method is robust because it leverages the underlying spatial relationships in the raw sensor input very effectively.

Kookmin University

cs.CV

Submitted: 2025-09-12

Updated: 2026-11-03

Comments: 30 pages. Published in Journal of Computational Design and Engineering (JCDE). Final published version

Journal ref: Journal of Computational Design and Engineering, 13(11), 1-30 (2026)

DOI: 10.1093/jcde/qwag088

Code: https://github.com/Kongminsang/bevtraj

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 82/100

The gist: BEVTraj proposes a map-free framework that uses deformable attention and sparse goal proposals to predict trajectories in Bird’s-Eye View space without relying on pre-built HD maps.

Key concepts

Map-Free Framework
This approach predicts paths without needing any existing high-definition (HD) maps. Instead, it learns the necessary spatial understanding directly from raw sensor data, making it adaptable to new environments without prior mapping.
Deformable Attention
This mechanism allows the model to selectively focus on important parts of the scene features. It doesn't just look at every point equally; it intelligently shifts its attention to relevant spatial locations based on what is needed for trajectory prediction.
Sparse Goal Proposals
Instead of considering all possible destinations, this technique identifies and focuses only on a small, critical set of potential goal locations. This reduces computational load while ensuring the model considers the most relevant future targets.
Scene Context Encoder
This module processes raw sensor data to build a rich understanding of the current environment. It combines historical agent states with spatial features to create contextual information that helps predict where an agent will go next.

Terminology

Summary

BEVTraj proposes a map-free framework that uses deformable attention and sparse goal proposals to predict trajectories in Bird’s-Eye View space without relying on pre-built HD maps. p. 0

Where the paper says it

8, AUGUST 2021 1 BEVTraj: Map-Free End-to-End Trajectory Prediction in Bird’s-Eye View with Deformable Attention and Sparse Goal Proposals Minsang Kong, Myeongjun Kim, Sang Gu Kang, Hejiu Lu, Yupeng Zhong, Sang Hun Lee, Member, IEEE, Abstract—In autonomous driving, trajectory prediction is essential for safe and efficient navigation. While recent methods often rely on high-definition (HD) maps to provide structured environmental priors, such maps are costly to maintain, geographically limited, and unreliable in dynamic or unmapped scenarios.

This method is significant because it achieves performance comparable to HD map-based methods while offering greater robustness and flexibility by directly leveraging raw sensor data

Core Framework Components

The BEVTraj framework is structured around two main modules: the Scene Context Encoder and the Iterative Deformable Decoder. The Scene Context Encoder builds scene-level context features by capturing complex interactions within the scene, utilizing historical agent states and BEV features from raw sensor data. p. 3

Where the paper says it

8, AUGUST 2021 4 Context Encoder generates scene-level context features by capturing complex interactions within the scene, utilizing the historical states of agents and the BEV features extracted from raw sensor data. Subsequently, the Iterative Deformable Decoder sequentially predicts multiple goal candidates and their corresponding initial trajectories for the target agent, and refines these predictions iteratively.

This encoder consists of three submodules:

  1. Sensor Encoder uses BEVFusion to generate a BEV feature map B, aligning with the BEV paradigm.

  2. Pre-Encoder applies temporal self-attention and social self-attention prior to temporal compression.

  3. BEV Deformable Aggregation (BDA) selectively attends to a compact set of key spatial locations in B to produce the BEV aggregated feature.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Deformable attention for Bird’s-Eye View Trajectory Prediction allows selective aggregation of task-relevant context from sparse locations in dense BEV features, enabling efficient, geometry-aware, and agent-centric information extraction from dense BEV representations.

  2. Sparse Goal Candidate Proposal (SGCP) module eliminates the need for heuristic post-processing like non-maximum suppression (NMS) by directly predicting a small set of sample adaptive and realistic goal candidates conditioned on the scene context.

  3. The framework can perform multimodal forecasting without relying on pre-built maps by directly leveraging raw sensor data, achieving performance comparable to state-of-the-art HD map-based methods while providing greater robustness and flexibility.

  4. The system can achieve competitive performance in future occupancy prediction by converting multiagent trajectory outputs into occupancy grids through a deterministic post-processing step, capturing transportation-system concerns like space utilization and interaction feasibility.

  5. The model can maintain stable lane-aligned future trajectories even under challenging conditions such as sharp curves and cluttered intersections by directly leveraging raw sensor cues that are not available in map-based representations.

Abstract

Trajectory prediction is a key task in autonomous driving, where a model must forecast plausible future motions of surrounding agents from dynamic histories and scene context. Many recent methods use high-definition (HD) maps to provide explicit road topology and semantic constraints, but such maps are costly to maintain, geographically limited, and difficult to update in changing environments. This paper presents Bird's-Eye View Trajectory Prediction (BEVTraj), a map-free trajectory prediction framework that directly uses BEV features constructed from raw multi-sensor data. Since sensor-derived BEV features are dense and only implicitly structured, BEVTraj employs deformable attention to selectively aggregate agent-relevant spatial context from sparse locations rather than processing the entire BEV grid uniformly. In addition, a Sparse Goal Candidate Proposal module combines overlapping mode-wise anchor clusters with attention-based goal prediction to generate compact, scene-adaptive goal hypotheses. These goals are decoded and iteratively refined into multimodal trajectories, while trajectory-aware mode scoring and temporal consistency modeling improve confidence estimation and suppress abrupt, physically implausible refinement updates. Experiments on nuScenes and Argoverse 2 Sensor show that BEVTraj achieves competitive results under a map-free setting without using pre-built HD maps, while ablation studies and qualitative analyses illustrate the roles of BEV context aggregation and sparse goal reasoning. The source code is available at https://github.com/Kongminsang/bevtraj.

Sources

Related papers