Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation".
Jane: Here's the gist of the paper. LiDAR scene flow estimation is essential for autonomous driving as it provides 3D motion for each point.
Tom: TrackCue introduces a tracking-guided framework that improves dynamic object representation in LiDAR scene flow estimation by repurposing point tracking to obtain dense image-space trajectories anchored to LiDAR points.
Jane: This framework also presents a visually consistent motion compensation strategy to compare tracked trajectories with ego-induced rigid trajectories, effectively isolating true object motion from egoinduced apparent motion.
Tom: Finally, it performs visual motion cue lifting to transfer these isolated motion cues back to the LiDAR domain for static-dynamic label refinement.
Title and authors: Tom: So we're looking at this paper today called "Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation". This is about getting three dee motion information from LiDAR data without having those perfect three dee annotations, which is a big problem for autonomous driving <ref:2605.16922#pg1>.
Jane: That's right, Tom. And the core idea here seems to be that instead of just relying on the sparse geometric observations from LiDAR points, they want to bring in some dense visual motion cues. What exactly does this paper propose?
Lu: They introduce something called TrackCue, which is a tracking-guided framework designed to improve how dynamic objects are represented in LiDAR scene flow estimation by using image-based point tracking.
Meng: Tracking guided, okay. So it’s using the camera images to help label the LiDAR points as static or dynamic? I wonder if that adds too much complexity for deployment at scale.
Tom: The mechanism is pretty clever; TrackCue repurposes existing point tracking techniques to get dense image-space trajectories that are anchored directly to those original LiDAR points, which gives them motion cues they didn't have before.
Jane: So, what does this mean for someone driving? It sounds like it helps the system figure out exactly how much an object is moving in the image plane versus just looking like it's moving because of how the camera is positioned.
Lu: They also have this visually consistent motion compensation strategy where they compare those tracked image-space trajectories with the ego-induced rigid trajectories, which lets them isolate the true object motion from any apparent motion caused by how your car moves.
Meng: Isolating true motion versus ego-motion sounds like a necessary step for robustness, but that comparison itself needs to be very reliable. What's the next part of this process?
Tom: They follow that up with a visual motion cue lifting process, which takes those ego-compensated image trajectories and transfers them back onto the original LiDAR points using nearest-neighbor search to generate refined static-dynamic labels.
Jane: So you're taking motion cues from the image, correcting for your vehicle's own movement, and then applying that corrected information back to the three dee LiDAR structure <ref:2605.16922#pg1>? That sounds like a really thorough way to create better supervision.
Title and authors: Lu: It’s meant to produce these refined static-dynamic labels that complement those initial LiDAR-only observations and give more reliable supervision for scene flow learning. They are essentially converting dense visual evidence into better labels for the three dee scene flow task <ref:2605.16922#pg1>.
Tom: Now, looking at the results, they show that TrackCue significantly improves both the precision and F1 score of those dynamic labels compared to their baseline method, DUFOMap nine <ref:2605.16922#pg2>.
Jane: That’s a big jump in quality; can you tell us what those numbers actually look like for the precision and F1 scores? Because a better score doesn't always mean it's more useful in real life.
Tom: They reduce false-positive dynamic labels by thirty-five point six percent while improving precision from twenty-four point six percent up to thirty-one point four percent and the F1 score from thirty-six point seven percent to forty-two point four percent. That’s a noticeable lift in how accurately the model identifies what's actually moving versus what looks like it's moving due to noise or sensor artifacts.
Meng: A reduction of thirty-five point six percent in false positives sounds substantial, but I have to ask, how does this perform when we move from testing on clean datasets to running on real traffic data where things are messy?
Jane: Well, the paper also shows consistent benefits across different datasets and distance ranges for flow predictions, specifically noting that TrackCue reduces the overall dynamic bucket normalized error by zero point two three when integrated into SeFlow, SeFlow++, and TeFlow.
Lu: The key insight they draw from this is that well-separated static and dynamic clusters are really crucial for making self-supervision reliable in these flow estimation tasks. If the separation isn't clean, the supervision gets noisy again.
Tom: They also quantified how much better this method is by comparing it to DUFOMap for auto-label generation in both the SeFlow and SeFlow++ pipelines, showing an F1 score improvement of five point six three and four point one zero percentage points for those two pipelines, respectively <ref:2605.16922#pg3>.
Jane: So, what about the potential problems? I noticed they mention that tracking over three dee to 2D flattened points has a limitation because projection inevitably loses some of that original three-dimensional geometric information <ref:2605.16922#pg1>.
Lu: They acknowledge that limitation upfront, and they suggest that future work could explore richer cross-modal information exchange where RGB and LiDAR cues can mutually refine both the supervision signals and the feature representations for three dee motion learning <ref:2605.16922#pg1>.
Title and authors: Meng: That sounds like a lot of engineering overhead to build—integrating two different modalities with mutual refinement—but if it makes the model more robust, I might be interested. What about deployment costs?
Tom: For now, they show that high-resolution images preserve finer visual details and provide more reliable point trajectories, which allows them to retain more true dynamic points as dynamic, meaning better generalization across different scenarios.
Jane: So it seems like this paper provides a solid way to bridge the gap between sparse LiDAR geometry and the dense visual data needed for accurate motion understanding. Where does this leave us in terms of scene flow research?
Lu: It opens a promising direction for future research by integrating RGB images not just to enhance feature representations but also specifically to provide that cross-modal supervision for three dee scene flow estimation <ref:2605.16922#pg1>.
Tom: So, to wrap up the "Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation" paper, they’ve shown a tracking-guided framework that uses image tracking and motion compensation to lift visual cues back onto LiDAR points, leading to a thirteen point zero nine point reduction on AEVAScenes in the zero-shot evaluation setting.
Jane: It really shows how leveraging visual consistency can clean up the labels generated by LiDAR alone, which is vital when you're trying to build systems that rely on self-supervision and don't have massive amounts of labeled data.
Lu: Exactly, and I think the move towards using these dense visual trajectories as motion cues is a solid step toward making scene flow estimation more reliable than just relying on sparse geometry.
Meng: From an engineering standpoint, the performance gains are good, but we still need to figure out how stable this tracking component is when the camera quality changes drastically, like in bad weather or low light.
Tom: That’s a fair point; the reliability of that point tracker is definitely something to monitor closely as we scale these models up for real deployment.
Jane: So, for anyone driving or walking, this means the underlying AI can get a much clearer picture of what's truly moving in front of it by looking at both what the LiDAR sees and what the camera sees in motion.
The paper's summary: Tom: So we're looking at how this paper puts all these pieces together—the tracking, the compensation, and then lifting those visual cues back onto your LiDAR points—to create a much better way to tell if something is static or moving in a scene.
Jane: Basically, they took motion information from images where it was dense and tried to translate that dense visual motion into something that helps the LiDAR data make smarter decisions about what's moving.
Tom: Right. The big deal here is how they handle the problem of those labels being too noisy because you just rely on the geometry of the LiDAR points alone, which often leads to a lot of false positives for dynamic objects.
Jane: They solve that by comparing their tracked image paths to what your car's own movement would cause—that’s isolating real object motion from just seeing something move because you were driving.
Tom: And then they use that corrected visual information to refine the labels for those LiDAR points, which means the final supervision for learning scene flow is much more accurate than before.
Jane: What this really changes for you on the road or in a parking garage is that your AI gets a clearer picture of what's actually going around it because it’s cross-referencing two different kinds of motion data—the three dee structure and the 2D visual movement <ref:2605.16922#pg1>.
Tom: It also means that if an object isn't perfectly clear from the LiDAR shape alone, those visual cues can help confirm it’s dynamic, which is especially important for small or slow things like pedestrians.
Jane: We saw some really solid numbers showing this helps precision and F1 scores improve significantly on those dynamic labels when compared to previous methods.
Tom: Yes, and they showed that this method actually brings down the overall error in flow predictions when you put it into larger flow estimation systems, like SeFlow or TeFlow.
Jane: It seems like this isn't just a small tweak; it’s a fundamental shift toward using multi-modal data to make scene understanding more robust, even if that means dealing with some tricky tracking limitations.
Tom: Exactly. The challenge now is how stable that tracking component stays when the camera quality changes drastically, because if the tracker breaks in bad weather, all those refined labels go out the window.
The paper's improvements: Tom: So we're looking at exactly what they suggest to make this whole system even better—it’s not just about getting labels, but about making sure those labels are actually trustworthy for the final scene flow learning model.
Jane: They’re focusing on how you can improve that balance between filtering out the noise and keeping all the actual dynamic objects in your data.
Tom: They show that by using this whole tracking and lifting process, you get a much better F1 score for those dynamic labels, which means less confusion between what's actually moving and what just looks like it’s moving randomly.
Jane: They also highlight that the way they compare the tracked image motion to your car's ego motion helps you truly isolate the object’s movement, which is a huge step toward reliable scene understanding.
Tom: The authors found that this approach improves precision significantly, bringing it up from around twenty-five percent to over thirty-one percent for those dynamic points.
Jane: It’s not just about the accuracy of the final label; it’s about how much cleaner the supervision signal is for training these flow estimation networks, which is what really matters for scaling up.
Tom: They also showed that this method works well across different datasets and even when you test it on completely new scenes, like a specific reduction in error on AEVAScenes in the zero-shot setting.
Jane: That’s a big deal because it means the system isn't just good at one specific type of traffic or environment; it has better generalization capabilities.
Tom: But they did admit there are some hurdles, and they pointed out that the core problem is still that you're projecting those three dee points down to 2D images, which means you’re always losing some of that original three dimensional geometry <ref:2605.16922#pg1>.
Jane: So while this method gives you better labels, it doesn't fix the fundamental issue of losing depth information when you move from LiDAR to a flat image plane.
Tom: That’s the caveat they give—you can't track perfectly in that sense, so future work needs to focus on finding richer ways for those RGB and LiDAR cues to help each other out without losing all the depth data.
Conclusion: Tom: So we’re wrapping up our look at "Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation," summarizing how they use image tracking to get better labels for dynamic objects in your three dee scene data.
Jane: Exactly, and the main thing to remember is that this whole framework—the tracking, the compensation, and lifting—is a way to give your AI much more reliable information when it’s trying to figure out where things are moving in three dee space.
Tom: The implication for us is that we can build scene flow models that aren't just guessing based on sparse geometry; we can use visual motion cues to confirm what’s actually happening.
Jane: It gives us a better way to supervise those learning models, which means they learn faster and more accurately about the real world.
Lu: From a representation learning standpoint, this is interesting because it shows how combining modalities can create a richer input for the network, potentially reducing the reliance on just one type of sensor data.
Meng: For us in production, what matters is stability; we need to know how much it costs to run this tracking component and what breaks when you try to deploy it on a really messy road.
Lalam: I’m confident that this paper shows a very solid path forward for integrating visual data into three dee perception tasks like scene flow estimation.
Tom: Yeah, so the paper proves that leveraging image-based point tracking actually leads to significant improvements in dynamic label accuracy and overall performance metrics across various tests.
Jane: It really shows how much better things get when you move beyond just looking at the LiDAR points on their own and start incorporating that dense visual motion information.
Tom: We’ve seen a lot of work on sensor fusion, but this specifically details a mechanism to make sure those two different data sources talk to each other effectively in terms of motion cues.
Jane: It opens up some interesting avenues for future research on how these modalities can mutually refine the features used by the AI model.
Lu: The limitation they flag is that tracking points from three dee to 2D images inherently loses some of that original three dimensional geometric information, which is a big thing to consider for accuracy <ref:2605.16922#pg1>.
Meng: That loss of depth information means we still have to be careful when we scale this up and try to run it on millions of LiDAR points because the tracking error could compound fast.
Lalam: I’m sure the authors are thinking about that trade-off between getting better labels and maintaining geometric fidelity in those future iterations.
Tom: So, it’s a solid paper for anyone working on self-supervised scene flow estimation right now, because it gives you a concrete way to boost your supervision signals.
Jane: It's a great step toward making three dee scene flow estimation more robust by using the visual world to help clarify the LiDAR data.
Tom: That’s all for today’s paper on "Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation." Next up, we're going to look at how other AI researchers are tackling sensor fault robustness in forecasting.
Korea University · Purdue University · KAIST
cs.CV
Submitted: 2026-05-16
Updated: 2026-11-04
Code: https://github.com/KTH-RPL/OpenSceneFlow
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 90/100
The gist: LiDAR scene flow estimation is essential for autonomous driving as it provides 3D motion for each point.
Key concepts
- Image-based Point Tracking
- This technique uses a point tracker on multi-frame images to follow specific points across time. It extracts dense trajectories in the image plane, which serve as rich visual motion cues that help classify points as static or dynamic.
- Visually Consistent Motion Compensation
- This strategy compares the tracked point trajectories with rigid motions caused by the ego vehicle's movement. By subtracting or compensating for ego-induced apparent motion, it effectively isolates the actual motion of surrounding objects.
- Visual Motion Cue Lifting
- This process transfers the isolated, compensated image motion cues back to their original locations in the 3D LiDAR domain using nearest-neighbor search. This converts dense visual evidence into refined static-dynamic labels for better training.
Terminology
Summary
LiDAR scene flow estimation is essential for autonomous driving as it provides 3D motion for each point. p. 0 Motion Cues from Image-based Point Tracking
for LiDAR Scene Flow Estimation
Youngdong Jang*,1 Gyeongrok Oh*,1 Jong Wook Kim1 Hyunju Ryu1
Hyung-gun Chi2 SeungHyeon Kim1 Seungryong Kim3 Jonghyun Choi4 Sangpil Kim†,1
1 Korea University 2 Purdue University 3 KAIST 4 Hyundai Motor Company
Abstract
LiDAR scene flow estimation is essential for autonomous driving, as it provides 3D
motion for each point. Self-supervised approaches use static-dynamic classification
to mitigate the imbalance between static and dynamic points, deriving targeted
supervision. To address this, we introduce TrackCue, a tracking-guided framework
for improving dynamic object representation in LiDAR scene flow estimation. In particular, TrackCue repurposes point tracking to obtain dense image-space
trajectories anchored to LiDAR points, providing motion cues beyond sparse
geometric observations. In particular, TrackCue repurposes point tracking to obtain dense image-space
trajectories anchored to LiDAR points, providing motion cues beyond sparse
geometric observations. Furthermore, we present a visually consistent motion
compensation strategy that compares the tracked trajectories with ego-induced rigid
trajectories in the image plane, effectively isolating true object motion from egoinduced apparent motion. Furthermore, we present a visually consistent motion
compensation strategy that compares the tracked trajectories with ego-induced rigid
trajectories in the image plane, effectively isolating true object motion from egoinduced apparent motion. To transfer these isolated motion cues back to the LiDAR
domain, we perform visual motion cue lifting, which associates ego-compensated
image trajectories with LiDAR points for static-dynamic label refinement. To transfer these isolated motion cues back to the LiDAR
domain, we perform visual motion cue lifting, which associates ego-compensated
image trajectories with LiDAR points for static-dynamic label refinement. As
a result, TrackCue produces more accurate static-dynamic classification and
provides more reliable supervision for scene flow learning.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The paper addresses the challenge of severe imbalance between static and dynamic points in self-supervised LiDAR scene flow estimation, which biases the model toward predicting zero flow. Existing methods rely on sparse geometric observations for static-dynamic classification, such as DUFOMap [9], which can be vulnerable to data sparsity and occlusions leading to noisy labels. p. 0 However, existing methods rely on sparse geometric observations for
this classification, making them vulnerable to data sparsity and occlusions. The
resulting noisy labels provide incorrect motion guidance and degrade scene flow
learning. Sparse and occluded LiDAR
observations often make dynamic object traces discontinuous across frames, whereas dense image
observations provide better visual continuity for tracking. To exploit this complementarity, TrackCue
leverages image-based point tracking [12, 13, 14] to obtain dense image-space trajectories as motion
cues for static-dynamic classification.Where the paper says it
Where the paper says it
How the training signal is built
The overall self-supervised objective is formulated by combining static and dynamic losses, where Ls constrains static points to have near-zero residual motion, and Ld supervises dynamic points through Chamfer distance and dynamic clustering terms. TrackCue refines the initial point clouds Pdufo into PTrackCue by associating image-space trajectories with LiDAR points through nearest-neighbor search to produce refined static-dynamic labels. p. 1 Finally, TrackCue lifts
these ego-compensated trajectory cues back to the LiDAR domain. It associates their image-space
locations with projected LiDAR points through nearest-neighbor search, thereby converting dense
visual motion evidence into refined static-dynamic labels in the 3D point cloud.Where the paper says it
Where the gains come from
The framework consists of three main components:
-
Image-based Point Tracking, which extracts temporally consistent point trajectories from multi-frame images using a point tracker, providing dense visual motion cues. p. 3
Where the paper says it
The pipeline consists of three main components: • Image-based Point Tracking: Extracts temporally consistent point trajectories from multi-frame images using a point tracker. These trajectories provide dense visual motion cues for identifying dynamic points that are ambiguous from LiDAR alone.
-
Visually Consistent Motion Compensation, which compares tracked trajectories with ego-induced rigid trajectories to distinguish true object motion from ego-induced apparent motion. p. 0
Where the paper says it
In particular, TrackCue repurposes point tracking to obtain dense image-space trajectories anchored to LiDAR points, providing motion cues beyond sparse geometric observations. Furthermore, we present a visually consistent motion compensation strategy that compares the tracked trajectories with ego-induced rigid trajectories in the image plane, effectively isolating true object motion from egoinduced apparent motion.
-
Visual Motion Cue Lifting, which lifts ego-compensated image motion cues back to LiDAR points through nearest-neighbor search to produce refined static-dynamic labels. p. 3
Where the paper says it
It filters out static points from the 2D trajectories, retaining only the true dynamic points. • Visual Motion Cue Lifting: Lifts ego-compensated image motion cues back to LiDAR points through nearest-neighbor search, converting dense visual evidence into static-dynamic labels.
Experimental results show that TrackCue significantly improves the precision and F1 score of dynamic labels, leading to performance gains in self-supervised scene flow estimation. Specifically, TrackCue reduces false-positive dynamic labels by 35. p. 8 As shown in Table 3, TrackCue significantly reduces false-positive dynamic predictions
and improves precision, while its conservative filtering leads to lower recall. Specifically, TrackCue
improves the F1 score by 5. It reduces
false-positive dynamic labels by 35. 6%, while improving precision from 24. As
a result, TrackCue produces more accurate static-dynamic classification and
provides more reliable supervision for scene flow learning. Experimental results
show that TrackCue significantly improves the precision and F1 score of dynamic
labels, leading to performance gains in self-supervised scene flow estimation. TrackCue suppresses static regions, such as
walls, while retaining points that correspond to actual object motion. (Right) TrackCue reduces
dynamic-point over-prediction compared with DUFOMap [9], achieving higher Dynamic IoU. proposed TrackCue consistently brings benefits in flow predictions regardless of the specific datasets
and distance range. Notably, we observe that TrackCue brings a larger improvement to SeFlow++
than to the other baselines. Specifically, TrackCue reduces the overall Dynamic Bucket-Normalized
EPE by 0. 68 points when integrated into SeFlow, SeFlow++, and TeFlow, respectively. 68 points when integrated into SeFlow, SeFlow++, and TeFlow, respectively. This observation highlights an important takeaway: well-separated static and dynamic clusters are
crucial for reliable self-supervision. Note that TrackCue further alleviates the sparsity and ambiguity inherent to LiDAR observations
through dense motion cues from RGB images, ultimately leading to larger performance gains. Furthermore, TrackCue yields substantial improvements across diverse datasets under the zero-shot
evaluation setting. While the gain on the in-domain Argoverse 2 subset is relatively modest, the
improvement becomes much more pronounced on unseen datasets; for example, TrackCue achieves a
13. 09 point reduction on AEVAScenes, despite its different sensor configuration.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The paper quantifies the improvement in dynamic clustering fidelity by comparing DUFOMap [9] and TrackCue for auto-label generation of both SeFlow [8] and SeFlow++ [7] pipelines. Table 3 shows that TrackCue improves the F1 score by 5. p. 8 As shown in Table 3, TrackCue significantly reduces false-positive dynamic predictions
and improves precision, while its conservative filtering leads to lower recall. Specifically, TrackCue
improves the F1 score by 5. 10 percentage points for SeFlow and SeFlow++, respectively. Since the F1 score captures the harmonic mean of precision and recall, the improved F1 score shows
that TrackCue achieves a better balance between filtering noisy dynamic labels and retaining true
dynamic points. 10 percentage points for SeFlow and SeFlow++, respectively. Since the F1 score captures the harmonic mean of precision and recall, the improved F1 score shows
that TrackCue achieves a better balance between filtering noisy dynamic labels and retaining true
dynamic points.Where the paper says it
Where the paper says it
Where the paper says it
The analysis on point tracker choice indicates that points misclassified by the LiDAR-only method are consistently refined by visual motion cues, improving precision by 5.61–10.68 percentage points and F1 score by 4. p. 8 Specifically, TrackCue
improves the F1 score by 5. 10 percentage points for SeFlow and SeFlow++, respectively. TrackCue suppresses static regions, such as
walls, while retaining points that correspond to actual object motion. (Right) TrackCue reduces
dynamic-point over-prediction compared with DUFOMap [9], achieving higher Dynamic IoU. High-resolution images
preserve finer visual details and provide more reliable point trajectories, allowing more true dynamic
points to be retained as dynamic. In contrast, low-resolution images introduce larger projection
errors and tracking ambiguities when 3D points are projected onto the 2D image plane. Through these experiments, we observe that
TrackCue brings substantial benefits for 3D motion understanding. Ultimately, our work opens a
promising direction for future research by integrating RGB images not only to enhance multimodal
feature representations, but also to provide cross-modal supervision for 3D scene flow estimation.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
While TrackCue shows promising results, a limitation is that tracking over 3D-to-2D flattened points is inherently limited since projection inevitably loses 3D geometric information. Future works could explore richer cross-modal information exchange where RGB and LiDAR cues mutually refine both supervision signals and feature representations for 3D motion learning. p. 9 Second, the scope of our work is limited to generating high-fidelity
pseudo labels for clear self-supervised signals. Future works could further explore richer cross-modal
information exchange, where RGB and LiDAR cues mutually refine both supervision signals and
feature representations for 3D motion learning. Through these experiments, we observe that
TrackCue brings substantial benefits for 3D motion understanding. Ultimately, our work opens a
promising direction for future research by integrating RGB images not only to enhance multimodal
feature representations, but also to provide cross-modal supervision for 3D scene flow estimation. † Corresponding author. arXiv:2605. 16922v1 [cs. p. 0 arXiv:2605. 16922v1 [cs. † Corresponding author. arXiv:2605. 16922v1 [cs. CV] 16 May 2026 16922v1 [cs. CV] 16 May 2026 † Corresponding author. arXiv:2605. † Corresponding author. arXiv:2605. † Corresponding author. arXiv:2605. 16922v1 [cs. CV] 16 May 2026Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation
16 May 2026. The paper is arXiv:2605. p. 0Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Tracking-guided framework for self-supervised LiDAR scene flow that leverages image-based point tracking to identify dynamic points with dense visual motion cues. This allows the system to obtain
dense image-space trajectories as motion cues for static-dynamic classification
even in sparse or occluded regions, improving the reliability of dynamic labels. -
Visually consistent motion compensation strategy that compares tracked image-space trajectories with ego-induced rigid trajectories to
isolate true object motion from egoinduced apparent motion.
This enables the system todistinguish true object motion from ego-induced apparent motion in the image plane,
leading to more accurate flow estimation by filtering out spurious motions. -
Visual motion cue lifting process that transfers ego-compensated image-space trajectories back to LiDAR points through nearest-neighbor search,
converting dense visual evidence into refined static-dynamic labels.
This producesrefined static-dynamic labels
thatcomplement LiDAR-only observations and provide more reliable supervision for scene flow learning.
-
Improved dynamic clustering fidelity by replacing DUFOMap with TrackCue, which shows a significant improvement in precision and F1 score for both SeFlow and SeFlow++ pipelines. This leads to a better balance between filtering noisy dynamic labels and retaining true dynamic points, strengthening the supervision for scene flow estimation.
-
Enhanced generalization across datasets by achieving substantial improvements on unseen datasets, such as a 13.09 point reduction on AEVAScenes in the zero-shot evaluation setting. This demonstrates that TrackCue
yields substantial improvements across diverse datasets under the zero-shot evaluation setting,
enhancing the network's ability to handle different sensor configurations and environments.
Abstract
Self-supervised LiDAR scene flow approaches rely on pseudo-labels that distinguish static and dynamic points, providing targeted supervision. These labels are typically derived from LiDAR geometry, making them unreliable in sparse or occluded regions. The resulting noise misguides motion supervision and degrades scene flow learning, yet existing methods remain largely dependent on the same geometric evidence. To address this, we introduce TrackCue, a tracking-guided framework that transfers dense image trajectories into reliable static-dynamic supervision. TrackCue anchors tracking queries to projected LiDAR points, yielding image-space trajectories that provide motion evidence beyond sparse LiDAR geometry. Since image-space displacement also reflects ego-vehicle motion, we present a visually consistent motion compensation strategy that compares each tracked trajectory with the rigid trajectory induced by ego-motion alone. It admits motion evidence only where both trajectories are valid and the deviation persists across frames, isolating true object motion while suppressing transient tracking errors. Finally, we perform visual motion cue lifting, which associates image-space object motion cues with LiDAR points for label refinement. As a result, TrackCue substantially improves the precision of dynamic pseudo-labels, providing more reliable supervision and leading to better self-supervised scene flow estimation.
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs