MetricTrail: Revisiting the Power of Metric Depth and Local Optimization for Long-Sequence Streaming 3D Reconstruction
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "MetricTrail: Revisiting the Power of Metric Depth and Local Optimization for Long-Sequence Streaming 3D Reconstruction".
Jane: Here's the gist of the paper. MetricTrail presents a streaming system for accurate camera motion estimation and dense reconstruction from long monocular videos by revisiting DPVO’s sparse local estimator with single-image metric depth and visual foundation representations to maintain efficiency over long sequences System Overview The core of MetricTrail is the bounded sparse local estimator DPVO, which is strengthened by incorporating single-image metric depth predictions and learned adapters for correspondence estimation.
Tom: This system initializes patch depths with MoGe-2 predictions and retains them as soft constraints during optimization to provide geometry specific to each image location and a learned reference for physical scale rather than just propagating the estimator’s current scale.
Jane: Furthermore, native matching features are augmented with frozen DINOv3 representations through learned adapters to predict correspondence targets and confidence weights by providing complementary appearance and contextual cues.
Tom: How the training signal is built The system utilizes a modular design where the correspondence estimator and adapters are trained on diverse short clips without additional long-sequence training.
Title and authors: Tom: So the title of this paper is "MetricTrail: Revisiting the Power of Metric Depth and Local Optimization for Long-Sequence Streaming three dee Reconstruction <ref:2611.07555#pg1,MetricTrail: Revisiting the Power of Metric Depth and Local Optimization for Long>."
Jane: It sounds like they are saying that by bringing back metric depth and local optimization, they can make long-sequence reconstruction work better without needing to do all that extra training.
Lu: They focus on how single-image metric depth initialization and persistent constraints improve long-sequence accuracy without retraining the correspondence network or enlarging the optimization window.
Meng: I wonder how much of an optimization window is actually needed for these systems to run efficiently on real traffic data, not just test scenes.
Lalam: The paper shows that they can achieve leading trajectory accuracy under evaluated protocols, strong dense reconstruction, and efficient local inference without any additional long-sequence training.
The paper's summary: Tom: So the summary says that MetricTrail uses metric depth and pretrained visual representations to reconstruct long videos through efficient sparse geometric estimation.
Jane: They are using this metric depth initialization to set patch depths and then keeping those as soft constraints while they optimize the whole system.
Lu: They also improve correspondence estimation by using learned adapters that boost the matching features and recurrent context with frozen DINOv3 representations.
Meng: So, they're not just relying on what the current optimization step sees; they are adding a prior based on a single image depth and some visual guidance from DINOv3.
Lalam: This setup allows the correspondence estimator and adapters to be trained on diverse short clips without needing any additional long-sequence training, which is a big deal for how we train these things.
The paper's improvements: Tom: The paper points out that the main gain comes from using single-image metric depth priors to initialize patch depths with MoGe-two predictions <ref:2611.07555#pg1>.
Jane: That initialization substantially improves long-sequence accuracy, and they note this happens without retraining the correspondence network or enlarging the optimization window.
Lu: They also mention that this metric depth initialization and its constraints improve long-sequence accuracy without requiring retraining the correspondence network or a larger optimization window.
Meng: If you're running this on a production server, initializing depths based on something from one image sounds like it saves a lot of computation during the optimization process.
Lalam: Plus, they combine multi-dataset adaptation and DINOv3, which further reduces the absolute trajectory error to six point zero five meters on KITTI and five point two two meters on VBR.
Conclusion: Tom: So to wrap up, MetricTrail shows that revisiting DPVO with metric depth initialization makes a big difference in long-sequence accuracy, even without retraining the correspondence network.
Jane: It also supports supplied camera intrinsics and calibrated stereo observations without retraining, which is really flexible for different sensor setups.
Lu: The system's ability to maintain a coherent scene and camera trajectory throughout the sequence demonstrates stable geometric composition beyond a fixed network context.
Meng: But they also show that revisiting both historical states and their measurements converts those accurate local estimates into a globally consistent trajectory.
Lalam: So, the final results show that revisiting both historical states and their measurements converts accurate local estimates into a globally consistent trajectory with the MetricTrail system.
NVIDIA · Shanghai Jiao Tong University · University of Toronto · Vector Institute · Stanford University
cs.CV
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: Project Page: https://research.nvidia.com/labs/sil/projects/metrictrail/, Code: https://github.com/nv-tlabs/vipe
Code: https://github.com/OpenDriveLab/LightwheelOcc
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 97/100
The gist: MetricTrail presents a streaming system for accurate camera motion estimation and dense reconstruction from long monocular videos by revisiting DPVO’s sparse local estimator with single-image…
Key concepts
- Bounded Sparse Local Estimator (DPVO)
- This is the core estimator in MetricTrail that handles local camera motion estimation. It is designed to be efficient for long sequences by focusing on local patches rather than trying to model the entire video at once. It uses constraints derived from metric depth and learned features to guide its predictions.
- Single-Image Metric Depth Priors
- This involves using predictions for depth derived from a single image (like MoGe-2) to set the initial depths for small patches in the 3D reconstruction. Instead of guessing these depths randomly, this prior provides a geometrically informed starting point, which greatly stabilizes and improves the accuracy of the subsequent long-sequence optimization.
- Learned Adapters with DINOv3
- These are modules added to existing matching features that use frozen visual representations from DINOv3. They learn how to best combine these appearance and contextual cues to predict which pixels correspond between frames. This allows the system to better estimate correspondence targets and confidence weights without needing extensive retraining on long video sequences.
- Geometric Optimization Constraints
- This refers to the process where local estimations are refined over time by composing measurements from multiple views. MetricTrail uses metric depth initialization as a constraint during this optimization, ensuring that the final trajectory remains geometrically consistent across the entire sequence, even when relying on short-clip training data.
Terminology
Summary
MetricTrail presents a streaming system for accurate camera motion estimation and dense reconstruction from long monocular videos by revisiting DPVO’s sparse local estimator with single-image metric depth and visual foundation representations to maintain efficiency over long sequences
System Overview
The core of MetricTrail is the bounded sparse local estimator DPVO, which is strengthened by incorporating single-image metric depth predictions and learned adapters for correspondence estimation. This system initializes patch depths with MoGe-2 predictions and retains them as soft constraints during optimization to provide geometry specific to each image location and a learned reference for physical scale rather than just propagating the estimator’s current scale. p. 1 Building on metric-depth regularization in learned odometry [3, 5], we initialize patch
depths with MoGe-2 [6] predictions and retain them as robust soft constraints during optimization. These
predictions provide both geometry specific to each image location and a learned reference for physical scale,
rather than simply propagating the estimator’s current scale as the camera advances. We therefore augment native matching
features and recurrent context with frozen DINOv3 representations [7] through learned adapters. These
representations provide complementary appearance and contextual cues for predicting correspondence targets
and confidence weights.Where the paper says it
Where the paper says it
How the training signal is built
The system utilizes a modular design where the correspondence estimator and adapters are trained on diverse short clips without additional long-sequence training. The correspondence estimator and adapters are trained on diverse public datasets using short clips, preserving the sparse patch graph and local solver without additional long-sequence training. p. 1 These
representations provide complementary appearance and contextual cues for predicting correspondence targets
and confidence weights. The correspondence estimator and adapters are trained on diverse public datasets
using short clips, preserving the sparse patch graph and local solver without additional long-sequence training. The network learns local correspondence corrections, while repeated geometric optimization composes these
measurements over time. This division allows us to train on diverse short clips without requiring long-trajectory
supervision.Where the paper says it
Where the paper says it
Where the gains come from
The primary gains stem from using single-image metric depth priors to initialize patch depths with MoGe-2 predictions, which substantially improves the long-sequence accuracy of the pretrained sparse estimator without retraining its correspondence network or enlarging its optimization window. The metric depth initialization and constraints improve long-sequence accuracy without correspondence-network retraining or a larger optimization window. p. 2 • Sparse local estimator. Metric-depth initialization and persistent constraints improve long-sequence
accuracy without correspondence-network retraining or a larger optimization window. 61 m and
improve revisit alignment without frontend retraining. With
the same prior, multi-dataset adaptation and DINOv3 further reduce ATE to 6. This modular design also allows
the depth provider to be independently upgraded and supports supplied camera intrinsics and calibrated
stereo observations without retraining. Under the reported RGB-only protocols, MetricTrail achieves
the lowest average trajectory error among evaluated methods on KITTI, Oxford Spires, and VBR.Where the paper says it
Where the paper says it
Where the paper says it
Performance Metrics
MetricTrail achieves the lowest average full-sequence absolute trajectory error (ATE) among evaluated methods on KITTI, Oxford Spires, and VBR under the reported RGB-only protocols <ref:2611.07555#pg2>. For dense reconstruction, it achieves the lowest Chamfer distance on 7Scenes and Oxford Spires using single-image predictions without dense depth optimization. p. 1 MetricTrail achieves the lowest average full-sequence absolute trajectory error (ATE) among the evaluated methods on KITTI, Oxford Spires, and VBR under the reported RGB-only protocols, while retaining
efficient local inference. Its dense reconstructions also achieve the lowest Chamfer distance on 7Scenes and
Oxford Spires using single-image predictions without dense depth optimization. The correspondence estimator and adapters are
trained on diverse short clips without additional long-sequence training. This modular design also allows
the depth provider to be independently upgraded and supports supplied camera intrinsics and calibrated
stereo observations without retraining. Ablation Study
Tab. 5 shows that metric depth yields the largest improvement across all three datasets. With supplied
intrinsics, the system processes KITTI-02 at 12. 70 FPS with 3.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Ablation and Refinement
The ablation study confirms that metric depth yields the largest improvement across all three datasets, and adding DINOv3 further improves all three datasets <ref:2611.07555#pg10>. The performance of MetricTrail is shown to be robust across different depth providers, with the full model improving over DPVO with the same prior in 14 of 15 prior–dataset combinations <ref:2611. p. 9 For the five alternative providers,
the full model improves over DPVO with the same prior in 14 of the 15 prior–dataset combinations, including
all five providers on both KITTI and Oxford Spires. These results show that the estimator enhancements remain
effective across different depth providers, although the gains depend on the provider and dataset. We then fix K, discard the auxiliary
state, and replay the prefix through the local estimator before continuing online. Revisit-based focal refinement
remains an optional retrospective step. The known rig transform provides a same-time reprojection constraint on patch geometry, without
introducing right-camera pose variables or requiring stereo-specific training. Loop closure. Classical systems solve offline SfM [8, 9] or maintain online estimates [10, 11]. Local windows bound update costs; global refinement and loop closure address errors
accumulated beyond them. The highlighted indoor and in-the-wild regions also show finer pointcloud detail and more accurate local shape in the depth maps. These results illustrate the complementary roles
of depth and camera motion: depth captures local geometry, while the optimized trajectory aligns observations
into a coherent scene.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Metric depth initialization substantially improves long-sequence accuracy by using MoGe-2 predictions to initialize patch depths rather than relying on heuristic initialization or simple scale propagation, stating
We find that single-image metric depth substantially improves its trajectory accuracy, without retraining the correspondence network or enlarging the optimization window.
-
Correspondence estimation is enhanced through learned adapters that augment matching features and recurrent context with frozen DINOv3 representations, allowing for better prediction of correspondence targets and confidence weights without additional long-sequence training.
-
The system supports supplied camera intrinsics and calibrated stereo observations without retraining, enabling flexible deployment across different sensor setups, as it
supports supplied camera intrinsics and calibrated stereo observations without retraining.
-
Trajectory accuracy is improved by utilizing metric-depth regularization during optimization, where the optimization objective includes a term like
Huber delta(log rho k - log rho hat k),
which allows multi-view evidence to override unreliable monocular predictions. -
The system can achieve low Chamfer distance on dense reconstruction even without dense depth optimization, as it
achieves the lowest Chamfer distance on 7Scenes and Oxford Spires using single-image predictions without dense depth optimization.
Abstract
We present MetricTrail, a streaming system for accurate camera motion estimation and dense reconstruction from long monocular videos. To keep inference efficient over long sequences, we revisit DPVO's sparse local estimator. We find that single-image metric depth substantially improves its trajectory accuracy, without retraining the correspondence network or enlarging the optimization window. Metric predictions initialize patch depths and remain as soft constraints during optimization. We further improve correspondence estimation through learned adapters that augment matching features and recurrent context with frozen DINOv3 representations. The correspondence estimator and adapters are trained on diverse short clips without additional long-sequence training. This modular design also allows the depth provider to be independently upgraded and supports supplied camera intrinsics and calibrated stereo observations without retraining. Under the reported RGB-only protocols, MetricTrail achieves the lowest average trajectory error among evaluated methods on KITTI, Oxford Spires, and VBR. Pairing the estimated poses with single-image depth predictions also produces strong dense reconstruction on indoor and outdoor benchmarks and coherent reconstructions of in-the-wild videos. With supplied intrinsics, the system processes KITTI-02 at 12.70 FPS with 3.21 GB peak GPU memory.
Sources
- Geometric Context Transformer for Streaming 3D Reconstruction
- RoMeO: Robust Metric Visual Odometry
- DINOv3
- VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion
- DINO-VO: Learning Where to Focus for Enhanced State Estimation
- Depth Anything 3: Recovering the Visual Space from Any Views
- FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry
- D'ej\`a View: Looping Transformers for Multi-View 3D Reconstruction
- FastVGGT: Training-Free Acceleration of Visual Geometry Transformer
- Mem3R: Streaming 3D Reconstruction with Hybrid Memory
- LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory
- Streaming 4D Visual Geometry Transformer
- Towards Consistent Video Geometry Estimation
- $R^3$: 3D Reconstruction via Relative Regression
- HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction
- VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences
- VGGT-SLAM 2.0: Real-time Dense Feed-forward Scene Reconstruction
- Marginalized Bundle Adjustment: Multi-View Camera Pose from Monocular Depth Estimates
- TartanGround: A Large-Scale Dataset for Ground Robot Perception and Navigation
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs