MASt3R-Fusion: Integrating Feed-Forward Visual Model with IMU, GNSS for High-Functionality SLAM
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "MASt3R-Fusion: Integrating Feed-Forward Visual Model with IMU, GNSS for High-Functionality SLAM".
Dev: Here's the gist of the paper. MASt3R-Fusion proposes a multi-sensor-assisted visual SLAM framework that tightly integrates feed-forward pointmap regression with complementary sensor information, including inertial measurements and GNSS data, to achieve globally consistent pose estimation and mapping <ref:2509.20757#pg2> The system introduces Sim(3)-based visual alignment constraints into a universal metric-scale SE(3) factor graph for effective information fusion <ref:2509.20757#pg5> Framework Integration The framework is mainly composed of two parts: real-time SLAM and global optimization <ref:2509.20757#pg2> In the real-time SLAM stage, input images are processed through a feed-forward neural network to recover the pointmaps and achieve further dense matching <ref:2509.20757#pg2> Pose tracking is performed by aligning pointmaps between consecutive frames <ref:2509.20757#pg2> In the global optimization stage, the information stored from real-time SLAM are processed in a global manner with extra loop closures and GNSS information <ref:2509.20757#pg2> Visual Measurement Paradigm The typical feed-forward visual model directly regresses the 3D pointmaps of RGB images in a common reference frame, and then estimates the camera poses through optimization <ref:2509.20757#pg4> The pipeline of pointmap regression is depicted in Fig.
Rosa: 3 <ref:2509.20757#pg2> This involves encoding images into feature tokens and decoding them to produce the pointmaps and descriptors <ref:2509.20757#pg2> Dense matching based on ray proximity is achieved through the optimization of point correspondences with large depth residuals <ref:2509.20757#pg4> Metric-Scale State Estimation To bridge the gap between Sim(3)-based visual alignment constraints and a more common metric-scale SE(3) factor graph, the isomorphic group transformation is introduced <ref:2509.20757#pg5> This transformation allows Sim(3)-based visual constraints to be consistently applied to SE(3) poses, enabling joint optimization with IMU, GNSS, and other metric-scale sensor information <ref:2509.20757#pg5> The system maintains a sliding-window system state where each keyframe state is represented by Ti ∈ SE(3), si ∈ R+, vi ∈ R3, and bi = (bg i, ba i) is the IMU bias vector <ref:2509.20757#pg5> Global Consistency and Robustness The global optimization stage incorporates loop closure and GNSS data for consistent pose estimation through efficient global optimization <ref:2509.20757#pg2> Loop closure detection uses image feature tokens obtained from the feed-forward encoder to construct an image retrieval system <ref:2509.20757#pg2> The loop closure candidate filtering is based on positional uncertainty, using a criterion such as d(t¯i, t¯j) The global factor graph optimization minimizes a cost function that includes visual odometry, visual alignment constraints, and GNSS measurements to achieve nearly optimal estimation results <ref:2509.20757#pg2> Experimental Validation Experimental results on both public benchmarks and self-collected datasets indicate substantial improvements over existing visual and multi-sensor SLAM systems <ref:2509.20757#pg2> On the KITTI-360 dataset, the proposed MASt3R-Fusion achieves accurate pose estimation, yielding lower relative translation errors (RTEs) compared to both traditional indirect/direct VIO methods and learning-based VIO approaches <ref:2509.20757#pg2> On the Wuhan urban dataset, the proposed MASt3R-Fusion can effectively control and correct the scale drift by combining probabilistic marginalization with Sim(3)-based visual alignment constraints <ref:2509.20757#pg2> The system demonstrates robust performance even under 100-second GNSS outages, achieving mostly sub-meter level trajectory smoothing <ref:2509.20757#pg5> Reconstruction Performance The proposed method can effectively recover the dense structure of the scene with metric scale, and for revisited trajectories, the consistency of scene reconstruction can be achieved based on high accuracy pose estimation <ref:2509.20757#pg2> The system preserves the overall 3D structure while leveraging binocular geometry, attaining relatively stable and complete realtime 3D perception performance <ref:2509.20757#pg2> In this work, MASt3R-Fusion presents a novel multisensor-assisted visual SLAM framework that tightly integrates feed-forward pointmap regression with complementary sensor information, including inertial and GNSS measurements <ref:2509.20757#pg2> By introducing Sim(3)-based visual alignment constraints into a universal metric-scale SE(3) factor graph, our system achieves accurate pose tracking, metric-scale 3D structure perception and globally consistent SLAM <ref:2509.20757#pg2> This work facilitates robust and accurate pose estimation and mapping in ubiquitous scenarios >.
Dev: References
1: S.
Rosa: Mokssit, D.
Title and authors: Rosa: So we're looking at this paper now, MASt3R-Fusion: Integrating Feed-Forward Visual Model with IMU, GNSS for High-Functionality SLAM. It sounds like they’re putting a lot of different tools together to make visual simultaneous localization and mapping more robust than what we usually see out there.
Dev: Yeah, it combines the idea of using these deep learning pointmap regressions with traditional multi-sensor information, specifically inertial measurements and GPS data. The title suggests this is about high functionality SLAM because they are tackling things like scale ambiguity and low texture environments that classical systems struggle with.
Taro: I wonder if bringing in GNSS at all changes the fundamental problem of visual odometry, or if it's just an extra layer of correction when the visual data itself is weak?
Rosa: That’s a good question. The paper seems to be arguing that the pointmap regression part gives you some strong three dee structure priors, which helps with scale estimation, and then fusing that with IMU and GNSS data makes it much more consistent globally <ref:2509.20757#pg1>. It’s about getting better pose estimation by not relying on just the cameras in isolation.
Dev: Exactly. The key innovation they introduce is this Sim(three)-based visual alignment constraints into a universal metric-scale SE(three) factor graph, which lets them combine these different information sources in a coherent way for both real-time tracking and global optimization <ref:2509.20757#pg1,Sim(3)-based visual alignment constraints>.
Taro: So, the main idea is bridging that gap between the three dee visual constraints and the standard pose estimation framework <ref:2509.20757#pg1>? That sounds like a lot of engineering work to get right in real-time.
Rosa: It definitely is. They are trying to make sure that whatever this powerful pointmap regression gives us, it actually translates into accurate, metric-scale poses when you combine it with the other sensors and optimize everything together.
The paper's summary: Dev: Okay, so looking at the summary of MASt3R-Fusion, they’re basically saying that instead of just relying on traditional methods where you regress pointmaps and then estimate poses separately, this system tightens everything up.
Rosa: Right. They take the output from a feed-forward neural network that regresses three dee pointmaps from images and use those rich three dee priors to help estimate camera poses through optimization <ref:2509.20757#pg1>. But then they integrate this into a probabilistic multi-sensor fusion framework that also includes IMU and GNSS data.
Taro: It sounds like the core summary is about taking a powerful vision component, making it metric-scale aware, and then using that awareness as a strong constraint within a larger system that benefits from all the sensor inputs.
Dev: That’s right. They are describing how they use feed-forward pointmap regression to get instant access to scaleless three dee structure information, which leads to aggressive data association and better dense matching, and then they plug those into the fusion framework <ref:2509.20757#pg1>.
Rosa: So, it’s not just a better visual SLAM system; it’s framed as a multi-sensor-assisted visual SLAM framework that uses these complementary sensors—the IMU and GNSS—to achieve globally consistent pose estimation.
Taro: So what does that mean for the listener driving? It means if the camera gets lost or sees a boring road, the system doesn't just fail; it uses that three dee information and the other sensors to keep track of where you are accurately over long distances <ref:2509.20757#pg1>.
The paper's improvements: Rosa: Now let’s talk about what they actually improved. They suggest a few key things, starting with how they handle the fusion. They introduce this Sim(three)-based visual alignment constraints into a universal metric-scale SE(three) factor graph <ref:2509.20757#pg1,Sim(3)-based visual alignment constraints>.
Dev: That’s the technical trick that lets them apply those visual constraints consistently to the standard SE(three) poses, which is what allows for joint optimization with IMU and GNSS information simultaneously <ref:2509.20757#pg1>. It solves the problem of how to merge these different mathematical frameworks together.
Taro: So, this transformation they introduce between Sim(three) and SE(three) is crucial because it enables that joint optimization across all the sensors at once <ref:2509.20757#pg1>?
Rosa: Precisely. And they also improved their global consistency by incorporating loop closures and GNSS data into the global optimization stage using efficient methods. They use image feature tokens from their encoder to build an image retrieval system for finding those loop closures.
Dev: That means they’re not just doing local tracking; they are aggressively looking for places in the map where they've been before, using that three dee awareness to filter potential loop closure candidates based on positional uncertainty <ref:2509.20757#pg1>.
Taro: So, when the world misbehaves—like you drive into a visually confusing area or lose GPS signal—the system has this mechanism to recover globally because it knows where it was before. That sounds like a solid safety case for real-world use.
Conclusion: Rosa: To wrap up, MASt3R-Fusion is presented as a novel multisensor-assisted visual SLAM framework that tightly integrates feed-forward pointmap regression with complementary sensor information, including inertial and GNSS measurements. They’ve shown this combination leads to accurate pose tracking and metric-scale three dee structure perception while maintaining globally consistent SLAM <ref:2509.20757#pg1>.
Dev: The main implication is that by introducing those Sim(three)-based visual alignment constraints into the SE(three) factor graph, they achieve a system that can handle real-time state estimation, complete-view structure perception, and globally consistent mapping at once <ref:2509.20757#pg1,Sim(3)-based visual alignment constraints>. It’s about making sure the visual information isn't just local; it’s part of a bigger global picture supported by all those other sensors.
Taro: From my side, what I think is that they manage to preserve the overall three dee structure while using binocular geometry to get that stable, complete real-time three dee perception performance, which is important when you’re driving and need reliable depth information even if the visual cues are a little tricky <ref:2509.20757#pg1>.
Rosa: Exactly. They show robust performance even under ten-hundred second GNSS outages, achieving mostly sub-meter level trajectory smoothing there. It proves that this integrated approach works when the world throws curveballs at it.
Dev: So, to finish up with MASt3R-Fusion, it’s a framework that achieves high precision pose estimation and mapping by tightly integrating multi-sensor information with the feed-forward pointmap regression paradigm. This work opens doors for more robust and accurate pose estimation in situations where we have access to IMU and GNSS data.
Taro: I just think the ability to recover globally when the world misbehaves, thanks to those loop closures filtered by positional uncertainty, is what makes this framework really interesting for real-world autonomy.
Wuhan University
cs.RO, cs.CV
Submitted: 2025-09-25
Updated: 2025-11-16
Journal ref: ISPRS Journal of Photogrammetry and Remote Sensing 239 (2026) 698-712
DOI: 10.1016/j.isprsjprs.2026.06.022
Code: https://github.com/GREAT-WHU/MASt3R-Fusion
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 82/100
The gist: MASt3R-Fusion proposes a multi-sensor-assisted visual SLAM framework that tightly integrates feed-forward pointmap regression with complementary sensor information…
Key concepts
- Feed-forward Pointmap Regression
- This technique uses a neural network to directly predict the 3D point cloud (pointmap) from an RGB image. Instead of traditional methods that estimate depth first, this model learns to generate the 3D structure instantly. This allows for fast, dense matching and accurate scale estimation during real-time SLAM.
- Sim(3)-based Visual Alignment Constraints
- This involves using constraints derived from Sim(3) geometry to align visual data with the metric-scale SE(3) poses. This transformation bridges the gap between visual representations and standard pose graphs, ensuring that visual measurements are consistently applied across different sensor inputs like IMU and GNSS.
- Metric-Scale State Estimation
- This approach uses a universal metric-scale SE(3) factor graph to estimate robot poses. It integrates various sensors (IMU, GNSS) by applying geometric transformations that allow Sim(3) visual constraints to work seamlessly with the metric scale of the state estimation, leading to joint optimization.
Terminology
Summary
MASt3R-Fusion proposes a multi-sensor-assisted visual SLAM framework that tightly integrates feed-forward pointmap regression with complementary sensor information, including inertial measurements and GNSS data, to achieve globally consistent pose estimation and mapping <ref:2509. p. 020757#pg2> The system introduces Sim(3)-based visual alignment constraints into a universal metric-scale SE(3) factor graph for effective information fusion <ref:2509. p. 020757#pg5>
Framework Integration
The framework is mainly composed of two parts: real-time SLAM and global optimization <ref:2509.20757#pg2> In the real-time SLAM stage, input images are processed through a feed-forward neural network to recover the pointmaps and achieve further dense matching <ref:2509. p. 120757#pg2> Pose tracking is performed by aligning pointmaps between consecutive frames <ref:2509. p. 120757#pg2> In the global optimization stage, the information stored from real-time SLAM are processed in a global manner with extra loop closures and GNSS information <ref:2509. p. 120757#pg2>
Visual Measurement Paradigm
The typical feed-forward visual model directly regresses the 3D pointmaps of RGB images in a common reference frame, and then estimates the camera poses through optimization <ref:2509.20757#pg4> The pipeline of pointmap regression is depicted in Fig. p. 2 3 <ref:2509. p. 020757#pg2> This involves encoding images into feature tokens and decoding them to produce the pointmaps and descriptors <ref:2509. p. 220757#pg2> Dense matching based on ray proximity is achieved through the optimization of point correspondences with large depth residuals <ref:2509. p. 220757#pg4>
Metric-Scale State Estimation
To bridge the gap between Sim(3)-based visual alignment constraints and a more common metric-scale SE(3) factor graph, the isomorphic group transformation is introduced <ref:2509.20757#pg5> This transformation allows Sim(3)-based visual constraints to be consistently applied to SE(3) poses, enabling joint optimization with IMU, GNSS, and other metric-scale sensor information <ref:2509. p. 420757#pg5> The system maintains a sliding-window system state where each keyframe state is represented by Ti ∈ SE(3), si ∈ R+, vi ∈ R3, and bi = (bg i, ba i) is the IMU bias vector <ref:2509. p. 420757#pg5>
Global Consistency and Robustness
The global optimization stage incorporates loop closure and GNSS data for consistent pose estimation through efficient global optimization <ref:2509.20757#pg2> Loop closure detection uses image feature tokens obtained from the feed-forward encoder to construct an image retrieval system <ref:2509. p. 620757#pg2> The loop closure candidate filtering is based on positional uncertainty, using a criterion such as d(t¯i, t¯j) The global factor graph optimization minimizes a cost function that includes visual odometry, visual alignment constraints, and GNSS measurements to achieve nearly optimal estimation results <ref:2509. p. 720757#pg2>
Experimental Validation
Experimental results on both public benchmarks and self-collected datasets indicate substantial improvements over existing visual and multi-sensor SLAM systems <ref:2509.20757#pg2> On the KITTI-360 dataset, the proposed MASt3R-Fusion achieves accurate pose estimation, yielding lower relative translation errors (RTEs) compared to both traditional indirect/direct VIO methods and learning-based VIO approaches <ref:2509. p. 820757#pg2> On the Wuhan urban dataset, the proposed MASt3R-Fusion can effectively control and correct the scale drift by combining probabilistic marginalization with Sim(3)-based visual alignment constraints <ref:2509. p. 120757#pg2> The system demonstrates robust performance even under 100-second GNSS outages, achieving mostly sub-meter level trajectory smoothing <ref:2509.20757#pg5>
Reconstruction Performance
The proposed method can effectively recover the dense structure of the scene with metric scale, and for revisited trajectories, the consistency of scene reconstruction can be achieved based on high accuracy pose estimation <ref:2509.20757#pg2> The system preserves the overall 3D structure while leveraging binocular geometry, attaining relatively stable and complete realtime 3D perception performance <ref:2509. p. 920757#pg2>
Conclusion
In this work, MASt3R-Fusion presents a novel multisensor-assisted visual SLAM framework that tightly integrates feed-forward pointmap regression with complementary sensor information, including inertial and GNSS measurements <ref:2509.20757#pg2> By introducing Sim(3)-based visual alignment constraints into a universal metric-scale SE(3) factor graph, our system achieves accurate pose tracking, metric-scale 3D structure perception and globally consistent SLAM <ref:2509. p. 020757#pg2> This work facilitates robust and accurate pose estimation and mapping in ubiquitous scenarios >. p. 1
References
[1] S. Mokssit, D. B. Licea, B. Guermah, and M. Ghogho, “Deep learning techniques for visual slam: A survey,” IEEE Access, vol. p. 0 11, pp. 20 026–20 050, 2023 >
[2] S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud, “DUSt3R: Geometric 3D Vision Made Easy,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 3 IEEE, Jun. 2024 >
[3] V. Leroy, Y. Cabon, and J. Revaud, “Grounding Image Matching in 3D with MASt3R,” arXiv preprint arXiv:2406. p. 009756, 2024 >
[4] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny, “VGGT: Visual Geometry Grounded Transformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 0 IEEE, 2025 >
[5] R. Murai, E. Dexheimer, and A. J. Davison, “MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 0 IEEE, Jun. 2025 >
[6] D. Maggio, H. Lim, and L. Carlone, “VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold,” arXiv preprint arXiv:2505. p. 112549, 2025 >
[7] S. Leutenegger, S. Lynen, M. Bosse, R. Siegwart, and P. T. Furgale, “Keyframe-Based Visual-Inertial SLAM Using Nonlinear Optimization,” The International Journal of Robotics Research, vol. p. 0 34, no. 3, pp. 314–334, 2015 >
[8] T. Qin, S. Cao, J. Pan, P. Li, and S. Shen, “VINS-Fusion: An Optimization-Based Multi-Sensor State Estimator,” 2020 >
[9] X. Li, S. Li, Y. Zhou, Z. Shen, X. Wang, X. Li, and W. Wen, “Continuous and precise positioning in urban environments by tightly coupled integration of GNSS, INS and vision,” IEEE Robotics and Automation Letters, vol. p. 0 7, no. 4, pp. 11 458–11 465 >
[10] H. Wang and L. Agapito, “3D Reconstruction with Spatial Memory,” arxiv, vol. 2408.16061, pp. 1–14, Aug. 2024 >
[11] J. Zhang, C. Herrmann, J. Hur, V. Jampani, T. Darrell, F. Cole, D. Sun, and M.-H. Yang, “MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion,” arXiv preprint arxiv:2410.03825, 2024 >
[12] Y. Wang, J. Zhou, H. Zhu, W. Chang, Y. Zhou, Z. Li, J. Chen, J. Pang, C. Shen, and T. He, “Pi3: Scalable Permutation-Equivariant Visual Geometry Learning,” arXiv preprint arXiv:2507. p. 013347, 2025 >
[13] W. Jang, P. Weinzaepfel, V. Leroy, L. Agapito, and J. Revaud, “Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors,” 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. p. 3 1071–1081 >
[14] C. Campos, R. Elvira, J. J. G. Rodr´ıguez, J. M. M. Montiel, and J. D. Tardos, “ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM,” IEEE Transactions on Robotics, vol. p. 0 37, no. 6, pp. 1874–1890, 2021 >
[15] D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self-supervised interest point detection and description,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, pp. p. 6 224–236 >
[16] D. Li, X. Shi, Q. Long, S. Liu, W. Yang, F. Wang, Q. Wei, and F. Qiao, “DXSLAM: A robust and efficient visual SLAM system with deep features,” in 2020 IEEE/RSJ International conference on intelligent robots and systems (IROS). p. 2 IEEE, 2020 >
[17] Z. Zhan, Y. Yu, R. Xia, W. Gan, H. Xie, G. Perda, L. Morelli, F. Remondino, and X. Wang, “SfM on-the-fly: A robust near real-time SfM for spatiotemporally disordered high-resolution imagery from multiple agents,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 224, pp. 202–221, 2025 >
[18] K. Tateno, F. Tombari, I. Laina, and N. Navab, “Cnnslam: Real-time dense monocular slam with learned depth prediction,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. p. 0 6243–6252 >
[19] X. Wang, Y. Zhuang, X. Cao, J. Huai, Z. Zhang, Z. Zheng, and N. El-Sheimy, “GAT-LSTM: A feature point management network with graph attention for feature-based visual SLAM in dynamic environments,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. p. 2 224, pp. 75–93, 2025 >
[20] A. Adkins, T. Chen, and J. Biswas, “ObVi-SLAM: Long-Term Object-Visual SLAM,” IEEE Robotics and Automation Letters, vol. p. 0 9, no. 3, pp. 2909–2916, Mar. 2024 >
[21] S. Yang and S. Scherer, “Cubeslam: Monocular 3-d object slam,” IEEE Transactions on Robotics, vol. p. 0 35, no. 4, pp. 925–938, 2019 >
[22] H. Zhou, Z. Guo, S. Liu, L. Zhang, Q. Wang, Y. Ren, and M. Li, “MoD-SLAM: Monocular Dense Mapping for Unbounded 3D Scene Reconstruction,” arXiv preprint arXiv:2402. p. 103762, 2024 >
[23] H. Matsuki, R. Murai, P. H. J. Kelly, and A. J. Davison, “Gaussian Splatting SLAM,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. p. 1 1234–1243 >
[24] Z. Teed and J. Deng, “Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras,” Advances in neural information processing systems, vol. p. 1 34, pp. 16 558–16 569, 2021 >
[25] Y. Liu, S. Dong, S. Wang, Y. Yin, Y. Yang, Q. Fan, and B. Chen, “SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos,” arXiv preprint arXiv:2412. p. 109401, 2024 >
[26] T. Qin, P. Li, and S. Shen, “Vins-mono: A robust and versatile monocular visual-inertial state estimator,” IEEE transactions on robotics, vol. p. 8 34, no. 4, pp. 1004–1020, 2018 >
[27] L. Von Stumberg and D. Cremers, “Direct sparse visualinertial odometry using dynamic marginalization,” in 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018, pp. 2510–2517 >
[28] F. Jiang, J. Chen, and S. Ji, “Panoramic Visual-Inertial SLAM Tightly Coupled with a Wheel Encoder,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. p. 0 182, pp. 96–111, 2021 >
[29] D. Wang, J. Wang, Y. Tian, Y. Fang, Z. Yuan, and M. Xu, “PAL-SLAM2: Visual and visual–inertial monocular SLAM for panoramic annular lens,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. p. 0 211, pp. 35–48, 2024 >
[30] L. v. Stumberg and D. Cremers, “DM-VIO: Delayed Marginalization Visual-Inertial Odometry,” IEEE Robotics and Automation Letters (RAL) & International Conference on Robotics and Automation (ICRA), vol. p. 0 7, no. 2, pp. 1408–1415, 2022 >
[31] L. Wang, H. Tang, T. Zhang, Y. Wang, Q. Zhang, and X. Niu, “POKF: A Pose-Only Representation-Based Kalman Filter for Visual Inertial Odometry,” IEEE Internet of Things Journal, vol. p. 1 12, no. 10, pp. 14 856–14 875, 2025 >
[32] Z. Xiao and S. Li, “A real-time, robust and versatile visualSLAM framework based on deep learning networks,” 2024, eprint: 2405. p. 003413 >
[33] H. Luo, Y. Liu, C. Guo, Z. Li, and W. Song, “SuperVINS: A Real-Time Visual-Inertial SLAM Framework for Challenging Imaging Conditions,” arXiv, vol. p. 0 2407.21348 >
[34] Y. Zhou, X. Li, S. Li, X. Wang, S. Feng, and Y. p. 0 Tan, “DBA-Fusion: Tightly Integrating Deep Dense Visual Bundle Adjustment With Multiple Sensors for Large-Scale Localization and Mapping,” IEEE Robotics and Automation Letters, vol. p. 0 9, no. 7, pp. 6138–6145 >
[35] X. Peng, Z. Liu, W. Li, P. Tan, S. Y. Cho, and Q. Wang, “Dvi-slam: A dual visual inertial slam network,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). p. 0 IEEE, 2024 >
[36] L. Han, Y. Lin, G. Du, and S. Lian, “Deepvio: Self-supervised deep learning of monocular visual inertial odometry using 3d geometric constraints,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). p. 1 IEEE, 2019 >
[37] S. Cao, X. Lu, and S. Shen, “GVINS: Tightly Coupled GNSS–Visual–Inertial Fusion for Smooth and Consistent State Estimation,” IEEE Transactions on Robotics, vol. p. 0 38, no. 4, pp. 2004–2021, 2021 >
[38] B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment—a modern synthesis,” in International workshop on vision algorithms. p. 1 Springer, 1999, pp. 298–372 >
[39] C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza, “IMU preintegration on manifold for efficient visual-inertial maximum-aposteriori estimation,” 2015 >
[40] Y. Liao, J. Xie, and A. Geiger, “Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. p. 8 45, no. 3, pp. 3292–3310, 2022 >
[41] S. Zhao, Y. Gao, T. Wu, D. Singh, R. Jiang, H. Sun, M. Sarawata, Y. Qiu, W. Whittaker, I Higginslneiraeiraeiraeiraeiraeiraeiraeiraeireireireireireireireireiraretiretiretiretiretire tire tire tire tire tire tire>
[42] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? p. 0 The KITTI vision benchmark suite,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, Jun. p. 8 2012 >
[43] M. Grupp, “evo: Python package for the evaluation of odometry and SLAM. p. 7” 2017 >
[44] K. Deng, Z. Ti, J. Xu, J. Yang, and J Xie, “VGGT-Long: Chunk it, Loop it, Align it — Pushing VGGT’s Limits on Kilometer-scale Long RGB Sequences,” arXiv preprint arXiv:2507. p. 116443, 2025 >
[45] M. Hu, W. Yin, C. Zhang, Z. Cai, X. Long, H. Chen, K Wang, G Yu, C Shen, and S Shenlneiraeiraeiraeiraeiraeireireireireireiretiretiretire tire tire>
[46] A. Martinelli, “Visual-inertial structure from motion: Observability and resolvability,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nov. p. 2 2013, pp.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Integration of Sim(3)-based visual alignment constraints into a universal metric-scale SE(3) factor graph allows for effective information fusion across real-time sliding-window optimization and global optimization with aggressive loop closures. This enables real-time pose tracking, metric-scale structure perception, and globally consistent mapping by bridging the gap between Sim(3)-based visual alignment constraints and metric-scale SE(3) states and factors.
-
Development of a feedforward model based real-time visual inertial SLAM system supports metric scale pose estimation and dense scene structure perception through the use of pointmap regression. This system is capable of handling arbitrarily long sequences based on 8 GB GPU memory, leveraging
Powerful 3D priors, which enable instant access to the scaleless 3D structure information.
-
Implementation of a globally consistent SLAM system that leverages loop closure and GNSS information utilizes geometry-based loop closure candidate filtering and full-information iterative optimization. This results in globally consistent and drift-free pose estimation by incorporating
geometry-based loop closure candidate filtering
and utilizingfullinformation iterative optimization.
-
The framework enables real-time, high-precision pose estimation, dense perception, and globally consistent mapping by tightly integrating multi-sensor information with the feedforward pointmap regression paradigm. This is achieved through the design of a system that supports
real-time state estimation, complete-view structure perception and globally consistent mapping.
-
The isomorphic group transformation between Sim(3) and SE(3) allows Sim(3)-based visual constraints to be consistently applied to SE(3) poses, enabling joint optimization with IMU, GNSS, and other metric-scale sensor information. This transformation facilitates the relationship
sR t = s 0 0 1 t
being equivalent to a factorization into an SE(3) transform followed by a scalar scaling. -
The system achieves high odometry accuracy on KITTI-360 data, yielding lower relative translation errors (RTEs) compared to both traditional indirect/direct VIO methods and learning-based VIO approaches, with the average RTE being 43.0% lower than DM-VIO and 17.7% lower than DBA-Fusion.
-
The proposed method achieves a significantly lower absolute translation error (ATE) on KITTI-360 data when global SLAM is activated, with the average ATE/length achieving 0.05%, which is
quite impressive considering that only monocular V-I data is used.
-
The system preserves the overall 3D structure while leveraging binocular geometry, attaining
relatively stable and complete realtime 3D perception performance
by maintaining metric scale awareness, unlike methods that rely solely on inference whereobjects with ambiguous depth cues can exhibit significant depth/scale errors.
Sources
- Grounding Image Matching in 3D with MASt3R
- VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold
- 3D Reconstruction with Spatial Memory
- MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion
- $\pi^3$: Permutation-Equivariant Visual Geometry Learning
- MoD-SLAM: Monocular Dense Mapping for Unbounded 3D Scene Reconstruction
- Retrieved Atmospheres and Inferred Surface Properties for Exoplanets Using Transmission and Reflected Light Spectroscopy
- SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos
- A real-time, robust and versatile visual-SLAM framework based on deep learning networks
- SuperVINS: A Real-Time Visual-Inertial SLAM Framework for Challenging Imaging Conditions
- GVINS: Tightly Coupled GNSS-Visual-Inertial Fusion for Smooth and Consistent State Estimation
- VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation