DBA-Fusion: Tightly Integrating Deep Dense Visual Bundle Adjustment with Multiple Sensors for Large-Scale Localization and Mapping

arXiv:2403.13714 · cs.RO, cs.CV · Submitted 2024-03-20 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "DBA-Fusion: Tightly Integrating Deep Dense Visual Bundle Adjustment with Multiple Sensors for Large-Scale Localization and Mapping".

Dev: Here's the gist of the paper. Visual simultaneous localization and mapping (VSLAM) has broad applications, with state-of-the-art methods leveraging deep neural networks for better robustness and applicability.

Rosa: In this paper, we tightly integrate the trainable deep dense bundle adjustment (DBA) with multi-sensor information through a factor graph.

Dev: The results validate the superior localization performance of our approach, which enables real-time dense mapping in large-scale environments.

Rosa: System Architecture The framework proposes a generic factor graph to tightly integrate DROID-SLAM-like deep DBA with multi-sensor information The system is designed to enhance the practicality of deep VSLAM through multi-sensor fusion and make it suitable for real-time large-scale applications.

Title and authors: Rosa: So we're looking at this paper, "DBA-Fusion: Tightly Integrating Deep Dense Visual Bundle Adjustment with Multiple Sensors for Large-Scale Localization and Mapping." It sounds like they are taking the deep dense bundle adjustment, which is a really powerful way to refine the map from visual data, and tying it together with other sensor information through a factor graph.

Dev: Right. The title itself tells you the big idea: they're integrating that deep dense bundle adjustment with multiple sensors for large-scale localization and mapping. It suggests they're trying to solve the problem of making these learning-based methods more robust when you need to map huge environments or use different types of input data than just cameras.

Taro: I wonder what that means in practice for someone who has to drive or walk around outside a lab setting, Rosa? Does this system handle things when the world gets messy?

Rosa: That's the first question I have. This paper suggests they developed a pipeline for visual-inertial integration first, which is their way of getting that minimum level of metric-scale localization before adding all those extra sensor factors. It’s about making sure you have a solid starting point before you complicate things further with fusion.

Dev: Exactly. The summary explains the core design: they use a recurrent optical flow module and then feed the resulting bundle adjustment information into a generic factor graph that uses a sliding window mechanism and supports probabilistic marginalization for fusing those different sensors. It’s about creating a flexible framework for visual-inertial integration, which is usually good for having scale awareness and continuity when you only have visual data.

Taro: So the recurrent optical flow module is based on DROID-SLAM, which they say is derived from RAFT. Does that mean the system relies heavily on those specific deep learning components for tracking?

Rosa: Yes, it does. The paper mentions that the structure of this recurrent optical flow adheres to how DROID-SLAM is implemented, which itself comes from the RAFT work. They use a correlation volume based on encoded feature maps and then inject that into a convolutional GRU with a hidden state to get the corrected optical flow residual in its residual form.

Title and authors: Dev: That sounds like they are using learned methods for tracking motion between images, which is where the deep learning comes in. But what's the actual gain here compared to just running DROID-SLAM alone?

Taro: The paper points out that they model the visual factor using a Hessian factor where they temporarily eliminate the point depths. They build a combined Hessian by positionally stacking or summing up blocks derived from that formulation, which is then used in the factor graph optimization.

Rosa: And Dev, this is a key part of what I find interesting—they mentioned that because they eliminated the dense depth states, they can perform the factor graph optimization efficiently on the CPU. It’s like solving a much bigger problem in two steps instead of one massive calculation.

Dev: That efficiency gain is important for real-time applications, especially when you're dealing with large-scale mapping where computational resources are limited. But how do they handle the different sensors beyond just vision?

Taro: The multi-sensor integration part is handled by a generic pose-centered factor graph. They include several specific sensor factors there: the visual constraint factor, an IMU preintegration factor, a GNSS position factor, and a wheel speed factor.

Rosa: That GNSS integration is important because it enables driftless and geo-referencing functionality. The paper claims that this DBA-Fusion scheme achieves stable decimeter-level position estimation when you use GNSS RTK integration alongside the wheel speed integration.

Dev: Stable decimeter level position with RTK and wheel speed—that's a strong claim, Rosa. It suggests a very tight coupling between the visual tracking and the inertial measurements, which is what we need for reliable real-time control loops where latency matters.

Taro: If you look at the experimental validation, they show good performance across most sequences but highlight that it's more significant on outdoor sequences. They also mention that DBA-VIO achieves good local map consistency without needing any loop closure techniques.

Title and authors: Rosa: That’s encouraging because local consistency is a big deal for mapping in complex areas like cities or rough terrain, and they achieved this without needing those extra loop closure methods they usually rely on to fix drift over long distances.

Dev: I also see them showing dramatically better translation and attitude estimation compared to the visual-only DROID-SLAM system. That directly addresses the core issue of visual odometry needing external help for accurate pose estimation.

Taro: So, for someone just listening to this, what does it change? It means you can expect more stable, georeferenced positioning in large areas, even when you combine visual tracking with things like GNSS and wheel speed.

Rosa: Right. The whole point of this DBA-Fusion paper is proposing a framework that tightly integrates deep dense visual bundle adjustment with multiple sensors in a probabilistic way to enable real-time localization and dense mapping. It shows how flexible this approach can be for large-scale, georeferenced applications using GNSS and wheel speed.

Dev: It validates the superior localization performance of their visual-inertial odometry implementation based on this framework, showing its flexibility to fuse GNSS and wheel speed for big scale applications. What I'm still thinking about is that they mention their limitation regarding the dynamic scenarios; they haven't fully detailed how well it handles objects moving unpredictably compared to what we see in other papers.

Taro: That’s a fair point, Dev. They do mention that future work will focus on extending this system to dynamic scenarios and neural map representations, which tells you where the current limits of this specific implementation lie right now.

Rosa: So, to wrap up on DBA-Fusion: it's a framework that tightly integrates deep dense visual bundle adjustment with multiple sensors in a probabilistic manner for real-time localization and mapping. The results validate its superior performance when combining GNSS and wheel speed for large-scale applications. We'll be keeping an eye on how they extend it to handle moving objects next.

The paper's summary: Rosa: So, we're looking at how they took that deep dense bundle adjustment—the thing that refines the map from your camera feeds—and they tightly linked it up with several different sensors using a factor graph for real-time mapping in big areas.

Dev: Right, the summary says the whole goal is to make deep visual simultaneous localization and mapping actually practical for huge environments by fusing multi-sensor information. It’s about taking that heavy learning component and making it work on a moving system.

Taro: What I find interesting is how they use this factor graph structure with a sliding window mechanism and probabilistic marginalization to fuse everything together. That sounds like they're building a very flexible way to combine the different data streams without losing track of what’s going on.

Rosa: And you're right, that flexibility is key because it lets them plug in different sensors—like GPS, wheel speed, or even just an IMU—into one unified system. It’s not just about vision anymore; it's about using everything to get a very stable picture of where you are.

Dev: Exactly. The paper shows that by integrating factors like the visual constraint, the IMU preintegration, and the GNSS position factor, they can achieve decimeter level accuracy when you use RTK integration along with wheel speed. That's a solid metric for real-time control loops.

Taro: It sounds promising because they show good performance on outdoor sequences where things get messy. But I want to ask what happens when the world misbehaves, like suddenly an object moves unpredictably in front of the camera? How does this system handle that kind of unexpected change?

Rosa: That’s a fair question, Taro. The paper mentions they have this recurrent optical flow module based on DROID-SLAM architecture, which uses deep learning to compute dense pixel association. They also mention an iterative update structure for the bundle adjustment formulation itself based on the edges in the graph.

Dev: So, while they haven't given a full breakdown of that dynamic handling yet, it suggests they are trying to refine their visual tracking continuously by feeding that information back into the optimization process. It’s meant to keep things consistent even when there are visual glitches or fast motion.

Taro: It sounds like the recurrent nature of the optical flow is supposed to help with that consistency, but I'm still curious about the latency involved in all these deep neural network computations running inside a factor graph optimization loop. How fast can this actually run on real hardware?

Rosa: The paper does show real-time performance evaluation on an Nvidia RTX three thousand ninety GPU, which suggests they've done their homework on the computational cost. They aim for real-time dense mapping in large-scale environments by doing the heavy lifting efficiently.

Dev: That efficiency is what makes it useful for large areas, but I’m still waiting to see if that real-time performance holds up when you push it to the limits of scale or sensor density. Because they focused on outdoor sequences, we should check out how those results compare against indoor data next.

The paper's improvements: Tom: So, we’ve talked about how they put deep bundle adjustment and multiple sensors into one factor graph for big scale mapping. Now we're looking at what they actually improved in their design and methodology to make it better than just stitching things together.

Rosa: The paper suggests a few major improvements for the way they structure the visual factor itself, which is where they use that Hessian formulation. They model the visual uncertainty with this learned information, so the optimization doesn't just trust the vision blindly.

Dev: That makes sense because having learned uncertainty in your factors means you can weight those sensor inputs more intelligently during the optimization process. It’s like telling your system, "Hey, I'm less sure about this specific visual measurement right now."

Taro: And they also improved how they handle the overall factor graph structure by using positionally stacking and summing up blocks when constructing that combined Hessian. That’s a clever way to ensure the geometric information is rich across all those frames.

Rosa: It means that even though they eliminated the dense depth states to save computation time on the CPU, they still managed to keep a very detailed geometric picture in their optimization setup. That’s a big win for speed and accuracy together.

Dev: The implication here is that this framework is designed for high-throughput scenarios, which is exactly what you need when you want real-time dense mapping across a huge area without the system freezing up. The safety case there revolves around keeping that loop rate stable despite those complex visual factors being fed in.

Taro: It also points toward a more robust system because they explicitly state that this approach achieves good local map consistency even without using loop closure techniques for small areas. That suggests the core tracking mechanism is fundamentally sound.

Rosa: And when you look at the overall implication, it’s making DROID-SLAM much more practical for real-world, outdoor use cases where you need to integrate GPS and wheel speed for geo-referencing. It moves this kind of deep VSLAM out of pure academic testing and into something that could actually be deployed on a robot.

Dev: Exactly. It shows the flexibility to fuse GNSS and wheel speed in a way that gives you stable decimeter level position estimation, which is crucial for any real-time navigation system. But we still need to see how these improvements hold up when things get truly dynamic, like sudden object movements or complex urban environments.

Conclusion: Rosa: So we've got our wrap up for DBA-Fusion, which is this framework that tightly integrates deep dense visual bundle adjustment with multiple sensors for large scale localization and mapping. It really shows how powerful combining those things can be when you need real time performance in big places.

Dev: Right, the main implication is that we're moving toward more robust systems where visual tracking isn't the only thing doing the heavy lifting; it’s about having a layered approach to localization. This means better safety cases for mobile robots operating outside controlled settings.

Taro: I just want to stress that while they show stability with GNSS and wheel speed, we still need more data on how this system recovers when the world gets truly unpredictable or dynamic. It’s good for a known environment, but real autonomy is about handling the unexpected stuff gracefully.

Rosa: True, Taro. The authors themselves flagged that future work will focus specifically on extending this system to handle those dynamic scenarios and also looking at neural map representations instead of just dense point clouds.

Dev: That’s where the engineering challenge lies—making sure that when you move from a stable, known environment to a chaotic one, the control loop doesn't fail due to high latency or unexpected state changes. The safety case has to cover that transition period.

Taro: I agree with Dev there; handling those transitions is where the real autonomy work happens, and I’m looking forward to seeing how they tackle that next.

Rosa: Well, for now, DBA-Fusion gives us a really solid foundation for georeferenced applications where stability and scale matter. It proves that deep learning can be tightly coupled with classical optimization methods effectively.

Dev: It’s definitely a step up from the visual-only systems we were looking at earlier in the day, proving that combining deep vision with inertial measurements yields dramatically better translation and attitude estimation.

Taro: I think it sets a good bar for what multi-sensor integration looks like in VSLAM research moving forward, especially when thinking about how we can build systems that are both fast and resilient.

Rosa: Exactly. We'll keep an eye on those future work plans to see how they tackle the dynamic stuff next, because that’s where the real world is waiting for this kind of integration to be tested.

Wuhan University

cs.RO, cs.CV

Submitted: 2024-03-20

Updated: 2024-03-20

Journal ref: IEEE Robotics and Automation Letters 9(7) (2024) 6138-6145

DOI: 10.1109/LRA.2024.3400156

Code: https://github.com/GREAT-WHU/DBA-Fusion

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 78/100

The gist: Visual simultaneous localization and mapping (VSLAM) has broad applications, with state-of-the-art methods leveraging deep neural networks for better robustness and applicability.

Key concepts

Deep Dense Bundle Adjustment (DBA)
This is a trainable deep method used for refining the 3D structure of the scene by adjusting camera poses and point depths simultaneously. It uses a Hessian factor to model visual constraints, which provides rich geometric information about the environment.
Factor Graph
A mathematical framework used to represent and optimize complex systems like SLAM. Here, it tightly integrates visual data (DBA) with sensor measurements (IMU, GNSS) into a single graph structure for robust estimation.
Recurrent Optical Flow Module
This module computes dense pixel association between image pairs using a recurrent convolutional GRU structure. It follows DROID-SLAM principles to generate corrected optical flow, which is crucial for tracking movement between frames.
Multi-sensor Integration
The system fuses data from various sensors, including visual constraints, IMU preintegration, GNSS position factors, and wheel speed factors. This fusion allows the system to achieve stable positioning even when relying on visual data alone.

Terminology

Summary

Visual simultaneous localization and mapping (VSLAM) has broad applications, with state-of-the-art methods leveraging deep neural networks for better robustness and applicability. p. 0 In this paper, we tightly integrate the trainable deep dense bundle adjustment (DBA) with multi-sensor information through a factor graph. p. 0 The results validate the superior localization performance of our approach, which enables real-time dense mapping in large-scale environments. p. 0

System Architecture

The framework proposes a generic factor graph to tightly integrate DROID-SLAM-like deep DBA with multi-sensor information The system is designed to enhance the practicality of deep VSLAM through multi-sensor fusion and make it suitable for real-time large-scale applications.

The overall design aims to flexibly fuses a trainable VSLAM system with multi-sensor information. p. 1

For the visual frontend, a recurrent optical flow module is utilized to compute dense pixel association for image pairs in a co-visibility graph. p. 1

The DBA information is then tightly fused with multi-sensor through a factor graph, where a sliding window mechanism with probabilistic marginalization is employed. p. 1

How the training signal is built

The recurrent optical flow module adheres to the implementation of DROID-SLAM, which is derived from the well-developed RAFT.

The structure of the recurrent optical flow is depicted in Fig. p. 1 2.

For a co-visible image pair (source frame i projected to target frame j), a correlation volume is formed based on their encoded feature maps. p. 1

The results are then injected into the convolutional GRU with hidden state hij to obtain the corrected flow δuij in the residual form. p. 1

The DBA formulation need to be iteratively updated for edges in the graph, leading to the hierarchical iteration structure shown in Fig. p. 2 3.

Where the gains come from

The visual factor is modeled using a Hessian factor where the point depths are temporarily eliminated.

The combined Hessian is constructed by positionally stacking/summing the blocks in Eq. p. 2 (4) as shown in Fig. p. 2 3.

Considering a bundle of edges anchored on frame i and projected to co-visible frames (numbered 1, 2,. p. 2.., N as example), the combined Hessian is constructed by positionally stacking/summing the blocks in Eq. p. 2 (5).

The factor graph optimization is efficiently performed on CPU, considering that the dense depth states have been eliminated. p. 2

Multi-sensor integration

The generic pose-centered factor graph is used for multi-sensor fusion.

The overall factor graph is depicted in Fig. p. 2 5.

The sensor factors include the Visual constraint factor (quadratic cost function [30]), the IMU preintegration factor, the GNSS position factor, and the Wheel speed factor. p. 3

The GNSS integration enables driftless and geo-referencing functionality. p. 0

The DBA-Fusion (G) scheme achieves stable decimeter-level position estimation with GNSS RTK integration. p. 6

Experimental Validation

The proposed DBA-Fusion shows good performance on most of the sequences, which is more significant on the “outdoor” squences.

DBA-VIO achieves good local map consistency without any loop closure techniques. p. 5

The proposed method shows dramatically better translation and attitude estimation than the visual-only DROID-SLAM. p. 5

The DBA-Fusion (G) scheme achieves stable decimeter-level position estimation. p. 6

The results are consistent with Sect. p. 6 V-A, as the proposed DBAVIO achieves locally consistent mapping performance based on the accurate pose estimation and significantly improves the practicability of DROID-SLAM in outdoor scenarios. p. 6

The real-time performance evaluated on Nvidia RTX 3090 GPU is shown in TABLE III. p. 6

Conclusion

In this paper, we propose a framework that tightly integrates deep dense visual bundle adjustment with multiple sensors in a probabilistic manner, enabling real-time localization and dense mapping.

The experimental results validate the superior localization performance of the VIO implementation based on this framework, and show its flexibility to fuse GNSS, WSS for large-scale, georeferenced applications. p. 6

Future work will focus on extending this system to dynamic scenarios and neural map representations. p. 6

Acknowledgments

The implemented DBA-Fusion is developed by the GREAT Group, School of Geodesy and Geomatics, Wuhan University.

References

[1] C. Cadena et al. p. 7, “Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age,” IEEE Trans. p. 7 Robot., vol. 32, no. 6, pp. 1309-1332, Dec. p. 7 2016.

[2] K. Wang, S. Ma, J. Chen, F. Ren and J. Lu, “Approaches, Challenges, and Applications for Deep Visual Odometry: Toward Complicated and Emerging Areas,” IEEE Trans. p. 7 Cogn. Develop. Syst., vol. 14, no. 1, pp. 35-49.

[3] J. Czarnowski, T. p. 7 Laidlow, R. p. 7 Clark and A. p. 7 J. Davison, “DeepFactors: Real-Time Probabilistic Dense Monocular SLAM,” IEEE Robot. p. 7 Automat. Lett., vol. 5, no. 2, pp. 721-728.

[4] Z. Teed and J. p. 7 Deng, “Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras,” in NeurIPS 2021, pp. p. 7 16558-16569. p. 7

[5] T. Qin, P. Li, and S. Shen, “Vins-mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Trans. p. 7 Robot., vol. 34, no. 4, pp. 1004-1020.

[6] X. Li et al., “Continuous and precise positioning in urban environments by tightly coupled integration of GNSS, INS and vision,” IEEE Robot. p. 7 Automat. Lett., vol. 7, no. 4, pp. 11458-11465. p. 7

[7] K. Wu, C. Guo, G. Georgiou, and S. p. 7 I. Roumeliotis. p. 7 p. 7, “Vins on wheels,” in ICRA 2017, pp. p. 7 5155-5162.

[8] D. Li et al., “DXSLAM: A Robust and Efficient Visual SLAM System with Deep Features,” in IROS 2020, pp. p. 7 4958-4965.

[9] L. Sun et al., “Improving Monocular Visual Odometry Using Learned Depth,” IEEE Trans. p. 7 Robot., vol. 38, no. 5, pp. 3173-3186.

[10] S. Yang and S. p. 7 Scherer, “Cubeslam: Monocular 3-d object slam,” IEEE Trans. p. 7 Robot., vol. 35, no. 4, pp. 925-938.

[11] J. Zhang, M. Henein, R. Mahony, and V. p. 7 Ila, “VDO-SLAM: a visual dynamic object-aware SLAM system,” arXiv:2005. p. 711052.

[12] S. Wang, R. Clark, H. Wen, and N. p. 7 Trigoni., “Deepvo: Towards end-toend visual odometry with deep recurrent convolutional neural networks,” in ICRA 2017, pp. p. 7 2043-2050.

[13] J. Jiao, J. Jiao, Y. Mo, W. Liu and Z. Deng., “MagicVO: An End-to-End Hybrid CNN and Bi-LSTM Method for Monocular Visual Odometry,” IEEE Access, vol. p. 7 7, pp. 94118-94127. p. 7

[14] H. Wang, J. Wang, and L. p. 0 Agapito., “Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM,” in CVPR 2023, pp. p. 7 13293-13302. p. 7

[15] Z. Zhu et al., “Nicer-slam: Neural implicit scene encoding for rgb slam,” arXiv:2302. p. 703594.

[16] A. I. Mourikis and S. p. 7 I. Roumeliotis., “A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation,” in ICRA 2007, pp. p. 7 3565-3572.

[17] S. Leutenegger, S. p. 7 Lynen, M. Bosse, R. Siegwart, P. p. 7 Furgale., “Keyframebased visual-inertial odometry using nonlinear optimization,” International J. p. 7 Robot. Research, vol. p. 7 34, no. 3, pp. 314-334.

[18] M. Li and A. I. Mourikis., “Improving the accuracy of EKF-based visualinertial odometry,” in ICRA 2012, pp. p. 7 828-835.

[19] L. Stumberg, V. p. 7 Usenko, and D. p. 7 Cremers., “Direct sparse visual-inertial odometry using dynamic marginalization,” in ICRA 2018, pp. p. 7 2510-2517.

[20] C. Campos, R. Elvira, J. J. G. Rodr´ıguez, J. p. 7 M. M., Montiel and J D. p. 7, “ORB-SLAM3: An Accurate Open-Source Library for Visual, ´ Visual-Inertial, and Multimap SLAM,” IEEE Trans. p. 7 Robot., vol. 37, no. 6, pp. 1874-1890.

[21] L. Stumberg and D. p. 7 Cremers., “Dm-vio: Delayed marginalization visualinertial odometry,” IEEE Robot. p. 7 and Automat. p. 7 Lett., vol. 7, no. 2, pp. 1408-1415.

[22] W. Zhang, S. Wang, X. Dong, R Guo, and N Haala. p. 7, “BAMF-SLAM: Bundle Adjusted Multi-Fisheye Visual-Inertial SLAM Using Recurrent Field Transforms,” arXiv:2306. p. 701173, 2023. p. 7

[23] X. Peng et al. p. 7, “DVI-SLAM: A Dual Visual Inertial SLAM Network,” arXiv:2309. p. 713814, 2023. p. 7

[24] M. Zhang, X. Zuo, Y. Chen, Y Liu and M Li., “Pose Estimation for Ground Robots: On Manifold Representation, Integration, Reparameterization, and Optimization,” IEEE Trans. p. 7 Robot., vol. 37, no. 4, pp. 1081-1099.

[25] T. Qin et al., “A general optimizationbased framework for global pose estimation with multiple sensors,” arXiv:1901. p. 703642, 2019. p. 7

[26] S. Cao, X. Lu, and S Shen. p. 7, ”GVINS: Tightly coupled GNSS-VisualInertial fusion for smooth and consistent state estimation,” IEEE Trans. p. 7 Robot., vol. 38, no. 4 2022, pp. p. 6 2004-2021.

[27] Z. Teed and J Deng. p. 7, “Raft: Recurrent all-pairs field transforms for optical flow,” in ECCV 2020, pp. p. 7 402-419.

[28] W. Wang et al. p. 0, “Tartanair: A dataset to push the limits of visual slam,” in IROS 2020, pp. p. 7 4909-4916.

[29] F. Dellaert and M. p. 7 Kaess., Factor Graphs for Robot Perception. p. 7, Foundations, 2017. p. 7

[30] F. Dellaert and GTSAM Contributors, GTSAM (Version 4. p. 72a8), Georgia Tech Borg Lab [Online]. p. 7 Available: https://github. p. 7com/borglab/gtsam. p. 7

[31] C. Forster, L. p. 7 Carlone, F. p. 7 Dellaert and D Scaramuzza. p. 7, “On-manifold preintegration for real-time visual–inertial odometry,” IEEE Trans. p. 7 Robot., vol. 33, no. 1, pp. 1-21, Feb. 2017.

[32] D. Schubert et al. p. 7, “The TUM VI benchmark for evaluating visualinertial odometry,” in IROS 2018, pp. p. 7 1680-1687.

[33] Y. Liao, J. Xie, and A. p. 7 Geiger., “KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d,” IEEE Trans. p. 7 Pattern Anal. p. 7 Mach. Intell., vol. 45, no. 3, pp. 3292-3310, 2022. p. 7

[34] M Grupp. p. 7, “evo: Python package for the evaluation of odometry and SLAM,” url: https://github. p. 7com/MichaelGrupp/evo, 2017. p. 7

[35] A. Geiger, P. Lenz and R. p. 7 Urtasun., “Are we ready for autonomous driving? p. 7 The KITTI vision benchmark suite,” in CVPR 2012, pp. p. 7 3354-3361.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Deep dense bundle adjustment can be tightly integrated with multi-sensor information through a factor graph, enabling real-time dense mapping in large-scale environments.

  2. The system provides a pipeline for tight visual-inertial integration which introduces the recurrent deep DBA into a sliding window factor graph optimization framework.

  3. The proposed system supports flexible integration of multiple sensors (e.g., GNSS, WSS), thus can be applied to largescale, georeferenced navigation and mapping applications.

  4. The recurrent optical flow module adheres to the implementation of DROID-SLAM, which is derived from the well-developed RAFT and tightly leverages camera poses and depths for iterative refinement.

  5. The factor graph optimization is performed on CPU, considering that the dense depth states have been eliminated, which is equal to solving the full factor graph optimization in a two-step way.

  6. The DBA Hessian provides rich geometric information (with learned uncertainty) to the factor graph while the optimization results are fed back to refine/re-weight the optical flow recurrently.

  7. The system can generate dense, accurate and georeferenced point clouds by utilizing GNSS RTK integration for stable decimeter-level position estimation when combined with wheel speed integration.

Sources

Related papers