Track-Free Dense Structure from Motion
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Track-Free Dense Structure from Motion".
Jane: Here's the gist of the paper. Classical Bundle Adjustment (BA) is fundamentally limited by its reliance on precise metric initialization and prior camera intrinsics By replacing fragile point tracks with a flexible kinematic pose graph and representing landmarks as 3D Gaussians, our framework explicitly models spatial uncertainty through a unified Negative Log-Likelihood (NLL) objective This volumetric formulation smooths the non-convex optimization landscape and naturally weights correspondences by their statistical confidence To maintain global consistency, we optimize over a sparse view graph using an iterative, adaptive edgeweighting mechanism to prune erroneous topological links Furthermore, we resolve mirror ambiguities inherent to prior-free SfM via a dualhypothesis regularization strategy Extensive evaluations show that our approach significantly expands the basin of attraction and achieves superior accuracy over both classical and learning-based baselines.
Tom: Framework Overview ProBA operates over a view graph G = V, E and its Minimum Spanning Tree T Bypassing exhaustive O(N 2) matching, we establish this topology using LoFTR
34: and extract dense correspondence fields via DKM
9: Over this structure, we stabilize prior-free optimization via a kinematic tree parameterization (Section 3.1).
Jane: We then formulate our unified probabilistic objective using continuous 3D Gaussians (Section 3.2), dynamically regulated by an adaptive edge weighting scheme (Section 3.3) and a novel mirror symmetry regularization that definitively resolves depth-inversion ambiguities (Section 3.4).
Tom: Pruning redundant reasoning tokens The framework employs several mechanisms to ensure stability and robustness during optimization.
Title and authors: Tom: Alright, let’s talk about who wrote this. We’ve got Jason Chui, Hector Andrade-Loarca, and Daniel Cremers from Technical University of Munich leading the charge here with ProBA (Probabilistic Bundle Adjustment).
Jane: It’s interesting that they are combining these approaches—using LoFTR to build a topological prior graph and DKM for dense correspondences—and then feeding all that into this new probabilistic optimization layer.
Lu: What I find compelling is how they frame the entire process as a re-parameterization of the bundle adjustment manifold, which allows them to jointly optimize extrinsics, focal lengths, and geometry from a strict cold start without needing those precise metric initializations.
Meng: So they’re saying that instead of starting with perfect numbers for camera positions and focal lengths, they can actually solve for them alongside the three dee structure itself <ref:2505.20858#pg1>. That sounds like a lot of computational complexity to manage at scale.
Lalam: It seems like the paper is trying to solve that initial setup problem by making the entire optimization process probabilistic from the very beginning, which should make it more resilient to poor initialization choices.
The paper's summary: Tom: So, what’s the actual summary of "Track-Free Dense Structure from Motion"? They show a three-stage pipeline: first building a view graph using LoFTR to get that topological prior, then finding dense correspondences with DKM, and finally running the ProBA optimization.
Jane: The key part they highlight is moving from rigid tracks to treating corresponding Gaussians as probabilistic entities; instead of enforcing exact three dee coordinate sharing immediately, they treat them as probabilities <ref:2505.20858#pg1>.
Lu: That’s where the unified negative log-likelihood objective comes in, which smooths the non-convex optimization landscape and naturally weights those correspondences by their statistical confidence. That’s a big shift from how we usually handle constraints.
Meng: When I think about deployment, that probabilistic weighting is smart because it means if one part of the scene has a low-confidence match, it won't derail the whole reconstruction as badly as a hard constraint would. But how do they manage the cost of calculating that unified objective function?
Lalam: They handle that by using an iterative, adaptive edge weighting scheme to prune those topological links that aren't adding much confidence to the global structure. It keeps the search space manageable.
The paper's improvements: Tom: Now for the improvements they list—they focus on how this framework handles things we always struggle with, specifically mirror ambiguities and scale drift during optimization.
Jane: They introduce a novel mirror symmetry regularization strategy that’s designed to definitively resolve those depth-inversion ambiguities that pop up in unconstrained SfM when you start from scratch.
Lu: Plus, they use an adaptive edge weighting mechanism where the instantaneous reliability of each edge is determined by how its negative log-likelihood compares to its local neighborhood, and the final symmetric reliability score combines confidence from both directions using a sigmoid function.
Meng: I’m curious about that mirror symmetry part. If it can resolve depth inversion without metric priors, that’s a huge win for applications where you don't have those initial calibration steps done perfectly.
Lalam: And they also use the three dee consistency loss to optimize the probabilistic overlap of those Gaussians by either minimizing their distance or adaptively expanding their spatial variances, which acts like a probabilistic shock absorber <ref:2505.20858#pg1>.
Conclusion: Tom: So, wrapping up on this paper "Track-Free Dense Structure from Motion," they’re showing that by replacing rigid point tracks with flexible kinematic pose graphs of three dee Gaussians and using that unified NLL objective, they achieve superior metric accuracy and structural density across diverse scenarios compared to COLMAP variants <ref:2505.20858#pg1>.
Jane: It seems the main implication for us is that we can tackle unconstrained or unstructured real-world scenes much more effectively because the method explicitly models spatial uncertainty rather than assuming perfect initial conditions.
Lu: The use of isotropic three dee Gaussians is particularly important here; it prevents rapid overfitting to incorrect early poses by providing smoother gradients during those initial pose updates <ref:2505.20858#pg1>.
Meng: For production, if this scales well and maintains that accuracy, it could mean we can deploy these reconstruction tools in environments where the scene geometry isn't perfectly mapped beforehand.
Lalam: And I think the dual-hypothesis mechanism to resolve mirror ambiguities from a strict cold start is a very concrete thing that makes this approach viable for practical use without needing heavy prior knowledge about reflection symmetries.
Tom: So, "Track-Free Dense Structure from Motion" gives us a mathematically grounded foundation for dense three dee reconstruction by making the optimization landscape much smoother and more robust to noise <ref:2505.20858#pg1>. We’ll be looking at how these volumetric representations change the way we approach pose estimation next.
Technical University of Munich · Munich Center for Machine Learning (MCML)
cs.CV
Submitted: 2025-05-27
Updated: 2026-11-09
Comments: 17 pages, 14 figures, 7 tables
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 96/100
The gist: Classical Bundle Adjustment (BA) is fundamentally limited by its reliance on precise metric initialization and prior camera intrinsics By replacing fragile point tracks with a flexible kinematic pose…
Key concepts
- Continuous 3D Gaussians
- Instead of treating landmarks as single points, this method models them as continuous 3D Gaussian distributions. This allows the system to represent uncertainty around a landmark's true position. When correspondences are incorrect, the Gaussian expands its spatial variance instead of causing catastrophic errors in optimization.
- Unified Negative Log-Likelihood (NLL) Objective
- The framework uses a single objective function that combines 2D reprojection loss and 3D consistency loss. This unified approach ensures that the optimization process considers both how well the points project onto images and how consistent their inferred 3D positions are with each other, leading to a smoother convergence.
- Adaptive Edge Weighting Mechanism
- This mechanism dynamically assesses the reliability of connections (edges) in the view graph. It compares an edge's local NLL against its neighbors, assigning a confidence score. This allows the system to prune erroneous topological links during optimization, ensuring global consistency by down-weighting unreliable data.
- Mirror Symmetry Regularization
- This technique introduces a dual-hypothesis approach to resolve inherent ambiguities in prior-free structure from motion. By optimizing two independent pose hypotheses simultaneously, it effectively resolves depth-inversion errors that plague standard methods when starting from scratch.
Terminology
Summary
Classical Bundle Adjustment (BA) is fundamentally limited by its reliance on precise metric initialization and prior camera intrinsics By replacing fragile point tracks with a flexible kinematic pose graph and representing landmarks as 3D Gaussians, our framework explicitly models spatial uncertainty through a unified Negative Log-Likelihood (NLL) objective This volumetric formulation smooths the non-convex optimization landscape and naturally weights correspondences by their statistical confidence To maintain global consistency, we optimize over a sparse view graph using an iterative, adaptive edgeweighting mechanism to prune erroneous topological links Furthermore, we resolve mirror ambiguities inherent to prior-free SfM via a dualhypothesis regularization strategy Extensive evaluations show that our approach significantly expands the basin of attraction and achieves superior accuracy over both classical and learning-based baselines. p. 0 By replacing fragile point tracks with a flexible kinematic
pose graph and representing landmarks as 3D Gaussians, our framework explicitly models spatial uncertainty through a unified Negative
Log-Likelihood (NLL) objective. This volumetric formulation smooths
the non-convex optimization landscape and naturally weights correspondences by their statistical confidence.Where the paper says it
Framework Overview
ProBA operates over a view graph G = V, E and its Minimum Spanning Tree T Bypassing exhaustive O(N 2) matching, we establish this topology using LoFTR [34] and extract dense correspondence fields via DKM [9] Over this structure, we stabilize prior-free optimization via a kinematic tree parameterization (Section 3.1). We then formulate our unified probabilistic objective using continuous 3D Gaussians (Section 3. p. 5 Cremers
structure, we stabilize prior-free optimization via a kinematic tree parameterization (Section 3. We then formulate our unified probabilistic objective using
continuous 3D Gaussians (Section 3. We then formulate our unified probabilistic objective using
continuous 3D Gaussians (Section 3. 2), dynamically regulated by an adaptive
edge weighting scheme (Section 3. 2), dynamically regulated by an adaptive
edge weighting scheme (Section 3. 3) and a novel mirror symmetry regularization
that definitively resolves depth-inversion ambiguities (Section 3.Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The framework employs several mechanisms to ensure stability and robustness during optimization. Specifically, the adaptive edge weighting mechanism determines the instantaneous reliability of each edge based on how its NLL compares to its local neighborhood. p. 7 We determine the instantaneous reliability
of each edge eij based on how its NLL (Lij = L
i→j
2D + L
j→i
2D + L
ij
3D) compares to
its local neighborhood N (i). Specifically, we compute the per-node average NLL
µi =
1
|N(i)|
P
k∈N(i) Lik and its standard deviation σi =
1
|N(i)|
P
k∈N(i)
(Lik −
µi)
2
1/2
. Since our view graph edges are undirected, we compute the final symmetric reliability score by combining the localized confidence from both
directions:
w_{ij} = 0. 5 * \operatorname {sigmoid}(-\bar {\mathcal {L}}_{ij}) + 0. While these methods improve
convergence, they often suffer from severe non-linearities when attempting to
enforce multi-view point sharing. ProBA bypasses these constraints, utilizing
a unified probabilistic objective to actively resolve depth-inversion (mirror) ambiguities—via a continuous dual-hypothesis regularization—without relying on
prior metric pose or intrinsic estimates.Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The unified objective function L total is defined as the sum of structural NLL loss evaluated on both primary and secondary mirror hypotheses plus a regularization term. The structural NLL loss incorporates two components: the 2D reprojection loss L 2D i to j and the 3D consistency loss L 3D ij. p. 8 Final Unified Objective. Let L
(1)
world and L
(2)
world denote the structural NLL
loss (Equation. Instead of prematurely enforcing exact 3D coordinate sharing, corresponding Gaussians are
treated as probabilistic clusters that converge spatially through a dynamic optimization: they iteratively shift their 3D centroids to minimize distance or adaptively expand their spatial variances to maintain overlap in high-uncertainty
regions. This volumetric uncertainty smooths the highly non-convex optimization landscape, enabling a strictly initialization-free regime.Where the paper says it
Where the paper says it
Where the gains come from
The key to robust convergence is modeling scene landmarks as continuous 3D Gaussians instead of discrete points. Utilizing isotropic 3D Gaussians prevents rapid overfitting to incorrect poses by providing smoother gradients during early pose updates. p. 6 Crucially,
we strictly employ isotropic rather than anisotropic Gaussians to prevent rapid
overfitting to the erroneous initial poses, intrinsics, and geometry. During early
optimization, this rotation invariance effectively flattens the loss basin, yielding
a broader convergence radius. Gray insets report Relative Translation Accuracy (RTA@5◦
) and runtime (s) against
standard and dense COLMAP baselines. ProBA achieves superior metric accuracy
and structural density. ProBA distinguishes itself by maintaining a strictly geometric, probabilistic optimization framework. By operating on image-pair correspondences without rigid track-building, our
method achieves a level of cold-start flexibility similar to regression-based models, while retaining the mathematical transparency and metric precision of classical geometric solvers.Where the paper says it
Where the paper says it
Where the paper says it
Limitations
One limitation is the focal-depth ambiguity in uncalibrated multi-view reconstruction which can cause scaling drift and degrade Relative Translation Accuracy (RTA) on planar scenes. This occurs because under the pinhole model, scaling focal length f and depth Z by a factor α leaves the 2D projection identical. p. 1 One prominent direction
involves the use of pseudo-object space errors (pOSE) [17], which lift 2D observations into a 3D proxy space and apply quadratic regularization to avoid
the singularities of the pinhole projection model. Similarly, expOSE [18] utilizes
an exponential depth parameterization to prevent geometric degeneracies, while
other approaches rely on rotation averaging and translation synchronization to
initialize a global coordinate frame [6,44].Where the paper says it
long tail
convergence characteristic of first-order optimization methods during final refinement. The method requires 30,000 iterations to guarantee full convergence across all experiments.
Quantitative Results
Performance on ETH3D shows ProBA achieves an average Relative Translation Accuracy (RTA@5°) of 91.4%, outperforming standard COLMAP and recent foundation models like VGGT. p. 4 Furthermore, "all-in-one" foundation models like DUST3r [40] and MAST3r [24]
directly regress 3D point maps from image pairs, while subsequent extensions
like MASt3R-SfM [8] adapt these pairwise predictions for global Structure-fromMotion. Similarly, differentiable pipelines like VGGSfM [39] and VGGT [38]
extract global camera geometry without relying on classical multi-view tracks. – ETH3D [31]: 13 high-resolution indoor/outdoor scenes within complex structures, featuring ground-truth poses alongside reference LiDAR scans for evaluating map accuracy and completeness. – IMC2021 [20]: 1,575 in-the-wild, crowdsourced image sub-sampled from 9
scenes to test robustness against severe occlusions and diverse, uncalibrated
intrinsics. Furthermore,
we resolve mirror ambiguities inherent to prior-free SfM via a dualhypothesis regularization strategy. Extensive evaluations show that our
approach significantly expands the basin of attraction and achieves superior accuracy over both classical and learning-based baselines, providing
a scalable foundation that greatly benefits SfM and SLAM robustness in
unstructured environments. Beyond its foundational role
in Structure-from-Motion (SfM) [30] and Simultaneous Localization and Mapping (SLAM) [23,27], BA has recently become a necessary step for high-fidelity
neural rendering techniques, such as Neural Radiance Fields [26] and 3D Gaussian Splatting [22]. Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36].Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Implementation Details
To ensure computational tractability, we downsample the dense pairwise matches to a 64×64 spatial grid and optimize the 3D Gaussian depth and variance fields as 32 × 32 parametric maps. Our custom CUDA implementation runs on a single NVIDIA L40S GPU using AdamW for 30,000 iterations. p. 3 Global
consistency is maintained via an adaptive edge weighting mechanism that
dynamically filters erroneous constraints. – Efficient CUDA Implementation: We developed a custom, highly optimized solver that will be released to the community. 2: System Overview. Our framework processes unordered images in three
stages: (1) View Graph Construction via LoFTR [34] to build a topological prior
(solid: minimum spanning tree edges, dash: auxiliary edges); (2) Finding Correspondences via DKM [9] to extract dense pairwise constraints; and (3) ProBA Optimization, which iteratively refines the edge weights and the global scene from a strict cold
start.Where the paper says it
Where the paper says it
Conclusion
ProBA provides a scalable, mathematically grounded foundation for robust 3D reconstruction and SLAM in unstructured environments. By replacing fragile multi-view heuristics with a flexible kinematic view graph of volumetric 3D Gaussians, ProBA smooths the narrow basins of attraction inherent to classical solvers. p. 0 By replacing fragile point tracks with a flexible kinematic
pose graph and representing landmarks as 3D Gaussians, our framework explicitly models spatial uncertainty through a unified Negative
Log-Likelihood (NLL) objective. This volumetric formulation smooths
the non-convex optimization landscape and naturally weights correspondences by their statistical confidence. – Unified Initialization-Free Objective: We derive a Negative Log-Likelihood (NLL) objective integrating 2D reprojection and 3D consistency. A
novel dual-hypothesis regularization robustly resolves mirror ambiguities
from a strict cold start. ProBA distinguishes itself by maintaining a strictly geometric, probabilistic optimization framework. By operating on image-pair correspondences without rigid track-building, our
method achieves a level of cold-start flexibility similar to regression-based models, while retaining the mathematical transparency and metric precision of classical geometric solvers. Furthermore,
we resolve mirror ambiguities inherent to prior-free SfM via a dualhypothesis regularization strategy. Extensive evaluations show that our
approach significantly expands the basin of attraction and achieves superior accuracy over both classical and learning-based baselines, providing
a scalable foundation that greatly benefits SfM and SLAM robustness in
unstructured environments.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
REFERENCE RULES
Baren J T Mildenhall B Verbin D Srinivasan P Hedman P Mip-NeRF 360 Unbounded anti-aliased neural radiance fields In CVPR 2024 <ref:2505.20858#pg2>
Edstedt J Athanasiadis I Wadenbäck M Felsberg M Dkm Dense kernelized feature matching for geometry estimation In CVPR 2023 <ref:2505.20858#pg2>
Harris M Sengupta S Owens JD Parallel prefix sum (scan) with cuda In GPU gems 3, vol. 3, pp. 851–876 Addison-Wesley Professional (2007) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Hartley R Zisserman A Multiple View Geometry in Computer Vision Cambridge University Press (2003) <ref:2505.20858#pg2>
Schönberger JL Frahm JM Structure-from-motion revisited In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 4104–4113 (2016) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Fu Y Liu S Kulkarni A Kautz J Efros A A Wang X COLMAP-Free 3d gaussian splatting In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024 <ref:2505.20858#pg2>
Wang J Karaev N Rupprecht C Novotny D Vggt Visual geometry grounded transformer In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2025) <ref:2505.20858#pg2>
LoFTR Detector-free local feature matching with transformers In CVPR 2021 <ref:2505.20858#pg2>
Edstedt J Sun Q Bökman G Wadenbäck M Felsberg M RoMa Robust dense feature matching In CVPR 2024 <ref:2505.20858#pg2>
Triggs B McLauchlan PF Hartley RI Fitzgibbon AW Bundle adjustment — a modern synthesis In International Workshop on Vision Algorithms pp. 298–372 Springer (1999) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505. Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505. Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Where the paper says it
Where the paper says it
Jensen R Dahl A Vogiatzis G Tola E Aanaes H Large scale multi-view stereopsis evaluation In CVPR (2014) <ref:2505.20858#pg2>
Snavely N Seitz SM Szeliski R Bundle adjustment in the large In European Conference on Computer Vision (ECCV). pp. 29–42 Springer (2010) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Chatterjee A Govindu VM Efficient and robust large-scale rotation averaging In ICCV. pp. 521–528 (2013) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Davison AJ Reid ID Molton ND Stasse O MonoSLAM: Real-time single camera SLAM IEEE TPAMI 29(6), 1052–1067 (2007) <ref:2505.20858#pg2>
Duisterhof B Žust L Weinzaepfel P Leroy V Cabon Y Revaud J MASt3R-SfM A fully-integrated solution for unconstrained structure-frommotion In International Conference on 3D Vision (3DV) 2025 <ref:2505.20858#pg2>
Wilson K Wehrwein S Visualizing spectral bundle adjustment uncertainty In Int. Conf. 3D Vis. pp. 663–671 (2020) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Wang J Chen M Karaev N Vedaldi A Rupprecht C Novotny D Vggsfm Visual geometry group structure-from-motion In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024) <ref:2505.20858#pg2>
Weber S Hong JH Cremers D Power variable projection for initialization-free large-scale bundle adjustment In ECCV (2024) <ref:2505.20858#pg2>
Jensen R Dahl A Vogiatzis G Tola E Aanaes H Large scale multi-view stereopsis evaluation In CVPR (2014) <ref:2505.20858#pg2>
Kaiser M Johannsson H Roberts R Ila V Leonard J Dellaert F iSAM2 Incremental smoothing and mapping with fluid relinearization and incremental variable reordering In IEEE Int. Conf. Robot. Autom. pp. 3281–3288 (2011) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505. Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Where the paper says it
Sünderhauf N Protzel P Switchable constraints for robust pose graph slam In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp. 1879–1884 IEEE (2012) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505. Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Where the paper says it
Triggs B McLauchlan PF Hartley RI Fitzgibbon AW Bundle adjustment — a modern synthesis In International Workshop on Vision Algorithms pp. 298–372 Springer (1999) <ref:2505.20858#pg2>
He X Sun J Wang Y Peng S Huang Q Bao H Zhou X Detector-Free Structure from Motion In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 21594–21603 (2024) <ref:2505. p. 0 Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505. Despite its success, traditional BA depends on several strong
assumptions: the availability of accurate initial pose estimates [1, 25], known
camera intrinsics, and the treatment of 2D observations as noise-free measurements [36]. Standard solvers are highly prone to getting stuck in local minima,
arXiv:2505.Where the paper says it
Where the paper says it
Wang J Karaev N Rupprecht C Novotny D Vggt Visual geometry grounded transformer In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2025) <ref:2505.20858#pg2>
Wang S Leroy V Cabon Y Chidlovskii B Revaud J DUSt3R Geometric 3D vision made easy In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024) <ref:2505.20858#pg2>
Lourakis MI Argyros AA Sba A software package for generic sparse bundle adjustment ACM Transactions on Mathematical Software (TOMS) 36(1), 1–30 (2009) <ref:2505.20858#pg2>
Kaess M Johannsson H Roberts R Ila V Leonard J Dellaert F iSAM2 Incremental smoothing and mapping with fluid relinearization and incremental variable reordering In IEEE Int. Conf. Robot. Autom. pp. 3281–3288 (2011) <ref:2505.20858#pg2>
Sünderhauf N Protzel P Switchable constraints for robust pose graph slam In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp. 1879–1884 IEEE (2012) <ref:2505.20858#pg2>
Triggs B McLauchlan PF Hartley RI Fitzgibbon AW Bundle adjustment — a modern synthesis In International Workshop on Vision Algorithms pp. 298–372 Springer (1999) <ref:2505.20858#pg2>
He X Sun J Wang Y Peng S Huang Q Bao H Zhou X Detector-Free Structure from Motion In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 21594–21603 (2024) <ref:2505.20858#pg2>
Wang J Karaev N Rupprecht C Novotny D Vggt Visual geometry grounded transformer In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2025) <ref:2505.20858#pg2>
Wang S Leroy V Cabon Y Chidlovskii B Revaud J DUSt3R Geometric 3D vision made easy In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024) <ref:2505.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Probabilistic Landmark Representation: The system can model scene landmarks as continuous 3D Gaussians instead of fragile point tracks, allowing it to
accommodate incorrect correspondences by expanding their spatial variance,
which smooths the optimization landscape. -
Unified Initialization-Free Objective: The framework can resolve mirror ambiguities from a strict cold start via a
novel dual-hypothesis regularization robustly resolves mirror ambiguities from a strict cold start,
ensuring correct geometry is found without metric priors. -
Scalable View-Graph Optimization: The system can maintain global consistency by optimizing over a sparse kinematic pose graph using an
adaptive edge weighting mechanism that dynamically down-weights erroneous topological links.
-
Robust Outlier Handling: The system demonstrates strong tolerance to corrupted correspondences, showing that
high-variance Gaussians act as probabilistic shock absorbers
by expanding their spatial variance, which prevents catastrophic shattering of the optimization process. -
Enhanced Geometric Fidelity: By using isotropic 3D Gaussians, the system avoids overfitting to inaccurate early poses and achieves superior reconstruction accuracy compared to anisotropic formulations where
anisotropic Gaussians rapidly overfit to these erroneous viewpoints.
Abstract
Classical Bundle Adjustment (BA) relies on precise metric initialization and known camera intrinsics, and traditional Structure-from-Motion (SfM) pipelines struggle to exploit modern dense matchers because rigid track-building heuristics break down under their inherent noise. We present Track-Free Dense Structure from Motion (TFD-SfM), which jointly optimizes camera extrinsics, focal lengths, and geometry from a strict cold start. Our central idea is to relax the rigidities of classical SfM into soft, probabilistic decisions under Negative Log-Likelihood (NLL) objectives. Instead of forcing matched pixels onto a single shared 3D point, we model each match as a 3D Gaussian whose variance is optimized jointly with the geometry. Unreliable matches take on large variance and contribute little to the loss. This smooths the optimization landscape, so the solver converges even from identity cameras and uniform depths. A kinematic relative-pose parameterization stabilizes the cold start, and a dual-hypothesis mechanism resolves the mirror-symmetry ambiguity on object-centric scenes. Across DTU, Mip-NeRF 360, ETH3D, and IMC2021, TFD-SfM is competitive with both classical and learning-based SfM baselines and leads on ETH3D, recovering accurate camera poses, intrinsics, and dense geometry from unordered, uncalibrated image collections. Project page: https://cphjsn.github.io/tfd-sfm/
Sources
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs