VCS-SLAM: Visibility-Consistent Semantic Fusion for 3D Gaussian SLAM
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "VCS-SLAM: Visibility-Consistent Semantic Fusion for 3D Gaussian SLAM".
Jane: Here's the gist of the paper. Visual SLAM addresses the deterioration of visual SLAM performance in complex real-world applications by proposing a geometry-validated semantic evidence fusion framework for RGB-D 3D Gaussian SLAM Pruning redundant reasoning tokens VCS-SLAM evaluates the geometric reliability of 2D semantic priors through visibility consistency, surface-supported boundary evidence, and ray-level conflict uncertainty instead of treating all observations as uniformly valid supervision The resulting reliability-aware objective suppresses occluded semantic updates, reduces unsupported semantic bleeding, and delays premature label assignment in ambiguous regions How the training signal is built The framework formulates online semantic 3D Gaussian SLAM as a geometry-validated semantic evidence fusion problem where 2D semantic priors are assimilated according to their geometric reliability rather than treated as uniformly valid supervision This reliability model uses three complementary geometric cues: visibility consistency, surfacesupported boundary evidence, and ray-level conflict uncertainty The final semantic loss is formulated as L ours sem = L fid sem + lambda edgeLedge where the structural boundary constraint acts as an additive penalty Where the gains come from The authors introduce three core modules to validate geometric reliability: 1.
Tom: Visibility-Consistent Semantic Updates (VCSU) provides fvis, which evaluates visibility consistency and generates a depth-gated mask Mvis applying a Gaussian weighting function to the depth residual This module acts as a dynamic depth gating mechanism during optimization Mvis(p) is applied to the semantic loss calculation, down-weighting gradients at pixels suffering from depth misalignment 2.
Jane: Surface-Coupled Edge Alignment (SCEA) introduces a structural regularizer Lsuraface which discourages unsupported semantic gradients in geometrically smooth and low-confidence regions, reducing semantic bleeding while preserving reliable co-planar semantic transitions This loss evaluates semantic edge magnitudes weighted by the lack of geometric support 3.
Tom: Conflict-Aware Uncertainty Weighting (CAUW) converts ray-wise depth variance into a local semantic confidence score Cpix, which modulates semantic residuals according to geometric reliability per pixel CAUW prevents ambiguous regions from prematurely committing to incorrect semantic labels by receiving smaller semantic gradients when high depth variance indicates geometric conflicts Performance and Validation Experiments on Replica demonstrate improved semantic consistency, boundary preservation, rendering quality, and reconstruction quality compared to baselines The system achieved a Depth L1 error of 0.321 cm on the Replica dataset In terms of semantic performance, VCS-SLAM attained a mean mIoU of 95.87% on four Replica scenes under the per-frame visible-class protocol Targeted analyses confirm that VCSU provides an absolute 4.62% improvement in depth-inconsistent mIoU over the SGS-SLAM baseline Similarly, SCEA achieved a 0.98 Boundary F1 score, indicating that surface-coupled constraints help preserve crisp labels transitions CAUW showed an Error Detection AUROC of 0.85, suggesting the proposed uncertainty weighting better identifies ambiguous geometric regions and down-weights their semantic gradients during optimization VCS-SLAM maintains a highly competitive overall SLAM speed of 7.27 FPS on ScanNet, ensuring practical applicability alongside top-tier geometric accuracy This performance is validated across tracking, mapping, and rendering metrics The proposed framework successfully validates per-frame semantic supervision before fusing it into the persistent Gaussian map by integrating depth-gated visibility masking, surfacesupported edge alignment, and conflict-aware uncertainty weighting This reliability-aware semantic objective reduces occlusion-induced artifacts, unsupported semantic bleeding, and premature label assignment in ambiguous regions Future work will study robustness to noisy semantic predictions from openvocabulary or pre-trained 2D segmentation models in more complex real-world environments The results show that the fusion of geometry-validated evidence delivers competitive global semantic accuracy, with the targeted contributions per-module isolated in Tab.
Paper summary: Jane: So to wrap up this discussion on VCS-SLAM, the authors proposed this geometry validated semantic evidence fusion framework for RGB D three dee Gaussian SLAM <ref:2606.29494#pg1,geometry validated semantic evidence fusion framework for RGB D 3D Gaussian SLAM>. They focused on replacing uniform weights with a reliability model based on visibility consistency, surface supported boundary evidence, and ray level conflict uncertainty.
Tom: The title itself tells you what they did: they are validating the geometric reliability of 2D semantic priors before fusing them into the persistent Gaussian map <ref:2606.29494#pg1>. This whole framework is about suppressing occluded updates, reducing unsupported semantic bleeding, and delaying premature label assignment in ambiguous regions.
Lu: Ultimately, the paper shows that when you integrate these three modules—VCSU, SCEA, and CAUW—the resulting reliability aware objective leads to improved semantic consistency and reconstruction quality compared to baselines on Replica data.
Meng: For deployment, the key number is that VCS-SLAM maintains a speed of seven point twenty-seven frames per second on ScanNet. That's a practical figure for real time mapping applications.
Jane: So, what does this mean for us in the long run? It means that when building these next generation mapping systems, we should probably stop treating semantic labels as just data points and start treating them as inputs that require geometric scrutiny before they can influence the final three dee world model <ref:2606.29494#pg1>.
Conclusion: Tom: So we’re wrapping up on VCS-SLAM, this paper by Raman Jha and Shuaihang Yuan, titled "Visibility-Consistent Semantic Fusion for three dee Gaussian SLAM." It’s basically taking that old idea of just dumping 2D semantic labels into a three dee map and making it smart about *when* to trust those labels <ref:2606.29494#pg1>.
Jane: Right, so if you’re driving or walking, the naive question is, why does it matter if we treat every single 2D label the same way <ref:2606.29494#pg1>? Because in real-world mapping with Gaussian SLAM, those labels can be wrong—maybe an object is momentarily hidden by a pole.
Lu: Exactly. The core idea here is to stop treating all semantic observations as equally valid supervision. The authors introduce three specific ways to check the reliability of those 2D priors before they get baked into the permanent three dee Gaussian map <ref:2606.29494#pg1>.
Meng: What are these checks? Because I’m building this at scale, I need to know what stops the system from getting corrupted by bad data. Does it use something simple like just checking if a pixel is occluded?
Tom: It's more detailed than that. They use visibility consistency, which checks how consistent the view is; surface-supported boundary evidence; and ray level conflict uncertainty, which looks at how much the geometry itself disagrees with the semantic label.
Jane: That means when there’s a big gap in data or a confusing shadow, the system down-weights those semantic updates. It delays premature labeling in those ambiguous spots until it gets more geometric support.
Lu: Precisely. Their final loss function is built to penalize unsupported gradients and push the system toward structurally sound semantic boundaries rather than just blindly following every 2D pixel label they see <ref:2606.29494#pg1>. It’s about making the geometry validate the semantics, not the other way around.
Meng: If it works as advertised, that means we can get a much cleaner three dee reconstruction of indoor spaces without those weird floating labels sticking around everywhere. That reduces noise when we try to use this map for something like robotics navigation.
Tom: It’s a big step because they show that you don't need perfect input data to build a robust semantic map, you just need a smart way to filter the noise from the geometry itself.
Jane: So, while they show strong results on Replica data—getting high mIoU scores and good depth error—the real test is how this holds up when we throw it at messy, unpredictable real-world environments where those conflicts are frequent.
New York University · NYU Abu Dhabi
cs.CV
Submitted: 2026-06-28
Updated: 2026-11-04
Comments: Accepted at RAL 2026
License: http://creativecommons.org/licenses/by-sa/4.0/
Importance score: 100/100
The gist: Visual SLAM addresses the deterioration of visual SLAM performance in complex real-world applications by proposing a geometry-validated semantic evidence fusion framework for RGB-D 3D Gaussian SLAM…
Key concepts
- Visibility-Consistent Semantic Updates (VCSU)
- This module evaluates how visible a pixel is during the optimization process. It generates a depth-gated mask that applies a Gaussian weighting function to the depth residual. This dynamically reduces the influence of semantic updates at pixels that are poorly visible or misaligned in depth, preventing noisy data from corrupting the map.
- Surface-Coupled Edge Alignment (SCEA)
- SCEA introduces a structural regularizer that penalizes semantic gradients in areas where there is no clear geometric support. This loss encourages the model to maintain smooth semantic transitions along surfaces while actively discouraging unsupported semantic changes, which helps reduce 'semantic bleeding' in geometrically flat or low-confidence regions.
- Conflict-Aware Uncertainty Weighting (CAUW)
- CAUW converts the variance in ray-wise depth measurements into a local confidence score for each pixel. This score modulates the semantic residuals during optimization; pixels with high geometric conflict (high depth variance) receive smaller semantic gradients, meaning the system is less likely to commit to an incorrect semantic label when geometry is ambiguous.
Terminology
Summary
Visual SLAM addresses the deterioration of visual SLAM performance in complex real-world applications by proposing a geometry-validated semantic evidence fusion framework for RGB-D 3D Gaussian SLAM
Pruning redundant reasoning tokens
VCS-SLAM evaluates the geometric reliability of 2D semantic priors through visibility consistency, surface-supported boundary evidence, and ray-level conflict uncertainty instead of treating all observations as uniformly valid supervision The resulting reliability-aware objective suppresses occluded semantic updates, reduces unsupported semantic bleeding, and delays premature label assignment in ambiguous regions
How the training signal is built
The framework formulates online semantic 3D Gaussian SLAM as a geometry-validated semantic evidence fusion problem where 2D semantic priors are assimilated according to their geometric reliability rather than treated as uniformly valid supervision This reliability model uses three complementary geometric cues: visibility consistency, surfacesupported boundary evidence, and ray-level conflict uncertainty The final semantic loss is formulated as L ours sem = L fid sem + lambda edgeLedge where the structural boundary constraint acts as an additive penalty
Where the gains come from
The authors introduce three core modules to validate geometric reliability:
-
Visibility-Consistent Semantic Updates (VCSU) provides fvis, which evaluates visibility consistency and generates a depth-gated mask Mvis applying a Gaussian weighting function to the depth residual This module acts as a dynamic depth gating mechanism during optimization Mvis(p) is applied to the semantic loss calculation, down-weighting gradients at pixels suffering from depth misalignment
-
Surface-Coupled Edge Alignment (SCEA) introduces a structural regularizer Lsuraface which discourages unsupported semantic gradients in geometrically smooth and low-confidence regions, reducing semantic bleeding while preserving reliable co-planar semantic transitions This loss evaluates semantic edge magnitudes weighted by the lack of geometric support
-
Conflict-Aware Uncertainty Weighting (CAUW) converts ray-wise depth variance into a local semantic confidence score Cpix, which modulates semantic residuals according to geometric reliability per pixel CAUW prevents ambiguous regions from prematurely committing to incorrect semantic labels by receiving smaller semantic gradients when high depth variance indicates geometric conflicts
Performance and Validation
Experiments on Replica demonstrate improved semantic consistency, boundary preservation, rendering quality, and reconstruction quality compared to baselines The system achieved a Depth L1 error of 0.321 cm on the Replica dataset In terms of semantic performance, VCS-SLAM attained a mean mIoU of 95. p. 0 Results on ScanNet
further show that VCS-SLAM maintains competitive tracking
performance under real RGB-D inputs. Index Terms— Semantic SLAM, Semantic Scene Understanding, Semantic Mapping, 3D Gaussian Splatting
I. The results are reported from [21],
VCS-SLAM results are ours. 4) Semantic Performance: Following the evaluation protocol of recent Gaussian-based semantic SLAM methods [9],
[10], [21], we benchmark semantic segmentation accuracy
under the per-frame visible-class (subset) protocol on four
Replica [25] scenes. Module Targeted Metric SGS-SLAM Ours Improvement
VCSU Depth-Inconsistent mIoU (%) ↑ 86. 62
SCEA Boundary F1 Score (BF) ↑ 0. 98 score (a 0. 16 absolute
increase), indicating that surface-coupled constraints help
preserve crisp labels transitions. 85, corresponding to a 0. 12 absolute improvement,
indicates that the proposed uncertainty weighting better identifies ambiguous geometric regions and down-weights their
semantic gradients during optimization.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion and Limitation
The proposed framework successfully validates per-frame semantic supervision before fusing it into the persistent Gaussian map by integrating depth-gated visibility masking, surfacesupported edge alignment, and conflict-aware uncertainty weighting This reliability-aware semantic objective reduces occlusion-induced artifacts, unsupported semantic bleeding, and premature label assignment in ambiguous regions Future work will study robustness to noisy semantic predictions from openvocabulary or pre-trained 2D segmentation models in more complex real-world environments The results show that the fusion of geometry-validated evidence delivers competitive global semantic accuracy, with the targeted contributions per-module isolated in Tab. VII The paper concludes that VCS-SLAM maintains a highly competitive overall SLAM speed of 7. p. 4 32
VCS-SLAM (Ours) 10. 62
VCS-SLAM maintains a highly competitive overall SLAM
speed of 7.Where the paper says it
References
[1] R. A. Newcombe, S. p. 7 References
[1] R. Newcombe, S. Newcombe, S. Hilliges, D. Hilliges, D. Molyneaux, D. Molyneaux, D. Davison, P. Davison, P. Shotton, S. Hodges, and A. Fitzgibbon,
“Kinectfusion: Real-time dense surface mapping and tracking,” in 2011
10th IEEE International Symposium on Mixed and Augmented Reality,
2011, pp.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[2] R. Jha, A. Lenka, M. Ramanagopal, A Sankaranarayanan, and K Mitra “Rt-x net: Rgb-thermal cross attention network for low-light image enhancement” in 2025 IEEE International Conference on Image Processing (ICIP) 2025 pp. p. 7 Sankaranarayanan, and K. Mitra,
“Rt-x net: Rgb-thermal cross attention network for low-light image
enhancement,” in 2025 IEEE International Conference on Image
Processing (ICIP), 2025, pp.Where the paper says it
[3] B Mildenhall, P P Srinivasan, M Tancik, J T Barron, R Ramamoorthi, and R Ng “Nerf: Representing scenes as neural radiance fields for view synthesis” Communications of the ACM vol. 65 no. 1 pp. 99–106
[4] B Kerbl, G Kopanas, T Leimk¨uhler, G Drettakis et al “3d gaussian splatting for real-time radiance field rendering” ACM Trans Graph vol. 42 no. 4 pp. 139–1
[5] E Sucar, S Liu, J Ortiz, and A J Davison “imap: Implicit mapping and positioning in real-time” in Proceedings of the IEEE/CVF international conference on computer vision 2021 pp. 6229–6238
[6] Z Zhu, S Peng, V Larsson, W Xu, H Bao, Z Cui, M R Oswald, and M Pollefeys “Nice-slam: Neural implicit scalable encoding for slam” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2022 pp. 12 786–12 796
[7] H Matsuki, R Murai, P H Kelly, and A J Davison “Gaussian splatting slam” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2024 pp. 18 039–18 048
[8] N Keetha, J Karhade, K M Jatavallabhula, G Yang, S Scherer, D Ramanan, and J Luiten “Splatam: Splat track & map 3d gaussians for dense rgb-d slam” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2024 pp. 21 357–21 366
[9] M Li, S Liu, H Zhou, G Zhu, N Cheng, T Deng, and H Wang “Sgsslam: Semantic gaussian splatting for neural dense slam” in European Conference on Computer Vision Springer 2024 pp. 163–179
[10] S Zhu, R Qin, G Wang, J Liu, and H Wang “Semgauss-slam: Dense semantic gaussian splatting slam” in 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE 2025 pp. 21 174–21 181
[11] D Yang, Y Gao, X Wang, Y Yue, Y Yang, and M Fu “Opengs-slam: Open-set dense semantic slam with 3d gaussian splatting for objectlevel scene understanding” in 2025 IEEE International Conference on Robotics and Automation (ICRA) IEEE 2025 pp. 8486–8492
[12] G Narita, T Seno, T Ishikawa, and Y Kaji “Panopticfusion: Online volumetric semantic mapping at the level of stuff and things” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE 2019 pp. 4205–4212
[13] A Hermans, G Floros, and B Leibe “Dense 3d semantic mapping of indoor scenes from rgb-d images” in 2014 IEEE International Conference on Robotics and Automation (ICRA) 2014 pp. 2631–2638
[14] R Jha, Y Zhou, and G Loianno “Adaptive keyframe selection for scalable 3d scene reconstruction in dynamic environments” arXiv preprint arXiv:2510.23928 2025
[15] S Zhu, G Wang, H Blum, J Liu, L Song, M Pollefeys, and H Wang “Sni-slam: Semantic neural implicit slam” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 pp. 21 167–21 177
[16] Y Haghighi, S Kumar, J P Thiran, and L Van Gool “Neural implicit dense semantic slam” arXiv preprint arXiv:2304.14560 2023
[17] K Li, M Niemeyer, N Navab, and F Tombari “Dns-slam: Dense neural semantic-informed slam” in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE 2024 pp. 7839–7846
[18] F Tosi, Y Zhang, Z Gong, S Mattoccia, M R Oswald, E Sandstrom, and M Poggi “How nerfs and 3d gaussian splatting are reshaping slam: a survey” IEEE Transactions on Robotics 2026
[19] C Yan, D Qu, D Xu, B Zhao, Z Wang, D Wang, and X Li “Gsslam: Dense visual slam with 3d gaussian splatting” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2024 pp. 19 595–19 604
[20] V Yugay, Y Li, T Gevers, and M R Oswald “Gaussian-slam: Photo-realistic dense slam with gaussian splatting” arXiv preprint arXiv:2312.10070 2023
[21] B Li, Z Cai, Y F Li, I Reid, and H Rezatofighi “Hier-slam: Scaling-up semantics in slam with a hierarchically categorical gaussian splatting” in 2025 IEEE International Conference on Robotics and Automation (ICRA) IEEE 2025 pp. 9748–9754
[22] Y Ji, Y Liu, G Xie, B Ma, Z Xie, and H Liu “Neds-slam: A neural explicit dense semantic slam framework using 3d gaussian splatting” IEEE Robotics and Automation Letters vol. 9 no. 10 pp. 8778–8785
[23] B Dou, T Zhang, Z Wang, Y Ma, Z Yuan, Y Ma, and N Zheng “Learning segmented 3d gaussians via efficient feature unprojection for zero-shot neural scene segmentation” in International Conference on Neural Information Processing Springer 2024 pp. 398–412
[24] M Caron, H Touvron, I Misra, H Jegou, J Mairal, P Bojanowski, and A Joulin “Emerging properties in self-supervised vision transformers” in Proceedings of the IEEE/CVF international conference on computer vision 2021 pp. 9650–9660
[25] J Straub, T Whelan, L Ma, Y Chen, E Wijmans, S Green, J J Engel, R Mur-Artal et al “The replica dataset: A digital replica of indoor spaces” arXiv preprint arXiv:1906.05797 2019
[26] A Dai, M Nießner, M Zollhofer, S Izadi, and C Theobalt “Bundle-fusion: Real-time globally consistent 3d reconstruction using on-the-fly surface reintegration” ACM Transactions on Graphics ToG vol. 36 no. 4 p. 1 2017
[27] Z Wang, A C Bovik, H R Sheikh, and E P Simoncelli “Image quality assessment: from error visibility to structural similarity” IEEE transactions on image processing vol. 13 no. 4 pp. 600–612 2004
[28] R Zhang, P Isola, A A Efros, E Shechtman, and O Wang “The unreasonable effectiveness of deep features as a perceptual metric” in Proceedings of the IEEE conference on computer vision and pattern recognition 2018 pp. 586–595
[29] J Sturm, N Engelhard, F Endres, W Burgard, and D Cremers “A benchmark for the evaluation of rgb-d slam systems” in 2012 IEEE/RSJ international conference on intelligent robots and systems IEEE 2012 pp. 573–580
[30] H Wang, J Wang, and L Agapito “Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2023 pp. 13 293–13 302
[31] M Johari, C Carta, and F Fleuret “Eslam: Efficient dense slam system based on hybrid representation of signed distance fields” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2023 pp. 17 408–17 419
[32] A Dai, A X Chang, M Savva, M Halber, T Funkhouser, and M Nießner “Scannet: Richly-annotated 3d reconstructions of indoor scenes” in Proceedings of the IEEE conference on computer vision and pattern recognition 2017 pp. 5828–5839
[33] X Yang, H Li, H Zhai, Y Ming, Y Liu, and G Zhang “Voxfusion: Dense tracking and mapping with voxel-based neural implicit representation” in 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) IEEE 2022 pp. 499–507
[34] E Sandstrom, Y Li, L Van Gool, and M R Oswald “Point-slam: ¨ Dense neural point cloud-based slam” in Proceedings of the IEEE/CVF international conference on computer vision 2023 pp. 18 433–18 444
--- Page 1 ---
VCS-SLAM: Geometry-Validated Semantic Evidence Fusion for 3D Gaussian SLAM
Gaussian SLAM Raman Jha, Shuaihang Yuan, and Yi Fang Visual SLAM performance often deteriorates in complex real-world applications Semantic 3D Gaussian SLAM commonly fuses 2D semantic priors into a persistent 3D map using uniform optimization weights However, such priors are not equally reliable in online mapping: occlusions, unsupported semantic boundaries, and ambiguous ray geometry can introduce persistent semantic artifacts into the global Gaussian map We propose VCS-SLAM, a geometry-validated semantic evidence fusion framework for RGB-D 3D Gaussian SLAM Instead of treating all semantic observations as uniformly valid supervision, VCS-SLAM evaluates their geometric reliability through visibility consistency, surface-supported boundary evidence, and ray-level conflict uncertainty The resulting reliability-aware objective suppresses occluded semantic updates, reduces unsupported semantic bleeding, and delays premature label assignment in ambiguous regions
--- Page 2 ---
I INTRODUCTION Visual SLAM fundamentally challenges robotics by requiring simultaneous camera pose estimation and incremental environment mapping [1] While traditional frameworks perform robustly in static settings their performance deteriorates during complex real-world applications Although multimodal sensor fusion addresses visual degradation in extreme conditions [2] ensuring spatial and geometric consistency of semantic representations in 3D environments remains a critical bottleneck Recent breakthroughs in Neural Radiance Fields (NeRF) [3] and 3D Gaussian Splatting (3DGS) [4] prompted their integration into visual SLAM mitigating surface continuity limitations inherent in discrete mapping Notably NeRFbased architectures (e g iMAP NICE-SLAM NICE-SLAM [6]) and 3DGS frameworks (e g MonoGS SplaTAM [8] SGSSLAM SGSSLAM [9]) achieve state-of-the-art performance producing consistent geometric and photorealistic maps Semantic mapping increasingly relies on 2D data-driven models Lacking inherent spatial and temporal continuity these models introduce severe semantic inconsistencies during online mapping
--- Page 3 ---
II RELATED WORKS A. Traditional Semantic SLAM-based methods Conventional semantic SLAM frameworks struggle to seamlessly integrate semantic information into mapping processes Historically these methods relied on discrete or implicit geometries like point clouds voxels, and signed distance fields (SDFs) [12], [13], [14] Point clouds and voxel grids are highly vulnerable to sparsity and degradation during rapid camera motion Conversely SDFs offer continuous surfaces but incur severe computational overhead restricting real-time scalability Consequently a critical need remains for efficient continuous mapping methods providing robust semantic-geometric representations
--- Page 4 ---
III METHODOLOGY A. Semantic Scene Representation using Gaussian Splatting Our framework extends 3D Gaussian Splatting [4] by integrating semantic information The environment is modeled as 3D Gaussians characterized by position μ, opacity σ, and geometric scale For dense semantic mapping we augment each Gaussian with channels encoding visual appearance ci and a C-dimensional semantic logit vector si where C is the number of semantic classes Per-frame semantic labels are converted into one-hot class targets for cross-entropy supervision During rendering semantic logits are alphacomposited and supervised using pixel-wise cross-entropy To optimize parameters against 2D observations 3D Gaussians are differentiably projected onto the image plane Overlapping Gaussians are aggregated via depth-sorted frontto-back volume rendering Let αi denote the evaluated 2D influence of a Gaussian including its occlusion term The semantic map Sp is rendered by blending semantic vectors of contributing Gaussians: Sp = ∑i=1 siαi Ö i−1 j=1 (1 − αj) Similarly expected depth Dp is synthesized by accumulating depth di of each Gaussian center in camera coordinates: Dp = ∑i=1 diαi Ö i−1 j=1 (1 − αj)
--- Page 5 ---
B.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Visibility-Consistent Semantic Updates: The system can suppress occluded semantic updates by applying a depth-gated visibility mask Mvis, which is defined as exp (- Dp - Dobs p / 2 sigma v vis 2), effectively filtering out unreliable semantic observations from regions where the rendered expected depth and sensor depth differ significantly.
-
Surface-Coupled Edge Alignment: The system can reduce unsupported semantic bleeding by introducing a structural regularizer Ledge that evaluates semantic edge magnitudes weighted by the lack of geometric support, penalizing gradients mainly when geometric support is weak, thereby
discouraging unsupported semantic gradients in geometrically smooth and low-confidence regions.
-
Conflict-Aware Uncertainty Weighting: The system can prevent premature label assignment in ambiguous regions by modulating semantic residuals using a reliability score rho p combining the depth-gated mask Mvis and the local confidence score Cp, where Cp = 1 / (1 + gamma Vpx), preventing
ambiguous regions from prematurely committing to incorrect semantic labels.
Abstract
Semantic 3D Gaussian SLAM commonly fuses 2D semantic priors into a persistent 3D map using uniform optimization weights. However, such priors are not equally reliable in online mapping: occlusions, unsupported semantic boundaries, and ambiguous ray geometry introduce persistent semantic artifacts into the global Gaussian map. We propose VCS-SLAM, a visibility-consistent semantic fusion framework for RGB-D 3D Gaussian SLAM. Instead of treating all semantic priors as uniformly valid signals, VCS-SLAM evaluates their geometric reliability through visibility consistency, surface-supported boundary evidence, and ray-level conflict uncertainty. The resulting reliability-aware objective suppresses occluded semantic updates, reduces unsupported semantic bleeding, and delays premature label assignment in ambiguous regions. Experiments on Replica show improved semantic consistency, boundary preservation, and reconstruction quality, while ScanNet results confirm competitive tracking under real RGB-D input. Under structured corruption of the 2D priors, VCS-SLAM degrades at little more than half the rate of a comparable semantic Gaussian SLAM baseline and outperforms naive gating, smoothness, and entropy-weighting variants of its own modules, indicating that the gains follow from the specific formulation rather than from reweighting alone. Code is available at https://github.com/jhakrraman/VCS-SLAM
Sources
- Adaptive Keyframe Selection for Scalable 3D Scene Reconstruction in Dynamic Environments
- Neural Implicit Dense Semantic SLAM
- Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting
- The Replica Dataset: A Digital Replica of Indoor Spaces
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs