SCOPE-4D: Endoscopic 4D Geometry Foundation Models
cs.CV
Submitted: 2026-10-01
Updated: 2026-10-01
Comments: Project page: https://chaoyizh.github.io/SCOPE-4D-page/
Project page: https://chaoyizh.github.io/SCOPE-4D-page
License: http://creativecommons.org/licenses/by/4.0/
The gist: Geometric understanding supports endoscopic navigation and robotic assistance, but learning reliable endoscopic geometry faces two challenges: scarce geometric annotations and ambiguity between
Terminology
Abstract
Geometric understanding supports endoscopic navigation and robotic assistance, but learning reliable endoscopic geometry faces two challenges: scarce geometric annotations and ambiguity between camera motion and tissue deformation. We present SCOPE-4D, an endoscopic 4D geometry foundation model that jointly predicts camera parameters, dense geometry, and 3D tissue trajectories from monocular RGB video in a single forward pass. Our curation and annotation pipeline constructs SCOPE-5K, a collection of approximately 5,000 clips spanning real and synthetic gastrointestinal endoscopy and laparoscopy. The collection provides rich geometric supervision and includes newly collected phantom and real-colonoscopy evaluation sets. Geometric supervised fine-tuning on SCOPE-5K learns endoscopic priors that improve camera and depth estimation. Common--Residual Motion (CRM) further constrains local deformation relative to common tissue movement. Together with geometric supervision, CRM and trajectory supervision further improve camera and depth estimation over geometric fine-tuning alone while enabling dense 3D tissue tracking. Evaluations on public and newly collected benchmarks demonstrate strong in-domain and out-of-domain geometry, superior 3D tracking, and more stable long-sequence colon reconstruction. A blinded user study further supports the perceived reconstruction quality on real clinical video. Together, these results demonstrate the value of large-scale endoscopic supervision and motion constraints for joint geometry estimation and tissue tracking.
Sources
- Stereo Correspondence and Reconstruction of Endoscopic Data Challenge
- Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Cameras
- VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences
- Any4D: Unified Feed-Forward Metric 4D Reconstruction
- RealSynCol: a high-fidelity synthetic colon dataset for 3D reconstruction applications
- Depth Anything 3: Recovering the Visual Space from Any Views
- SM4RT: Learning Structured Motion Geometry for 4D Reconstruction
- EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstruction
- 4RC: 4D Reconstruction via Conditional Querying Anytime and Anywhere
- Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics
- LoMa: Local Feature Matching Revisited
- SAM 2: Segment Anything in Images and Videos
- NRGS-SLAM: Monocular Non-Rigid SLAM for Endoscopy via Deformation-Aware 3D Gaussian Splatting
- A micro Lie theory for state estimation in robotics
- VGGT-$\Omega$
- SpatialTrackerV2: 3D Point Tracking Made Easy
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models