SceneBound: How Much Should a Streaming 3D Reconstruction Model Remember?
cs.CV, cs.RO
Submitted: 2026-10-31
Updated: 2026-10-31
Comments: 34 pages, 8 figures, 27 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Streaming 3D reconstruction models keep past video frames in a recurrent state, fast weights from test-time training, or a key-value (KV) cache.
Terminology
Abstract
Streaming 3D reconstruction models keep past video frames in a recurrent state, fast weights from test-time training, or a key-value (KV) cache. Each written frame shifts the coordinate frame of later cameras or occupies GPU memory. Training-free rules decide what to write from a signal of the current frame. How much they write accounts for their mean accuracy: spending the update budget of a gate at uniformly spaced frames, with no signal, matches nine published gates, and the budget bounds the drift of the state wherever the writes fall. We propose SceneBound, a training-free rule that writes a frame only when its predicted camera pose lies at least a fixed radius from every stored keyframe. Its keyframe count stays within a constant factor of the covering number of the visited poses, the fewest views that keep a keyframe near every camera, so the count grows with the places visited and stays flat when the camera returns. In large scenes, SceneBound closes a submap once decoding its first frame again reveals drift. SceneBound uses one radius across four released models and three memory types. In the recurrent state of CUT3R and the KV cache of StreamVGGT, it achieves the lowest mean trajectory error among the published training-free rules we evaluate, in one pass and over nine passes of a scene, where its keyframes grow by 3% to 6%.
Sources
- Info3R: Information-Adaptive Test-Time Training for 3D Reconstruction
- SpecFit: Not All Draft Tokens Cost the Same in Offloaded MoE Inference
- SSR: A Training-Free Approach for Streaming 3D Reconstruction
- MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction
- S2Tok: Streaming 3D Gaussian Reconstruction with Persistent Spatial Tokens
- ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction
- Mem3R: Streaming 3D Reconstruction with Hybrid Memory
- StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression
- Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers
- MUT3R: Motion-aware Updating Transformer for Dynamic 3D Reconstruction
- The Replica Dataset: A Digital Replica of Indoor Spaces
- AMB3R-SLAM: Kilometer-scale SLAM with Hierarchical Backend
- Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
- TOKI: A Bitemporal Operator Algebra for Contradiction Resolution in LLM-Agent Persistent Memory
- PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences
- BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training
- When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents
- InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams
- Slot3R: Set-Associative Spatial Memory for Streaming 3D Reconstruction
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs