Fast and Reliable Open-Vocabulary Object Search with Absence Detection from Robot Memory
cs.RO
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: 7 pages, 7 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Mobile manipulators deployed in everyday human environments, such as homes for assistive care or flexible automation, often need to locate specific objects on demand from accumulated observations.
Terminology
Abstract
Mobile manipulators deployed in everyday human environments, such as homes for assistive care or flexible automation, often need to locate specific objects on demand from accumulated observations. Equally important, they must reliably determine when a target is absent before committing to navigation or interaction. Existing approaches demand either slow exhaustive re-processing or expensive 3D representations, and are evaluated on recall and accuracy rather than specificity, leaving their ability to abstain on missing targets unmeasured. We find existing systems that search using text or visual embedding almost never abstain on absent objects achieving at most 9% specificity. To support accurate search at speed, we present a two-stage architecture that decouples fast candidate retrieval from verified selection, enabling both high recall and reliable absence detection from sequential RGB observations with pose. We evaluate eight retrieval-selection configurations plus an exhaustive search baseline across ScanNet apartment scenes, including significant numbers of absent objects. Adding a visual verification step to caption-based retrieval matches or exceeds exhaustive search reliability while requiring an order of magnitude fewer detector calls. We reproduce ScanNet scores on a physical robot operating entirely on-board with a single consumer GPU running in real time.
Sources
- SAM 3: Segment Anything with Concepts
- Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation
- CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
- OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics
- Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
- Gemma 3 Technical Report
- Nomic Embed: Training a Reproducible Long Context Text Embedder
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation