OmniRobotHome: A Multi-Camera Home Platform for Real-Time Human-Robot Interaction

arXiv:2604.28197 · cs.RO, cs.CV · Submitted 2026-04-30 · Read on arXiv

cs.RO, cs.CV

Submitted: 2026-04-30

Updated: 2026-10-13

Comments: Project Page: https://junc0ng.github.io/omnirobothome

Code: https://github.com/snakers4/silero-vad

Project page: https://junc0ng.github.io/omnirobothome

License: http://creativecommons.org/licenses/by/4.0/

The gist: Robots that assist people must respond promptly and appropriately to human activity, which requires low-latency processing, accurate perception, and anticipation of human motion.

Terminology

Abstract

Robots that assist people must respond promptly and appropriately to human activity, which requires low-latency processing, accurate perception, and anticipation of human motion. These requirements are especially difficult to meet in living environments, where perception must cover wide areas, remain robust to occlusion, and capture natural behavior without wearable devices. We present OmniRobotHome, a testbed that addresses these challenges in a fully furnished home that supports natural everyday activity. Its 48 hardware-synchronized cameras provide room-wide coverage, enabling accurate markerless perception under occlusion. A distributed perception pipeline processes these streams in parallel and delivers persistent human and object states to robot controllers with low latency. We demonstrate OmniRobotHome through two human-robot interaction studies: proactive yielding for safety and nonverbal intent recognition for assistance. Through these studies, we analyze how camera coverage, state latency, multi-view pose, hand, and gaze estimation, and behavior-memory-based forecasting affect overall task performance. Our results highlight the value of OmniRobotHome as a testbed for studying human-robot assistance in real living environments.

Sources

Related papers