VisionSIM: Beyond Idealized Sensors - A Multi-Modal Simulator for Robot Perception

arXiv:2611.08368 · cs.RO, cs.AI, cs.CV · Submitted 2026-11-05 · Read on arXiv

cs.RO, cs.AI, cs.CV

Submitted: 2026-11-05

Updated: 2026-11-05

Code: https://github.com/Genesis-Embodied-AI/Genesis

License: http://creativecommons.org/licenses/by/4.0/

The gist: Robotic perception increasingly relies on heterogeneous sensor suites combining cameras with emerging modalities (event, SPAD, iToF/dToF, and thermal).

Terminology

Abstract

Robotic perception increasingly relies on heterogeneous sensor suites combining cameras with emerging modalities (event, SPAD, iToF/dToF, and thermal). Yet existing simulators idealize these sensors, hiding critical modality-specific failure modes that limit real-world deployment. We present VisionSIM, a unified, physically grounded simulation platform that elevates sensor realism to a first-class concern by modeling the raw physical measurement process for diverse conventional and emerging modalities. We expose fine-grained control over physical sensor parameters, reproducing failure modes overlooked by general-purpose simulators: pileup and glare for LiDAR; phase wrapping, saturation, and motion artifacts for iToF; refractory drops for event cameras; and history-dependent emission for thermal cameras. This enables principled cross-modality comparison on identical scenes and trajectories. We release the platform and a large indoor dataset with synchronized raw measurements and rich ground-truth annotations.

Sources

Related papers