EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards
This benchmark tests vision-language models (VLMs) as real-time safety guards for robots by evaluating their ability to detect hazards and trust visual information. It uses a unique design featuring two axes: physical situation safety and whether in-scene signs are misleading. Findings show that models often miss specific contextual dangers and that deceptive signs cause all guards to fail, either by missing hazards or overreacting.