Blackout vs. Freeze: Analyzing Physical Failure Modes of VLAs under Camera Faults
cs.RO, cs.AI
Submitted: 2026-09-30
Updated: 2026-09-30
Code: https://github.com/NVIDIA/Isaac-GR00T
Project page: https://frankarobotics.github.io/docs
License: http://creativecommons.org/licenses/by/4.0/
The gist: Unreliable visual inputs can harm task performance and cause potential physical safety risks for vision-language-action (VLA) models.
Terminology
Abstract
Unreliable visual inputs can harm task performance and cause potential physical safety risks for vision-language-action (VLA) models. We analyze how π0.5 and GR00T models act under input faults such as image blackouts and freezing. We find that blackout and freezing produce distinct physical failure modes even when task-success rates are similarly low: freezing causes more extreme joint behavior, whereas blackout after gripper closure can cause more object drops, most markedly without proprioception. Selective intervention studies reveal that proprioception (current robot state) partly compensates for the removed robot depictions and reduces non-target contact. However, it cannot sufficiently restore task success when wrist-view object information is removed, even when aided by the remaining scene view. We then evaluate two mitigation approaches: camera-blackout training and training-free replacement of faulty visual embeddings. Both improve task success in selected conditions, but can increase unintended contact or disturbance to surrounding objects. Real-robot trials further show that successful execution under camera faults can still involve unintended physical interactions. These findings motivate designing VLA policies that use the robot and object information still available under camera faults to limit hazardous motion.
Sources
- ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
- SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
- VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing
- ActFovea: Runtime Safeguarding for VLA Policies via Spatiotemporal Visual-Action Consistency
- Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration
- Do You Need Proprioceptive States in Visuomotor Policies?
- LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies