RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation
cs.RO, cs.CV
Submitted: 2026-09-29
Updated: 2026-09-29
Project page: https://shuhongll.github.io/rawvla
License: http://creativecommons.org/licenses/by/4.0/
The gist: Vision-language-action (VLA) models typically operate on RGB images produced by a fixed camera image signal processor (ISP), leaving the imaging pipeline outside the learning and evaluation loop.
Terminology
Abstract
Vision-language-action (VLA) models typically operate on RGB images produced by a fixed camera image signal processor (ISP), leaving the imaging pipeline outside the learning and evaluation loop. We systematically examine the consequences of this overlooked design choice across five fundamental ISP dimensions: gain, sensor noise, chromatic response, tonal response, and bit depth. Our analysis reveals that RAW-to-RGB processing materially shapes both action prediction and manipulation success, with different ISP dimensions exerting substantially different effects. Guided by these findings, we introduce RawVLA, a streaming neural ISP that adaptively renders RAW observations for frozen VLA policies while concentrating its capacity on the imaging factors relevant to embodied behavior. We further present RawVLA-Bench, a RAW-domain manipulation benchmark to expose image processing as an explicit evaluation variable across clean and adverse acquisition conditions. Experiments on RawVLA-Bench show that RawVLA preserves performance under standard conditions while substantially improving robustness under degraded imaging, establishing adaptive RAW processing as an effective interface between physical cameras and embodied policies.
Sources
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- RADAR: Benchmarking Vision-Language-Action Generalization via Real-World Dynamics, Spatial-Physical Intelligence, and Autonomous Evaluation
- Gemini Robotics: Bringing AI into the Physical World
- Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization
- Learned Image Compression for Vision-Language-Action Models
- Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
- Embodied Image Compression
- UniISP: A Unified ISP Framework for Both Human and Machine Vision
- What Matters in Building Vision-Language-Action Models for Generalist Robots
- Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action Model
- RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling
- RoVLA: Multi-Consistency Constraints for Robust Vision-Language-Action Models
- Colosseum V2: Benchmarking Generalization for Vision-Language-Action Models
- Improving Robustness of Vision-Language-Action Models by Restoring Corrupted Visual Inputs
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing
- LADEV: A Language-Driven Testing and Evaluation Platform for Vision-Language-Action Models in Robotic Manipulation
- Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color
- STRONG-VLA: Decoupled Robustness Learning for Vision-Language-Action Models under Multimodal Perturbations
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies