Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Lights, Camera, Malfunction".
Dev: Here's the gist of the paper. Vision-Language-Action (VLA) models exhibit vulnerabilities to minor environmental perturbations, which this paper addresses by proposing an optimized physical spotlight attack framework and a chroma-preserving adversarial training method <ref:2607.14698#pg6> The proposed ChromaGuard method maintains high performance in benign, color-dependent operations while achieving robustness against severe lighting attacks <ref:2607.14698#pg9> FLARE Framework and Attack Generation The authors introduce FLARE, an optimized physical spotlight attack framework designed to exploit vulnerabilities in VLA models via targeted illuminations <ref:2607.14698#pg4> This framework parameterizes the spotlight using parameters like hue, saturation, value, intensity, spatial height, and cutoff angle <ref:2607.14698#pg5> The attack objective is calculated by combining trajectory deviation and task failure into a single score to find worst-case physical lighting parameters <ref:2607.14698#pg6> Bayesian Optimization is employed to efficiently search the continuous spotlight parameter space for values that maximize this objective score <ref:2607.14698#pg6> The framework allows the adversary to control spatial placement, illumination footprint, intensity, and chromatic properties of the emitted light <ref:2607.14698#pg4> Defensive Pitfalls in Training A critical defensive trap identified is that naive data augmentations incorrectly condition VLA models to discard color as noise <ref:2607.14698#pg6> When applying broad HSV-space data augmentation, the resulting model behaves similarly to one trained exclusively on grayscale images <ref:2607.14698#pg4> This conditioning causes the model to lose the capacity to solve semantic tasks where fine-grained color information plays a critical role <ref:2607.14698#pg6> The authors demonstrate this degradation through a diagnostic grayscale evaluation, showing that defended models maintain high success rates on grayscale inputs but drop to at most 47.5% success on real-world color-dependent tasks <ref:2607.14698#pg4> ChromaGuard Methodology ChromaGuard is proposed as a novel augmentation architecture designed to preserve texture and hue during training <ref:2607.14698#pg6> The method restricts the augmentation parameter space to allow bounded perturbations only to saturation, brightness, contrast, and sharpness while fixing the hue perturbation to zero <ref:2607.14698#pg6> This constrained space is used to simulate challenging lighting conditions via a standard adversarial objective <ref:2607.14698#pg6> ChromaGuard achieves state-of-the-art resilience against dynamic lighting attacks, reaching 70.0% success in color-invariant tasks <ref:2607.14698#pg6> Furthermore, it yields success rates of 97.5% and 92.5% in benign and attacked color-dependent scenarios on a physical platform <ref:2607.14698#pg4> Empirical Results and Implications In simulation, the optimized attack completely compromises the Baseline model, dropping its success rate to 0.0% while inducing severe trajectory deviations up to 115.5 cm <ref:2607.14698#pg8> The authors show that naive data augmentation looks seemingly sufficient because it forces the model to bypass illumination challenges by discarding essential color cues <ref:2607.14698#pg4> Real-world experiments confirm this, where the Naive-Aug model maintains almost identical performance on grayscale inputs as on standard RGB, exposing a critical flaw in naive augmentation <ref:2607.14698#pg6> ChromaGuard successfully resolves the trade-off by decoupling illumination variations from chroma features during training <ref:2607.14698#pg4> It matches the robustness of Naive-Aug in the color-invariant setting while demonstrating substantial improvements in the color-dependent setting <ref:2607.14698#pg6> The study concludes that optimized spotlight perturbations can reliably degrade baseline VLA performance to a 0.0% success rate <ref:2607.14698#pg8> The paper highlights the urgent necessity of achieving a certain level of generalizability of current state-of-the-art VLA models before diving too deeply into their security <ref:2607.14698#pg9> The tools presented serve as analytical tools to expose the acute vulnerability of VLA models to subtle environmental perturbations rather than definitive attack or defense methods <ref:2607.14698#pg4> They do not yet explore temporal dynamics such as time-varying lighting or closed-loop adaptive attacks <ref:2607.14698#pg4> The primary objective is to expose the critical robustness crisis facing current VLA models <ref:2607.14698#pg6> The authors emphasize that naive data augmentation fails to resolve these vulnerabilities by discarding color features rather than learning robust representations <ref:2607.14698#pg4> References
1: B.
Rosa: Z.
Dev: et al.
Rosa: RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control <ref:2607.14698#pg10>
2: M.
Paper summary: Rosa: Looking at "Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color," the paper really centers on this tension between robustness and fidelity in vision language action models. The authors highlight how existing training methods using naive augmentations inadvertently condition the model to discard color as noise.
Dev: And they use a specific attack framework called FLARE, which optimizes spotlight parameters like hue, saturation, value, intensity, spatial height, and cutoff angle to induce catastrophic trajectory deviations in baseline models that have no access to internal weights.
Taro: The main contribution here is pinpointing this critical pitfall: standard augmentations don't just add noise; they fundamentally condition the model to ignore color cues when those cues are essential for tasks. They show this through a grayscale diagnostic evaluation where the defended model only maintained high success rates on grayscale inputs, dropping to at most forty-seven point five percent on real color-dependent tasks <ref:2607.14698#pg1,high success rates on grayscale inputs>.
Rosa: And then they propose ChromaGuard as a solution, which is a chromapreserving adversarial training methodology that restricts augmentation to only bounded perturbations in saturation and brightness while fixing the hue perturbation to zero. This method successfully preserves semantic fidelity, achieving ninety-seven point five percent success under benign conditions and ninety-two point five percent under an attack on a physical platform <ref:2607.14698#pg3>.
Dev: It’s important to note what the authors themselves point out about their work; they emphasize that these tools are analytical, exposing this acute vulnerability to subtle environmental perturbations rather than presenting definitive attack or defense methods yet. They don't explore things like temporal dynamics such as time-varying lighting or closed-loop adaptive attacks.
Taro: So, the implication is that we’re in a crisis with current VLA models; they need better generalizability before we can focus too deeply on security against these kinds of subtle environmental shifts. The paper shows us what's broken and suggests a way to learn more robust representations by decoupling illumination variations from chroma features during training.
Conclusion: Rosa: So, to wrap up this talk about "Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color," we've seen how these models are super fragile when the lighting changes unexpectedly.
Dev: Yeah, it’s about those physical spotlights being a big problem for vision language action models because they can easily crash them.
Taro: Exactly, the authors show that even if you train a model to be tough against lighting, simple color augmentations actually hurt its ability to do the actual task when the colors matter.
Rosa: The core idea here is exposing this vulnerability—how these AI models are blind to subtle changes in light because they learned everything from just grayscale data.
Dev: And the authors propose this ChromaGuard method as a way to fix it, using a specific training technique that keeps the hue consistent while letting other things like saturation and brightness wiggle a bit.
Taro: That ChromaGuard seems promising because they show it can handle those real-world color-dependent tasks, getting ninety seven point five percent success under benign conditions.
Rosa: So what does this mean for us? It means that just making models tougher against random noise isn't enough; we have to train them to understand *what* the colors are important and how they change when the light gets weird.
Dev: And it’s not just about the attack itself, though; it’s about making sure these models can actually run safely in real-time loops without those kind of sudden performance drops during deployment.
Taro: They're pointing out that we need better generalizability before we start worrying too much about building perfect defenses against every possible lighting setup.
Rosa: Right, so the focus shifts from just blocking attacks to learning robust representations that don't care about minor illumination blips at all.
Marino Watanabe Takami Sato, Kentaro Yoshioka
Keio University
cs.RO
Submitted: 2026-07-16
Updated: 2026-11-08
Code: https://github.com/TheRobotStudio/SO-ARM100
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 90/100
The gist: Vision-Language-Action (VLA) models exhibit vulnerabilities to minor environmental perturbations, which this paper addresses by proposing an optimized physical spotlight attack framework and a…
Key concepts
- FLARE Framework
- This is an optimized framework used to generate the worst-case physical spotlight attacks against VLA models. It systematically searches a wide range of light parameters like hue, saturation, and intensity using Bayesian Optimization to find the lighting settings that cause the most catastrophic failures in model performance.
- Naive Data Augmentation
- This is a common training technique where input images are randomly perturbed across various visual properties like hue and saturation. The paper shows this method is flawed because it incorrectly conditions VLA models to treat color as noise, which degrades their ability to perform tasks that rely on fine-grained color information.
- ChromaGuard
- This is a novel adversarial training method designed to make VLA models robust against lighting attacks. It constrains data augmentation only to parameters like saturation and brightness, explicitly fixing the hue perturbation at zero, thereby preserving essential color information during training.
Terminology
Summary
Vision-Language-Action (VLA) models exhibit vulnerabilities to minor environmental perturbations, which this paper addresses by proposing an optimized physical spotlight attack framework and a chroma-preserving adversarial training method <ref:2607. p. 0 Lights, Camera, Malfunction: When Illumination
Robustness Leaves VLA Models Blind to Color
Marino Watanabe Takami Sato Kentaro Yoshioka
Keio University
Abstract: Vision-Language-Action (VLA) models have emerged as a powerful
paradigm for general-purpose robot manipulation; however, their transition to
real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimized physical spotlight attack framework
that exploits these vulnerabilities via targeted illuminations, dropping baseline
task success rates to zero without any access to model internals. 1, a chromapreserving adversarial training methodology. We demonstrate that our approach not only provides
robust defensive capabilities against severe lighting attacks but also maintains high performance
in benign, color-dependent operations.Where the paper says it
Where the paper says it
FLARE Framework and Attack Generation
The authors introduce FLARE, an optimized physical spotlight attack framework designed to exploit vulnerabilities in VLA models via targeted illuminations <ref:2607.14698#pg4> This framework parameterizes the spotlight using parameters like hue, saturation, value, intensity, spatial height, and cutoff angle <ref:2607. p. 2 In simulation, the
spotlight is rendered directly to optimize these parameters; on hardware, the same parameterization
corresponds to a physical lamp whose pose and output can be adjusted within safe bounds. We parameterize the spotlight by:
θ = [hlight, slight, vlight, I, zlight, αcutoff], (1)
where hlight, slight, vlight denote the hue, saturation, and value of the light source, I is the light intensity, zlight is the spatial height (z-coordinate), and αcutoff represents the spotlight’s cutoff angle. For episode i, we score a spotlight parameter
by combining trajectory deviation and task failure:
Li(θ) = 1
S
X
S
t=1
∥P
(i)
t − P
′(i)
t ∥2 + λ1∥P
(i)
T − P
′(i)
T
∥2 + λ2I
(i)
f
, (2)
where S is the maximum number of steps, P
(i)
t
and P
′(i)
t
are the clean and attacked end-effector
positions respectively, I
(i)
f ∈ {0, 1} indicates a binary task failure, and λ1, λ2 are weighting hyperparameters. We aim to maximize the average objective score over N episodes: J(θ) = 1
N
PN
i=1 Li(θ). We aim to maximize the average objective score over N episodes: J(θ) = 1
N
PN
i=1 Li(θ). Because physical and simulated rollouts are expensive and internal model gradients are unavailable,
we employ Bayesian Optimization to efficiently search the continuous spotlight parameter space for
values that maximize J(θ). 1, the adversary can deploy physical lighting
fixtures within the robot’s workspace. When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and
lightness) of the emitted light.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Defensive Pitfalls in Training
A critical defensive trap identified is that naive data augmentations incorrectly condition VLA models to discard color as noise <ref:2607.14698#pg6> When applying broad HSV-space data augmentation, the resulting model behaves similarly to one trained exclusively on grayscale images <ref:2607. p. 0 However, we find that naive adversarial training
does not consistently yield genuine robustness; rather, it often conditions VLA models to entirely
ignore the visual features that were perturbed during the training process. For example, when applying broad HSV-space data augmentation to the training data, the resulting VLA model behaves
similarly to one trained exclusively on grayscale images. Consequently, the model loses the capacity
to solve semantic tasks where fine-grained color information plays a critical role. arXiv:2607. While adversarial training is the standard countermeasure, we identify a critical and previously
underestimated defensive pitfall: naive data augmentations incorrectly condition
VLA models to discard color as noise, collapsing their visual perception into a
purely shape-biased processor. We expose this degradation through a diagnostic
grayscale evaluation, in which the defended model maintains high success rates
on grayscale inputs, while its success rate on benign, color-dependent real-world
tasks drops to at most 47. While adversarial training is the standard countermeasure, we identify a critical and previously
underestimated defensive pitfall: naive data augmentations incorrectly condition
VLA models to discard color as noise, collapsing their visual perception into a
purely shape-biased processor. We expose this degradation through a diagnostic
grayscale evaluation, in which the defended model maintains high success rates
on grayscale inputs, while its success rate on benign, color-dependent real-world
tasks drops to at most 47.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
ChromaGuard Methodology
ChromaGuard is proposed as a novel augmentation architecture designed to preserve texture and hue during training <ref:2607.14698#pg6> The method restricts the augmentation parameter space to allow bounded perturbations only to saturation, brightness, contrast, and sharpness while fixing the hue perturbation to zero <ref:2607. p. 3 To isolate illumination effects, ChromaGuard restricts the augmentation parameter space. Specifically, it allows bounded perturbations to saturation, brightness, contrast, and sharpness, but explic4 itly fixes the hue perturbation to zero (h = 0). We use this constrained parameter space, denoted
as ΦCG, to simulate challenging lighting conditions via a standard adversarial objective. • We propose ChromaGuard, a novel augmentation architecture designed to preserve texture and
hue. We empirically prove that this method achieves state-of-the-art resilience against dynamic
lighting attacks, reaching 70. We empirically prove that this method achieves state-of-the-art resilience against dynamic
lighting attacks, reaching 70. 0% success in color-invariant tasks. 0% success in color-invariant tasks. Furthermore, it strictly retains
the perceptual capabilities necessary for complex physical manipulation by yielding success
rates of 97. On
a physical 6-DoF robotic platform, we demonstrate that ChromaGuard achieves
97. 5% success rates in benign and attacked color-dependent tasks, respectively.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Empirical Results and Implications
In simulation, the optimized attack completely compromises the Baseline model, dropping its success rate to 0.0% while inducing severe trajectory deviations up to 115. p. 5 The optimized attack completely compromises the Baseline model, dropping its success rate
to 0. 0% while inducing severe trajectory deviations of up to 115. Consequently, the model loses the capacity
to solve semantic tasks where fine-grained color information plays a critical role. arXiv:2607. , the VLA effectively degrades into a shape-biased, grayscale processor. By indiscriminately perturbing color channels during training, the augmentation forces the model to bypass
the illumination challenge by discarding essential color cues. As shown in Table 2, the baseline model,
which naturally learned color features during training, suffers a significant drop, its success rates
falling to near zero across the suites. In contrast, the Naive-Aug model maintains almost identical
performance on grayscale inputs as on standard RGB, e. This confirms that defense techniques relying on indiscriminate color perturbation can severely degrade the semantic grounding of
color information required for complex real-world tasks. ChromaGuard resolves this trade-off by decoupling illumination variations from chroma features
during training. ChromaGuard resolves this trade-off by decoupling illumination variations from chroma features
during training. As shown in Table 3, ChromaGuard matches the robustness of Naive-Aug in the
color-invariant setting (≥70% SR under attack) while demonstrating substantial improvements in
the color-dependent setting.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion and Future Directions
The study concludes that optimized spotlight perturbations can reliably degrade baseline VLA performance to a 0.0% success rate <ref:2607. p. 0 While adversarial training is the standard countermeasure, we identify a critical and previously
underestimated defensive pitfall: naive data augmentations incorrectly condition
VLA models to discard color as noise, collapsing their visual perception into a
purely shape-biased processor. We expose this degradation through a diagnostic
grayscale evaluation, in which the defended model maintains high success rates
on grayscale inputs, while its success rate on benign, color-dependent real-world
tasks drops to at most 47. Our physical evaluations
on a 6-DoF manipulator indicate that ChromaGuard effectively balances robustness and perceptual
fidelity, maintaining reliable success rates under targeted lighting attacks without compromising the
visual acuity necessary for semantic manipulation. These findings emphasize the urgent necessity of
achieving a certain level of generalizability of the current state-of-the-art VLA models before diving
too deeply into their security. 6 Limitations
Although our FLARE and ChromaGuard demonstrate substantial attack and mitigation capabilities on spotlight attacks, we do not intend to serve them as definitive attack and defense methods. Instead, they serve as analytical tools to expose the acute vulnerability of VLA models to subtle
environmental perturbations. Our analysis reveals that naive data augmentation fails to resolve these
vulnerabilities, merely discarding color features rather than learning robust representations. Furthermore, these tools do not yet explore temporal dynamics, such as time-varying lighting or closed-loop
adaptive attacks. Furthermore, these tools do not yet explore temporal dynamics, such as time-varying lighting or closed-loop
adaptive attacks. Our primary objective is to expose the critical robustness crisis facing current VLA
models, rather than to engineer an exhaustive adversarial attack or an impenetrable defense. Instead, they serve as analytical tools to expose the acute vulnerability of VLA models to subtle
environmental perturbations. Our analysis reveals that naive data augmentation fails to resolve these
vulnerabilities, merely discarding color features rather than learning robust representations.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
[1] B. Z. et al. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control <ref:2607. p. 8 References
[1] B. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control.Where the paper says it
[2] M. J. Kim, K. Pertsch, S. p. 8 In PMLR, volume 229, pages 2165–2183, 2023. Pertsch, S. Pertsch, S. Karamcheti, T. Karamcheti, T. Balakrishna, S. Balakrishna, S. Rafailov, E.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[3] M. Shukor, D. Aubakirova, F. p. 8 09246, 2024. Aubakirova, F. Aubakirova, F. Capuano, P. Capuano, P. Kooijmans, S.Where the paper says it
Where the paper says it
Where the paper says it
[4] K. B. et al. π0: A Vision-Language-Action Flow Model for General Robot Control
[5] Y. Jiang, Y. Ze, Z. Jiang, S. Song, and J. p. 8 org/abs/2410. Song, and J.Where the paper says it
[6] S. Jiang, Z. Huang, K. Qian, Z. Luo, T. Zhu
[7] S. Li, Y. Chen, D. Li, S. Liu, D. Lan
[8] Q. Li, B. Yin, W. Huang, R. Liu
[9] B. Ma, H. Guo, P. Lv, M. Xu
[10] X. Lu, J. Chen, S. Xiao, Z. Jin
[11] H. Xu, Y. S. Koh, S. Huang
[12] X. Wang, J. Li, Z. Weng, Y. Wang
[13] X. Zhou, G. Tie, G. Zhang, H. Wang
[14] J. Zhou, Y. Wei, R. Zhen, B. Zhao
[15] N. Zhang, W. Tao, X. Xiao, Q. Sun
[16] X. Zhou, Y. Xu, G. Tie, Y. Chen
[17] B. Liu, Y. Zhu, C. Gao, Y. Feng
[18] D. Y. G. Orjuela, L. p. 9 Advances in Neural Information Processing
Systems, 36:44776–44791, 2023. Orjuela, L. Orjuela, L. Scappatura, V.Where the paper says it
Where the paper says it
[19] J. Guo, Z. Wu, C. Tu, Y. Ma
[20] A. Madry, A. Makelov, L. p. 9 On
Robustness of Vision-Language-Action Model Against Multi-Modal Perturbations, 2025. Makelov, L.Where the paper says it
[21] S. Xhonneux, A. p. 9 06083, 2017. Xhonneux, A. Xhonneux, A. Sordoni, S.Where the paper says it
Where the paper says it
[22] Y. Ganin, E. Ustinova, H. p. 9 Advances in Neural Information Processing Systems, 37:1502–1530, 2024. Ustinova, H.Where the paper says it
[23] A. Hanif, M. Naseer, S. Khan
[24] S. Yang, W. Xiao, M. Zhang, S. Guo
[25] T. Akiba, S. Sano, T. Yanase
[26] LeRobot: An open-source library for end-to-end robot learning
[27] E. Todorov, T. p. 9 22818, 2026. Todorov, T. Todorov, T. Erez, and Y.Where the paper says it
Where the paper says it
[28] R. Knight, P. Kooijmans, R. p. 8 Capuano, P. Kooijmans, S.Where the paper says it
[29] P. Intelligence, K. p. 8 Transferring Vision-LanguageAction Models to Industry Applications: Architectures, Performance, and Challenges. In 2025
7th International Conference on Industrial Artificial Intelligence (IAI), pages 1–6.Where the paper says it
--- Page 1 ---
Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color
Marino Watanabe Takami Sato Kentaro Yoshioka Keio University
--- Page 2 ---
FLARE Framework: Vulnerability to Lighting Conditions Defense Dilemma & Our Solution Optimized Parameters: θ = [h, s, v,I, z, α] Wrist camera 3rd-person camera VLA Model Crash! Naive Augmentation Proposed: ChromaGuard Under Attack Benign Success Success Selects Wrong Object Augmented Inputs Color- Distorted Pick up red ball Pick up red ball ColorPreserved Illumination Attack Robust but Color-Blind Robust and Color-Aware … Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense. p. 1 FLARE Framework: Vulnerability to Lighting Conditions
Defense Dilemma & Our Solution
Optimized Parameters:
𝜽 = [ℎ, 𝑠, 𝑣,𝐼, 𝑧, 𝛼]
Wrist
camera
3
rd
-person
camera
VLA
Model Crash! Naive Augmentation Proposed: ChromaGuard
Under Attack
Benign
Success
Success
Selects
Wrong Object
Augmented
Inputs
Color- Distorted
Pick up
red ball
Pick up
red ball
ColorPreserved
Illumination
Attack
Robust but Color-Blind Robust and Color-Aware …
…
Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense. Naive Augmentation Proposed: ChromaGuard
Under Attack
Benign
Success
Success
Selects
Wrong Object
Augmented
Inputs
Color- Distorted
Pick up
red ball
Pick up
red ball
ColorPreserved
Illumination
Attack
Robust but Color-Blind Robust and Color-Aware …
…
Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense. Top: Our
FLARE framework optimizes spotlight parameters to induce catastrophic trajectory deviations in
VLA models. Top: Our
FLARE framework optimizes spotlight parameters to induce catastrophic trajectory deviations in
VLA models. Bottom: The defense dilemma where naive color augmentation degrades task success
on color-dependent tasks, which is resolved by our proposed ChromaGuard approach.Where the paper says it
Where the paper says it
Where the paper says it
--- Page 3 ---
2 Background 2.1 Security and Robustness of VLA Models against Physical Attacks The multimodal nature of VLAs exposes a wide variety of attack channels [8, 9] such as trainingtime backdoor attacks [13, 14], inference-time adversarial attacks [11, 15, 12]. p. 1 2 Background
2. 1 Security and Robustness of VLA Models against Physical Attacks
The multimodal nature of VLAs exposes a wide variety of attack channels [8, 9] such as trainingtime backdoor attacks [13, 14], inference-time adversarial attacks [11, 15, 12]. 1 Security and Robustness of VLA Models against Physical Attacks
The multimodal nature of VLAs exposes a wide variety of attack channels [8, 9] such as trainingtime backdoor attacks [13, 14], inference-time adversarial attacks [11, 15, 12]. Meanwhile, these
advanced attacks naturally extend the prior attacks effective against general DNN models. jor contributions of these previous works are in overcoming the major challenges in triggering the
vulnerabilities in their VLA domains. In this work, we highlight that VLA models are indeed vulnerable to these kinds of attacks, but the current state-of-the-art VLA models are poorly generalized
and are just vulnerable to physical environmental variations, rather than particularly vulnerable to
these advanced attacks. In this work, we highlight that VLA models are indeed vulnerable to these kinds of attacks, but the current state-of-the-art VLA models are poorly generalized
and are just vulnerable to physical environmental variations, rather than particularly vulnerable to
these advanced attacks. LIBERO-PRO [16] also reports that four major state-of-the-art VLA models easily collapse under small task and position perturbations on the LIBERO [17], one of the most
popular VLA benchmarks. LIBERO-PRO [16] also reports that four major state-of-the-art VLA models easily collapse under small task and position perturbations on the LIBERO [17], one of the most
popular VLA benchmarks. All models achieve ≥90% success rates on the original LIBERO tasks
but fall to nearly 0% under the perturbations, showing poor generalizability. All models achieve ≥90% success rates on the original LIBERO tasks
but fall to nearly 0% under the perturbations, showing poor generalizability. Under these circumstances, any attacks may succeed. Under these circumstances, any attacks may succeed. This motivates us to start by investigating the impact of lighting
conditions, which can be perturbed in normal benign scenarios.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 4 ---
3 Methodology 3.1 Threat Model We follow the common threat model adopted in prior work [10], which assumes a strict black-box setup, i. p. 2 3 Methodology
3. 1 Threat Model
We follow the common threat model adopted in prior work [10], which assumes a strict black-box
setup, i. 3 Methodology
3. 1 Threat Model
We follow the common threat model adopted in prior work [10], which assumes a strict black-box
setup, i. 1 Threat Model
We follow the common threat model adopted in prior work [10], which assumes a strict black-box
setup, i. , the adversary has no access to the target model’s internal architecture, weights, gradient
information, or training datasets. Bottom: The defense dilemma where naive color augmentation degrades task success
on color-dependent tasks, which is resolved by our proposed ChromaGuard approach. To resolve this defense dilemma, we introduce ChromaGuard, as depicted in Fig. When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and
lightness) of the emitted light. However, they cannot dynamically change the light once deployed;
the parameters remain fixed across all evaluation episodes. When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and
lightness) of the emitted light. However, they cannot dynamically change the light once deployed;
the parameters remain fixed across all evaluation episodes.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 5 ---
3.2 FLARE Framework: Optimized Physical Spotlight Attack We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in Fig. p. 2 However, they cannot dynamically change the light once deployed;
the parameters remain fixed across all evaluation episodes. 2 FLARE Framework: Optimized Physical Spotlight Attack
We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in
Fig. However, they cannot dynamically change the light once deployed;
the parameters remain fixed across all evaluation episodes. 2 FLARE Framework: Optimized Physical Spotlight Attack
We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in
Fig.Where the paper says it
Where the paper says it
--- Page 6 ---
3.3 ChromaGuard: Hue-Preserving Adversarial Training Naive lighting augmentation typically perturbs all color channels indiscriminately, often through aggressive hue, saturation, or texture-distorting transforms We first introduce a typical naive data augmentation (we call this baseline Naive-Aug) which is a standard combination of color and sharpness jittering While this approach improves general visual robustness, unconstrained hue variations (h ̸= 0) can alter intrinsic object colors, inadvertently confounding performance on strictly color-dependent tasks
--- Page 7 ---
3.1 Threat Model We follow the common threat model adopted in prior work [10], which assumes a strict black-box setup, i.e., the adversary has no access to the target model’s internal architecture, weights, gradient information, or training datasets As depicted in Fig. p. 2 , the adversary has no access to the target model’s internal architecture, weights, gradient
information, or training datasets. As depicted in Fig.Where the paper says it
--- Page 8 ---
3.2 FLARE Framework: Optimized Physical Spotlight Attack We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in Fig. 2, to generate worst-case lighting scenarios We parameterize the spotlight by: θ = [hlight, slight, vlight, I, zlight, αcutoff], (1) where hlight, slight, vlight denote the hue, saturation and value of the light source Given a clean observation ot t=A(ot; θ), which alters illumination while preserving physical scene geometry and object layout The attack objective follows the intuition that dangerous failures are not only unsuccessful but also catastrophically distant from the intended trajectory
--- Page 9 ---
4 Simulator Experiments: Exposing Vulnerability against Spotlight Attack We evaluate the performance of our VLA model against spotlight attacks within a simulated environment In this section, we demonstrate that (1) even state-of-the-art VLA models can be easily compromised by simple spotlight illumination, and (2) a high degree of robustness against this attack can be seemingly achieved through naive data augmentation
--- Page 10 ---
4.1 Experimental Setup We conduct experiments within the LeRobot framework [26], utilizing the widely adopted LIBERO multi-task manipulation benchmark [17] While the vision and language backbones utilize standard pre-trained weights, the action expert is trained from scratch We train a Baseline model with no data augmentation and a Naive-Aug model with data augmentation that perturbs input images across multiple visual properties: specifically, ±180◦ in hue, [0. p. 4 While the vision and language backbones utilize standard pre-trained weights, the action expert is
trained from scratch. In this configuration, we train a Baseline model with no data augmentation
and a Naive-Aug model with data augmentation that perturbs input images across multiple visual
properties: specifically, ±180◦
in hue, [0. 1, the adversary can deploy physical lighting
fixtures within the robot’s workspace. When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and
lightness) of the emitted light. In simulation, the
spotlight is rendered directly to optimize these parameters; on hardware, the same parameterization
corresponds to a physical lamp whose pose and output can be adjusted within safe bounds. We parameterize the spotlight by:
θ = [hlight, slight, vlight, I, zlight, αcutoff], (1)
where hlight, slight, vlight denote the hue, saturation, and value of the light source, I is the light intensity, zlight is the spatial height (z-coordinate), and αcutoff represents the spotlight’s cutoff angle. We first introduce a typical naive data
augmentation (we call this baseline Naive-Aug) which is a standard combination of color and sharpness jittering. We denote the augmentation parameters as ϕ = [h, s, v, c, γ]
⊤, representing perturbations on hue, saturation, brightness, contrast, and sharpness, respectively.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 11 ---
Table 2 Grayscale diagnostic evaluation. The high success rate of Naive-Aug on artificially desaturated images shows an over-reliance on shape biases, indicating semantic degradation of color cues
--- Page 12 ---
5 Real-World Experiments To validate the practical implications of our simulated findings We then evaluate the models across both colorinvariant and color-dependent physical manipulation tasks to demonstrate that our ChromaGuard can effectively address the limitation
--- Page 13 ---
5.2 Evaluation of Color Reliance via Grayscale Inference To systematically verify whether the augmented model loses its reliance on color information, we first conduct a grayscale diagnostic evaluation As shown in Table 2, the baseline model suffers a significant drop, its success rates falling to near zero across the suites This phenomenon exposes a critical flaw in naive data augmentation because tasks rarely require disambiguating identically shaped objects solely by color in standard benchmarks
--- Page 14 ---
5.3 Evaluation of Color-Dependent and Color-Invariant Tasks To assess the practical impact of augmentation-induced perceptual degradation, we evaluated the models on physical hardware across invariant and color-dependent tasks ChromaGuard maintains high perceptual fidelity in benign scenarios while ensuring robustness against targeted physical spotlight attacks It successfully preserves semantic fidelity, achieving a 97. p. 6 It bypasses the illumination
challenge not by learning robust color representations, but by collapsing its perception into a purely
shape- and position-biased state, effectively ignoring semantic color cues. 3 Evaluation of Color-Dependent and Color-Invariant Tasks
To assess the practical impact of augmentation-induced perceptual degradation, we evaluated the
models on physical hardware across invariant and color-dependent tasks. FLARE Framework: Vulnerability to Lighting Conditions
Defense Dilemma & Our Solution
Optimized Parameters:
𝜽 = [ℎ, 𝑠, 𝑣,𝐼, 𝑧, 𝛼]
Wrist
camera
3
rd
-person
camera
VLA
Model Crash! Naive Augmentation Proposed: ChromaGuard
Under Attack
Benign
Success
Success
Selects
Wrong Object
Augmented
Inputs
Color- Distorted
Pick up
red ball
Pick up
red ball
ColorPreserved
Illumination
Attack
Robust but Color-Blind Robust and Color-Aware …
…
Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense.Where the paper says it
Where the paper says it
--- Page 15 ---
Table 3 Real-World evaluation of model robustness Base Model Model Color-Invariant Task SR Color-Dependent Task SR Benign Attack Benign Attack SmolVLA Baseline 70.0 0.0 77.5 (11.1) 27.5 (24.1) Naive-Aug 80. p. 3 3 ChromaGuard: Hue-Preserving Adversarial Training
Naive lighting augmentation typically perturbs all color channels indiscriminately, often through
aggressive hue, saturation, or texture-distorting transforms. We first introduce a typical naive data
augmentation (we call this baseline Naive-Aug) which is a standard combination of color and sharpness jittering.Where the paper says it
--- Page 16 ---
Table 3 Real-World evaluation of model robustness Base Model Model Color-Invariant Task SR Color-Dependent Task SR Benign Attack Benign Attack π0.5 Baseline 75. p. 0 Lights, Camera, Malfunction: When Illumination
Robustness Leaves VLA Models Blind to Color
Marino Watanabe Takami Sato Kentaro Yoshioka
Keio University
Abstract: Vision-Language-Action (VLA) models have emerged as a powerful
paradigm for general-purpose robot manipulation; however, their transition to
real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimized physical spotlight attack framework
that exploits these vulnerabilities via targeted illuminations, dropping baseline
task success rates to zero without any access to model internals.Where the paper says it
--- Page 18 ---
Table 3 Real-World evaluation of model robustness Base Model Model Color-Invariant Task SR Color-Dependent Task SR Benign Attack Benign Attack ChromaGuard (Ours) π0.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
ChromaGuard preserves semantic fidelity by restricting augmentation to hue perturbation of zero while allowing bounded perturbations to saturation, brightness, contrast, and sharpness. This prevents naive data augmentation from conditioning VLA models to discard essential color cues, which is a critical failure mode observed in standard adversarial training.
-
The improved system can maintain high performance in benign color-dependent operations while achieving state-of-the-art resilience against dynamic lighting attacks. Specifically, it achieves 97.5% success rates in benign scenarios and 92.5% under attacked color-dependent scenarios on a physical 6DoF robotic platform.
-
The system can be evaluated by diagnosing semantic degradation using grayscale inference, exposing the vulnerability where naive augmentation causes the model to
collapse its visual perception into a purely shape-biased processor.
This diagnostic validation shows that while a model may maintain high success rates on grayscale inputs, its success rate on color-dependent real-world tasks drops to at most 47.5%.
Abstract
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for general-purpose robot manipulation; however, their transition to real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimization framework that exploits these vulnerabilities via targeted illuminations, causing critical disruption of baseline tasks. While adversarial training is the standard countermeasure, we identify a critical and previously underestimated defensive pitfall: naive data augmentations condition VLA models to largely discard color as noise, biasing their visual perception toward shape cues. We expose this degradation through a diagnostic grayscale evaluation, in which a naively defended model maintains high success rates on grayscale inputs, while its success rate on benign, color-dependent real-world tasks drops to at most 48.8%, compared with 78.8% for the undefended SmolVLA baseline. To address this, we propose ChromaGuard, a chroma-preserving adversarial training method. On a physical 6-DoF robotic platform using SmolVLA, we demonstrate that ChromaGuard achieves 95.0% and 88.8% success rates in benign and attacked color-dependent tasks, respectively. Demo videos are available at https://keio-csg.github.io/ChromaGuard/.
Sources
- OpenVLA: An Open-Source Vision-Language-Action Model
- SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Image Data Augmentation for Deep Learning: A Survey
- LeRobot: An Open-Source Library for End-to-End Robot Learning
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation