Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color

arXiv:2607.14698 · cs.RO · Submitted 2026-07-16 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Lights, Camera, Malfunction".

Dev: Here's the gist of the paper. Vision-Language-Action (VLA) models exhibit vulnerabilities to minor environmental perturbations, which this paper addresses by proposing an optimized physical spotlight attack framework and a chroma-preserving adversarial training method <ref:2607.14698#pg6> The proposed ChromaGuard method maintains high performance in benign, color-dependent operations while achieving robustness against severe lighting attacks <ref:2607.14698#pg9> FLARE Framework and Attack Generation The authors introduce FLARE, an optimized physical spotlight attack framework designed to exploit vulnerabilities in VLA models via targeted illuminations <ref:2607.14698#pg4> This framework parameterizes the spotlight using parameters like hue, saturation, value, intensity, spatial height, and cutoff angle <ref:2607.14698#pg5> The attack objective is calculated by combining trajectory deviation and task failure into a single score to find worst-case physical lighting parameters <ref:2607.14698#pg6> Bayesian Optimization is employed to efficiently search the continuous spotlight parameter space for values that maximize this objective score <ref:2607.14698#pg6> The framework allows the adversary to control spatial placement, illumination footprint, intensity, and chromatic properties of the emitted light <ref:2607.14698#pg4> Defensive Pitfalls in Training A critical defensive trap identified is that naive data augmentations incorrectly condition VLA models to discard color as noise <ref:2607.14698#pg6> When applying broad HSV-space data augmentation, the resulting model behaves similarly to one trained exclusively on grayscale images <ref:2607.14698#pg4> This conditioning causes the model to lose the capacity to solve semantic tasks where fine-grained color information plays a critical role <ref:2607.14698#pg6> The authors demonstrate this degradation through a diagnostic grayscale evaluation, showing that defended models maintain high success rates on grayscale inputs but drop to at most 47.5% success on real-world color-dependent tasks <ref:2607.14698#pg4> ChromaGuard Methodology ChromaGuard is proposed as a novel augmentation architecture designed to preserve texture and hue during training <ref:2607.14698#pg6> The method restricts the augmentation parameter space to allow bounded perturbations only to saturation, brightness, contrast, and sharpness while fixing the hue perturbation to zero <ref:2607.14698#pg6> This constrained space is used to simulate challenging lighting conditions via a standard adversarial objective <ref:2607.14698#pg6> ChromaGuard achieves state-of-the-art resilience against dynamic lighting attacks, reaching 70.0% success in color-invariant tasks <ref:2607.14698#pg6> Furthermore, it yields success rates of 97.5% and 92.5% in benign and attacked color-dependent scenarios on a physical platform <ref:2607.14698#pg4> Empirical Results and Implications In simulation, the optimized attack completely compromises the Baseline model, dropping its success rate to 0.0% while inducing severe trajectory deviations up to 115.5 cm <ref:2607.14698#pg8> The authors show that naive data augmentation looks seemingly sufficient because it forces the model to bypass illumination challenges by discarding essential color cues <ref:2607.14698#pg4> Real-world experiments confirm this, where the Naive-Aug model maintains almost identical performance on grayscale inputs as on standard RGB, exposing a critical flaw in naive augmentation <ref:2607.14698#pg6> ChromaGuard successfully resolves the trade-off by decoupling illumination variations from chroma features during training <ref:2607.14698#pg4> It matches the robustness of Naive-Aug in the color-invariant setting while demonstrating substantial improvements in the color-dependent setting <ref:2607.14698#pg6> The study concludes that optimized spotlight perturbations can reliably degrade baseline VLA performance to a 0.0% success rate <ref:2607.14698#pg8> The paper highlights the urgent necessity of achieving a certain level of generalizability of current state-of-the-art VLA models before diving too deeply into their security <ref:2607.14698#pg9> The tools presented serve as analytical tools to expose the acute vulnerability of VLA models to subtle environmental perturbations rather than definitive attack or defense methods <ref:2607.14698#pg4> They do not yet explore temporal dynamics such as time-varying lighting or closed-loop adaptive attacks <ref:2607.14698#pg4> The primary objective is to expose the critical robustness crisis facing current VLA models <ref:2607.14698#pg6> The authors emphasize that naive data augmentation fails to resolve these vulnerabilities by discarding color features rather than learning robust representations <ref:2607.14698#pg4> References

1: B.

Rosa: Z.

Dev: et al.

Rosa: RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control <ref:2607.14698#pg10>

2: M.

Paper summary: Rosa: Looking at "Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color," the paper really centers on this tension between robustness and fidelity in vision language action models. The authors highlight how existing training methods using naive augmentations inadvertently condition the model to discard color as noise.

Dev: And they use a specific attack framework called FLARE, which optimizes spotlight parameters like hue, saturation, value, intensity, spatial height, and cutoff angle to induce catastrophic trajectory deviations in baseline models that have no access to internal weights.

Taro: The main contribution here is pinpointing this critical pitfall: standard augmentations don't just add noise; they fundamentally condition the model to ignore color cues when those cues are essential for tasks. They show this through a grayscale diagnostic evaluation where the defended model only maintained high success rates on grayscale inputs, dropping to at most forty-seven point five percent on real color-dependent tasks <ref:2607.14698#pg1,high success rates on grayscale inputs>.

Rosa: And then they propose ChromaGuard as a solution, which is a chromapreserving adversarial training methodology that restricts augmentation to only bounded perturbations in saturation and brightness while fixing the hue perturbation to zero. This method successfully preserves semantic fidelity, achieving ninety-seven point five percent success under benign conditions and ninety-two point five percent under an attack on a physical platform <ref:2607.14698#pg3>.

Dev: It’s important to note what the authors themselves point out about their work; they emphasize that these tools are analytical, exposing this acute vulnerability to subtle environmental perturbations rather than presenting definitive attack or defense methods yet. They don't explore things like temporal dynamics such as time-varying lighting or closed-loop adaptive attacks.

Taro: So, the implication is that we’re in a crisis with current VLA models; they need better generalizability before we can focus too deeply on security against these kinds of subtle environmental shifts. The paper shows us what's broken and suggests a way to learn more robust representations by decoupling illumination variations from chroma features during training.

Conclusion: Rosa: So, to wrap up this talk about "Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color," we've seen how these models are super fragile when the lighting changes unexpectedly.

Dev: Yeah, it’s about those physical spotlights being a big problem for vision language action models because they can easily crash them.

Taro: Exactly, the authors show that even if you train a model to be tough against lighting, simple color augmentations actually hurt its ability to do the actual task when the colors matter.

Rosa: The core idea here is exposing this vulnerability—how these AI models are blind to subtle changes in light because they learned everything from just grayscale data.

Dev: And the authors propose this ChromaGuard method as a way to fix it, using a specific training technique that keeps the hue consistent while letting other things like saturation and brightness wiggle a bit.

Taro: That ChromaGuard seems promising because they show it can handle those real-world color-dependent tasks, getting ninety seven point five percent success under benign conditions.

Rosa: So what does this mean for us? It means that just making models tougher against random noise isn't enough; we have to train them to understand *what* the colors are important and how they change when the light gets weird.

Dev: And it’s not just about the attack itself, though; it’s about making sure these models can actually run safely in real-time loops without those kind of sudden performance drops during deployment.

Taro: They're pointing out that we need better generalizability before we start worrying too much about building perfect defenses against every possible lighting setup.

Rosa: Right, so the focus shifts from just blocking attacks to learning robust representations that don't care about minor illumination blips at all.

Marino Watanabe Takami Sato, Kentaro Yoshioka

Keio University

cs.RO

Submitted: 2026-07-16

Updated: 2026-11-08

Code: https://github.com/TheRobotStudio/SO-ARM100

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 90/100

The gist: Vision-Language-Action (VLA) models exhibit vulnerabilities to minor environmental perturbations, which this paper addresses by proposing an optimized physical spotlight attack framework and a…

Key concepts

FLARE Framework
This is an optimized framework used to generate the worst-case physical spotlight attacks against VLA models. It systematically searches a wide range of light parameters like hue, saturation, and intensity using Bayesian Optimization to find the lighting settings that cause the most catastrophic failures in model performance.
Naive Data Augmentation
This is a common training technique where input images are randomly perturbed across various visual properties like hue and saturation. The paper shows this method is flawed because it incorrectly conditions VLA models to treat color as noise, which degrades their ability to perform tasks that rely on fine-grained color information.
ChromaGuard
This is a novel adversarial training method designed to make VLA models robust against lighting attacks. It constrains data augmentation only to parameters like saturation and brightness, explicitly fixing the hue perturbation at zero, thereby preserving essential color information during training.

Terminology

Summary

Vision-Language-Action (VLA) models exhibit vulnerabilities to minor environmental perturbations, which this paper addresses by proposing an optimized physical spotlight attack framework and a chroma-preserving adversarial training method <ref:2607. p. 0

Where the paper says it

Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color Marino Watanabe Takami Sato Kentaro Yoshioka Keio University Abstract: Vision-Language-Action (VLA) models have emerged as a powerful paradigm for general-purpose robot manipulation; however, their transition to real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimized physical spotlight attack framework that exploits these vulnerabilities via targeted illuminations, dropping baseline task success rates to zero without any access to model internals.

14698#pg6> The proposed ChromaGuard method maintains high performance in benign, color-dependent operations while achieving robustness against severe lighting attacks <ref:2607. p. 1
Where the paper says it

1, a chromapreserving adversarial training methodology. We demonstrate that our approach not only provides robust defensive capabilities against severe lighting attacks but also maintains high performance in benign, color-dependent operations.

14698#pg9>

FLARE Framework and Attack Generation

The authors introduce FLARE, an optimized physical spotlight attack framework designed to exploit vulnerabilities in VLA models via targeted illuminations <ref:2607.14698#pg4> This framework parameterizes the spotlight using parameters like hue, saturation, value, intensity, spatial height, and cutoff angle <ref:2607. p. 2

Where the paper says it

In simulation, the spotlight is rendered directly to optimize these parameters; on hardware, the same parameterization corresponds to a physical lamp whose pose and output can be adjusted within safe bounds. We parameterize the spotlight by: θ = [hlight, slight, vlight, I, zlight, αcutoff], (1) where hlight, slight, vlight denote the hue, saturation, and value of the light source, I is the light intensity, zlight is the spatial height (z-coordinate), and αcutoff represents the spotlight’s cutoff angle.

14698#pg5> The attack objective is calculated by combining trajectory deviation and task failure into a single score to find worst-case physical lighting parameters <ref:2607. p. 3
Where the paper says it

For episode i, we score a spotlight parameter by combining trajectory deviation and task failure: Li(θ) = 1 S X S t=1 ∥P (i) t − P ′(i) t ∥2 + λ1∥P (i) T − P ′(i) T ∥2 + λ2I (i) f , (2) where S is the maximum number of steps, P (i) t and P ′(i) t are the clean and attacked end-effector positions respectively, I (i) f ∈ {0, 1} indicates a binary task failure, and λ1, λ2 are weighting hyperparameters. We aim to maximize the average objective score over N episodes: J(θ) = 1 N PN i=1 Li(θ).

14698#pg6> Bayesian Optimization is employed to efficiently search the continuous spotlight parameter space for values that maximize this objective score <ref:2607. p. 3
Where the paper says it

We aim to maximize the average objective score over N episodes: J(θ) = 1 N PN i=1 Li(θ). Because physical and simulated rollouts are expensive and internal model gradients are unavailable, we employ Bayesian Optimization to efficiently search the continuous spotlight parameter space for values that maximize J(θ).

14698#pg6> The framework allows the adversary to control spatial placement, illumination footprint, intensity, and chromatic properties of the emitted light <ref:2607. p. 2
Where the paper says it

1, the adversary can deploy physical lighting fixtures within the robot’s workspace. When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and lightness) of the emitted light.

14698#pg4>

Defensive Pitfalls in Training

A critical defensive trap identified is that naive data augmentations incorrectly condition VLA models to discard color as noise <ref:2607.14698#pg6> When applying broad HSV-space data augmentation, the resulting model behaves similarly to one trained exclusively on grayscale images <ref:2607. p. 0

Where the paper says it

However, we find that naive adversarial training does not consistently yield genuine robustness; rather, it often conditions VLA models to entirely ignore the visual features that were perturbed during the training process. For example, when applying broad HSV-space data augmentation to the training data, the resulting VLA model behaves similarly to one trained exclusively on grayscale images.

14698#pg4> This conditioning causes the model to lose the capacity to solve semantic tasks where fine-grained color information plays a critical role <ref:2607. p. 0
Where the paper says it

Consequently, the model loses the capacity to solve semantic tasks where fine-grained color information plays a critical role. arXiv:2607.

14698#pg6> The authors demonstrate this degradation through a diagnostic grayscale evaluation, showing that defended models maintain high success rates on grayscale inputs but drop to at most 47. p. 0
Where the paper says it

While adversarial training is the standard countermeasure, we identify a critical and previously underestimated defensive pitfall: naive data augmentations incorrectly condition VLA models to discard color as noise, collapsing their visual perception into a purely shape-biased processor. We expose this degradation through a diagnostic grayscale evaluation, in which the defended model maintains high success rates on grayscale inputs, while its success rate on benign, color-dependent real-world tasks drops to at most 47.

5% success on real-world color-dependent tasks <ref:2607. p. 0
Where the paper says it

While adversarial training is the standard countermeasure, we identify a critical and previously underestimated defensive pitfall: naive data augmentations incorrectly condition VLA models to discard color as noise, collapsing their visual perception into a purely shape-biased processor. We expose this degradation through a diagnostic grayscale evaluation, in which the defended model maintains high success rates on grayscale inputs, while its success rate on benign, color-dependent real-world tasks drops to at most 47.

14698#pg4>

ChromaGuard Methodology

ChromaGuard is proposed as a novel augmentation architecture designed to preserve texture and hue during training <ref:2607.14698#pg6> The method restricts the augmentation parameter space to allow bounded perturbations only to saturation, brightness, contrast, and sharpness while fixing the hue perturbation to zero <ref:2607. p. 3

Where the paper says it

To isolate illumination effects, ChromaGuard restricts the augmentation parameter space. Specifically, it allows bounded perturbations to saturation, brightness, contrast, and sharpness, but explic4

14698#pg6> This constrained space is used to simulate challenging lighting conditions via a standard adversarial objective <ref:2607. p. 4
Where the paper says it

itly fixes the hue perturbation to zero (h = 0). We use this constrained parameter space, denoted as ΦCG, to simulate challenging lighting conditions via a standard adversarial objective.

14698#pg6> ChromaGuard achieves state-of-the-art resilience against dynamic lighting attacks, reaching 70. p. 1
Where the paper says it

• We propose ChromaGuard, a novel augmentation architecture designed to preserve texture and hue. We empirically prove that this method achieves state-of-the-art resilience against dynamic lighting attacks, reaching 70.

0% success in color-invariant tasks <ref:2607. p. 1
Where the paper says it

We empirically prove that this method achieves state-of-the-art resilience against dynamic lighting attacks, reaching 70. 0% success in color-invariant tasks.

14698#pg6> Furthermore, it yields success rates of 97. p. 1
Where the paper says it

0% success in color-invariant tasks. Furthermore, it strictly retains the perceptual capabilities necessary for complex physical manipulation by yielding success rates of 97.

5% and 92.5% in benign and attacked color-dependent scenarios on a physical platform <ref:2607. p. 0
Where the paper says it

On a physical 6-DoF robotic platform, we demonstrate that ChromaGuard achieves 97. 5% success rates in benign and attacked color-dependent tasks, respectively.

14698#pg4>

Empirical Results and Implications

In simulation, the optimized attack completely compromises the Baseline model, dropping its success rate to 0.0% while inducing severe trajectory deviations up to 115. p. 5

Where the paper says it

The optimized attack completely compromises the Baseline model, dropping its success rate to 0. 0% while inducing severe trajectory deviations of up to 115.

5 cm <ref:2607. p. 0
Where the paper says it

Consequently, the model loses the capacity to solve semantic tasks where fine-grained color information plays a critical role. arXiv:2607.

14698#pg8> The authors show that naive data augmentation looks seemingly sufficient because it forces the model to bypass illumination challenges by discarding essential color cues <ref:2607. p. 4
Where the paper says it

, the VLA effectively degrades into a shape-biased, grayscale processor. By indiscriminately perturbing color channels during training, the augmentation forces the model to bypass the illumination challenge by discarding essential color cues.

14698#pg4> Real-world experiments confirm this, where the Naive-Aug model maintains almost identical performance on grayscale inputs as on standard RGB, exposing a critical flaw in naive augmentation <ref:2607. p. 6
Where the paper says it

As shown in Table 2, the baseline model, which naturally learned color features during training, suffers a significant drop, its success rates falling to near zero across the suites. In contrast, the Naive-Aug model maintains almost identical performance on grayscale inputs as on standard RGB, e.

14698#pg6> ChromaGuard successfully resolves the trade-off by decoupling illumination variations from chroma features during training <ref:2607. p. 7
Where the paper says it

This confirms that defense techniques relying on indiscriminate color perturbation can severely degrade the semantic grounding of color information required for complex real-world tasks. ChromaGuard resolves this trade-off by decoupling illumination variations from chroma features during training.

14698#pg4> It matches the robustness of Naive-Aug in the color-invariant setting while demonstrating substantial improvements in the color-dependent setting <ref:2607. p. 7
Where the paper says it

ChromaGuard resolves this trade-off by decoupling illumination variations from chroma features during training. As shown in Table 3, ChromaGuard matches the robustness of Naive-Aug in the color-invariant setting (≥70% SR under attack) while demonstrating substantial improvements in the color-dependent setting.

14698#pg6>

Conclusion and Future Directions

The study concludes that optimized spotlight perturbations can reliably degrade baseline VLA performance to a 0.0% success rate <ref:2607. p. 0

Where the paper says it

While adversarial training is the standard countermeasure, we identify a critical and previously underestimated defensive pitfall: naive data augmentations incorrectly condition VLA models to discard color as noise, collapsing their visual perception into a purely shape-biased processor. We expose this degradation through a diagnostic grayscale evaluation, in which the defended model maintains high success rates on grayscale inputs, while its success rate on benign, color-dependent real-world tasks drops to at most 47.

14698#pg8> The paper highlights the urgent necessity of achieving a certain level of generalizability of current state-of-the-art VLA models before diving too deeply into their security <ref:2607. p. 7
Where the paper says it

Our physical evaluations on a 6-DoF manipulator indicate that ChromaGuard effectively balances robustness and perceptual fidelity, maintaining reliable success rates under targeted lighting attacks without compromising the visual acuity necessary for semantic manipulation. These findings emphasize the urgent necessity of achieving a certain level of generalizability of the current state-of-the-art VLA models before diving too deeply into their security.

14698#pg9> The tools presented serve as analytical tools to expose the acute vulnerability of VLA models to subtle environmental perturbations rather than definitive attack or defense methods <ref:2607. p. 7
Where the paper says it

6 Limitations Although our FLARE and ChromaGuard demonstrate substantial attack and mitigation capabilities on spotlight attacks, we do not intend to serve them as definitive attack and defense methods. Instead, they serve as analytical tools to expose the acute vulnerability of VLA models to subtle environmental perturbations.

14698#pg4> They do not yet explore temporal dynamics such as time-varying lighting or closed-loop adaptive attacks <ref:2607. p. 7
Where the paper says it

Our analysis reveals that naive data augmentation fails to resolve these vulnerabilities, merely discarding color features rather than learning robust representations. Furthermore, these tools do not yet explore temporal dynamics, such as time-varying lighting or closed-loop adaptive attacks.

14698#pg4> The primary objective is to expose the critical robustness crisis facing current VLA models <ref:2607. p. 7
Where the paper says it

Furthermore, these tools do not yet explore temporal dynamics, such as time-varying lighting or closed-loop adaptive attacks. Our primary objective is to expose the critical robustness crisis facing current VLA models, rather than to engineer an exhaustive adversarial attack or an impenetrable defense.

14698#pg6> The authors emphasize that naive data augmentation fails to resolve these vulnerabilities by discarding color features rather than learning robust representations <ref:2607. p. 7
Where the paper says it

Instead, they serve as analytical tools to expose the acute vulnerability of VLA models to subtle environmental perturbations. Our analysis reveals that naive data augmentation fails to resolve these vulnerabilities, merely discarding color features rather than learning robust representations.

14698#pg4>

References

[1] B. Z. et al. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control <ref:2607. p. 8

Where the paper says it

References [1] B. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control.

14698#pg10>

[2] M. J. Kim, K. Pertsch, S. p. 8

Where the paper says it

In PMLR, volume 229, pages 2165–2183, 2023. Pertsch, S.

Karamcheti, T. p. 8
Where the paper says it

Pertsch, S. Karamcheti, T.

Xiao, A. Balakrishna, S. p. 8
Where the paper says it

Karamcheti, T. Balakrishna, S.

Nair, R. Rafailov, E. p. 8
Where the paper says it

Balakrishna, S. Rafailov, E.

Foster, G. Lam, P. Sanketi

[3] M. Shukor, D. Aubakirova, F. p. 8

Where the paper says it

09246, 2024. Aubakirova, F.

Capuano, P. p. 8
Where the paper says it

Aubakirova, F. Capuano, P.

Kooijmans, S. p. 8
Where the paper says it

Capuano, P. Kooijmans, S.

Palma, A. Zouitine

[4] K. B. et al. π0: A Vision-Language-Action Flow Model for General Robot Control

[5] Y. Jiang, Y. Ze, Z. Jiang, S. Song, and J. p. 8

Where the paper says it

org/abs/2410. Song, and J.

Wu

[6] S. Jiang, Z. Huang, K. Qian, Z. Luo, T. Zhu

[7] S. Li, Y. Chen, D. Li, S. Liu, D. Lan

[8] Q. Li, B. Yin, W. Huang, R. Liu

[9] B. Ma, H. Guo, P. Lv, M. Xu

[10] X. Lu, J. Chen, S. Xiao, Z. Jin

[11] H. Xu, Y. S. Koh, S. Huang

[12] X. Wang, J. Li, Z. Weng, Y. Wang

[13] X. Zhou, G. Tie, G. Zhang, H. Wang

[14] J. Zhou, Y. Wei, R. Zhen, B. Zhao

[15] N. Zhang, W. Tao, X. Xiao, Q. Sun

[16] X. Zhou, Y. Xu, G. Tie, Y. Chen

[17] B. Liu, Y. Zhu, C. Gao, Y. Feng

[18] D. Y. G. Orjuela, L. p. 9

Where the paper says it

Advances in Neural Information Processing Systems, 36:44776–44791, 2023. Orjuela, L.

Scappatura, V. p. 9
Where the paper says it

Orjuela, L. Scappatura, V.

Di Gennaro

[19] J. Guo, Z. Wu, C. Tu, Y. Ma

[20] A. Madry, A. Makelov, L. p. 9

Where the paper says it

On Robustness of Vision-Language-Action Model Against Multi-Modal Perturbations, 2025. Makelov, L.

Schmidt

[21] S. Xhonneux, A. p. 9

Where the paper says it

06083, 2017. Xhonneux, A.

Sordoni, S. p. 9
Where the paper says it

Xhonneux, A. Sordoni, S.

Gunnemann

[22] Y. Ganin, E. Ustinova, H. p. 9

Where the paper says it

Advances in Neural Information Processing Systems, 37:1502–1530, 2024. Ustinova, H.

Ajakan

[23] A. Hanif, M. Naseer, S. Khan

[24] S. Yang, W. Xiao, M. Zhang, S. Guo

[25] T. Akiba, S. Sano, T. Yanase

[26] LeRobot: An open-source library for end-to-end robot learning

[27] E. Todorov, T. p. 9

Where the paper says it

22818, 2026. Todorov, T.

Erez, and Y. p. 9
Where the paper says it

Todorov, T. Erez, and Y.

Tassa

[28] R. Knight, P. Kooijmans, R. p. 8

Where the paper says it

Capuano, P. Kooijmans, S.

Cadene

[29] P. Intelligence, K. p. 8

Where the paper says it

Transferring Vision-LanguageAction Models to Industry Applications: Architectures, Performance, and Challenges. In 2025 7th International Conference on Industrial Artificial Intelligence (IAI), pages 1–6.

Black, N. Brown

--- Page 1 ---

Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color

Marino Watanabe Takami Sato Kentaro Yoshioka Keio University

--- Page 2 ---

FLARE Framework: Vulnerability to Lighting Conditions Defense Dilemma & Our Solution Optimized Parameters: θ = [h, s, v,I, z, α] Wrist camera 3rd-person camera VLA Model Crash! Naive Augmentation Proposed: ChromaGuard Under Attack Benign Success Success Selects Wrong Object Augmented Inputs Color- Distorted Pick up red ball Pick up red ball ColorPreserved Illumination Attack Robust but Color-Blind Robust and Color-Aware … Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense. p. 1

Where the paper says it

FLARE Framework: Vulnerability to Lighting Conditions Defense Dilemma & Our Solution Optimized Parameters: 𝜽 = [ℎ, 𝑠, 𝑣,𝐼, 𝑧, 𝛼] Wrist camera 3 rd -person camera VLA Model Crash! Naive Augmentation Proposed: ChromaGuard Under Attack Benign Success Success Selects Wrong Object Augmented Inputs Color- Distorted Pick up red ball Pick up red ball ColorPreserved Illumination Attack Robust but Color-Blind Robust and Color-Aware … … Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense.

Top: Our FLARE framework optimizes spotlight parameters to induce catastrophic trajectory deviations in VLA models. p. 1
Where the paper says it

Naive Augmentation Proposed: ChromaGuard Under Attack Benign Success Success Selects Wrong Object Augmented Inputs Color- Distorted Pick up red ball Pick up red ball ColorPreserved Illumination Attack Robust but Color-Blind Robust and Color-Aware … … Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense. Top: Our FLARE framework optimizes spotlight parameters to induce catastrophic trajectory deviations in VLA models.

Bottom: The defense dilemma where naive color augmentation degrades task success on color-dependent tasks, which is resolved by our proposed ChromaGuard approach. p. 1
Where the paper says it

Top: Our FLARE framework optimizes spotlight parameters to induce catastrophic trajectory deviations in VLA models. Bottom: The defense dilemma where naive color augmentation degrades task success on color-dependent tasks, which is resolved by our proposed ChromaGuard approach.

--- Page 3 ---

2 Background 2.1 Security and Robustness of VLA Models against Physical Attacks The multimodal nature of VLAs exposes a wide variety of attack channels [8, 9] such as trainingtime backdoor attacks [13, 14], inference-time adversarial attacks [11, 15, 12]. p. 1

Where the paper says it

2 Background 2. 1 Security and Robustness of VLA Models against Physical Attacks The multimodal nature of VLAs exposes a wide variety of attack channels [8, 9] such as trainingtime backdoor attacks [13, 14], inference-time adversarial attacks [11, 15, 12].

Meanwhile, these advanced attacks naturally extend the prior attacks effective against general DNN models. p. 1
Where the paper says it

1 Security and Robustness of VLA Models against Physical Attacks The multimodal nature of VLAs exposes a wide variety of attack channels [8, 9] such as trainingtime backdoor attacks [13, 14], inference-time adversarial attacks [11, 15, 12]. Meanwhile, these advanced attacks naturally extend the prior attacks effective against general DNN models.

The ma2jor contributions of these previous works are in overcoming the major challenges in triggering the vulnerabilities in their VLA domains. p. 2
Where the paper says it

jor contributions of these previous works are in overcoming the major challenges in triggering the vulnerabilities in their VLA domains. In this work, we highlight that VLA models are indeed vulnerable to these kinds of attacks, but the current state-of-the-art VLA models are poorly generalized and are just vulnerable to physical environmental variations, rather than particularly vulnerable to these advanced attacks.

LIBERO-PRO [16] also reports that four major state-of-the-art VLA models easily collapse under small task and position perturbations on the LIBERO [17], one of the most popular VLA benchmarks. p. 2
Where the paper says it

In this work, we highlight that VLA models are indeed vulnerable to these kinds of attacks, but the current state-of-the-art VLA models are poorly generalized and are just vulnerable to physical environmental variations, rather than particularly vulnerable to these advanced attacks. LIBERO-PRO [16] also reports that four major state-of-the-art VLA models easily collapse under small task and position perturbations on the LIBERO [17], one of the most popular VLA benchmarks.

All models achieve ≥90% success rates on the original LIBERO tasks but fall to nearly 0% under the perturbations, showing poor generalizability. p. 2
Where the paper says it

LIBERO-PRO [16] also reports that four major state-of-the-art VLA models easily collapse under small task and position perturbations on the LIBERO [17], one of the most popular VLA benchmarks. All models achieve ≥90% success rates on the original LIBERO tasks but fall to nearly 0% under the perturbations, showing poor generalizability.

Under these circumstances, any attacks may succeed. p. 2
Where the paper says it

All models achieve ≥90% success rates on the original LIBERO tasks but fall to nearly 0% under the perturbations, showing poor generalizability. Under these circumstances, any attacks may succeed.

This motivates us to start by investigating the impact of lighting conditions, which can be perturbed in normal benign scenarios. p. 2
Where the paper says it

Under these circumstances, any attacks may succeed. This motivates us to start by investigating the impact of lighting conditions, which can be perturbed in normal benign scenarios.

--- Page 4 ---

3 Methodology 3.1 Threat Model We follow the common threat model adopted in prior work [10], which assumes a strict black-box setup, i. p. 2

Where the paper says it

3 Methodology 3. 1 Threat Model We follow the common threat model adopted in prior work [10], which assumes a strict black-box setup, i.

p. 2
Where the paper says it

3 Methodology 3. 1 Threat Model We follow the common threat model adopted in prior work [10], which assumes a strict black-box setup, i.

e., the adversary has no access to the target model’s internal architecture, weights, gradient information, or training datasets. p. 2
Where the paper says it

1 Threat Model We follow the common threat model adopted in prior work [10], which assumes a strict black-box setup, i. , the adversary has no access to the target model’s internal architecture, weights, gradient information, or training datasets.

As depicted in Fig. p. 1
Where the paper says it

Bottom: The defense dilemma where naive color augmentation degrades task success on color-dependent tasks, which is resolved by our proposed ChromaGuard approach. To resolve this defense dilemma, we introduce ChromaGuard, as depicted in Fig.

2, the adversary can deploy physical lighting fixtures within the robot’s workspace When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and lightness) of the emitted light However, they cannot dynamically change the light once deployed; the parameters remain fixed across all evaluation episodes. p. 2
Where the paper says it

When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and lightness) of the emitted light. However, they cannot dynamically change the light once deployed; the parameters remain fixed across all evaluation episodes.

p. 2
Where the paper says it

When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and lightness) of the emitted light. However, they cannot dynamically change the light once deployed; the parameters remain fixed across all evaluation episodes.

--- Page 5 ---

3.2 FLARE Framework: Optimized Physical Spotlight Attack We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in Fig. p. 2

Where the paper says it

However, they cannot dynamically change the light once deployed; the parameters remain fixed across all evaluation episodes. 2 FLARE Framework: Optimized Physical Spotlight Attack We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in Fig.

p. 2
Where the paper says it

However, they cannot dynamically change the light once deployed; the parameters remain fixed across all evaluation episodes. 2 FLARE Framework: Optimized Physical Spotlight Attack We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in Fig.

2, to generate worst-case lighting scenarios We parameterize the spotlight by: θ = [hlight, slight, vlight, I, zlight, αcutoff], (1) where hlight, slight, vlight denote the hue, saturation and value of the light source Given a clean observation ot t=A(ot; θ), which alters illumination while preserving physical scene geometry and object layout The attack objective follows the intuition that dangerous failures are not only unsuccessful but also catastrophically distant from the intended trajectory

--- Page 6 ---

3.3 ChromaGuard: Hue-Preserving Adversarial Training Naive lighting augmentation typically perturbs all color channels indiscriminately, often through aggressive hue, saturation, or texture-distorting transforms We first introduce a typical naive data augmentation (we call this baseline Naive-Aug) which is a standard combination of color and sharpness jittering While this approach improves general visual robustness, unconstrained hue variations (h ̸= 0) can alter intrinsic object colors, inadvertently confounding performance on strictly color-dependent tasks

--- Page 7 ---

3.1 Threat Model We follow the common threat model adopted in prior work [10], which assumes a strict black-box setup, i.e., the adversary has no access to the target model’s internal architecture, weights, gradient information, or training datasets As depicted in Fig. p. 2

Where the paper says it

, the adversary has no access to the target model’s internal architecture, weights, gradient information, or training datasets. As depicted in Fig.

2, the adversary can deploy physical lighting fixtures within the robot’s workspace When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and lightness) of the emitted light However, they cannot dynamically change the light once deployed; the parameters remain fixed across all evaluation episodes.

--- Page 8 ---

3.2 FLARE Framework: Optimized Physical Spotlight Attack We propose the Framework for Lighting Adversarial Robustness Evaluation (FLARE), illustrated in Fig. 2, to generate worst-case lighting scenarios We parameterize the spotlight by: θ = [hlight, slight, vlight, I, zlight, αcutoff], (1) where hlight, slight, vlight denote the hue, saturation and value of the light source Given a clean observation ot t=A(ot; θ), which alters illumination while preserving physical scene geometry and object layout The attack objective follows the intuition that dangerous failures are not only unsuccessful but also catastrophically distant from the intended trajectory

--- Page 9 ---

4 Simulator Experiments: Exposing Vulnerability against Spotlight Attack We evaluate the performance of our VLA model against spotlight attacks within a simulated environment In this section, we demonstrate that (1) even state-of-the-art VLA models can be easily compromised by simple spotlight illumination, and (2) a high degree of robustness against this attack can be seemingly achieved through naive data augmentation

--- Page 10 ---

4.1 Experimental Setup We conduct experiments within the LeRobot framework [26], utilizing the widely adopted LIBERO multi-task manipulation benchmark [17] While the vision and language backbones utilize standard pre-trained weights, the action expert is trained from scratch We train a Baseline model with no data augmentation and a Naive-Aug model with data augmentation that perturbs input images across multiple visual properties: specifically, ±180◦ in hue, [0. p. 4

Where the paper says it

While the vision and language backbones utilize standard pre-trained weights, the action expert is trained from scratch. In this configuration, we train a Baseline model with no data augmentation and a Naive-Aug model with data augmentation that perturbs input images across multiple visual properties: specifically, ±180◦ in hue, [0.

0, 4.0] in saturation, [0. p. 2
Where the paper says it

1, the adversary can deploy physical lighting fixtures within the robot’s workspace. When they place the light, they can control the spatial placement, illumination footprint, intensity, and the specific chromatic properties (hue, saturation, and lightness) of the emitted light.

2, 3.0] in value, [0. p. 2
Where the paper says it

In simulation, the spotlight is rendered directly to optimize these parameters; on hardware, the same parameterization corresponds to a physical lamp whose pose and output can be adjusted within safe bounds. We parameterize the spotlight by: θ = [hlight, slight, vlight, I, zlight, αcutoff], (1) where hlight, slight, vlight denote the hue, saturation, and value of the light source, I is the light intensity, zlight is the spatial height (z-coordinate), and αcutoff represents the spotlight’s cutoff angle.

8, 1.2] in contrast, and [0. p. 3
Where the paper says it

We first introduce a typical naive data augmentation (we call this baseline Naive-Aug) which is a standard combination of color and sharpness jittering. We denote the augmentation parameters as ϕ = [h, s, v, c, γ] ⊤, representing perturbations on hue, saturation, brightness, contrast, and sharpness, respectively.

5, 1.5] in sharpness We then evaluate these models across three diverse task suites: LIBERO-Spatial, LIBERO-Object, and LIBERO-10

--- Page 11 ---

Table 2 Grayscale diagnostic evaluation. The high success rate of Naive-Aug on artificially desaturated images shows an over-reliance on shape biases, indicating semantic degradation of color cues

--- Page 12 ---

5 Real-World Experiments To validate the practical implications of our simulated findings We then evaluate the models across both colorinvariant and color-dependent physical manipulation tasks to demonstrate that our ChromaGuard can effectively address the limitation

--- Page 13 ---

5.2 Evaluation of Color Reliance via Grayscale Inference To systematically verify whether the augmented model loses its reliance on color information, we first conduct a grayscale diagnostic evaluation As shown in Table 2, the baseline model suffers a significant drop, its success rates falling to near zero across the suites This phenomenon exposes a critical flaw in naive data augmentation because tasks rarely require disambiguating identically shaped objects solely by color in standard benchmarks

--- Page 14 ---

5.3 Evaluation of Color-Dependent and Color-Invariant Tasks To assess the practical impact of augmentation-induced perceptual degradation, we evaluated the models on physical hardware across invariant and color-dependent tasks ChromaGuard maintains high perceptual fidelity in benign scenarios while ensuring robustness against targeted physical spotlight attacks It successfully preserves semantic fidelity, achieving a 97. p. 6

Where the paper says it

It bypasses the illumination challenge not by learning robust color representations, but by collapsing its perception into a purely shape- and position-biased state, effectively ignoring semantic color cues. 3 Evaluation of Color-Dependent and Color-Invariant Tasks To assess the practical impact of augmentation-induced perceptual degradation, we evaluated the models on physical hardware across invariant and color-dependent tasks.

5% SR under benign conditions and 92. p. 1
Where the paper says it

FLARE Framework: Vulnerability to Lighting Conditions Defense Dilemma & Our Solution Optimized Parameters: 𝜽 = [ℎ, 𝑠, 𝑣,𝐼, 𝑧, 𝛼] Wrist camera 3 rd -person camera VLA Model Crash! Naive Augmentation Proposed: ChromaGuard Under Attack Benign Success Success Selects Wrong Object Augmented Inputs Color- Distorted Pick up red ball Pick up red ball ColorPreserved Illumination Attack Robust but Color-Blind Robust and Color-Aware … … Figure 1: Full Pipeline of Our Proposed FLARE Framework and ChromaGuard Defense.

5% under attack

--- Page 15 ---

Table 3 Real-World evaluation of model robustness Base Model Model Color-Invariant Task SR Color-Dependent Task SR Benign Attack Benign Attack SmolVLA Baseline 70.0 0.0 77.5 (11.1) 27.5 (24.1) Naive-Aug 80. p. 3

Where the paper says it

3 ChromaGuard: Hue-Preserving Adversarial Training Naive lighting augmentation typically perturbs all color channels indiscriminately, often through aggressive hue, saturation, or texture-distorting transforms. We first introduce a typical naive data augmentation (we call this baseline Naive-Aug) which is a standard combination of color and sharpness jittering.

0 70.0 47.5 (85.7) 40.0 (83.3) ChromaGuard (Ours) 80/92/97/55 SmolVLA

--- Page 16 ---

Table 3 Real-World evaluation of model robustness Base Model Model Color-Invariant Task SR Color-Dependent Task SR Benign Attack Benign Attack π0.5 Baseline 75. p. 0

Where the paper says it

Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color Marino Watanabe Takami Sato Kentaro Yoshioka Keio University Abstract: Vision-Language-Action (VLA) models have emerged as a powerful paradigm for general-purpose robot manipulation; however, their transition to real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimized physical spotlight attack framework that exploits these vulnerabilities via targeted illuminations, dropping baseline task success rates to zero without any access to model internals.

0 30.0 47.5 (66.7) 12.5 (25.7) Naive-Aug 75/75/40/47 SmolVLA

--- Page 18 ---

Table 3 Real-World evaluation of model robustness Base Model Model Color-Invariant Task SR Color-Dependent Task SR Benign Attack Benign Attack ChromaGuard (Ours) π0.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. ChromaGuard preserves semantic fidelity by restricting augmentation to hue perturbation of zero while allowing bounded perturbations to saturation, brightness, contrast, and sharpness. This prevents naive data augmentation from conditioning VLA models to discard essential color cues, which is a critical failure mode observed in standard adversarial training.

  2. The improved system can maintain high performance in benign color-dependent operations while achieving state-of-the-art resilience against dynamic lighting attacks. Specifically, it achieves 97.5% success rates in benign scenarios and 92.5% under attacked color-dependent scenarios on a physical 6DoF robotic platform.

  3. The system can be evaluated by diagnosing semantic degradation using grayscale inference, exposing the vulnerability where naive augmentation causes the model to collapse its visual perception into a purely shape-biased processor. This diagnostic validation shows that while a model may maintain high success rates on grayscale inputs, its success rate on color-dependent real-world tasks drops to at most 47.5%.

Abstract

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for general-purpose robot manipulation; however, their transition to real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimization framework that exploits these vulnerabilities via targeted illuminations, causing critical disruption of baseline tasks. While adversarial training is the standard countermeasure, we identify a critical and previously underestimated defensive pitfall: naive data augmentations condition VLA models to largely discard color as noise, biasing their visual perception toward shape cues. We expose this degradation through a diagnostic grayscale evaluation, in which a naively defended model maintains high success rates on grayscale inputs, while its success rate on benign, color-dependent real-world tasks drops to at most 48.8%, compared with 78.8% for the undefended SmolVLA baseline. To address this, we propose ChromaGuard, a chroma-preserving adversarial training method. On a physical 6-DoF robotic platform using SmolVLA, we demonstrate that ChromaGuard achieves 95.0% and 88.8% success rates in benign and attacked color-dependent tasks, respectively. Demo videos are available at https://keio-csg.github.io/ChromaGuard/.

Sources

Related papers