When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving

The gist The DriveDegrade benchmark targets detection or bird’s-eye-view perception, not the planning output that drives the car

Introduction and Motivation

Camera-only end-to-end (E2E) driving models are nearing deployment where the camera stream is degraded by blur, noise, low light, weather, frame loss, and memory faults How much a policy tolerates before its driving breaks is unclear Corruption-robustness benchmarks target detection or bird’s-eye-view perception rather than the planning output that drives the car DriveDegrade is a benchmark for image-degradation tolerance in camera-only E2E driving

DriveDegrade Methodology

The benchmark injects sixteen corruption families at five severities on the fly inside the image loader, one operator reaching fifteen policies, and evaluates open-loop planning on nuScenes and NAVSIM plus a CARLA closed-loop anchor The injector depends only on numpy/cv2 which is what lets one implementation serve six model environments and two simulators unchanged Six families are deterministic functions of the input frame so every model receives bit-identical pixels

Corruption Taxonomy and Severity

The taxonomy follows ImageNet-C and nuScenesC/RoboBEV across four groups: sensor/quality, weather/environmental, temporal/multi-camera, and approximate-memory A fifth group specific to a camera carried on a vehicle rather than inherited from generic image robustness comprises occlusion, mud splash, raindrop, and lens dirt Every family is defined by a single physical parameter and the ladder is therefore a specification rather than an ordinal label The endpoints were fixed before running any model by one rule where S1 is a condition a deployed camera meets routinely, S5 is where a human would call the view unusable, bracketing the knee rather than centering it

Evaluation and Findings

The evaluation uses retention rather than raw degraded L2 because the planners have different clean baselines and in two cases different official valid-planning subsets The primary comparison is the shape of degradation The analysis locates, per corruption, the severity at which planning starts to break over sixteen families Blur and noise are most fragile (s∗10 ≈ 2.2–2.5 for motion/gaussian blur) while weather and bit-error are tolerated across the whole range Perception reliance differs sharply across architectures, with VAD and LAW crossing no threshold at all Quality versus removal is ambiguous at its flat end, which is exactly where one would otherwise declare a winner The gap between the axes is itself informative, largest for planners that tolerate bad pixels but not missing ones

Closed-Loop Validation

The CARLA anchor confirms camera loss remains load-bearing under feedback Complete case averaging is optimistic exactly where the benchmark bites, so we bound the unscored runs rather than drop them Blackout S5 costs every planner a quarter to two thirds of its score The ordering within the image-quality group does not hold across all conditions

Conclusion

DriveDegrade turns camera degradation from a perception-only robustness question into a planning-level tolerance measurement It locates clear breakpoints for blur, JPEG, noise, and camera loss while weather and bit-error are often tolerated The CARLA anchor confirms camera loss remains load-bearing under feedback The injector, hooks, configs, scripts, and per-cell CSVs will be released at https://github.com/UGAMOBILITY- LAB/DriveDegrade, MIT licensed and with no raw sensor data Every cell is traceable to a checkpoint, split, corruption, severity, and seed The paper provides an artifact repository for reproducibility on CPU

References

[1] Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Joshua Tenenbaum and Boris Katz. When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving

[2] Holger Caesar et al. nuScenes: A multimodal dataset for autonomous driving

[3] Holger Caesar et al. DriveDegrade Overview. DriveDegrade overview.

[4] Ming-Fang Chang et al. Table 1. DriveDegrade versus existing robustness and evaluation benchmarks.

[5] Xuewu Lin et al. Quality versus removal

[6] Wei Jiang et al. Conclusion

--- Page 1 ---

When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving

Abstract

Camera-only end-to-end (E2E) driving models are nearing deployment where the camera stream is degraded by blur, noise, low light, weather, frame loss, and memory faults How much a policy tolerates before its driving breaks is unclear Corruption-robustness benchmarks target detection or bird’s-eye-view perception rather than the planning output that drives the car We present DriveDegrade, a benchmark for image-degradation tolerance in camera-only E2E driving Sixteen corruption families at five severities are injected on the fly inside the image loader, one operator reaching fifteen policies, and we evaluate open-loop planning on nuScenes and NAVSIM plus a CARLA closed-loop anchor First, mild degradation barely affects planning, and the families that break it have a clear threshold at mid severity Second, fragility is corruption-dependent: blur, JPEG, and raindrop damage planning most, while weather and bit error are tolerated far into the range Third, a flat curve is ambiguous so we separate corruptions that degrade the image from those that remove it A planner that reads its camera must lose accuracy when information is deleted whatever it does under quality loss On these two axes the planners separate sharply quantifying the ego-status shortcut without mistaking indifference for robustness A released visionlanguage-action planner is flat on both axes and blinding all six of its cameras costs it only 11.5%

--- Page 2 ---

Figure 1. DriveDegrade overview. Surround-view camera streams are degraded on the fly inside the image loader over a five-level severity ladder Six camera-input E2E planners are evaluated at the planning level then summarized through tolerance curves, a vision-reliance diagnostic separating quality degradation from information removal, and safety/system metrics The band beneath lists the nine further policies the identical injector reaches under their own protocols All sixteen corruption families are specified in Supp. B driving-specific contamination families whose damage is spatially concentrated rather than global

--- Page 3 ---

Table 1. DriveDegrade versus existing robustness and evaluation benchmarks. Perception benchmarks stop at perception; planning benchmarks do not corrupt the image RobustE2E [28] and Bench2Drive-Robust [78] evaluate planning under perturbation and both report graded severities Our plan counts evaluated E2E policies, “many” where a benchmark hosts an open leaderboard rather than a fixed set

--- Page 4 ---

Figure 2. Per-corruption image-degradation tolerance. (A) The sixteen families on one nuScenes front-camera frame at S4, clean for reference; the bottom row is the four driving-specific families L2-retention curves for all six planners ordered by UniAD fragility S5 retention for every corruption and planner Blur, JPEG, and noise damage planning most; weather and bit-error are tolerated OpenDriveVLA (squares) sits in [0.909, 0.989] on all sixteen

--- Page 5 ---

Table 3. Vision reliance on two axes. S5 L2 retention under corruptions that degrade the image versus those that remove it VAD and LAW are quality-immune yet still lose accuracy when a camera is removed, so their flat curves measure insensitivity to quality, not blindness The gap between the axes is itself informative largest for planners that tolerate bad pixels but not missing ones

--- Page 6 ---

Figure 3. Mean L2 retention vs. severity. The slope indexes camera reliance: the full-stack and diffusion planners fall fastest, the vectorized and latent-world-model planners stay flat, an insensitivity to image quality that Table 3 separates from disregard of the camera

--- Page 7 ---

Table 4. PDMS under degradation (NAVSIM navtest, 12,147 scenes per cell).

1. Robust planning against image degradation: The system can maintain accuracy when camera input is degraded by factors like blur, JPEG, and noise, as these families are identified as most fragile in Section 4.1. This allows for reliable operation under real-world sensor impairments where perception might otherwise fail or provide misleading information.

2. Distinguishing quality degradation from information removal: The improved system can determine if a failure is due to quality loss versus information removal, as the paper shows that a planner must lose accuracy when information is deleted, whatever it does under quality loss (Section 4.2). This enables designers to understand whether the planner's failure is due to poor image detail or complete occlusion.

3. Adaptive reliance assessment: The system can quantify ego-status shortcut without mistaking indifference for robustness by analyzing the flatcurve ambiguity (Section 4.2). This provides an interpretable measure of camera reliance, separating planners that ignore input from those that are merely insensitive to image quality.

4. Targeted mitigation strategy: Based on the identified corruption families, the system can prioritize mitigation efforts; for instance, it can be designed to handle weather and bit-error [that] are tolerated far into the range while focusing engineering resources on mitigating failures like raindrop, which is identified as the most damaging family in the benchmark for camera-reliant planners (Section 4.5).

5. Enhanced closed-loop performance: When operating in a feedback loop, the system can leverage its robustness; the CARLA anchor shows that camera loss remains load-bearing under feedback (Section 4.9), meaning planning accuracy is preserved when visual input is degraded, unlike open-loop evaluations where reliance on ego status can be misleading.

Camera-only end-to-end (E2E) driving models are nearing deployment, where the camera stream is degraded by blur, noise, low light, weather, frame loss, and memory faults. How much a policy tolerates before its driving breaks is unclear. Corruption-robustness benchmarks target detection or bird's-eye-view perception, not the planning output that drives the car. We present DriveDegrade, a benchmark for image-degradation tolerance in camera-only E2E driving. Sixteen corruption families at five severities are injected on the fly inside the image loader, one operator reaching fifteen policies, and we evaluate open-loop planning on nuScenes and NAVSIM plus a CARLA closed-loop anchor. First, mild degradation barely affects planning, and the families that break it have a clear threshold at mid severity. Second, fragility is corruption-dependent: blur, JPEG, and raindrop damage planning most, while weather and bit error are tolerated far into the range. Third, a flat curve is ambiguous, so we separate corruptions that degrade the image from those that remove it. A planner that reads its camera must lose accuracy when information is deleted, whatever it does under quality loss. On these two axes the planners separate sharply, quantifying the ego-status shortcut without mistaking indifference for robustness. A released vision-language-action planner is flat on both axes, and blinding all six of its cameras costs it only 11.5 percent.