FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog".
Jane: Here's the gist of the paper. Perception under adverse weather conditions remains a critical bottleneck for reliable autonomous driving, yet existing benchmarks lack the systematic multi-modal alignments required to evaluate robust sensor fusion The paper presents FogDrive, a rigorously calibrated, multi-modal autonomous driving dataset designed to bridge data-centric engineering and robust machine learning Dataset Composition and Calibration FogDrive is a CARLA-based synthetic multi-modal benchmark compiled using the CARLA simulator It contains 660 scenes (approximately 133k fully annotated frames with a 50:50 day/night split) across four synchronized cameras (RGB, depth, semantic segmentation), a LiDAR and semantic-LiDAR pair, and front radar Crucially, physically consistent fog is independently modeled on camera channels (via the Koschmieder model) and LiDAR channels (via the Beer-Lambert law) across three calibrated visibility densities (160m, 100m, and 50m) Every scene is provided in four perfectly matched variants (clean plus three graded fog levels) with cross-calibrated 2D and 3D bounding boxes Data Quality Assurance To ensure the highest standard of open science reliability, a semantic-segmentation-based quality audit across 8k images validates our annotations at 95.1% precision and over 99% recall for vehicles within 40m The dataset provides synchronized multi-modal observations, including multi-view RGB images with corresponding depth and semantic segmentation, LiDAR, semantic LiDAR, and radar measurements The depth PNG encodes distance into a 24-bit-per-pixel value with sub-millimetre precision which is relevant in fog because depth errors propagate into image transmission via t = exp(−β d) Benchmarking Findings in 3D Detection Experiment 1 investigated whether training a 3D LiDAR / camera-LiDAR detector on a single fog density generalises across fog densities, and whether mixing densities at training time helps Results showed that mixing fog densities at training time outperforms single-density Light training on raw detection rate at every test fog level Furthermore, scaling the mix specifically improves the geometric accuracy of each detection, with Mixed-L sweeping Mean TP across all test densities and posts a clean ATE sweep with the largest single gain being a ∼4cm reduction in mean translation error under moderate fog Benchmarking Findings in 2D Detection Experiment 2 examined the 2D image-detection counterpart, finding that training directly on foggy data approaches the clean upper bound and consistently outperforms defog-thendetect Standard reference-based image-quality metrics (PSNR, SSIM) do not predict downstream detection utility The paper concludes that the best option remains direct foggy training (Pipeline D) when retraining on the target domain is infeasible Limitations of the Dataset The dataset's scope is bounded by several decisions, including focusing on vehicle classes 0, 1, and 4 while excluding two-wheelers from the spawnable blueprint pool Fog is modeled for the camera and LiDAR channels at three calibrated densities while radar is released without an additional fog degradation by design The validation audit identifies a residual false-positive source of a ≈5 % true-phantom rate driven by multi-level geometry in CARLA’s urban maps Empty-frame frequency is high, with the front-camera view containing no annotated vehicles in roughly one frame in five FogDrive enables further investigation of perception under controlled visibility degradation through its paired structure and calibrated densities The dataset, access class, and audit protocol will be released upon acceptance Page 1 --- arXiv:2607.22698v1
cs.CV: 18 Jul 2026 PANWAR: FOGDRIVE 1 FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog Vansh Panwar p.vansh@iitg.ac.in Indian Institute of Technology, Guwahati Abstract Perception under adverse weather conditions remains a critical bottleneck for reliable autonomous driving, yet existing benchmarks lack the systematic multi-modal alignments required to evaluate robust sensor fusion Real-world weather datasets suffer from uncontrolled collection and single-level, uncalibrated conditions Meanwhile, synthetic alternatives either focus strictly on camera-only restoration or lack the paired cleanand-foggy structures needed to benchmark "defog-then-detect" pipelines We present FogDrive, a rigorously calibrated, multi-modal autonomous driving dataset designed to bridge data-centric engineering and robust machine learning Compiled using the CARLA simulator, FogDrive contains 660 scenes (approximately 133k fully annotated frames with a 50:50 day/night split) across four synchronized cameras (RGB, depth, semantic segmentation), a LiDAR and semantic-LiDAR pair, and front radar Crucially, physically consistent fog is independently modeled on camera channels (via the Koschmieder model) and LiDAR channels (via the Beer-Lambert law) across three calibrated visibility densities (160m, 100m, and 50m) Every scene is provided in four perfectly matched variants (clean plus three graded fog levels) with cross-calibrated 2D and 3D bounding boxes To ensure the highest standard of open science reliability, a semantic-segmentation-based quality audit across 8k images validates our annotations at 95.1% precision and over 99% recall for vehicles within 40m Finally, we establish comprehensive baseline benchmarks using state-of-the-art architectures (TransFusion, BEVFusion, and YOLOv8-m) across two distinct paradigms: 3D multi-modal fusion and 2D image restoration Our major contributions are: • We introduce FogDrive, a large-scale synthetic multi-modal benchmark for autonomousdriving perception under controlled fog • FogDrive provides a controlled benchmark for studying weather-induced distribution shifts through paired clean observations and three calibrated fog densities (visibility ∼160, 100, and 50 m) generated using physics-based camera and LiDAR degradation models • We establish the quality and reproducibility of FogDrive through a quantitative annotation audit against an independent semantic segmentation reference, achieving 95.1 % precision and above-99 % recall for vehicles within 40 m • We establish benchmark tasks for multi-modal 3D detection and image-based perception under graded fog and demonstrate two key empirical findings: (i) training across multiple fog densities consistently improves robustness compared with singledensity training, and (ii) directly learning from degraded observations consistently outperforms defog-then-detect pipelines, while conventional image-quality metrics (PSNR, SSIM) fail to predict downstream object detection performance Page 2 --- PANWAR: FOGDRIVE 1 Existing approaches generally follow two paradigms The defog-then-detect approach inserts an image-dehazing module before a clean-trained detector, leveraging recent advances in image restoration In contrast, the train-on-fog approach directly trains or fine-tunes perception models on degraded observations, treating fog as a domain shift rather than a degradation to invert Determining which strategy yields more robust perception-and how this trade-off varies with fog density and sensing modality-remains an open empirical question whose resolution requires carefully designed benchmark datasets Existing datasets address different aspects of the problem, but none simultaneously provide paired clean and degraded observations, multiple calibrated fog densities, a complete multi-modal sensor suite, and consistent annotations across identical driving scenes The need for such controlled evaluation is not unique to autonomous driving Page 3 --- PANWAR: FOGDRIVE 3 We introduce FogDrive, a CARLA-based synthetic multi-modal benchmark specifically designed for systematic evaluation of autonomous-driving perception under controlled fog conditions The benchmark is built around paired observations collected under controlled weather variation Every driving scene is released in four perfectly aligned variants comprising one clear-weather reference and three calibrated fog densities (approximately 160 m, 100 m, and 50 m visibility), while preserving identical scene geometry and annotations across all conditions This paired design enables direct comparison of perception algorithms under progressively increasing environmental degradation and supports systematic studies of robustness across sensing modalities Page 4 --- PANWAR: FOGDRIVE 4 As perception models continue to improve, benchmark datasets have become essential not only for measuring performance but also for understanding the robustness and failure modes of learning algorithms under challenging environmental conditions Existing datasets provide only limited support for systematically studying perception robustness under controlled fog degradation We introduce FogDrive, a CARLA-based synthetic multi-modal benchmark specifically designed for systematic evaluation of autonomous-driving perception under controlled fog conditions The benchmark is built around paired observations collected under controlled weather variation Page 5 --- PANWAR: FOGDRIVE 5 Across machine learning, carefully designed synthetic benchmarks have repeatedly complemented real-world datasets by isolating factors that are difficult or impossible to control in naturally collected data Rather than replacing real-world benchmarks, they enable precise diagnosis of model behavior by systematically varying individual factors while keeping all other variables fixed Likewise, a benchmark that controls fog severity while preserving identical scene geometry enables direct investigation of weather-induced distribution shifts, multimodal robustness, and competing perception strategies under matched environmental conditions Page 6 --- PANWAR: FOGDRIVE 6 We introduce FogDrive, a CARLA-based synthetic multi-modal benchmark specifically designed for systematic evaluation of autonomous-driving perception under controlled fog conditions The benchmark is built around paired observations collected under controlled weather variation Every driving scene is released in four perfectly aligned variants comprising one clear-weather reference and three calibrated fog densities (approximately 160 m, 100 m, and 50 m visibility), while preserving identical scene geometry and annotations across all conditions Page 7 --- PANWAR: FOGDRIVE 7 The rig itself is identical across every scenario in the dataset; full attribute tables for each sensor are reproduced in the supplementary material (§D5) The rig itself is identical across every scenario in the dataset; full attribute tables for each sensor are reproduced in the supplementary material (§D5) Page 8 --- PANWAR: FOGDRIVE 8 The two tools share a common physical basis: camera fog applies Koschmieder’s atmospheric optics model per pixel, I(x) = J(x)t(x) + A(1 −t(x)) LiDAR fog attenuates each return point using the Beer–Lambert law, Prec ∝ Pemit exp(−2βr) for a return range r, then resamples surviving energy to model fog backscatter and per-point noise Page 9 --- PANWAR: FOGDRIVE 9 The need for such controlled evaluation is not unique to autonomous driving Across machine learning, carefully designed synthetic benchmarks have repeatedly complemented real-world datasets by isolating factors that are difficult or impossible to control in naturally collected data Likewise, a benchmark that controls fog severity while preserving identical scene geometry enables direct investigation of weather-induced distribution shifts, multimodal robustness, and competing perception strategies under matched environmental conditions Page 10 --- PANWAR: FOGDRIVE 10 Existing datasets address different aspects of the problem, but none simultaneously provide paired clean and degraded observations, multiple calibrated fog densities, a complete multi-modal sensor suite, and consistent annotations across identical driving scenes The need for such controlled evaluation is not unique to autonomous driving Page 11 --- PANWAR: FOGDRIVE 11 Across machine learning, carefully designed synthetic benchmarks have repeatedly complemented real-world datasets by isolating factors that are difficult or impossible to control in naturally collected data Likewise, a benchmark that controls fog severity while preserving identical scene geometry enables direct investigation of weather-induced distribution shifts, multimodal robustness, and competing perception strategies under matched environmental conditions
Paper summary: Tom: So, wrapping up this discussion about FogDrive, it’s a large-scale synthetic multi-modal benchmark specifically designed for systematic evaluation of autonomous driving perception under controlled fog conditions <ref:2607.22698#pg1> The authors are Vansh Panwar, and they put this dataset out for acceptance.
Jane: What this means simply is that we have a controlled environment where we can systematically vary the weather—from one hundred sixty meters down to fifty meters visibility—while keeping the scene geometry exactly the same across all those conditions <ref:2607.22698#pg3> This paired design lets you directly compare algorithms as the degradation gets progressively worse and it supports systematic studies of robustness across sensing modalities.
Lu: The core finding they push is that training across multiple fog densities consistently improves robustness compared to single density training, and directly learning from degraded observations consistently outperforms defog-then-detect pipelines <ref:2607.22698#pg2> That’s a clear direction for how we should be thinking about model training under adverse conditions.
Meng: From a deployment standpoint, the implication is that if we want our perception models to handle real fog well, we need to train them on this kind of mixed, degraded data rather than just clean data or relying on an image restoration module beforehand <ref:2607.22698#pg2> It tells us which strategy yields more robust perception and how that trade-off changes with fog density and sensing modality.
Tom: Right. And the limitation they mentioned is that the dataset only focuses on vehicle classes zero, one, and four, excluding two-wheelers from the spawnable blueprint pool <ref:2607.22698#pg1> So while it's a powerful tool for testing heavy vehicles under fog, we know we still have to worry about those other classes when we go live.
Jane: Exactly. The paper sets up this controlled test bed so that as perception models get better, they can be rigorously checked against these specific, reproducible conditions <ref:2607.22698#pg3> It’s about providing the controlled environmental variation that is currently missing from real-world data for studying how algorithms succeed or fail under fog.
Conclusion: Tom: So, we're wrapping up on FogDrive, which is this big synthetic dataset they built for testing how AI handles fog in autonomous driving.
Jane: The title itself is pretty straightforward—"FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog." It tells you exactly what it is and what it does.
Lu: It’s essentially a controlled environment where you can test perception models when the weather gets progressively worse, from one hundred sixty meters of visibility down to fifty meters.
Meng: And they made sure the data is multi-modal, meaning it includes everything—cameras, depth sensors, LiDAR and semantic LiDAR data, plus even front radar measurements.
Tom: That's a lot of different ways to see the world at once while fog rolls in. So what does this mean for us right now?
Jane: It means we can finally do systematic studies on robustness across different sensing modalities under matched environmental conditions, something you just can't get from real-world data collection.
Lu: The authors did a quality audit on their annotations, validating the labels at ninety-five point one percent precision and over ninety-nine percent recall for vehicles within forty meters. That level of consistency is key when training models.
Meng: But they did flag some limitations, which I think is important for us to hear. They only focused on vehicle classes zero, one, and four, leaving out two-wheelers from the initial data pool.
Tom: Yeah, so it's a powerful tool for testing heavy vehicles under controlled fog conditions that we can reproduce exactly every time we run an experiment.
Jane: Exactly. This dataset sets up a very specific test bed for understanding how perception systems handle distribution shifts caused by weather degradation before they even hit the road.
Lu: And the two main experiments they ran showed that mixing different fog densities during training actually helps improve detection performance across all test levels, and that training directly on foggy data consistently beats those methods where you try to fix the image first.
Meng: That's a big deal for deployment because it points toward direct learning as the better path rather than relying on an image restoration pipeline that might fail in real traffic.
Tom: So, we've seen how they built this benchmark and what their initial results showed, but we still need to figure out exactly how this translates into a model that actually runs reliably when the fog isn't perfectly modeled.
Vansh Panwar
Indian Institute of Technology, Guwahati
cs.CV
Submitted: 2026-07-18
Updated: 2026-11-09
Code: https://github.com/ultralytics/ultralytics
Project page: https://mukhlasadib.github.io/CARLA-2DBBox
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 90/100
The gist: Perception under adverse weather conditions remains a critical bottleneck for reliable autonomous driving…
Key concepts
- Multi-modal Dataset
- FogDrive combines multiple sensor types—like cameras and LiDAR—into one dataset. This is crucial because real-world driving involves fusing information from these different sources. The dataset provides synchronized observations from all these sensors, allowing models to learn how to combine visual, depth, and point cloud data effectively under fog conditions.
- Calibrated Fog Densities
- The paper models fog using two different physical laws: the Koschmieder model for cameras and the Beer-Lambert law for LiDAR. This allows researchers to create three distinct levels of fog (160m, 100m, and 50m) that are physically realistic. This control is vital because it lets scientists study how performance changes predictably as visibility decreases.
- Defog-then-Detect vs. Train-on-Fog
- This refers to two main strategies for handling fog in perception: first, 'defog-then-detect' where an image restoration module cleans the picture before detection; second, 'train-on-fog' where the model is directly trained using foggy images. FogDrive allows researchers to compare these methods systematically across different sensor modalities and fog severities.
- Cross-Calibrated Bounding Boxes
- The dataset includes perfectly matched 2D and 3D bounding boxes for vehicles in every scene variant. These boxes are cross-calibrated, meaning they accurately represent the same object in both image planes (2D) and spatial coordinates (3D). This consistency is essential for accurately measuring detection accuracy, especially when comparing 2D image detection results to 3D LiDAR results.
Terminology
Summary
Perception under adverse weather conditions remains a critical bottleneck for reliable autonomous driving, yet existing benchmarks lack the systematic multi-modal alignments required to evaluate robust sensor fusion The paper presents FogDrive, a rigorously calibrated, multi-modal autonomous driving dataset designed to bridge data-centric engineering and robust machine learning
Dataset Composition and Calibration
FogDrive is a CARLA-based synthetic multi-modal benchmark compiled using the CARLA simulator It contains 660 scenes (approximately 133k fully annotated frames with a 50:50 day/night split) across four synchronized cameras (RGB, depth, semantic segmentation), a LiDAR and semantic-LiDAR pair, and front radar Crucially, physically consistent fog is independently modeled on camera channels (via the Koschmieder model) and LiDAR channels (via the Beer-Lambert law) across three calibrated visibility densities (160m, 100m, and 50m) Every scene is provided in four perfectly matched variants (clean plus three graded fog levels) with cross-calibrated 2D and 3D bounding boxes
Data Quality Assurance
To ensure the highest standard of open science reliability, a semantic-segmentation-based quality audit across 8k images validates our annotations at 95.1% precision and over 99% recall for vehicles within 40m The dataset provides synchronized multi-modal observations, including multi-view RGB images with corresponding depth and semantic segmentation, LiDAR, semantic LiDAR, and radar measurements The depth PNG encodes distance into a 24-bit-per-pixel value with sub-millimetre precision which is relevant in fog because depth errors propagate into image transmission via t = exp(−β d)
Benchmarking Findings in 3D Detection
Experiment 1 investigated whether training a 3D LiDAR / camera-LiDAR detector on a single fog density generalises across fog densities, and whether mixing densities at training time helps Results showed that mixing fog densities at training time outperforms single-density Light training on raw detection rate at every test fog level Furthermore, scaling the mix specifically improves the geometric accuracy of each detection, with Mixed-L sweeping Mean TP across all test densities and posts a clean ATE sweep with the largest single gain being a ∼4cm reduction in mean translation error under moderate fog
Benchmarking Findings in 2D Detection
Experiment 2 examined the 2D image-detection counterpart, finding that training directly on foggy data approaches the clean upper bound and consistently outperforms defog-thendetect Standard reference-based image-quality metrics (PSNR, SSIM) do not predict downstream detection utility The paper concludes that the best option remains direct foggy training (Pipeline D) when retraining on the target domain is infeasible
Limitations of the Dataset
The dataset's scope is bounded by several decisions, including focusing on vehicle classes 0, 1, and 4 while excluding two-wheelers from the spawnable blueprint pool Fog is modeled for the camera and LiDAR channels at three calibrated densities while radar is released without an additional fog degradation by design The validation audit identifies a residual false-positive source of a ≈5 % true-phantom rate driven by multi-level geometry in CARLA’s urban maps Empty-frame frequency is high, with the front-camera view containing no annotated vehicles in roughly one frame in five
Conclusion
FogDrive enables further investigation of perception under controlled visibility degradation through its paired structure and calibrated densities The dataset, access class, and audit protocol will be released upon acceptance
--- Page 1 ---
arXiv:2607.22698v1 [cs. p. 0 arXiv:2607. 22698v1 [cs.Where the paper says it
PANWAR: FOGDRIVE 1
FogDrive: A Multi-Modal Synthetic Driving Dataset for Perception under Graded Fog Vansh Panwar p.vansh@iitg. p. 0 CV] 18 Jul 2026
PANWAR: FOGDRIVE 1
FogDrive: A Multi-Modal Synthetic Driving
Dataset for Perception under Graded Fog
Vansh Panwar
p. vansh@iitg. 1% precision and over 99% recall for vehicles within 40m. Finally, we establish
comprehensive baseline benchmarks using state-of-the-art architectures (TransFusion,
BEVFusion, and YOLOv8-m) across two distinct paradigms: 3D multi-modal fusion
and 2D image restoration.Where the paper says it
defog-then-detect
pipelines We present FogDrive, a rigorously calibrated, multi-modal autonomous driving dataset designed to bridge data-centric engineering and robust machine learning Compiled using the CARLA simulator, FogDrive contains 660 scenes (approximately 133k fully annotated frames with a 50:50 day/night split) across four synchronized cameras (RGB, depth, semantic segmentation), a LiDAR and semantic-LiDAR pair, and front radar Crucially, physically consistent fog is independently modeled on camera channels (via the Koschmieder model) and LiDAR channels (via the Beer-Lambert law) across three calibrated visibility densities (160m, 100m, and 50m) Every scene is provided in four perfectly matched variants (clean plus three graded fog levels) with cross-calibrated 2D and 3D bounding boxes To ensure the highest standard of open science reliability, a semantic-segmentation-based quality audit across 8k images validates our annotations at 95.1% precision and over 99% recall for vehicles within 40m Finally, we establish comprehensive baseline benchmarks using state-of-the-art architectures (TransFusion, BEVFusion, and YOLOv8-m) across two distinct paradigms: 3D multi-modal fusion and 2D image restoration Our major contributions are: • We introduce FogDrive, a large-scale synthetic multi-modal benchmark for autonomousdriving perception under controlled fog • FogDrive provides a controlled benchmark for studying weather-induced distribution shifts through paired clean observations and three calibrated fog densities (visibility ∼160, 100, and 50 m) generated using physics-based camera and LiDAR degradation models • We establish the quality and reproducibility of FogDrive through a quantitative annotation audit against an independent semantic segmentation reference, achieving 95. p. 0Where the paper says it
--- Page 2 ---
PANWAR: FOGDRIVE 1
Existing approaches generally follow two paradigms The defog-then-detect approach inserts an image-dehazing module before a clean-trained detector, leveraging recent advances in image restoration In contrast, the train-on-fog approach directly trains or fine-tunes perception models on degraded observations, treating fog as a domain shift rather than a degradation to invert Determining which strategy yields more robust perception-and how this trade-off varies with fog density and sensing modality-remains an open empirical question whose resolution requires carefully designed benchmark datasets Existing datasets address different aspects of the problem, but none simultaneously provide paired clean and degraded observations, multiple calibrated fog densities, a complete multi-modal sensor suite, and consistent annotations across identical driving scenes The need for such controlled evaluation is not unique to autonomous driving
--- Page 3 ---
PANWAR: FOGDRIVE 3
We introduce FogDrive, a CARLA-based synthetic multi-modal benchmark specifically designed for systematic evaluation of autonomous-driving perception under controlled fog conditions The benchmark is built around paired observations collected under controlled weather variation Every driving scene is released in four perfectly aligned variants comprising one clear-weather reference and three calibrated fog densities (approximately 160 m, 100 m, and 50 m visibility), while preserving identical scene geometry and annotations across all conditions This paired design enables direct comparison of perception algorithms under progressively increasing environmental degradation and supports systematic studies of robustness across sensing modalities
--- Page 4 ---
PANWAR: FOGDRIVE 4
As perception models continue to improve, benchmark datasets have become essential not only for measuring performance but also for understanding the robustness and failure modes of learning algorithms under challenging environmental conditions Existing datasets provide only limited support for systematically studying perception robustness under controlled fog degradation We introduce FogDrive, a CARLA-based synthetic multi-modal benchmark specifically designed for systematic evaluation of autonomous-driving perception under controlled fog conditions The benchmark is built around paired observations collected under controlled weather variation
--- Page 5 ---
PANWAR: FOGDRIVE 5
Across machine learning, carefully designed synthetic benchmarks have repeatedly complemented real-world datasets by isolating factors that are difficult or impossible to control in naturally collected data Rather than replacing real-world benchmarks, they enable precise diagnosis of model behavior by systematically varying individual factors while keeping all other variables fixed Likewise, a benchmark that controls fog severity while preserving identical scene geometry enables direct investigation of weather-induced distribution shifts, multimodal robustness, and competing perception strategies under matched environmental conditions
--- Page 6 ---
PANWAR: FOGDRIVE 6
We introduce FogDrive, a CARLA-based synthetic multi-modal benchmark specifically designed for systematic evaluation of autonomous-driving perception under controlled fog conditions The benchmark is built around paired observations collected under controlled weather variation Every driving scene is released in four perfectly aligned variants comprising one clear-weather reference and three calibrated fog densities (approximately 160 m, 100 m, and 50 m visibility), while preserving identical scene geometry and annotations across all conditions
--- Page 7 ---
PANWAR: FOGDRIVE 7
The rig itself is identical across every scenario in the dataset; full attribute tables for each sensor are reproduced in the supplementary material (§D5) The rig itself is identical across every scenario in the dataset; full attribute tables for each sensor are reproduced in the supplementary material (§D5)
--- Page 8 ---
PANWAR: FOGDRIVE 8
The two tools share a common physical basis: camera fog applies Koschmieder’s atmospheric optics model per pixel, I(x) = J(x)t(x) + A(1 −t(x)) LiDAR fog attenuates each return point using the Beer–Lambert law, Prec ∝ Pemit exp(−2βr) for a return range r, then resamples surviving energy to model fog backscatter and per-point noise
--- Page 9 ---
PANWAR: FOGDRIVE 9
The need for such controlled evaluation is not unique to autonomous driving Across machine learning, carefully designed synthetic benchmarks have repeatedly complemented real-world datasets by isolating factors that are difficult or impossible to control in naturally collected data Likewise, a benchmark that controls fog severity while preserving identical scene geometry enables direct investigation of weather-induced distribution shifts, multimodal robustness, and competing perception strategies under matched environmental conditions
--- Page 10 ---
PANWAR: FOGDRIVE 10
Existing datasets address different aspects of the problem, but none simultaneously provide paired clean and degraded observations, multiple calibrated fog densities, a complete multi-modal sensor suite, and consistent annotations across identical driving scenes The need for such controlled evaluation is not unique to autonomous driving
--- Page 11 ---
PANWAR: FOGDRIVE 11
Across machine learning, carefully designed synthetic benchmarks have repeatedly complemented real-world datasets by isolating factors that are difficult or impossible to control in naturally collected data Likewise, a benchmark that controls fog severity while preserving identical scene geometry enables direct investigation of weather-induced distribution shifts, multimodal robustness, and competing perception strategies under matched environmental conditions
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved 3D detection systems can achieve superior localization quality by training on mixed fog densities at inference time, as
scaling the mix specifically improves the geometric accuracy of each detection.
This results in aclean ATE sweep with the largest single gain being a ~4cm reduction in mean translation error under moderate fog.
-
2D perception pipelines can be optimized by employing direct foggy training rather than relying on image restoration, as
training directly on foggy data approaches the clean upper bound and consistently outperforms defog-thendetect.
This approach is shown to be superior to methods like DCP or fine-tuned AOD-Net when compared against image quality metrics like PSNR or SSIM. -
Multi-modal fusion systems can benefit from dynamic training strategies by mixing fog densities during training, as
mixing fog densities at training time outperforms single-density Light training at every test fog level.
This strategy is shown to improve raw detection rate without requiring additional data, while scaling the mix further improves geometric accuracy.
Abstract
Fog degrades autonomous-driving perception across modalities: it lowers image contrast, attenuates and scatters LiDAR returns, and shifts the input distribution of every learned module downstream. Choosing between restoring the degraded input and training on it directly requires the same scenes observed with and without fog at known severities, and enough foggy data to train on. Real-world capture cannot supply this: fog arrives entangled with lighting and traffic, at severities recorded rather than chosen; existing synthetic datasets narrow the sensor suite, the scale, or the severity range. We present FogDrive, a CARLA dataset of 660 scenes (133k frames), each released as a clear reference plus three calibrated fog densities, with physics-based fog on camera and LiDAR and 2D/3D annotations consistent across all variants. Every frame carries four cameras (RGB, depth, semantic), LiDAR, semantic LiDAR and radar -- spanning fog's response spectrum from heavily degraded cameras to largely unaffected radar. Experiments trained on FogDrive show that mixing fog densities improves 3D detection at no extra data cost, that PSNR and SSIM poorly predict a dehazer's detection benefit, and that radar improves camera-LiDAR fusion at every density. The dataset and its generation framework will be released.
Sources
- Virtual KITTI 2
- KITTI-CARLA: a KITTI-like dataset generated by CARLA Simulator
- SCOPE: A Synthetic Multi-Modal Dataset for Collective Perception Including Physical-Correct Weather Conditions
- BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View
- Adver-City: Open-Source Multi-Modal Dataset for Collaborative Perception Under Adverse Weather Conditions
- SynFog: A Photo-realistic Synthetic Fog Dataset based on End-to-end Imaging Simulation for Advancing Real-World Defogging in Autonomous Driving
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs