Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language

arXiv:2609.03677 · cs.CV, cs.CL, cs.LG, cs.NE, cs.RO · Submitted 2026-09-03 · Read on arXiv

cs.CV, cs.CL, cs.LG, cs.NE, cs.RO

Submitted: 2026-09-03

Updated: 2026-09-03

Comments: 9 pages, 5 figures, submitted to the IEEE Open Journal of Intelligent Transportation Systems (OJ-ITS), our implementation and benchmark dataset are available at https://github.com/KIT-MRT/AD-Diff

Code: https://github.com/KIT-MRT/AD-Diff

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 21× in this paper · explained in 2 episodes
• Accuracy — 17× in this paper · explained in 1 episode
• Domain Shift — 7× in this paper · explained in 4 episodes
• Introspection — 6× in this paper · explained in 2 episodes
• Robustness — 6× in this paper · explained in 5 episodes
• Distribution Shift — 5× in this paper · explained in 5 episodes
• Embeddings — 5× in this paper · explained in 2 episodes
• Sparsity — 5× in this paper · explained in 4 episodes
• Aggregation — 4× in this paper · explained in 1 episode
• Instruction Tuning — 4× in this paper · explained in 2 episodes
• Cosine Similarity — 2× in this paper · explained in 3 episodes
• Accountability — 1× in this paper · explained in 2 episodes
• Active Learning — 1× in this paper · explained in 6 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Contrastive Learning — 1× in this paper · explained in 5 episodes
• Domain Adaptation — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Human-in-the-Loop — 1× in this paper · explained in 3 episodes
• Karlsruhe Institute of Technology — 1× in this paper
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Semantic Understanding — 1× in this paper · explained in 2 episodes

Sources

• Data-Centric Evolution in Autonomous Driving: A Comprehensive Survey of Big Data System, Data Mining, and Closed-Loop Technologies
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Visual Instruction Tuning
• Qwen3-VL Technical Report
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• MIMIC-IT: Multi-Modal In-Context Instruction Tuning
• Otter: A Multi-Modal Model with In-Context Instruction Tuning
• Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generation
• gpt-oss-120b & gpt-oss-20b Model Card
• SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models