Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning

arXiv:2603.16122 · cs.CV · Submitted 2026-03-17 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning".

Jane: The gist: SynOE-OD, a Synthetic Outlier-Exposure-based Object Detection framework, enables a single detector to detect OOD objects alongside ID objects by leveraging strong generative models and transfer learning.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Let's look at the title and who wrote this. "Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning." It sounds like a pretty specific problem they are trying to solve in a real environment, street scenes.

Jane: The authors are Sadia Ilyas, Annika Mutze, Klaus Friedrichs, Thomas Kurbiel, and Matthias Rottmann from the University of Wuppertal and Osnabruck.

Lu: They are focusing on bridging that gap between OOD detection research that usually stays in image classification or segmentation and the actual task of detecting objects in a scene.

Meng: It’s about making sure autonomous driving systems or medical imaging tools don't miss something critical because it looks too different from what they saw before.

Lalam: The implication is that we can move toward a unified way to handle known versus unknown objects in object detection tasks, which has been a big challenge.

The paper's summary: Tom: To summarize the core idea of this SynOE-OD framework, they are using Stable Diffusion and GroundingDINO to generate these synthetic outliers for street scenes.

Jane: So instead of just training on real objects, they create new, unusual objects in the images and use those as training examples to teach the detector what an outlier looks like.

Lu: This generated data is then used for transfer learning on both standard object detectors and OVODs to build this joint capability.

Meng: It’s essentially using the synthetic outliers to expand the model's detection capacity beyond just what was in the original dataset, which is what they call OOD object detection.

Lalam: The result is a framework where you have a single detector that can reliably find both the common objects and these novel objects together.

The paper's improvements: Tom: The main improvement they are pushing is this unified approach: adapting an existing detector to detect OOD while keeping it fine-tuned for the normal, in-distribution objects.

Jane: They explicitly define the problem by treating real objects as ID and anything else as OOD, and they show how the data generation pipeline directly supports that split.

Lu: They detail the data generation pipeline using Stable Diffusion to inpaint unusual objects in street scenes, followed by GroundingDINO to get precise bounding boxes and class labels for those inpainted items.

Meng: And they use this process for transfer learning, fine-tuning both closed-vocabulary detectors and OVODs on these mixed datasets to achieve state-of-the-art OOD detection performance.

Lalam: The authors also highlight that including a small proportion of these synthetic OOD samples during training yields the strongest improvement in both ID and OOD robustness.

Conclusion: Tom: So, to wrap up, this SynOE-OD framework consistently outperforms strong OVOD baselines across several benchmarks on datasets like RoadAnomaly and Fishyscapes.

Jane: The key point is that this method provides a significant performance gain, for example, a ten point seven five percent improvement in AP50 to ninety-five on the Fishyscapes dataset.

Lu: The ablation studies show that the synthetic outliers provide complementary supervision that helps OOD detection go beyond what zero-shot generalization can achieve.

Meng: When they looked at cross-domain scenarios, joint fine-tuning on combined datasets like NuImages and Cityscapes actually improved AP50 to thirty-seven percent and AR to forty-six percent for the RoadAnomaly dataset alone.

Lalam: The overall implication is that targeted synthetic outlier exposure combined with effective transfer learning can really specialize a vision language model for OOD object detection.

Tom: It suggests that having diverse data sources, where synthetic outliers provide this kind of supervision, genuinely enhances OOD detection beyond just zero-shot generalization.

Jane: It definitely points toward how essential it is to have reliable ID and OOD object detection in safety-critical applications where both are equally important.

University of Wuppertal · University of Osnabruck, Germany

cs.CV

Submitted: 2026-03-17

Updated: 2026-10-14

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: The gist: SynOE-OD, a Synthetic Outlier-Exposure-based Object Detection framework, enables a single detector to detect OOD objects alongside ID objects by leveraging strong generative models and

Key concepts

Out-of-Distribution (OOD) Object Detection: This is the task of detecting objects in a scene that are not typical or expected. Standard detectors often ignore these unusual items, which is dangerous in real-world applications like self-driving cars. The goal is to reliably find and correctly label these atypical objects so the system knows what it's seeing beyond normal expectations.
Synthetic Outlier Exposure (OE): This involves creating artificial, unusual objects for training by using generative models like Stable Diffusion. These synthetic outliers are inserted into real street scenes. This process exposes the detector to a wide variety of rare object appearances, helping it learn to recognize things that are outside its initial training distribution.
Open-Vocabulary Object Detectors (OVODs): These detectors can identify objects belonging to any category, not just those predefined in their original training set. In this framework, an OVOD helps generate refined bounding boxes and labels for the synthetic outliers created by Stable Diffusion, providing high-quality supervision for the detector during training.

Terminology

Ablation — 25× in this paper · explained in 1 episode
Benchmark — 22× in this paper · explained in 2 episodes
Fine-Tuning — 21× in this paper · explained in 5 episodes
Generalization — 19× in this paper · explained in 13 episodes
Transfer Learning — 14× in this paper · explained in 12 episodes
Semantic Segmentation — 11× in this paper · explained in 2 episodes
Robustness — 10× in this paper · explained in 5 episodes
Generative Model — 5× in this paper · explained in 2 episodes
Conference on Computer Vision and Pattern Recognition — 4× in this paper
Generative Models — 4× in this paper · explained in 3 episodes
Transformer — 4× in this paper · explained in 2 episodes
Zero-Shot Generalization — 4× in this paper · explained in 5 episodes
Zero-shot performance — 4× in this paper · explained in 2 episodes
Ablation Studies — 3× in this paper · explained in 1 episode
Diffusion Model — 3× in this paper · explained in 10 episodes
Instance Segmentation — 3× in this paper · explained in 3 episodes
Unified Framework — 3× in this paper · explained in 5 episodes
Adaptation — 2× in this paper · explained in 1 episode
Core — 2× in this paper · explained in 2 episodes
Foundation Model — 2× in this paper · explained in 3 episodes
Foundation Models — 2× in this paper · explained in 7 episodes
Two-stage training — 2× in this paper · explained in 2 episodes
Virtual Outlier Synthesis — 2× in this paper
Vision-language models — 2× in this paper · explained in 2 episodes
Ablation Study — 1× in this paper · explained in 3 episodes
Adversarial Attacks — 1× in this paper · explained in 1 episode
Benchmarking — 1× in this paper · explained in 4 episodes
Catastrophic Forgetting — 1× in this paper · explained in 5 episodes
Diffusion Models — 1× in this paper · explained in 9 episodes
Distribution Shift — 1× in this paper · explained in 9 episodes
Distribution Shifts — 1× in this paper · explained in 2 episodes
Domain Shift — 1× in this paper · explained in 6 episodes
Generalizability — 1× in this paper · explained in 6 episodes
Grounding — 1× in this paper · explained in 3 episodes
Multimodal Dataset — 1× in this paper · explained in 2 episodes
Prompt Engineering — 1× in this paper · explained in 5 episodes
Uncertainty Quantification — 1× in this paper · explained in 9 episodes

Summary

The gist: SynOE-OD, a Synthetic Outlier-Exposure-based Object Detection framework, enables a single detector to detect OOD objects alongside ID objects by leveraging strong generative models and transfer learning.

Introduction and Motivation

Out-of-distribution (OOD) object detection is an important yet underexplored task, where a reliable detector must localize and correctly classify atypical objects as OOD Under such conditions, object detectors may overlook objects that fall outside the semantic distribution of typical scenes, implicitly treating them as background. In safety-critical applications, e.g., autonomous driving or medicine, it becomes crucial to detect OOD objects accurately as in-distribution (ID) objects in a perceived scene. A significant amount of work has been carried out that addresses OOD image classification and OOD semantic segmentation, while OOD object detection remains comparatively understudied. The lack of unified benchmarks for ID/OOD object detection and the scarcity of realistic OOD data have hindered the progress on this topic. Recent study shows that OVODs exhibit limited performance in certain OOD detection scenarios.

Proposed Framework SynOE-OD

SynOE-OD is a framework that enables a single detector to detect OOD objects, which are otherwise silently overlooked, alongside ID objects. The framework leverages strong generative models, like Stable Diffusion, and Open-Vocabulary Object Detectors (OVODs) to generate semantically meaningful, object-level data that serve as outliers during training. This generated data is used for transfer-learning to establish strong ID task performance and supplement detection models with OOD object detection robustness. The approach combines two complementary paradigms, i.e., OE and transfer-learning, into a unified framework by utilizing a strong generative model, Stable Diffusion and an OVOD, e.g. GroundingDINO (GDINO) [37], to enable object-level OE that supplements object detectors with OOD object detection capability.

Data Generation Pipeline

The data-generation pipeline is composed of two components: Stable Diffusion, to generate synthetic object-level outliers for street-scene datasets, and GDINO for obtaining refined bounding boxes and class labels of the inpainted objects. To generate synthetic outliers, Stable Diffusion is used to inpaint unusual objects in street-scenes. To obtain diverse, atypical objects, ChatGPT is used to return a list of candidate object categories that are unusual for a given dataset. Selected regions of the image are modified by inserting synthetic outliers, and the input text for the corresponding image is sampled from the list Lunusual returned by ChatGPT. GDINO is used to obtain accurate bounding boxes and class labels, distinguishing between three scenarios: Bounding box refinement for correctly inpainted OOD objects, Label reassignment when the inpainting is a modified version of an ID object, and Bounding box removal in case of ‘empty inpainting’.

Transfer Learning and Joint Detection

To enable the joint optimization of ID and OOD objects, SynOE-OD leverages object detectors with strong backbones pre-trained on large-scale datasets, e.g., Objects365 and medium-scale datasets such as COCO. The detector is finetuned via transfer learning on our inpainted datasets composed of real and synthetic objects. Based on the discussion in Section 3.1, an additional n+1-st class bucket is added for OOD objects. The n ID classes are the specific set of object categories relevant to the original scenes, while all other objects are assigned to the n+1-st bucket, considered as the OOD class.

Experimental Setup and Results

The experiments use popular street-scenes datasets, NuImages [5], Cityscapes [8], and BDD100K [65]. The evaluation is performed on the OoDIS benchmark [48], which focuses exclusively on OOD object detection, reporting Average Precision (AP50 to 95) for the OOD class and AP50 to 95 for ID classes. SynOE-OD consistently demonstrates superior performance compared to strong OVOD baselines across all evaluated OOD datasets. For instance, the performance gain in terms of AP50 to 95 with SynOE-OD in Table I is 5.49%, 6.7% and 10.75%, on the RoadObstacle, RoadAnomaly and Fishyscapes datasets respectively.

Ablation Studies

Ablations reveal that including a small proportion of OOD samples yields the strongest improvement in OOD and ID robustness. Furthermore, when training on multiple datasets constitutes to crossdomain scenario, joint fine-tuning on the combined NuImages and Cityscapes datasets improves AP50 to 37% and AR to 46% for RoadObstacle only. The results suggest that controlled modifications to the data generation procedure have a measurable impact on OOD object detection performance.

Conclusion

SynOE-OD, a unified framework for ID and OOD object detection in street-scenes, consistently outperforms existing zero-shot OVOD baselines. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important. The results suggest that OVOD benefits from a wide variety of data sources, where synthetic outliers provide complementary supervision that enhances OOD object detection beyond zeroshot generalization. The extensive ablations highlight the potential of our method for safety-critical applications where reliable ID/OOD object detection is equally important.

Improvements for AI systems

Use SynOE-OD to enable a single detector to detect OOD objects alongside ID objects, addressing a critical issue where atypical objects are completely missed by the object detector and incorrectly treated as background.
Develop an object detection framework that integrates seamlessly with standard object detectors, adapting them to jointly detect OOD and ID objects by leveraging synthetic outliers directly into the image space of street-scene datasets.
Implement a data generation pipeline that systematically incorporates synthetic outliers using Stable Diffusion and GroundingDINO to generate object-level OE that supplements object detectors with OOD object detection capability.
Utilize transfer learning to establish strong ID task performance while simultaneously supplement[ing] detection models with OOD object detection robustness, as the paper states, The real objects ensure the ID task performance, whereas the synthetic outliers expand the model’s detection capacity to semantic concepts beyond the ID objects.
Enhance zero-shot generalization for Open-Vocabulary Object Detectors (OVODs) by fine-tuning them on inpainted datasets composed of real and synthetic objects, which demonstrates that targeted synthetic outlier exposure combined with effective transfer learning can specialize a strong VLM for OOD object detection beyond zero-shot generalization.
Improve robustness against domain shift by training models on multiple heterogeneous datasets, as the experiments show that while training on a combination improves performance, it can lead to a performance drop across all test sets if there is significant domain mismatch.
Refine annotation accuracy by using GDINO for label generation, specifically employing techniques like Label reassignment when the inpainting is a modified version of an ID object and discarding uninformative samples where GDINO produces no detections for the inpainted sample.

Abstract

Out-of-distribution (OOD) objects are often silently overlooked by object detectors, which produce no detection for them, leaving downstream modules with nothing to act on and no indication that anything was missed. A reliable object detector must detect OOD objects by localizing and correctly classifying them as OOD. Existing approaches primarily focus on OOD detection for image classification or rely on auxiliary branches, and typically do not treat in-distribution (ID) and OOD objects in a unified manner. We present SynOE-OD, a Synthetic Outlier-Exposure-based Object Detection method, which uses text-guided inpainting and an open-vocabulary object detector to generate and verify synthetic outliers offline. These are then used to fine-tune an object detector of choice with an added OOD class, which requires no auxiliary branch at inference time. On the OoDIS benchmark, SynOE-OD outperforms the strongest open-vocabulary baseline, reaching AP 50:95 of 57.6%, 29.4% and 39.2% on RoadObstacle, RoadAnomaly and Fishyscapes, against 48.5%, 22.7% and 22.7%. This state-of-the-art performance holds for both fine-tuned closed-set and open-vocabulary detectors while maintaining competitive ID performance in street scenes.