UniDG: Unified Defect Generation via Defect-Context Editing
cs.CV, cs.AI
Submitted: 2026-04-10
Updated: 2026-10-28
Comments: NeurIPS 2026 Accepted
Code: https://github.com/RetoFan233/UniDG
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generating realistic visual defects is crucial for alleviating the scarcity of abnormal samples in anomaly detection…
Terminology
Abstract
Generating realistic visual defects is crucial for alleviating the scarcity of abnormal samples in anomaly detection, yet existing methods often rely on category-specific few-shot adaptation and struggle to generalize across objects, defect types, and domains. This limitation stems in part from the lack of paired defect editing data and the fine-grained nature of defect appearance, where subtle variations in scale, texture, and morphology must be preserved while maintaining consistency with the target scene. To address this challenge, we introduce UDG, a dataset of 300K normal-abnormal-mask-caption quadruplets curated from diverse defect-related scenarios, and present UniDG, a unified defect generation model. UniDG formulates defect synthesis as Defect-Context Editing: it extracts reference defect context with adaptive cropping, organizes reference and target inputs in a structured diptych format, and fuses multimodal conditions through MM-DiT attention. We further develop a two-stage training strategy: Diversity-SFT learns diverse transferable defect priors from paired editing data, while optional Consistency-RFT can further improve reference adherence, local realism, and defect-category consistency. Without per-category fine-tuning or using MVTec-AD/VisA for training, UniDG outperforms prior few-shot anomaly generation and image insertion/editing baselines in both synthesis quality and downstream single- and multi-class anomaly detection/localization.
Sources
- Qwen3-VL Technical Report
- Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
- FreeEdit: Mask-free Reference-based Image Editing with Multi-modal Instruction
- Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
- Prodigy: An Expeditiously Adaptive Parameter-Free Learner
- SAM 2: Segment Anything in Images and Videos
- Dual-Grained Policy Optimization for Hallucination Reduction in Flow-Based RL
- IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs