UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction

arXiv:2608.04949 · cs.CV, cs.CL, cs.IT, cs.MM · Submitted 2026-08-05 · Read on arXiv

Bo Kong, Liruiz Jia, Yi Liang, Chao Liu, Dongfang Han, Tianwei Yan, Yuan Liu, Shengquan Liu

cs.CV, cs.CL, cs.IT, cs.MM

Submitted: 2026-08-05

Comments: Accepted at ACM MM2026

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Multimodal Relation Extraction — 18× in this paper
• Calibration — 17× in this paper · explained in 9 episodes
• Augmentation — 16× in this paper · explained in 2 episodes
• Aleatoric Uncertainty — 13× in this paper · explained in 7 episodes
• Information Bottleneck — 11× in this paper · explained in 3 episodes
• Consistency — 9× in this paper · explained in 3 episodes
• Latent Space — 8× in this paper · explained in 7 episodes
• Unified Multimodal Relation Extraction — 8× in this paper
• Optimal Transport — 7× in this paper · explained in 5 episodes
• Contrastive Learning — 6× in this paper · explained in 9 episodes
• Hyperparameters — 5× in this paper · explained in 2 episodes
• Benchmark — 4× in this paper · explained in 2 episodes
• Core — 4× in this paper · explained in 2 episodes
• Instruction Tuning — 4× in this paper · explained in 3 episodes
• MLLMs — 4× in this paper · explained in 2 episodes
• Multilevel Optimal Transport — 4× in this paper
• Transformer — 4× in this paper · explained in 2 episodes
• Uncertainty-Driven Unimodal Augmentation — 4× in this paper
• Cross-modal Alignment — 3× in this paper · explained in 2 episodes
• Embeddings — 3× in this paper · explained in 2 episodes
• Joint Aleatoric Uncertainty Alignment — 3× in this paper
• Knowledge Graph — 3× in this paper · explained in 8 episodes
• Multimodal Fusion — 3× in this paper · explained in 4 episodes
• Transformers — 3× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Ablation Study — 2× in this paper · explained in 3 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Computational Efficiency — 2× in this paper · explained in 2 episodes
• Data Augmentation — 2× in this paper · explained in 10 episodes
• Distillation — 2× in this paper · explained in 2 episodes
• F1 Score — 2× in this paper · explained in 4 episodes
• Multimodal Object-Entity Relation Extraction — 2× in this paper
• Robustness — 2× in this paper · explained in 6 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Data Fusion — 1× in this paper · explained in 3 episodes
• Diffusion Model — 1× in this paper · explained in 10 episodes
• Diffusion Models — 1× in this paper · explained in 9 episodes
• Federated Learning — 1× in this paper · explained in 19 episodes
• Fixed Point — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 14 episodes
• Graph Construction — 1× in this paper · explained in 3 episodes
• Hierarchical Cross-Modal Interaction — 1× in this paper
• Latent Diffusion Model — 1× in this paper · explained in 3 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Latent Variables — 1× in this paper · explained in 2 episodes
• Mixture of Experts — 1× in this paper · explained in 2 episodes
• Multimodal Dataset — 1× in this paper · explained in 2 episodes
• Multimodal Learning — 1× in this paper · explained in 2 episodes
• Multimodal Neural Relation Extraction — 1× in this paper
• Multimodal approach — 1× in this paper · explained in 2 episodes
• Mutual Information — 1× in this paper · explained in 2 episodes
• Papers — 1× in this paper · explained in 3 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Quantifying Uncertainty — 1× in this paper · explained in 5 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Representation Learning — 1× in this paper · explained in 3 episodes
• Semantic Alignment — 1× in this paper · explained in 6 episodes
• Semi-supervised Learning — 1× in this paper · explained in 4 episodes
• Topic Modeling — 1× in this paper · explained in 2 episodes
• Uncertainty Quantification — 1× in this paper · explained in 9 episodes
• Variational Bayes — 1× in this paper · explained in 2 episodes
• Weight Decay — 1× in this paper · explained in 2 episodes

Sources

• Qwen2.5-VL Technical Report
• Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models
• REMOTE: A Unified Multimodal Relation Extraction Framework with Multilevel Optimal Transport and Mixture-of-Experts
• The Llama 3 Herd of Models
• A Survey on Algorithmic Developments in Optimal Transport Problem with Applications
• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
• Trustworthy Federated Label Distribution Learning under Annotation Quality Disparity

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models