Gender Bias in Vision-Language In-Context Learning
cs.CV
Submitted: 2026-09-23
Updated: 2026-09-28
Comments: ECCV 2026
Code: https://github.com/mathfather/Gender-Bias-in-VL-ICL
License: http://creativecommons.org/licenses/by/4.0/
The gist: In-context learning (ICL) enables large vision-language models (LVLMs) to perform tasks by following patterns from in-context examples, yet its potential to amplify societal biases remains
Terminology
Abstract
In-context learning (ICL) enables large vision-language models (LVLMs) to perform tasks by following patterns from in-context examples, yet its potential to amplify societal biases remains underexplored. We systematically investigate how ICL influences gender bias in LVLMs through VL-BICLE, an evaluation framework comprising six ICL settings, three tasks, and four datasets. Our experiments on six LVLMs reveal that gendered ICL demonstrations act as a directional force, shifting model bias toward the demonstrated gender through a cross-gender mechanism that disproportionately degrades performance on the opposite gender. This effect appears in image captioning and pronoun prediction but not in visual question answering, indicating that gendered ICL influences bias only when the task output involves gendered language. Similarity-based retrieval methods inherit the training pool's gender imbalance and offer no debiasing advantage, while standard quality metrics remain blind to these bias shifts. To mitigate this bias, we replace real in-context images with synthetic ones from stable diffusion models while keeping captions unchanged. This simple intervention reduces gender bias without degrading caption quality.
Sources
- Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Debiasing Vision-Language Models via Biased Prompts
- Towards Multimodal In-Context Learning for Vision & Language Models
- Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals
- Building and better understanding vision-language models: insights and future directions
- MIMIC-IT: Multi-Modal In-Context Instruction Tuning
- Advancing Multimodal In-Context Learning in Large Vision-Language Models with Task-aware Demonstrations
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- On the Limitations of Steering in Language Model Alignment
- GPT-4 Technical Report
- Qwen3-VL Technical Report
- Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- MiniCPM-V: A GPT-4V Level MLLM on Your Phone
- The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs?
- Debiasing Multimodal Large Language Models via Penalization of Language Priors
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models