Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions
cs.AI
Submitted: 2026-07-16
Updated: 2026-11-07
Code: https://github.com/williamium3000/cedi
Terminology
Sources
- Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions
- Introducing Visual Perception Token into Multimodal Large Language Model
- Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models
- Gemma 3 Technical Report
- LLaMA: Open and Efficient Foundation Language Models
- Object Hallucination in Image Captioning
- Hallucination is Inevitable: An Innate Limitation of Large Language Models
- HalluLens: LLM Hallucination Benchmark
- A Survey on Hallucination in Large Vision-Language Models
- CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning
- VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models
- mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality
- SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension
- SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension
- MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
- ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models
- MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation
- Evaluation and Analysis of Hallucination in Large Vision-Language Models
- Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges
- LanP: Rethinking the Impact of Language Priors in Large Vision-Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents