Simile Understanding in Text-to-Image Models: An Evaluation Framework
Luecheng Wang, Shintaro Ozaki, Hidetaka Kamigaito, Katsuhiko Hayashi, Jingun Kwon, Manabu Okumura, Taro Watanabe
cs.CV, cs.CL, cs.MM
Submitted: 2026-08-05
Comments: Accepted as a full paper at ACM Multimedia 2026
Code: https://github.com/ultralytics/ultralytics
License: http://creativecommons.org/licenses/by/4.0/
The gist: Similes provide a compact and expressive way to describe visual characteristics in text prompts.
Terminology
Abstract
Similes provide a compact and expressive way to describe visual characteristics in text prompts. Recent text-to-image models (t2i models) can produce visually compelling outputs from simile prompts, yet even frontier models frequently misinterpret the metaphorical vehicle and confuse it with the object. These systematic failures reveal a gap between figurative language and object-level visual grounding in t2i models. To investigate this issue, we propose a scalable evaluation framework for simile understanding. Our framework includes (1) a controlled simile dataset in which metaphorical vehicles are drawn from a predefined set of object-detectable categories and combined with diverse templates, (2) automatic grounding metrics based on YOLO (You Only Look Once) detection, and (3) text encoder layer analysis using Diffusion Lens to track how metaphorical vehicles emerge during generation. Experiments across architecturally diverse t2i models reveal consistent literalization failure patterns. We further discuss potential mitigation strategies for improving simile grounding in t2i models.
Sources
- Phi-4 Technical Report
- MetaCLUE: Towards Comprehensive Visual Metaphors Research
- PixArt-$\alpha$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis
- Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
- GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment
- The Llama 3 Herd of Models
- A Survey on LLM-as-a-Judge
- T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation
- Large Language Model Displays Emergent Ability to Interpret Novel Literary Metaphors
- Mistral 7B
- Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models
- The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation
- TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation
- Learning Transferable Visual Models From Natural Language Supervision
- High-Resolution Image Synthesis with Latent Diffusion Models
- T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation
- Gemma 2: Improving Open Language Models at a Practical Size
- Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models
- Qwen-Image Technical Report
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models