Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges
cs.CL, cs.AI, cs.MM
Submitted: 2026-07-21
Updated: 2026-09-01
License: http://creativecommons.org/licenses/by/4.0/
The gist: Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, shared cultural knowledge, and communicative intent rather
Terminology
Abstract
Multimodal humor in memes, cartoons, and comics remains difficult for AI systems because intended meaning depends on non-literal mechanisms, shared cultural knowledge, and communicative intent rather than literal scene description. This survey focuses on visual humor understanding in single-image and multi-panel artifacts, while treating humor generation as an emerging downstream frontier. We position the literature against prior humor, sarcasm, and general MLLM surveys and organize it using a capability-centric hierarchy spanning recognition, interpretation and reasoning, and generation. Under this lens, we synthesize benchmark design, evaluation protocols, and modeling paradigms, tracing the field's shift from task-specific fusion models to large-model approaches based on multimodal alignment, evidence-grounded reasoning, and controlled generation. We conclude by highlighting the main barriers to progress: shortcut-prone evaluation, limited cultural and narrative coverage, weak evidence grounding, and unresolved safety and ownership concerns.
Sources
- MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing
- Qwen3-VL Technical Report
- StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos
- I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors
- Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Models
- Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you!
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- A Survey of Multimodal Sarcasm Detection
- From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment
- Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
- TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models
- Chumor 1.0: A Truly Funny and Challenging Chinese Humor Understanding Dataset from Ruo Zhi Ba
- Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning
- MemeCap: A Dataset for Captioning and Interpreting Memes
- BottleHumor: Self-Informed Humor Explanation using the Information Bottleneck Principle
- MemeGuard: An LLM and VLM-based Framework for Advancing Content Moderation via Meme Intervention
- Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations
- D-HUMOR: Dark Humor Understanding via Multimodal Open-ended Reasoning -- A Benchmark Dataset and Method
- Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study
- When 'YES' Meets 'BUT': Can Large Models Comprehend Contradictory Humor Through Comparative Reasoning?
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering