PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies
cs.AI
Submitted: 2026-10-01
Updated: 2026-10-04
Project page: https://abuseofnotation.github.io/category-theory-illustrated
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Analogical reasoning involves identifying and preserving relational structures across domains.
Abstract
Analogical reasoning involves identifying and preserving relational structures across domains. However, existing approaches to AI-driven multimodal analogy generation lack an interpretable measure of whether this structure is understood and maintained in the generated output. We address this gap with Pullback Refinement via Interpretable Structural Mapping (PRISM), a modality-agnostic framework for measuring and improving relational alignment in multimodal analogies, evaluated on visual metaphor generation. PRISM represents analogies as explicit relational mappings grounded in category theory and uses VLMs to instantiate these structures across modalities. Its first component, the pullback score, quantifies relational alignment from the resulting graph representation. On the AnaloBench benchmark, selecting the correct analogy purely by pullback score achieves 82.5% accuracy, demonstrating that the score captures meaningful relational information. PRISM's second component is an iterative refinement loop that uses the pullback score as an in-context feedback signal to iteratively revise the generated image towards greater relational depth. VLM-as-a-judge and human evaluations show that PRISM consistently improves metaphor consistency and analogy appropriateness over zero- shot generation, with human participants preferring the refined output in 57.65% of pairwise comparisons. However, a qualitative analysis reveals that refinement can favour visually crowded compositions rather than genuinely deeper relational correspondences.
Sources
- HealthBench: Evaluating Large Language Models Towards Improved Human Health
- CARV: A Diagnostic Benchmark for Compositional Analogical Reasoning in Multimodal LLMs
- Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narratives
- The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation
- Evaluating the Robustness of Analogical Reasoning in Large Language Models
- Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstraction
- Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning
- Multimodal Analogical Reasoning over Knowledge Graphs
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems