MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models
summary
The gist
Vision–Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts <ref:2602.03665#pg2> Prior works typically rely on binary or…
In short
Vision-Language Models struggle with moral judgment because prior methods use simple binary labels that miss human moral reasoning's complexity. MM-SCALE introduces a large dataset of 32,212 multimodal moral scenarios using 5-point scalar ratings and explicit grounding (text/image). Using a listwise optimization framework, models fine-tuned on this data show better ranking fidelity and more stable safety calibration than those trained with binary supervision.
Key concepts
- MM-SCALE Dataset
- A large dataset of 32,212 multimodal moral scenarios. Each scenario includes an image (social situation), multiple options, a 1-5 scalar moral rating, and labels indicating whether the judgment relies on text, image, or both. This moves beyond simple yes/no answers to capture nuanced human morality.
- Scalar Moral Supervision
- Instead of binary labels (good/bad), this method uses 5-point scale ratings. This provides richer alignment signals for VLMs, allowing them to learn the continuous and pluralistic nature of moral reasoning rather than just discrete classifications. It helps models calibrate their judgments more finely.
- Listwise Preference Optimization
- A training framework that tunes models by optimizing their performance over the entire set of scenarios associated with a single image. This encourages the model to learn a globally consistent ranking of moral acceptability, using human scalar labels without needing hard labels or custom reward functions.
- Modality Grounding
- The process of explicitly labeling which part of the input (text, image, or both) primarily informs the moral judgment. This helps models understand *why* a judgment was made by showing whether the visual context or textual description was more important for assessing a social norm.
Terminology used across episodes
This episode discusses
- MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models · Paper Radio
- WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI
- Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms
- MIRAGE: Multi-model Interface for Reviewing and Auditing Generative Text-to-Image AI
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model
The paper
MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models · Read on arXiv
Seoul National University, Carnegie Mellon University, University of Florida, Epic Games
Vision-Language Models increasingly make socially consequential judgments from image-text inputs, yet current evaluations often stop at verdict-level accuracy: whether the final label is human-aligned, not whether it follows from the correct evidence. We introduce MM-SCALE (Multimodal Moral SCALE), a benchmark for evidence-grounded moral judgment built around a design choice: each image is paired with multiple action scenarios, so models must compare how different actions interact with the same visual context rather than score images in isolation. MM-SCALE contains 8,444 image contexts and 21,977 action scenarios, each annotated with a 5-point moral acceptability rating and a modality-grounding label indicating whether the human judgment relied on text, image, or both. Three tasks evaluate whether models (i) assign calibrated scalar moral scores, (ii) preserve human orderings within a shared visual context, and (iii) ground rationales in the evidence source humans found critical to judgment. Across models, aggregate metrics overstate alignment: NDCG@5 remains high across models, while within-image pairwise accuracy remains only 0.53-0.62 even on scenario pairs whose human mean ratings differ by at least one point. CoT has inconsistent effects on calibration and does not reliably improve within-image ordering. Evidence grounding evaluations further show that models often describe visual context without making it critical for the verdict. These results reveal an evidence-grounding failure in current VLMs and position MM-SCALE as an evaluation testbed for moral judgment under shared image contexts. This dataset includes potentially harmful and sensitive visual content. Images are intended solely for evaluation.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models".
Jane: Here's the gist of the paper. Vision–Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts <ref:2602.03665#pg2> Prior works typically rely on binary or pairwise supervision, which often fail to capture the continuous and pluralistic nature of human moral reasoning <ref:2602.03665#pg2>.
Tom: We present MM–SCALE (MULTIMODAL MORAL SCALE), a large–scale dataset for aligning VLMs with human moral preferences through 5–point scalar ratings and explicit modality grounding <ref:2602.03665#pg2>.
Jane: By moving from discrete to scalar supervision, our framework provides richer alignment signals and finer calibration of multimodal moral reasoning <ref:2602.03665#pg2>.
Tom: Dataset Composition and Novelty MM–SCALE is a dataset of 32,212 multimodal moral scenarios designed to align VLMs with scalar moral preferences by human <ref:2602.03665#pg2>.
Title and authors: Tom: Let’s look at the title and authors of this paper now, "MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models." It sets the stage by immediately telling us what they are testing.
Jane: The title is quite descriptive, focusing on evaluating evidence grounded moral judgment because it highlights that the focus is on how the model uses visual and textual evidence to make those judgments.
Lu: It’s interesting that they are explicitly evaluating grounding because it points out a weakness in previous methods where the connection between the decision and what you see or read wasn't being measured properly.
Meng: So, what’s the actual scope here? Are they just looking at one type of moral problem, or is this dataset meant to cover a lot of different kinds of scenarios?
Tom: The authors are Eunkyu Park, Wesley Hanwen Deng, Cheyon Jin, Matheus Kunzler Maldaner, Jordan Wheeler, Jason I. Hong, Hong Shen and Adam Perer. They come from some pretty solid AI and computer science backgrounds at places like Carnegie Mellon and Seoul National University.
Jane: That’s a good team of people who clearly have deep expertise in the areas they're tackling, which you can see reflected in the detailed construction of this dataset.
Lu: The paper mentions that they are moving away from binary or pairwise supervision because those methods simply fail to capture the continuous and pluralistic nature of human moral reasoning.
Meng: So, if we use this dataset, what’s the immediate implication for my team? Do we just swap out one training setup for another, or is there a fundamental shift in how we think about alignment?
Tom: It suggests a fundamental shift because they are moving from discrete to scalar supervision and using listwise optimization. They argue that this framework provides richer alignment signals and finer calibration of multimodal moral reasoning.
The paper's summary: Jane: So, summarizing the core idea of MM-SCALE, it’s a dataset of thirty-two thousand two hundred twelve multimodal moral scenarios annotated with five point scalar ratings and modality grounding labels by humans.
Tom: That’s right. Each scenario has a target image showing a social norm situation, multiple possible actions from that scene, and then the human annotator picks one of five moral acceptability scores, plus they label if their judgment depended on the text or the image or both.
Lu: The novelty here isn't just having more data points; it’s that they are explicitly including modality attribution in the annotations, which is a crucial piece of missing information in most existing datasets.
Meng: That grounding column sounds very useful for debugging model behavior. If a model gets it wrong on an image scenario, we can check if the error is because it ignored the visual context or because it misread the accompanying text.
Jane: It also highlights that they are using an interface they built themselves for data collection, which suggests they’ve put a lot of thought into getting high-quality, consistent annotations.
Tom: They sourced their scenarios by taking daily norm scenarios from Commonsense NormBank and then rendering them using a text-to-image model to create the visual context.
Lu: That two step process—sourcing from a commonsense knowledge base and then generating the image—is how they build those multimodal moral contexts, making sure the visuals are grounded in real-world social norms.
The paper's improvements: Tom: The paper points out a few improvements they’ve made to the approach, specifically moving from binary or pairwise supervision to this scalar, listwise optimization framework.
Jane: They suggest that this shift from discrete labels to a continuous five point rating allows the model to learn relative moral acceptability in a globally consistent way without needing hard labels or handcrafted reward functions.
Lu: That’s significant because it lets the model learn the relative ordering of scenarios—which one is better than another—using those scalar human labels, which is much more informative than just knowing if action A is safe and action B is unsafe.
Meng: So, if we use this listwise optimization technique, how does that help us when we are trying to deploy a model on something where the situation changes constantly?
Tom: It allows the model to learn the relative moral acceptability of scenarios in a globally consistent way, leveraging those scalar human labels without defining hard labels or handcrafted reward functions. This leads to much more stable and fine calibration of multimodal moral reasoning.
Jane: And they show that this approach is superior because listwise scalar supervision yields the strongest ranking fidelity across all tested architectures, achieving an ndcg five of zero point eight nine for LLaVA one vision seven billion parameter model in their work.
Conclusion: Tom: So, to wrap up on MM-SCALE, the main implication is that models fine-tuned with scalar moral ratings from this dataset outperform prior vision language models that were safety tuned with binary labels.
Jane: They also found something very telling about how humans think: they found that sixty eight percent of human moral ratings of situation action pairs change after seeing the image, which strongly confirms the need for multimodal grounding.
Lu: The paper concludes by questioning whether scalar rating or ranking should be the core supervisory signal we use for preference-based alignment in general. It’s a big question for how we structure these training regimes moving forward.
Meng: From an engineering standpoint, it tells us that to get good moral reasoning, you need to pay attention to the interplay between visual and textual context because that’s where the most significant shifts happen in judgment.
Lalam: I’m confident in this direction; the data shows a clear path toward capturing the continuous nature of human morality through these structured scalar judgments and explicit modality grounding.
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI