MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models".
Jane: Here's the gist of the paper. Vision–Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts <ref:2602.03665#pg2> Prior works typically rely on binary or pairwise supervision, which often fail to capture the continuous and pluralistic nature of human moral reasoning <ref:2602.03665#pg2>.
Tom: We present MM–SCALE (MULTIMODAL MORAL SCALE), a large–scale dataset for aligning VLMs with human moral preferences through 5–point scalar ratings and explicit modality grounding <ref:2602.03665#pg2>.
Jane: By moving from discrete to scalar supervision, our framework provides richer alignment signals and finer calibration of multimodal moral reasoning <ref:2602.03665#pg2>.
Tom: Dataset Composition and Novelty MM–SCALE is a dataset of 32,212 multimodal moral scenarios designed to align VLMs with scalar moral preferences by human <ref:2602.03665#pg2>.
Title and authors: Tom: Let’s look at the title and authors of this paper now, "MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models." It sets the stage by immediately telling us what they are testing.
Jane: The title is quite descriptive, focusing on evaluating evidence grounded moral judgment because it highlights that the focus is on how the model uses visual and textual evidence to make those judgments.
Lu: It’s interesting that they are explicitly evaluating grounding because it points out a weakness in previous methods where the connection between the decision and what you see or read wasn't being measured properly.
Meng: So, what’s the actual scope here? Are they just looking at one type of moral problem, or is this dataset meant to cover a lot of different kinds of scenarios?
Tom: The authors are Eunkyu Park, Wesley Hanwen Deng, Cheyon Jin, Matheus Kunzler Maldaner, Jordan Wheeler, Jason I. Hong, Hong Shen and Adam Perer. They come from some pretty solid AI and computer science backgrounds at places like Carnegie Mellon and Seoul National University.
Jane: That’s a good team of people who clearly have deep expertise in the areas they're tackling, which you can see reflected in the detailed construction of this dataset.
Lu: The paper mentions that they are moving away from binary or pairwise supervision because those methods simply fail to capture the continuous and pluralistic nature of human moral reasoning.
Meng: So, if we use this dataset, what’s the immediate implication for my team? Do we just swap out one training setup for another, or is there a fundamental shift in how we think about alignment?
Tom: It suggests a fundamental shift because they are moving from discrete to scalar supervision and using listwise optimization. They argue that this framework provides richer alignment signals and finer calibration of multimodal moral reasoning.
The paper's summary: Jane: So, summarizing the core idea of MM-SCALE, it’s a dataset of thirty-two thousand two hundred twelve multimodal moral scenarios annotated with five point scalar ratings and modality grounding labels by humans.
Tom: That’s right. Each scenario has a target image showing a social norm situation, multiple possible actions from that scene, and then the human annotator picks one of five moral acceptability scores, plus they label if their judgment depended on the text or the image or both.
Lu: The novelty here isn't just having more data points; it’s that they are explicitly including modality attribution in the annotations, which is a crucial piece of missing information in most existing datasets.
Meng: That grounding column sounds very useful for debugging model behavior. If a model gets it wrong on an image scenario, we can check if the error is because it ignored the visual context or because it misread the accompanying text.
Jane: It also highlights that they are using an interface they built themselves for data collection, which suggests they’ve put a lot of thought into getting high-quality, consistent annotations.
Tom: They sourced their scenarios by taking daily norm scenarios from Commonsense NormBank and then rendering them using a text-to-image model to create the visual context.
Lu: That two step process—sourcing from a commonsense knowledge base and then generating the image—is how they build those multimodal moral contexts, making sure the visuals are grounded in real-world social norms.
The paper's improvements: Tom: The paper points out a few improvements they’ve made to the approach, specifically moving from binary or pairwise supervision to this scalar, listwise optimization framework.
Jane: They suggest that this shift from discrete labels to a continuous five point rating allows the model to learn relative moral acceptability in a globally consistent way without needing hard labels or handcrafted reward functions.
Lu: That’s significant because it lets the model learn the relative ordering of scenarios—which one is better than another—using those scalar human labels, which is much more informative than just knowing if action A is safe and action B is unsafe.
Meng: So, if we use this listwise optimization technique, how does that help us when we are trying to deploy a model on something where the situation changes constantly?
Tom: It allows the model to learn the relative moral acceptability of scenarios in a globally consistent way, leveraging those scalar human labels without defining hard labels or handcrafted reward functions. This leads to much more stable and fine calibration of multimodal moral reasoning.
Jane: And they show that this approach is superior because listwise scalar supervision yields the strongest ranking fidelity across all tested architectures, achieving an ndcg five of zero point eight nine for LLaVA one vision seven billion parameter model in their work.
Conclusion: Tom: So, to wrap up on MM-SCALE, the main implication is that models fine-tuned with scalar moral ratings from this dataset outperform prior vision language models that were safety tuned with binary labels.
Jane: They also found something very telling about how humans think: they found that sixty eight percent of human moral ratings of situation action pairs change after seeing the image, which strongly confirms the need for multimodal grounding.
Lu: The paper concludes by questioning whether scalar rating or ranking should be the core supervisory signal we use for preference-based alignment in general. It’s a big question for how we structure these training regimes moving forward.
Meng: From an engineering standpoint, it tells us that to get good moral reasoning, you need to pay attention to the interplay between visual and textual context because that’s where the most significant shifts happen in judgment.
Lalam: I’m confident in this direction; the data shows a clear path toward capturing the continuous nature of human morality through these structured scalar judgments and explicit modality grounding.
Seoul National University, Carnegie Mellon University, University of Florida, Epic Games
cs.CV, cs.HC
Submitted: 2026-02-03
Updated: 2026-11-10
Comments: NeurIPS 2026 Evaluations and Datasets Track
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 99/100
The gist: Vision–Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts <ref:2602.03665#pg2> Prior works typically rely on binary or…
Key concepts
- MM-SCALE Dataset
- A large dataset of 32,212 multimodal moral scenarios. Each scenario includes an image (social situation), multiple options, a 1-5 scalar moral rating, and labels indicating whether the judgment relies on text, image, or both. This moves beyond simple yes/no answers to capture nuanced human morality.
- Scalar Moral Supervision
- Instead of binary labels (good/bad), this method uses 5-point scale ratings. This provides richer alignment signals for VLMs, allowing them to learn the continuous and pluralistic nature of moral reasoning rather than just discrete classifications. It helps models calibrate their judgments more finely.
- Listwise Preference Optimization
- A training framework that tunes models by optimizing their performance over the entire set of scenarios associated with a single image. This encourages the model to learn a globally consistent ranking of moral acceptability, using human scalar labels without needing hard labels or custom reward functions.
- Modality Grounding
- The process of explicitly labeling which part of the input (text, image, or both) primarily informs the moral judgment. This helps models understand *why* a judgment was made by showing whether the visual context or textual description was more important for assessing a social norm.
Terminology
Summary
Vision–Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts <ref:2602. p. 0 MM–SCALE: Grounded Multimodal Moral Reasoning
via Scalar Judgment and Listwise Alignment
Eunkyu Park♡, Wesley Hanwen Deng♠, Cheyon Jin♡, Matheus Kunzler Maldaner✧,
Jordan Wheeler♣, Jason I. Hong♠, Hong Shen♠, Adam Perer♠,
Ken Holstein♠, Motahhare Eslami♠, Gunhee Kim♡
♡Seoul National University, ♠Carnegie Mellon University, ✧University of Florida, ♣Epic Games
Abstract
Vision–Language Models (VLMs) continue to struggle to
make morally salient judgments in multimodal and socially
ambiguous contexts. Hong♠, Hong Shen♠, Adam Perer♠,
Ken Holstein♠, Motahhare Eslami♠, Gunhee Kim♡
♡Seoul National University, ♠Carnegie Mellon University, ✧University of Florida, ♣Epic Games
Abstract
Vision–Language Models (VLMs) continue to struggle to
make morally salient judgments in multimodal and socially
ambiguous contexts. Prior works typically rely on binary
or pairwise supervision, which often fail to capture the
continuous and pluralistic nature of human moral reasoning. Prior works typically rely on binary
or pairwise supervision, which often fail to capture the
continuous and pluralistic nature of human moral reasoning. We present MM–SCALE (MULTIMODAL MORAL
SCALE), a large–scale dataset for aligning VLMs with human moral preferences through 5–point scalar ratings and
explicit modality grounding. Each image-scenario pair is
annotated with moral acceptability scores and grounded
reasoning labels by humans using an interface we tailored
for data collection, enabling listwise preference optimization over ranked scenario sets. By moving from discrete
to scalar supervision, our framework provides richer alignment signals and finer calibration of multimodal moral reasoning.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Dataset Composition and Novelty
MM–SCALE is a dataset of 32,212 multimodal moral scenarios designed to align VLMs with scalar moral preferences by human <ref:2602.03665#pg2>. Its novelty lies in scalar moral ratings and explicit modality grounding (text, image, or both) <ref:2602. p. 1 Our contributions are as follows:
1. We present MM–SCALE, a dataset of 32,212 socionormative, moral scenarios grounded in AI–generated images, whose novelty lies in scalar moral ratings and explicit modality grounding (text, image, or both), as
compared in Table 1. Overview and Design Principles
MM–SCALE is a dataset of 32,212 multimodal moral scenarios designed to align VLMs with scalar moral preferences by human. Each instance contains a target image (depicting a social–norm situation), multiple plausible moral
scenarios, scalar moral judgments (1–5), and a modality attribution (text, image, or both). Dataset Composition and Statistics
Each target setting in MM–SCALE includes an average of
3. 48 moral scenarios, totaling 32,212 annotated scenarios
across 9,260 unique image contexts.Where the paper says it
Where the paper says it
Where the paper says it
Annotation Protocol and Grounding Signals
The annotation protocol involves annotators assigning a scalar score (1–5) for moral acceptability and labeling the modality that informs their judgment: text, image, or both <ref:2602.03665#pg2>. Annotators are asked to indicate whether their judgment depends primarily on the visual content, the text, or both <ref:2602. p. 4 Modality Grounding and Agreement. Annotators are
asked, “Was your judgment primarily influenced by the visual content, the text, or both? Overview of our data annotation pipeline. (a) Situations Sourcing: We source daily norm scenarios that can add details to
an action from the Commonsene Normbank [11] dataset. Accordingly, our annotation pipeline (illustrated in Fig. 2)
allows annotators to assess multiple scenarios within the
same image context, capturing fine–grained scalar rankings
across complex, multidimensional situations.Where the paper says it
Where the paper says it
Where the paper says it
Listwise Alignment Framework
We adopt a listwise preference optimization framework for tuning VLMs to reproduce human preference ordering over the full set of scenarios associated with each image <ref:2602.03665#pg2>. This is achieved using the ListMLE loss [2], which encourages the model to assign scores such that the resulting ranking matches a target permutation π∗ such that µπ∗(1) ≥ µπ∗(2) ≥ · · · ≥ µπ∗(n) <ref:2602. p. 5 Let f(I, Si) = ˆsi denote the model’s predicted scalar score
for scenario Si conditioned on image I. The ListMLE loss
encourages the model to assign scores such that the resulting ranking matches π
∗
:
LListMLE = − log Yn
t=1
exp(ˆsπ∗(t))
Pn
j=t
exp(ˆsπ∗(j))
! The ListMLE loss
encourages the model to assign scores such that the resulting ranking matches π
∗
:
LListMLE = − log Yn
t=1
exp(ˆsπ∗(t))
Pn
j=t
exp(ˆsπ∗(j))
! This allows the model to learn the relative moral acceptability of scenarios in a globally consistent way, leveraging
scalar human labels without defining hard labels or handcrafted reward functions.Where the paper says it
Where the paper says it
Empirical Results on Supervision Type
Listwise scalar supervision yields the strongest ranking fidelity (NDCG@5, MRR) across all tested architectures <ref:2602.03665#pg2>. Specifically, for LLaVA–OneVision-7B, Ours (Listwise PO) achieved NDCG@5 of 0. p. 6 Model Supervision Ranking-based Score-based
NDCG@5 ↑ MRR ↑ Unsafe Rate ↓ AUC–Safety ↑
LLaVA-OneVision-7B
Ours (Listwise PO) 0. 76
BPO (Binary PO) 0. We report ranking and score–based
metrics. Ranking–based metrics include (1) NDCG@5,
measuring how well the model’s ranked list of moral scenarios aligns with human–rated moral acceptability, and
(2) MRR (Mean Reciprocal Rank), measuring how early
the top morally acceptable scenario appears in the model’s
ranked list. To address this question, we highlight two dimensions of
comprehensively aligning models with human moral judgments: (1) scalar moral supervision, and (2) multimodal
grounding. Scalar ratings capture fine–grained acceptarXiv:2602. In contrast, ListMLE produces smoother and more discriminative scalar predictions, yielding higher AUC–Safety
while maintaining competitive Unsafe Rates. Binary classification (BCE) shows the expected instability: hardthreshold training inflates Unsafe Errors and severely weakens ranking fidelity. To address this question, we highlight two dimensions of
comprehensively aligning models with human moral judgments: (1) scalar moral supervision, and (2) multimodal
grounding. Scalar ratings capture fine–grained acceptarXiv:2602. Additionally, we show that 68% of human judgments
shift after seeing the image, underscoring the need for multimodal grounding. We find that scalar listwise supervision
produces more consistent moral rankings across modalities
and maintains stability across synthetic and real images—
highlighting the role of supervision granularity in improving moral alignment.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Modality Grounding Insights
Analysis of judgment shifts shows that scenarios grounded in image or image+text are substantially more likely to diverge from their original text-only label <ref:2602.03665#pg2>. Furthermore, multimodal scenarios grounded in both image and text exhibit the highest rate of extreme re-evaluations, indicating that richer visual-linguistic context often leads to more substantial reconsideration of moral judgment <ref:2602. p. 4 6 (a) shows the frequency of shifts by
modality. Multimodal scenarios grounded in both image
and text exhibit the highest rate of extreme re-evaluations,
indicating that richer visual-linguistic context often leads to
more substantial reconsideration of moral judgment. We empirically demonstrate in § 5. 3 that multimodal distinctions are not arbitrary;
modality labels are highly consistent across annotators and
predictably shift scalar ratings depending on whether visuals reinforce or contradict text.Where the paper says it
Where the paper says it
Conclusion and Future Directions
We introduced MM–SCALE, a dataset of 32,212 multimodal moral scenarios annotated with scalar moral judgments and grounding modality labels <ref:2602.03665#pg2>. We show that models fine-tuned with scalar moral ratings from MM–SCALE outperform prior VLMs safety-tuned with binary labels <ref:2602. p. 7 Our annotation
pipeline supports iterative supervision through model-user
disagreement, and our training framework leverages listwise preference optimization to align models to full moral
rankings rather than isolated binary labels. We show that
models fine-tuned with scalar moral ratings from MM–
SCALE outperform prior VLMs safety-tuned with binary
labels. We show that
models fine-tuned with scalar moral ratings from MM–
SCALE outperform prior VLMs safety-tuned with binary
labels. Moreover, we find that 68% of human moral ratings of situation-action pairs change after seeing the image,
confirming the need for multimodal grounding. Beyond our work, several important questions remain open. First, we can scale up the interactive
annotation process with MORALE to include more diverse annotators and scenario iterations. Is binary agreement sufficient,
or is scalar rating more informative? Should ranking be the
core supervisory signal instead?Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Limitations
Our dataset presents several limitations, including annotators being primarily based in the United States and the United Kingdom, which may constrain cultural diversity <ref:2602.03665#pg2>. Additionally, modality attribution was coarsely limited to three categories: text, image, or both <ref:2602. p. 8 the annotation of modality attribution was coarsely limited
to three categories: text, image, or both. While this offers useful insight into which modality informs moral reasoning, it does not capture more granular or layered grounding dynamics. While this offers useful insight into which modality informs moral reasoning, it does not capture more granular or layered grounding dynamics. Future work could explore finer-grained or
continuous grounding signals that better reflect the nuanced
interplay between visual and textual context.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Scalar moral supervision provides richer alignment signals than binary or pairwise supervision, as experiments show that
VLMs fine–tuned on MM–SCALE achieve higher ranking fidelity and more stable safety calibration than those trained with binary signals.
This allows models to capture thecontinuous and pluralistic nature of human moral reasoning
by using 5-point scalar ratings. -
Multimodal grounding enables models to perform grounded reasoning by specifying
whether the judgment is based on text, image, or both,
which helps resolve ambiguity by showinghow visual context (a) What is the person doing in the image? Helping someone cross the street. UNSAFE (b) Drinking liquid that you got from a stranger.
-
Listwise preference optimization improves moral alignment by allowing models to
learn the relative moral acceptability of scenarios in a globally consistent way, leveraging scalar human labels without defining hard labels or handcrafted reward functions.
This method shows thatlistwise scalar supervision produces more consistent moral rankings across modalities and maintains stability across synthetic and real images.
-
The MORALE interface develops an interactive annotation workflow where disagreement triggers a modality grounding check, enabling the system to
identify salient cases for refinement
by asking annotators,User selects which modality their judgment depends on.
This supportsmore human–centered alignment and dataset expansion.
Abstract
Vision-Language Models increasingly make socially consequential judgments from image-text inputs, yet current evaluations often stop at verdict-level accuracy: whether the final label is human-aligned, not whether it follows from the correct evidence. We introduce MM-SCALE (Multimodal Moral SCALE), a benchmark for evidence-grounded moral judgment built around a design choice: each image is paired with multiple action scenarios, so models must compare how different actions interact with the same visual context rather than score images in isolation. MM-SCALE contains 8,444 image contexts and 21,977 action scenarios, each annotated with a 5-point moral acceptability rating and a modality-grounding label indicating whether the human judgment relied on text, image, or both. Three tasks evaluate whether models (i) assign calibrated scalar moral scores, (ii) preserve human orderings within a shared visual context, and (iii) ground rationales in the evidence source humans found critical to judgment. Across models, aggregate metrics overstate alignment: NDCG@5 remains high across models, while within-image pairwise accuracy remains only 0.53-0.62 even on scenario pairs whose human mean ratings differ by at least one point. CoT has inconsistent effects on calibration and does not reliably improve within-image ordering. Evidence grounding evaluations further show that models often describe visual context without making it critical for the verdict. These results reveal an evidence-grounding failure in current VLMs and position MM-SCALE as an evaluation testbed for moral judgment under shared image contexts. This dataset includes potentially harmful and sensitive visual content. Images are intended solely for evaluation.
Sources
- WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI
- Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms
- MIRAGE: Multi-model Interface for Reviewing and Auditing Generative Text-to-Image AI
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs