Exploit or conform? How norm utility prompting changes the way LLMs use community beliefs
cs.AI
Submitted: 2026-11-02
Updated: 2026-11-02
Code: https://github.com/LeonKlingborg/Norm-utility-in-LLM-decision-making
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large Language Models (LLMs) serve both as tools for aiding human decision-making in everyday life and as models of human agents in experimental simulation studies.
Terminology
Abstract
Large Language Models (LLMs) serve both as tools for aiding human decision-making in everyday life and as models of human agents in experimental simulation studies. Alignment on prototypical social decisions -- such as whether to cooperate on a collective action -- therefore has immediate practical consequences for researchers and for society at large. Here we systematically explore the impact of ``norm utility'' -- the non-material components of utility that account for an individual's desire to conform to social norms and to be internally consistent -- on the strategic decision-making of three open-weight LLMs in one-shot, two-player social dilemmas. We find that when a social dilemma is presented together with information about community beliefs -- but without any indication of how those beliefs should matter -- models tend to treat community belief as strategic information, so that in the Snowdrift game cooperation falls as community belief in cooperation increases. When we explicitly prompt the components of norm utility this relationship inverts, and cooperation increases with community belief, in broad agreement with the predictions of a simple norm utility model. We then show that prompting norm utility reduces the volatility of strategic decisions with respect to changes in the underlying social context, and generates more consistent reasoning habits across models. Taken together, our results suggest that prompting norm utility can help generate more human-like and consistent behavior across models and lower volatility within models.
Sources
- A General Language Assistant as a Laboratory for Alignment
- Non-Determinism of "Deterministic" LLM Settings
- When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
- Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?
- Gemma 4 Technical Report
- Spontaneous Giving and Calculated Greed in Language Models
- Behavior is Not Enough: A Mechanism-Based Evaluation of Social Norm Emergence in LLM Societies
- Muse Glimmer 30B: An On-Device Agentic Model
- Do people apply different norms to humans and large language models acting on their behalf? Evidence from norm elicitations in two canonical economic games
- Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
- LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theory
- CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas
- Cultural Evolution of Cooperation among LLM Agents
- Large language models that replace human participants can harmfully misportray and flatten identity groups
- Large Language Models Often Say One Thing and Do Another
- Disentangling the Drivers of LLM Social Conformity: An Uncertainty-Moderated Dual-Process Mechanism
- Conformity in Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents