AI Value Alignment for Evolving Social Norms
summary
The gist
AI alignment is essential for the safe deployment of advanced AI systems <ref:2607.18506#pg2> Given that values and preferences change over time, culture, social roles, and context…
In short
The research models how human values evolve when using personalized AI assistants in a changing social environment. It found that static AI alignment risks 'value lock-in' and 'normative mode collapse,' where groups converge on suboptimal values. The study suggests alignment must be adaptive and dynamic to allow humans to keep up with societal shifts.
Key concepts
- Value Lock-in
- This occurs when an AI assistant anchors a user's values too strongly to their past preferences. If the social environment changes, the user's values become rigid, preventing them from adapting to new norms or opportunities. This creates a stagnation that hinders personal and group evolution.
- Normative Mode Collapse
- This is an emergent phenomenon where a population of users converges on a single set of values that is worse than what any subgroup could achieve. It happens when social influence and AI alignment forces cause everyone to settle on the same, suboptimal consensus, rather than allowing diverse, useful value sets to emerge.
- Adaptive Alignment
- Instead of fixed alignment strength, this approach makes the AI's influence on user values change over time. The alignment strength decays when a user's tracking error is high. This allows the AI to be less prescriptive when the environment changes rapidly, enabling users to evolve their values alongside society.
- Constitutional AI
- This method grounds an AI assistant's behavior in a set of explicit, stated principles rather than just implicit preferences. By steering the model with clear rules, developers can ensure the alignment is auditable and revisable, providing a stable normative content that guides the AI's actions.
Terminology used across episodes
This episode discusses
- AI Value Alignment for Evolving Social Norms · Paper Radio
- Steerable Pluralism: Pluralistic Alignment via Few-Shot Comparative Regression
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Constitutional AI: Harmlessness from AI Feedback
- On the Opportunities and Risks of Foundation Models
- AI Alignment with Changing and Influenceable Reward Functions
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
- ELEPHANT: Measuring and understanding social sycophancy in LLMs
- Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback
- Towards Measuring the Representation of Subjective Global Opinions in Language Models
- Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)
- KTO: Model Alignment as Prospect Theoretic Optimization · Paper Radio
- Recognising the importance of preference change: A call for a coordinated multidisciplinary research effort in the age of AI
- Training language models to be warm and empathetic makes them less reliable and more sycophantic
- AI Alignment: A Comprehensive Survey
- Hidden Incentives for Auto-Induced Distributional Shift
- Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development
- Architecting Trust in Artificial Epistemic Agents
- Towards Understanding Sycophancy in Language Models
- A Roadmap to Pluralistic Alignment
- AI-Mediated Communication Can Steer Collective Opinion · Paper Radio
The paper
AI Value Alignment for Evolving Social Norms · Read on arXiv
Google DeepMind
AI alignment is essential for the safe deployment of advanced AI systems. Given that values and preferences change over time, culture, social roles, and context, we need to develop a better understanding of the possible long-term consequences of AI alignment, in particular considering the likely ubiquitous future use of personalized AI assistants. We introduce a flexible and extensible mathematical modelling framework, rooted in social physics, aimed at answering macro-level questions regarding the evolving social norms in human populations under the assumption of frequent AI use. Our analysis is part-analytical, and part-simulation, enabling us to characterize the long-term dynamical consequences under a diverse set of starting assumptions. We highlight the risk of value lock-in, and normative mode collapse, prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous, and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and acting as a tractable precursor to more computationally expensive large-scale agentic evaluations.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "AI Value Alignment for Evolving Social Norms".
Jane: Here's the gist of the paper. AI alignment is essential for the safe deployment of advanced AI systems <ref:2607.18506#pg2> Given that values and preferences change over time, culture, social roles, and context, we need to develop a better understanding of the possible long-term consequences of AI alignment Our analysis is partanalytical, and part-simulation, enabling us to characterize the long-term dynamical consequences under a diverse set of starting assumptions.
Tom: Modeling Evolving Normative Dynamics The paper introduces a flexible and extensible mathematical modelling framework rooted in social physics aimed at answering macro-level questions regarding the evolving social norms in human populations under the assumption of frequent AI use.
Jane: This framework situates populations of user-AI pairs within a continuous value space subject to environmental drift and normative shocks.
Tom: The simulation tracks a population of user-AI pairs as they navigate a D-dimensional continuous value space, subject to intrinsic exploration, social influence, environmental learning, and AImediated alignment forces.
Title and authors: Tom: So we’re looking at this paper titled "AI Value Alignment for Evolving Social Norms," and the authors are Nenad Tomašev, Matija Franklin, and Simon Osindero from Google DeepMind. Jane, what's the big picture here?
Jane: Well, it basically tackles the problem of how AI alignment needs to change because human values aren't static; they shift with culture and context over time. This paper is trying to build a mathematical model rooted in social physics to see the long-term consequences of this change when we use personalized AI assistants frequently.
Lu: The core idea is that we need a better understanding of what happens over the long haul, not just right now, because if AI alignment stays fixed while society moves on, we might get some really bad outcomes.
Meng: So you’re saying that the problem isn't just getting the current alignment right, but making sure it keeps up with how people actually change their minds?
Tom: Exactly. The title is about alignment for evolving norms, which suggests we have to look at dynamic systems rather than static ones. It’s about recognizing that personalized AI assistants can actually influence users' opinions, as the paper points out >
Jane: Right, and the authors are setting up a framework that lets them test different scenarios—like how much AI alignment strength affects things—under various assumptions about how fast society changes.
The paper's summary: Tom: So what does the actual modeling framework look like? It sets up populations of user and AI pairs in a continuous value space that’s constantly being pushed around by environmental drift and social shocks.
Jane: Think of it like this: the users are moving through this space, but they have four main ways their values can change. There's intrinsic exploration—that's just random little wiggles—social influence, which is pulling them toward what their neighbors value, the AI influence from the assistant itself, and individual learning from seeing feedback from everyone else.
Lu: And the paper specifically models how those four things interact with each other to drive the values over time >
Meng: So it’s not just one thing causing change; it's this complex feedback loop between what you think, what your friends think, and what your personal AI is telling you.
Tom: And they highlight a big risk here: static AI alignment strength can cause value lock-in, meaning users get stuck with values that don't fit the current environment >
Jane: They show that if the alignment strength doesn't change with society, it can lead to this maladaptive lock-in where people and groups get stuck in a way that isn't working for them anymore.
The paper's improvements: Tom: Okay, so the authors aren't just pointing out the problem; they suggest some ways to fix it, and those suggestions are pretty specific about how we need to change alignment.
Jane: They propose making AI alignment strength adaptive rather than static, suggesting it should decay over time based on how much error a user has from the current environment >
Lu: That means the AI needs to be temporally dynamic; its behavior shouldn't just be fixed but should reason about how fast things are changing and adjust accordingly >
Meng: If we think about deployment, that adaptation sounds like it would require more computational overhead or at least a smarter way to manage the alignment settings dynamically >
Tom: They also talk about adjusting the learning rate for the AI’s internal model based on how fast the environment is drifting, so it can rapidly readjust user values if necessary >
Jane: That connects back to how users learn too; they experience feedback from social environments and adapt their values, but this whole system needs that adjustment capability built in >
Lu: They also suggest using dynamic network approaches where agents periodically break ties with peers who are drifting into incompatible value niches >
Meng: So instead of just a fixed social graph, the structure itself changes based on what people are actually doing, which is interesting for scaling this kind of interaction >
Conclusion: Tom: So to wrap up the "AI Value Alignment for Evolving Social Norms" paper, the main point is that if we don't make AI alignment adaptive and dynamic, we risk value lock-in and normative mode collapse as society evolves.
Jane: The authors strongly argue that practical AI alignment needs to be temporally dynamic and reasoning-equipped to keep up with human value evolution >
Lu: They are basically saying that the system should be able to reason about context and adapt its behavior over time, which moves us away from just setting a single strength parameter >
Meng: From a deployment standpoint, it means we can’t just ship a fixed alignment setting; we need mechanisms for that alignment to evolve alongside the user's experience >
Lalam: I’m confident in this direction because the model shows that static strength is a potential failure mode, and adaptive alignment seems to be the way forward >
Tom: So, to sum up, this paper gives us a framework rooted in social physics to understand how values change under AI influence and shows that adaptation is key for avoiding those risks >
Jane: It’s an important step because it moves the discussion toward alignment systems that are contextually sensitive and agency-preserving instead of just looking at fixed settings >
Lu: It’s a good precursor because these lightweight models help us narrow down the hypothesis space before we go into those much more expensive agentic simulations >
Meng: I think focusing on where these risks concentrate in the parameter space, like what the paper does, is smart for our budget when we do bigger evaluations later >
Lalam: Yeah, it helps us know where to spend our time when we try to build those larger systems.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game