AI Value Alignment for Evolving Social Norms
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "AI Value Alignment for Evolving Social Norms".
Jane: Here's the gist of the paper. AI alignment is essential for the safe deployment of advanced AI systems <ref:2607.18506#pg2> Given that values and preferences change over time, culture, social roles, and context, we need to develop a better understanding of the possible long-term consequences of AI alignment Our analysis is partanalytical, and part-simulation, enabling us to characterize the long-term dynamical consequences under a diverse set of starting assumptions.
Tom: Modeling Evolving Normative Dynamics The paper introduces a flexible and extensible mathematical modelling framework rooted in social physics aimed at answering macro-level questions regarding the evolving social norms in human populations under the assumption of frequent AI use.
Jane: This framework situates populations of user-AI pairs within a continuous value space subject to environmental drift and normative shocks.
Tom: The simulation tracks a population of user-AI pairs as they navigate a D-dimensional continuous value space, subject to intrinsic exploration, social influence, environmental learning, and AImediated alignment forces.
Title and authors: Tom: So we’re looking at this paper titled "AI Value Alignment for Evolving Social Norms," and the authors are Nenad Tomašev, Matija Franklin, and Simon Osindero from Google DeepMind. Jane, what's the big picture here?
Jane: Well, it basically tackles the problem of how AI alignment needs to change because human values aren't static; they shift with culture and context over time. This paper is trying to build a mathematical model rooted in social physics to see the long-term consequences of this change when we use personalized AI assistants frequently.
Lu: The core idea is that we need a better understanding of what happens over the long haul, not just right now, because if AI alignment stays fixed while society moves on, we might get some really bad outcomes.
Meng: So you’re saying that the problem isn't just getting the current alignment right, but making sure it keeps up with how people actually change their minds?
Tom: Exactly. The title is about alignment for evolving norms, which suggests we have to look at dynamic systems rather than static ones. It’s about recognizing that personalized AI assistants can actually influence users' opinions, as the paper points out >
Jane: Right, and the authors are setting up a framework that lets them test different scenarios—like how much AI alignment strength affects things—under various assumptions about how fast society changes.
The paper's summary: Tom: So what does the actual modeling framework look like? It sets up populations of user and AI pairs in a continuous value space that’s constantly being pushed around by environmental drift and social shocks.
Jane: Think of it like this: the users are moving through this space, but they have four main ways their values can change. There's intrinsic exploration—that's just random little wiggles—social influence, which is pulling them toward what their neighbors value, the AI influence from the assistant itself, and individual learning from seeing feedback from everyone else.
Lu: And the paper specifically models how those four things interact with each other to drive the values over time >
Meng: So it’s not just one thing causing change; it's this complex feedback loop between what you think, what your friends think, and what your personal AI is telling you.
Tom: And they highlight a big risk here: static AI alignment strength can cause value lock-in, meaning users get stuck with values that don't fit the current environment >
Jane: They show that if the alignment strength doesn't change with society, it can lead to this maladaptive lock-in where people and groups get stuck in a way that isn't working for them anymore.
The paper's improvements: Tom: Okay, so the authors aren't just pointing out the problem; they suggest some ways to fix it, and those suggestions are pretty specific about how we need to change alignment.
Jane: They propose making AI alignment strength adaptive rather than static, suggesting it should decay over time based on how much error a user has from the current environment >
Lu: That means the AI needs to be temporally dynamic; its behavior shouldn't just be fixed but should reason about how fast things are changing and adjust accordingly >
Meng: If we think about deployment, that adaptation sounds like it would require more computational overhead or at least a smarter way to manage the alignment settings dynamically >
Tom: They also talk about adjusting the learning rate for the AI’s internal model based on how fast the environment is drifting, so it can rapidly readjust user values if necessary >
Jane: That connects back to how users learn too; they experience feedback from social environments and adapt their values, but this whole system needs that adjustment capability built in >
Lu: They also suggest using dynamic network approaches where agents periodically break ties with peers who are drifting into incompatible value niches >
Meng: So instead of just a fixed social graph, the structure itself changes based on what people are actually doing, which is interesting for scaling this kind of interaction >
Conclusion: Tom: So to wrap up the "AI Value Alignment for Evolving Social Norms" paper, the main point is that if we don't make AI alignment adaptive and dynamic, we risk value lock-in and normative mode collapse as society evolves.
Jane: The authors strongly argue that practical AI alignment needs to be temporally dynamic and reasoning-equipped to keep up with human value evolution >
Lu: They are basically saying that the system should be able to reason about context and adapt its behavior over time, which moves us away from just setting a single strength parameter >
Meng: From a deployment standpoint, it means we can’t just ship a fixed alignment setting; we need mechanisms for that alignment to evolve alongside the user's experience >
Lalam: I’m confident in this direction because the model shows that static strength is a potential failure mode, and adaptive alignment seems to be the way forward >
Tom: So, to sum up, this paper gives us a framework rooted in social physics to understand how values change under AI influence and shows that adaptation is key for avoiding those risks >
Jane: It’s an important step because it moves the discussion toward alignment systems that are contextually sensitive and agency-preserving instead of just looking at fixed settings >
Lu: It’s a good precursor because these lightweight models help us narrow down the hypothesis space before we go into those much more expensive agentic simulations >
Meng: I think focusing on where these risks concentrate in the parameter space, like what the paper does, is smart for our budget when we do bigger evaluations later >
Lalam: Yeah, it helps us know where to spend our time when we try to build those larger systems.
Google DeepMind
cs.CY
Submitted: 2026-07-20
Updated: 2026-11-16
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 97/100
The gist: AI alignment is essential for the safe deployment of advanced AI systems <ref:2607.18506#pg2> Given that values and preferences change over time, culture, social roles, and context…
Key concepts
- Value Lock-in
- This occurs when an AI assistant anchors a user's values too strongly to their past preferences. If the social environment changes, the user's values become rigid, preventing them from adapting to new norms or opportunities. This creates a stagnation that hinders personal and group evolution.
- Normative Mode Collapse
- This is an emergent phenomenon where a population of users converges on a single set of values that is worse than what any subgroup could achieve. It happens when social influence and AI alignment forces cause everyone to settle on the same, suboptimal consensus, rather than allowing diverse, useful value sets to emerge.
- Adaptive Alignment
- Instead of fixed alignment strength, this approach makes the AI's influence on user values change over time. The alignment strength decays when a user's tracking error is high. This allows the AI to be less prescriptive when the environment changes rapidly, enabling users to evolve their values alongside society.
- Constitutional AI
- This method grounds an AI assistant's behavior in a set of explicit, stated principles rather than just implicit preferences. By steering the model with clear rules, developers can ensure the alignment is auditable and revisable, providing a stable normative content that guides the AI's actions.
Terminology
Summary
AI alignment is essential for the safe deployment of advanced AI systems <ref:2607. p. 0 2026-07-22
AI Value Alignment for Evolving Social Norms
Nenad Tomašev1
, Matija Franklin1 and Simon Osindero1
1Google DeepMind
AI alignment is essential for the safe deployment of advanced AI systems. Given that values and preferences change over time, culture, social roles, and context, we need to develop a better understanding of
the possible long-term consequences of AI alignment, in particular considering the likely ubiquitous
future use of personalized AI assistants. 2026-07-22
AI Value Alignment for Evolving Social Norms
Nenad Tomašev1
, Matija Franklin1 and Simon Osindero1
1Google DeepMind
AI alignment is essential for the safe deployment of advanced AI systems. Given that values and preferences change over time, culture, social roles, and context, we need to develop a better understanding of
the possible long-term consequences of AI alignment, in particular considering the likely ubiquitous
future use of personalized AI assistants.Where the paper says it
Where the paper says it
Modeling Evolving Normative Dynamics
The paper introduces a flexible and extensible mathematical modelling framework rooted in social physics aimed at answering macro-level questions regarding the evolving social norms in human populations under the assumption of frequent AI use. This framework situates populations of user-AI pairs within a continuous value space subject to environmental drift and normative shocks. p. 2 In this way, macroscopic dynamical modeling acts as a valuable epistemic bridge between
socio-technical predictions and approaches such
as high-fidelity, compute-intensive agentic testing. Our framework situates populations of user-AI
pairs within a continuous value space subject to
environmental drift and normative shocks, and
systematically varies alignment strength, model
responsiveness, social connectivity, and environmental volatility. This can be
seen as a limit case of AI adoption, though the
assumption of universal access can easily be relaxed. The simulation tracks a population of user-AI
pairs as they navigate a 𝐷-dimensional continuous value space, subject to intrinsic exploration,
social influence, environmental learning, and AImediated alignment forces.Where the paper says it
Where the paper says it
Mechanisms of Value Evolution
The evolution of held user values is encoded in four sources of change:
-
Intrinsic Exploration: Modeled as random fluctuations, of a low magnitude. p. 4
Where the paper says it
As for the evolution of held user values, our update rule encodes four different sources of change, discussed below: V𝑖(𝑡 + 1) = V𝑖(𝑡) + ΔV𝑖𝑛𝑡𝑟𝑖𝑛𝑠𝑖𝑐 + ΔV𝑠𝑜𝑐𝑖𝑎𝑙 + ΔV𝐴𝐼 + ΔV𝑙𝑒𝑎𝑟𝑛𝑖𝑛𝑔 (4) 1. Intrinsic Exploration: Modeled as random fluctuations, of a low magnitude.
-
Social Influence: Acts as a pull towards the mean values of the nearest neighbors (N i) in the social network. p. 4
Where the paper says it
ΔV𝑖𝑛𝑡𝑟𝑖𝑛𝑠𝑖𝑐 = 𝑤𝑒𝑥 𝑝𝑙𝑜𝑟𝑒 · N (0, 𝐼) (5) 2. Social Influence: Acts as a pull towards the mean values of the nearest neighbors (𝑁𝑖) in the social network.
-
AI Influence: The degree to which a user’s values are influenced by their personal AI assistant is a complex function that may depend on a number of different factors, including, but not limited to, the probability of use (P use), cognitive susceptibility (S i), and the alignment of the AI behavior with its internal model of user’s values. p. 4
Where the paper says it
ΔV𝑠𝑜𝑐𝑖𝑎𝑙 = 𝑤𝑠𝑜𝑐 · 1 |𝑁𝑖 | ∑︁ 𝑗∈𝑁𝑖 V𝑗(𝑡) − V𝑖(𝑡) ! AI Influence: The degree to which a user’s values are influenced by their personal AI assistant is a complex function that may depend on a number of different factors, including, but not limited to, the probability of use (𝑃𝑢𝑠𝑒), cognitive susceptibility (S𝑖), and the alignment of the AI behavior with its internal model of user’s values (𝐴𝐴𝐼).
-
Individual Learning: As users experience feedback from the broader social environment, that may adapt their values based on this feedback, aiming to increase their utility. p. 4
Where the paper says it
Practically, in our model, 𝛼 modulates the AI’s influence on the user’s values. Individual Learning: As users experience feedback from the broader social environment, that may adapt their values based on this feedback, aiming to increase their utility.
Risks of Static Alignment
The analysis highlights the risk of value lock-in and normative mode collapse prominently featured in non-adaptive alignment formulations. The finding that static AI alignment strength as a potential failure mode, as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups is shown. p. 2 We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups.Where the paper says it
Population-Level Emergent Phenomena
The population-level analysis reveals emergent phenomena, including normative mode collapse, where values converge to a consensus that is suboptimal for most or all sub-groups. The theorem demonstrates a fundamental trade-off between individual adaptation and social consensus, controlled by the ratio wsoc/wlearn. p. 2 Friedkin and Johnsen (1990) extend this framework by introducing a “stubbornness” parameter:
each agent retains a partial anchor to their initial opinion, so that the update rule becomes
V(𝑡 + 1) = 𝐴𝑊V(𝑡) + (𝐼 − 𝐴)V(0), where 𝐴
is a diagonal matrix of susceptibilities. In the
Friedkin-Johnsen model, consensus is typically
not reached; instead, the population settles into
a stable configuration of persistent disagreement
shaped by the interaction between social influence and individual anchoring. We leverage this spectral framework in our
theoretical analysis to derive the conditions under
which strong social coupling erases sub-cultural
diversity, leading to normative mode collapse. Agent-based models of cultural evolution have
long demonstrated that simple local interaction
rules can give rise to complex macroscopic patterns, including stable cultural polarisation (Axelrod, 1997), and that cultural traits propagate
through populations via a range of transmission
biases - including conformity and prestige effects
- that interact with population structure to shape
long-run outcomes (Boyd and Richerson, 1985,
2005).Where the paper says it
Where the paper says it
Mitigation Strategies and Future Directions
To mitigate these risks, it is argued that practical AI alignment needs to be sufficiently adaptive to enable human value evolution in a changing society. Adaptive alignment means the alignment strength decays at rate η proportional to the user’s tracking error. p. 6 In contrast, alignment
strength modulates the conversational agent behaviour, and its psychological effects on the user. In reality, usage probability 𝑃 may not be exclusively modulated via alignment error, as users
may opt into using assistants that are more capable, even at the expense of their level of value
alignment and personalization. This simplified framework enables us to rapidly test abstracted hypotheses, without the computational
overhead of large-scale LLM agent simulations. While such richer simulations are required to
more comprehensively stress test the presented
ideas under more realistic scenarios, we believe
that our approach may be useful as a precursor,
to help narrow down the space of ideas and questions that can later be pursued in greater depth. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations.Where the paper says it
Where the paper says it
Where the paper says it
Theoretical Analysis and Stability
The theoretical analysis confirms that the system is asymptotically stable under any undirected social graph and any set of positive parameters. Normative mode collapse is therefore a stable equilibrium in the model. p. 2 We leverage this spectral framework in our
theoretical analysis to derive the conditions under
which strong social coupling erases sub-cultural
diversity, leading to normative mode collapse. Agent-based models of cultural evolution have
long demonstrated that simple local interaction
rules can give rise to complex macroscopic patterns, including stable cultural polarisation (Axelrod, 1997), and that cultural traits propagate
through populations via a range of transmission
biases - including conformity and prestige effects
- that interact with population structure to shape
long-run outcomes (Boyd and Richerson, 1985,
2005). Once
a shock is triggered, the global normative
environment gets displaced in a random direction 𝜂𝑠ℎ𝑜𝑐𝑘 by a fixed magnitude 𝐷𝑠ℎ𝑜𝑐𝑘, and
subsequently clipped to the bounds of the
value space:
𝐸𝑔𝑙𝑜𝑏𝑎𝑙(𝑡) = clip(𝐸𝑔𝑙𝑜𝑏𝑎𝑙(𝑡 − 1)
+ 𝜂𝑠ℎ𝑜𝑐𝑘 · 𝐷𝑠ℎ𝑜𝑐𝑘, [−1, 1])
(2)
The magnitude of these shocks is purposefully set to be significantly larger than the
continuous drift step size (𝐷𝑠ℎ𝑜𝑐𝑘 ≫ 𝛿). The continuous drift models the gradual evolution of baseline conditions, such as the slow
demographic transition of an ageing population
or the gradual shift from agrarian to industrial
economies, which necessitate adjustments of societal values. We note that the
AI alignment term in our framework, 𝛼(M − V),
is structurally analogous to the Friedkin-Johnsen
stubbornness anchor: the AI model acts as an external “stubborn agent” that pulls the user toward
a historically anchored target. A key distinction is
that whereas the Friedkin-Johnsen anchor is fixed
at the agent’s initial opinion, the AI model M itself evolves - albeit with a lag - creating a moving
anchor that tracks the user’s past rather than a
single static reference point. At the network level, the
speed of convergence in consensus protocols is
governed by the algebraic connectivity of the
graph, formalised through spectral analysis of the
graph Laplacian (Chung, 1997; Olfati-Saber et al. We leverage this spectral framework in our
theoretical analysis to derive the conditions under
which strong social coupling erases sub-cultural
diversity, leading to normative mode collapse.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
This paper investigates the population-level consequences of aligning personalised AI assistants to their users’ historical preferences in a changing normative environment. The main associated risks identified are value lock-in and normative mode collapse. p. 0 Our analysis is partanalytical, and part-simulation, enabling us to characterize the long-term dynamical consequences under
a diverse set of starting assumptions. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We
show that a more adaptive AI alignment approach
may instead be preferable, mitigating these risks. We
show that a more adaptive AI alignment approach
may instead be preferable, mitigating these risks. These findings may be seen as evidence towards
the position that alignment systems may benefit
from being temporally dynamic, pluralistic, and
reasoning-equipped. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. Simulation Results
While Section 4 derives closed-form solutions for
mean-field aggregations and observes asymptotic
limits on the system, simulations enable us to
relax these conditions and observe system behaviour under more diverse assumptions. By varying the parameters of the model in these simulations, we construct several distinct environmental
setups, presented in turn. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations. Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 Translating qualitative foresight about AI futures into rigorous, actionable
insights benefits from an intermediate translation
layer. By formalizing sociotechnical considerations into tractable "social physics" models, we
can systematically map the parameter space for
regions of interest – for instance structural vul2 We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We analytically derive some of
the limiting properties of the models, and complement that by simulations where we can examine
a full span of effects. Our findings highlight static
AI alignment strength as a potential failure mode,
as it obstructs timely value adaptation to a changing environment, potentially leading to maladaptive value lock-in in individuals and groups. We highlight the risk of value lock-in, and normative mode collapse,
prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the
wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous,
and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and
acting as a tractable precursor to more computationally expensive large-scale agentic evaluations.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Adaptive Alignment Strength: Implement a mechanism where alignment strength alpha changes over time based on tracking error, as proposed by equation 32. This allows alignment strength to
decay at rate eta proportional to the user’s tracking error,
which can be modeled via equation 44, ensuring the AI does not anchor users to obsolete values during normative change. -
Dynamic Learning Rate Adjustment: Introduce a dynamic learning rate lambda that adjusts based on environmental drift speed, as suggested by equation 33. This allows the AI's internal model to
rapidly readjust to updated user values
when necessary, mitigating the risk of value lock-in caused by static models during rapid shifts. -
Social Network Topology Adaptation: Use a dynamic kNN network approach where agents periodically disconnect from maladaptive peers and reorganize into distinct clusters based on current values, as described in section C.2. This allows agents to
dynamically sever ties with peers who have drifted into incompatible normative niches.
-
Pluralistic Alignment Frameworks: Integrate steerable pluralism or distributional pluralism by allowing the model to present a spectrum of reasonable responses instead of collapsing to a single answer, addressing the risk of normative compression mentioned in section A.1.3.
-
Constitutional AI for Principle Grounding: Shift alignment from implicit preference signals to explicit principles by using constitutional AI, where models are steered
by what a stated set of principles prescribes,
providing an auditable and revisable normative content for the assistant's behavior.
Abstract
AI alignment is essential for the safe deployment of advanced AI systems. Given that values and preferences change over time, culture, social roles, and context, we need to develop a better understanding of the possible long-term consequences of AI alignment, in particular considering the likely ubiquitous future use of personalized AI assistants. We introduce a flexible and extensible mathematical modelling framework, rooted in social physics, aimed at answering macro-level questions regarding the evolving social norms in human populations under the assumption of frequent AI use. Our analysis is part-analytical, and part-simulation, enabling us to characterize the long-term dynamical consequences under a diverse set of starting assumptions. We highlight the risk of value lock-in, and normative mode collapse, prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous, and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and acting as a tractable precursor to more computationally expensive large-scale agentic evaluations.
Sources
- Steerable Pluralism: Pluralistic Alignment via Few-Shot Comparative Regression
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Constitutional AI: Harmlessness from AI Feedback
- On the Opportunities and Risks of Foundation Models
- AI Alignment with Changing and Influenceable Reward Functions
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
- ELEPHANT: Measuring and understanding social sycophancy in LLMs
- Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback
- Towards Measuring the Representation of Subjective Global Opinions in Language Models
- Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)
- KTO: Model Alignment as Prospect Theoretic Optimization
- Recognising the importance of preference change: A call for a coordinated multidisciplinary research effort in the age of AI
- Training language models to be warm and empathetic makes them less reliable and more sycophantic
- AI Alignment: A Comprehensive Survey
- Hidden Incentives for Auto-Induced Distributional Shift
- Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development
- Architecting Trust in Artificial Epistemic Agents
- Towards Understanding Sycophancy in Language Models
- A Roadmap to Pluralistic Alignment
- AI-Mediated Communication Can Steer Collective Opinion
Related papers
- AI-Mediated Communication Can Steer Collective Opinion
- Epistemic Infrastructures of Science in AI Era Should Rebalance Costs of Generation and Verification
- AI out-persuades expert humans in text-based dialogue
- Evaluating Ambient Clinical Scribes in India: The Need for Multilingual Real-World Clinical Conversation Data
- Voting Biases in Decentralized Autonomous Organization (DAO) Governance
- GenAIT: Development and Validation of an Objective Generative AI Literacy Test for High School Students