Modelling Opinion Dynamics at Scale with Deep MARL

arXiv:2606.07487 · cs.MA, cs.GT, cs.SI · Submitted 2026-06-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Modelling Opinion Dynamics at Scale with Deep MARL".

Jane: Here's the gist of the paper. Modelling opinion dynamics typically relies on hand-crafted local interaction rules to study emergent macroscopic phenomena such as consensus and polarisation <ref:2606.07487#pg2>.

Tom: To explore the potential of MARL for opinion dynamics, we introduce a GPU-accelerated consensus and truth-finding game that scales to populations of up to 1000 agents, comparable to many real-world social sub-networks <ref:2606.07487#pg2>.

Jane: Model Formulation The paper models opinion dynamics as a networked partially observable stochastic game (N-POSG) defined by the 10-tuple (n, G, S, The paper models opinion dynamics as a networked partially observable stochastic game (N-POSG) given by the 10-tuple <ref:2606.07487#pg4>.

Tom: Agents sample local actions from a policy conditioned on their local action-observation history (AOH) <ref:2606.07487#pg4>.

Paper summary: Tom: So we’ve looked at the mechanics of this paper, which is "Modelling Opinion Dynamics at Scale with Deep MARL," and it shows how agents can learn consensus and truth-finding through reinforcement learning on a GPU-accelerated framework that handles up to one thousand agents.

Jane: The authors are really pointing toward the idea that in environments like social media, high levels of conformity aren't just neutral; they become maladaptive, leading to reduced collective accuracy and more lying behavior because the incentives are misaligned.

Lu: They used the structure of this networked partially observable stochastic game to show how agents learn these behaviors directly by optimizing simple rewards rather than needing pre-programmed interaction rules.

Meng: From a deployment standpoint, the scaling was impressive—training up to one thousand agents took about thirty-five minutes on a single NVIDIA A100 GPU—but they did admit that scaling deep MARL methods to much larger populations is still a significant computational challenge.

Lalam: The paper suggests that while conformity helps in small, dynamic networks like hunter-gatherers, that same conformity behavior becomes harmful when scaled up in massive social media environments.

Tom: So the title itself, "Modelling Opinion Dynamics at Scale with Deep MARL," highlights that they are tackling the problem of large-scale opinion dynamics using deep reinforcement learning to model how opinions evolve across a network.

Jane: What this means for us listening is that we might be seeing real evidence now, through these models, of why certain social media behaviors—like following trends or just agreeing with the majority—aren't always beneficial when there's a mix of truth-seeking and conformity incentives.

Lu: The paper’s main contribution is using this deep MARL approach to model human behavior using simple reward heuristics, which is a different way than just looking at how opinions spread through existing algorithms.

Meng: And they did provide some concrete evidence by showing that for agents with strong conformity, the importance structure they predict in a Bluesky network dataset matches what we see in real human data.

Lalam: They also showed that the model best predicts this node-importance structure in a Bluesky network when the conformity parameter alpha is set to zero point two, which suggests that even slightly stronger conformity can be predictive of human social structures.

Tom: And finally, they pointed out a limitation: their model doesn't account for things like recommendation systems or bounded human attention, which is where future research needs to go.

Jane: So the paper gives us a framework and some concrete comparisons—hunter-gatherers versus social media—to think about how we might need to rethink our models of online influence and group behavior.

Conclusion: Tom: So we've looked at how this paper uses deep reinforcement learning to model opinion dynamics at scale, and now we need to talk about what that actually means for us in plain English.

Jane: It basically takes a complex social interaction—how people form opinions on a network—and tries to learn the rules by just trying things out, using these machine learning agents.

Tom: Right. The title is "Modelling Opinion Dynamics at Scale with Deep MARL." That tells you two big things upfront: they are dealing with huge groups of agents and they are using deep reinforcement learning.

Jane: And what it means is that instead of writing a long list of rules for how people should behave, the AI agents learn the best way to act based on what happens in their neighborhood over time.

Tom: Exactly. They're not just guessing; they have this reward signal that tells them if they are being accurate with the truth and also if they are conforming to their neighbors.

Jane: That reward structure is key because it shows how human incentives—being right versus being accepted—drive the whole system forward.

Tom: And the authors found something interesting about where conformity works, specifically comparing massive social media networks against smaller hunter-gatherer groups.

Jane: Because they found that high conformity isn't always helpful in huge online communities, suggesting that what works for small groups might backfire in big social media environments.

Tom: It points to a mismatch between how we evolved to interact and the way we interact now, which could explain a lot of the misinformation we see online.

Jane: It raises a question for us about whether these learned behaviors are just descriptive or if they actually drive the world in unexpected ways.

Lukas Seier, Brandon Kaplowitz, Sebastian Towers, Richard Bailey, Jakob Foerster

University of Oxford

cs.MA, cs.GT, cs.SI

Submitted: 2026-06-05

Updated: 2026-11-09

Comments: 39 pages, 31 figures. Accepted at the 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

Code: https://github.com/flipbagels/OpiniMARL

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 77/100

The gist: Modelling opinion dynamics typically relies on hand-crafted local interaction rules to study emergent macroscopic phenomena such as consensus and polarisation <ref:2606.07487#pg2>.

Key concepts

Networked Partially Observable Stochastic Game (N-POSG)
This mathematical framework describes the opinion dynamics as a game played on a network where agents cannot see the true state of the environment directly. They only observe local actions and opinions, making it a complex system requiring learning to navigate.
Mixed Incentive Reward
Agents are rewarded based on two things: how accurately their guess matches the actual truth (accuracy) and how much they agree with their neighbors' guesses (conformity). A parameter 'alpha' controls the balance between these two incentives.
Conformity vs. Accuracy Trade-off
The study investigates how agents behave when incentivized to either be truthful or conform to the group. In large social media, high conformity hurts accuracy, while in small hunter-gatherer settings, it can lead to better collective agreement.

Terminology

Summary

Modelling opinion dynamics typically relies on hand-crafted local interaction rules to study emergent macroscopic phenomena such as consensus and polarisation <ref:2606. p. 007487#pg2>. To explore the potential of MARL for opinion dynamics, we introduce a GPU-accelerated consensus and truth-finding game that scales to populations of up to 1000 agents, comparable to many real-world social sub-networks <ref:2606. p. 007487#pg2>.

Model Formulation

The paper models opinion dynamics as a networked partially observable stochastic game (N-POSG) defined by the 10-tuple (n, G, S, The paper models opinion dynamics as a networked partially observable stochastic game (N-POSG) given by the 10-tuple <ref:2606.07487#pg4>. Agents sample local actions from a policy conditioned on their local action-observation history (AOH) <ref:2606. p. 207487#pg4>. The joint policy is defined as π = (πi, π−i, where π−i = The joint policy is defined as π = (πi, π−i) and samples actions at conditioned on the joint AOH τt = (τ1t,. p. 2.., τnt) with probability <ref:2606. p. 007487#pg4>.

Training Signal Construction

Agents have mixed incentives to make guesses that both match a partially observed true state of the environment and conform to the guesses of their neighbours <ref:2606.07487#pg3>. The reward is given by r i(t+1) = αT it + (1 − α)C it, where T it = The reward is given by r i(t+1) = αT it + (1 − α)C it <ref:2606. p. 707487#pg5>. This incentivises accuracy and conformity, with α controlling the weighting <ref:2606. p. 307487#pg5>. The connection between truth-finding and zero-shot coordination under conformity incentives is exploited in oracle consensus protocols <ref:2606. p. 107487#pg4>.

Empirical Validation and Findings

The model validates its ability to recover agent importance structures from graph topology alone via learned attention layers, finding that highly conforming populations most closely match human data <ref:2606.07487#pg3>. In large social media networks such high levels of conformity significantly reduce collective accuracy and promote dishonest agents that lie to fit in <ref:2606. p. 007487#pg2>. Conversely, small, dynamic hunter-gatherer networks are less affected; here, conformity can even improve collective agreement <ref:2606. p. 007487#pg3>.

Scaling and Performance

The GPU-accelerated framework enables training of up to 1000 agents for 106 training steps in 35 minutes on a single NVIDIA A100 GPU <ref:2606.07487#pg3>. The model is more parameter efficient than the baseline, scaling with population size n at a rate of 8n compared to 768n (Appendix C) <ref:2606. p. 707487#pg2>. The perceived importance (PI) centrality measure CPI(i) = The paper defines the perceived importance (PI) centrality measure CPI(i) = 1 NG[i] X j∈NG [i] NG[j]wji <ref:2606. p. 507487#pg2>.

Conclusion on Social Media vs. p. 0 Hunter-Gatherer Networks

High conformity appears maladaptive in social media networks, exhibiting sharp transitions in accuracy around α = 0.3 and α = 0.45, whereas it is advantageous for collective agreement in small, dynamic hunter–gatherer networks <ref:2606. p. 807487#pg2>. This suggests a mismatch between evolved human conformity heuristics and modern social media environments as a potential contributor to misinformation <ref:2606. p. 007487#pg2>.

Limitations

Our model does not account for factors such as recommendation systems, bounded human attention, influence-seeking behaviour, and heterogeneity in agent incentives <ref:2606.07487#pg2>. This presents a promising direction for future research in MARL for opinion dynamics modelling <ref:2606. p. 807487#pg2>.

Datasets Used

The validation used a 1000-node dataset of the Bluesky machine learning community, and also studied a 475-node X/Twitter network of the 117th U.S. Congress <ref:2606. p. 0 p. 007487#pg2>. Additionally, a 37-node GPS-derived proximity dataset of the Hadza hunter-gatherer tribe was used <ref:2606. p. 6 p. 607487#pg2>.

Hyperparameters

The hyperparameters used during training include H = The hyperparameters we use during training are given in Table 1 <ref:2606.07487#pg2>. We set σ (private signal uncertainty) to provide a sufficiently weak signal that requires agents to share information over multiple time steps, while remaining below the learnability threshold during training <ref:2606. p. 707487#pg2>. The entropy coefficient is tuned to have a significant effect on the learning for truth-seeking agents, with greater entropy improving learnability during training <ref:2606. p. 807487#pg2>.

Ablation Study

Ablating the GRU component decreases the performance of static graphs significantly <ref:2606.07487#pg2>. Ablating agent IDs decreases the performance of static graphs significantly <ref:2606.07487#pg2>. This highlights the importance of the time axis when repeatedly communicating with the same neighbours, and the benefits of knowing which neighbours are sending which signals <ref:2606. p. 507487#pg2>.

Final Conclusion

We develop a GPU-accelerated framework for modelling opinion dynamics on a consensus and truth-finding game at the scale of 1000 agents on a single GPU <ref:2606.07487#pg2>. We extend the OP algorithm to our general-sum setting, accurately modelling information diffusion by preventing the emergence of unrealistic conventions <ref:2606. p. 807487#pg2>. Our model best predicts node-importance structure in a Bluesky network dataset for agents with strong conformity (α = 0. p. 82) <ref:2606. p. 007487#pg2>. We show that high conformity provides benefits for collective agreement in hunter-gatherer networks, but becomes maladaptive in social media, suggesting an evolutionary mismatch that increases population-level distrust and susceptibility to misinformation <ref:2606. p. 807487#pg2>.

References

The references listed are: Daron Acemoglu, Munther A Dahleh, Ilan Lobel, and Asuman Ozdaglar. Bayesian learning in social networks. p. 9 The Review of Economic Studies, 78(4):1201–1236, 2011 <ref:2606. p. 907487#pg6>. Stéphane Airiau, Sandip Sen, and Daniel Villatoro. p. 9 Emergence of conventions through social learning: Heterogeneous learners in complex networks. p. 9 Autonomous Agents and Multi-Agent Systems, 28(5):779–804, 2014 <ref:2606. p. 907487#pg6>. Coren L Apicella, Frank W Marlowe, James H Fowler, and Nicholas A Christakis. p. 9 Social networks and cooperation in hunter-gatherers. p. 9 Nature, 481(7382):497–501, 2012 <ref:2606. p. 907487#pg6>. Solomon E. Asch. Effects of group pressure upon the modification and distortion of judgments. p. 9 In Harold Guetzkow, editor, Groups, leadership and men: Research in human relations, pages 177–190. p. 9 Carnegie Press, 1951 <ref:2606. p. 907487#pg6>. Christoph Aymanns, Jakob Foerster, Co-Pierre Georg, and Matthias Weber. p. 9 Fake news in social networks. p. 9 arXiv preprint arXiv:1708. p. 906233, 2017 <ref:2606. p. 007487#pg6>. Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. p. 9 Emergent tool use from multi-agent autocurricula. p. 9 In International Conference on Learning Representations, 2019 <ref:2606. p. 907487#pg6>. Abhijit V Banerjee. p. 9 A simple model of herd behavior. p. 9 The Quarterly Journal of Economics, 107(3):797–817, 1992 <ref:2606. p. 907487#pg6>. Sven Banisch and Eckehard Olbrich. p. 9 Opinion polarization by learning from social feedback. p. 9 The Journal of Mathematical Sociology, 43(2):76–103, 2019 <ref:2606. p. 907487#pg6>. Andrea Baronchelli. p. 9 The emergence of consensus: a primer. p. 9 Royal Society Open Science, 5(2):172189, 2018 <ref:2606. p. 0 p. 907487#pg6>. Daniel S Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. p. 9 The complexity of decentralized control of Markov decision processes. p. 9 Mathematics of Operations Research, 27(4):819–840, 2002 <ref:2606. p. 907487#pg6>. Sushil Bikhchandani, David Hirshleifer, and Ivo Welch. p. 9 A theory of fads, fashion, custom, and cultural change as informational cascades. p. 9 Journal of Political Economy, 100(5):992–1026, 1992 <ref:2606. p. 907487#pg6>. Avrim Blum, MohammadTaghi Hajiaghayi, Katrina Ligett, and Aaron Roth. p. 9 Regret minimization and the price of total anarchy. p. 9 In Proceedings of the fortieth annual ACM Symposium on Theory of Computing, pages 373–382, 2008 <ref:2606. p. 907487#pg6>. James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne and Qiao Zhang. p. 9 JAX: composable transformations of Python+NumPy programs. p. 9 2018 <ref:2606.07487#pg6>. Claudio Castellano, Santo Fortunato, and Vittorio Loreto. p. 9 Statistical physics of social dynamics. p. 9 Reviews of Modern Physics, 81(2):591–646, 2009 <ref:2606. p. 907487#pg6>. Damon Centola, Robb Willer, and Michael Macy. p. 9 The emperor’s dilemma: A computational model of self-enforcing norms. p. 9 American Journal of Sociology, 110(4):1009–1040, 2005 <ref:2606. p. 907487#pg6>. Arun G Chandrasekhar, Horacio Larreguy, and Juan Pablo Xandri. p. 9 Testing models of social learning on networks: Evidence from a lab experiment in the field. p. 9 Technical report, National Bureau of Economic Research, 2015 <ref:2606. p. 907487#pg6>. Justin Chen, Swarnadeep Saha, and Mohit Bansal. p. 9 Reconcile: Round-table conference improves reasoning via consensus among diverse LLMs. p. 9 In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7066–7085, 2024 <ref:2606. p. 907487#pg6>.

The paper constructs a dataset of the Bluesky machine learning community, and also studies a 475-node X/Twitter network of the 117th U.S. Congress <ref:2606.07487#pg2>. Additionally, a 37-node GPS-derived proximity dataset of the Hadza hunter-gatherer tribe was used <ref:2606.07487#pg2>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improve agent learning by using a GPU-accelerated consensus and truth-finding game that scales to populations of up to 1000 agents, allowing for training of up to 106 training steps in 35 minutes on a single NVIDIA A100 GPU. This enables the modeling of large social sub-networks.

  2. Implement an extended other-play algorithm for general-sum social interactions to prevent unrealistic conventions, ensuring that agents do not have a priori preferences over the underlying state when optimizing for conformity.

  3. Develop a model capable of recovering agent importance structures from graph topology alone via a learned attention layer, finding that highly conforming populations most closely match human data. This allows for predicting node-importance structures in real-world datasets like the Bluesky network.

  4. Design an AI system that can distinguish between the effects of reward weighting alpha on accuracy across different network topologies, showing that conformity appears maladaptive in social media networks while being advantageous in small, dynamic hunter-gatherer networks.

  5. Incorporate a mechanism to prevent the emergence of dishonest agents by requiring training and evaluation with the other-play symmetry operator (OP) to avoid unrealistic conventions. This ensures that agents learn information diffusion rather than simply converging on a convention like always output 0.

  6. Enhance graph analysis by calculating perceived importance (PI) using the measure CPI(i) = 1 / NG[i] X j in NG [i] NG[j] wji, which is designed such that if every agent divides their attention uniformly, every agent’s CPI is exactly 1. This provides a node-level metric for comparing learned weights against empirical data.

Sources

Related papers