Population Physics, Population Problems: Safety and Emergence in LLM Societies
cs.MA, cs.CL
Submitted: 2026-09-27
Updated: 2026-09-27
Code: https://github.com/openai/evals
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: The collective behaviour of large language model (LLM) societies is not the sum of their individual outputs.
Terminology
Abstract
The collective behaviour of large language model (LLM) societies is not the sum of their individual outputs. It yields statistically distinct, sometimes-unpredictable phenomena, for which the tools we use to study single agents may not scale. Due to recent incidents involving autonomous agentic systems, however, understanding these systems is paramount. For that we introduce a framework for measuring self-organisation in LLM social systems and apply it to three such systems: a Schelling grid, a social network (Moltbook), and a Twitter-like misinformation simulation ('Rogue'). All three exhibit statistically significant self-organisation. Moreover, their relaxation dynamics vary with the environmental information available to the agents, with open-ended systems (Moltbook, Rogue) exhibiting sharp, phase-transition-like dynamics. Further results show that population-level pathologies can emerge even when the LLMs are safety-tuned or monitored, being primarily driven by the coordinated activity of a population subset. We also show when self-organisation does not emerge under two additional scenarios (a commons dilemma, GovSim, and a LLM-as-a-judge deliberation scheme, ChatEval). We argue that measuring signatures of this kind offers a lightweight, agent-agnostic diagnostic layer for detecting coordinated collective behaviour in deployed multi-agent systems without relying on natural language or model versioning.
Sources
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
- What Do AI Agents Talk About? Discourse and Architectural Constraints in the First AI-Only Social Network
- When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems
- If LLMs Have Human-Like Attributes, Then So Does Age of Empires II
- Qwen3 Technical Report
- Form Without Function: Agent Social Behavior in the Moltbook Network
Related papers
- Highway Congestion Reduction through Reinforcement Learning Based Eulerian Headway Control
- ORACLE-SWE: Quantifying the Contribution of Critical Information Signals on SWE Agents
- You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agents
- Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems
- MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization
- PeroMAS: A Multi-agent System of Perovskite Material Discovery