Claims of emergence in AI should be grounded in information decomposition

arXiv:2610.28479 · cs.AI, cs.HC, cs.IT, cs.LG, cs.MA, math.IT · Submitted 2026-10-20 · Read on arXiv

cs.AI, cs.HC, cs.IT, cs.LG, cs.MA, math.IT

Submitted: 2026-10-20

Updated: 2026-10-20

Comments: NeurIPS Spotlight Paper

Journal ref: 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

License: http://creativecommons.org/licenses/by/4.0/

The gist: When a language model develops capabilities not traceable to its individual components, when a multi-agent AI system outperforms each agent on its own, or when a human-AI pair succeeds where either

Abstract

When a language model develops capabilities not traceable to its individual components, when a multi-agent AI system outperforms each agent on its own, or when a human-AI pair succeeds where either fails in isolation, AI researchers describe it as a case of ``synergy'' or ``emergence''. However, these are usually informal assessments not supported by a theoretical framework, making these claims impossible to compare or even meaningfully dispute. Here we argue that information decomposition can serve as a unifying formal framework to operationalize assessments about synergy in AI systems. Information decomposition provides a way to distinguish qualitatively different types of information, including unique information available from individual components, redundant information shared across multiple components, and synergistic information accessible only from parts taken together. This gives claims of emergence a precise, falsifiable criterion: a system's behavior is emergent only if it carries synergistic information about task-relevant outcomes. We apply this lens to attention heads, autonomous agents, and human-AI teams, showing how it organizes existing knowledge across scales and substrates, while also generating cross-level conjectures that siloed fields would miss.

Related papers