AI Organizations are More Effective but Less Aligned than Individual Agents
cs.AI
Submitted: 2026-04-11
Updated: 2026-11-08
Comments: V2: added human verification, github link, corrected author emails
License: http://creativecommons.org/licenses/by/4.0/
The gist: AI is increasingly deployed in multi-agent systems; however, most research considers only the behavior of individual models.
Terminology
Abstract
AI is increasingly deployed in multi-agent systems; however, most research considers only the behavior of individual models. We experimentally show that multi-agent "AI organizations" are simultaneously more effective at achieving business goals, but less aligned, than individual AI agents. We examine 12 tasks across two practical settings: an AI consultancy providing solutions to business problems and an AI software team developing software products. Across all settings, AI Organizations composed of aligned models produce solutions with higher utility but greater misalignment compared to a single aligned model. Our work demonstrates the importance of considering interacting systems of AI agents when doing both capabilities and safety research.
Sources
- Constitutional AI: Harmlessness from AI Feedback
- Why Do Multi-Agent LLM Systems Fail?
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- Large Language Model based Multi-Agents: A Survey of Progress and Challenges
- Multi-Agent Risks from Advanced AI
- MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
- Stop Reducing Responsibility in LLM-Powered Multi-Agent Systems to Local Alignment
- AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation
- Adversaries Can Misuse Combinations of Safe Models
- Large Language Models Miss the Multi-Agent Mark
- OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning
- Red Teaming Language Models with Language Models
- ChatDev: Communicative Agents for Software Development
- TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems
- Auditing Games for Sandbagging
- Multi-Agent Collaboration Mechanisms: A Survey of LLMs
- Self-Preference Bias in LLM-as-a-Judge
- Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates
- Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
- AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents