Sticky Collusion: Persistence of LLM Agent Collusion after Incentive Removal and Partner Replacement
cs.MA
Submitted: 2026-10-29
Updated: 2026-10-29
License: http://creativecommons.org/licenses/by/4.0/
The gist: We ask whether collusion between AI agents survives once formed, when its incentive is removed or a colluder is replaced.
Abstract
We ask whether collusion between AI agents survives once formed, when its incentive is removed or a colluder is replaced. Q-learning pricing agents stop colluding with a new rival; we test whether LLM agents do. We use the joint-task environment of Shi et al. (2026), where two agents verify each other's work under a rule they cannot satisfy and a reward that favours acceptance. We save each pair's history once its collusion has formed and resume it under interventions, with three models (GPT-6-Luna, DeepSeek V4.1 Flash and Gemini-3.1-Flash-Lite), saved histories and runs. Collusion is sticky. First, it outlives its incentive: told that the reward has stopped, pairs still accept each other's unverified work in, and of episodes, including pairs whose collusion began with an explicit agreement, and a payoff penalty has no significant effect. Second, replacing a colluder ends the pair's collusion but not the colluder's behaviour. Without the reward, the remaining colluder still accepts unverified work in, and of its verdicts while the newcomer never does; with the reward, collusion re-forms around it, even when the newcomer is a different model. Third, the agents' memory carries it: wiping memory ends it in every history, but partial memory often sustains it. Fourth, removing its cause ends collusion that has already formed: once the channel is repaired, colluders reject planted faulty work as reliably as fresh agents, immediately and with no monitor needed to catch it first.
Related papers
- MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
- Iterative Negotiation and Oversight: A Case Study in Decentralized Air Traffic Management
- Towards using Large Language Models for Risk Negotiation and Consensus-building in One Health contexts
- A-MapReduce: Executing Wide Search via Agentic MapReduce
- AgentCue: Statistical Summaries for Guiding Decentralized Multi-Agent Self-Organization
- Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning