ORBIT: A Framework for Multi-Agent Safety and Security Evaluations
cs.MA, cs.CR
Submitted: 2026-09-27
Updated: 2026-09-27
Code: https://github.com/wlanderson0/orbit
Project page: https://ukgovernmentbeis.github.io/inspect_evals/evals
License: http://creativecommons.org/licenses/by/4.0/
The gist: Multi-agent LLM systems are increasingly deployed for complex, long-horizon tasks or emerge as a natural consequence of agents interacting in the wild.
Terminology
Abstract
Multi-agent LLM systems are increasingly deployed for complex, long-horizon tasks or emerge as a natural consequence of agents interacting in the wild. Yet they give rise to significant safety and security risks: the flexible protocols that enable task generalization also expose novel threats, from cascading prompt injection to inter-agent collusion. Progress in defending against these threats has been slowed by a lack of shared empirical infrastructure, which forces bespoke environment development for every new defense and makes standardized comparison impossible. Existing evaluations address isolated threat models or single-agent settings, but none jointly vary attack, defense, and architecture across realistic multi-agent environments. To address this gap, we introduce ORBIT, a configurable evaluation framework for empirical multi-agent safety and security research, built on UK AISI's Inspect. ORBIT lets researchers configure communication topologies, memory, scheduling, and agent roles. It supports four threat types and four defense strategies, as well as non-adversarial failures, with a benchmark suite spanning five scenario families covering browser use, computer use, agentic coding, customer service, and cooperative allocation. Our central finding is a gap in defense transferability across threats: per-action defenses that cut a compromised agent's attack success by 60 points on multi-issue coding give no measurable protection against colluding agents, and none of the defenses we tested generalized over all attacks tested. We further demonstrate security-performance tradeoffs and interactions between architecture and defense effectiveness. We make ORBIT available open-source at https://github.com/wlanderson0/orbit.
Sources
- Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- Design Patterns for Securing LLM Agents against Prompt Injections
- Why Do Multi-Agent LLM Systems Fail?
- Defeating Prompt Injections by Design
- Architecting Resilient LLM Agents: A Guide to Secure Plan-then-Execute Implementations
- Vulnerability Detection with Code Language Models: How Far Are We?
- MASEval: Extending Multi-Agent Evaluation from Models to Systems
- A Systematic Review of Poisoning Attacks Against Large Language Models
- AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents
- Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents
- Accelerating scientific discovery with Co-Scientist
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast
- Architecture Matters for Multi-Agent Security
- Multi-Agent Risks from Advanced AI
- Adversaries Can Misuse Combinations of Safe Models
- Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Research
- Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents
Related papers
- Highway Congestion Reduction through Reinforcement Learning Based Eulerian Headway Control
- Auditable and Deployable Multi-Agent Motion Planning via Posetal Games from Legal Guidelines
- ORACLE-SWE: Quantifying the Contribution of Critical Information Signals on SWE Agents
- You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agents
- Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems
- MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization