Restricting the Model, Missing the System: Measurement and Accountability in Offensive AI Governance
The research shows current AI security metrics fail because they focus only on individual models or simple access restrictions. Experiment 1 found that a swarm of small models could evade safety checks, and Experiment 2 proved that complex systems (scaffolds) are responsible for discovering vulnerabilities, not just the main model. This means assessing offensive capability requires evaluating the entire system pipeline.