PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems
PiSAs is a new benchmark to test unintentional privacy leaks in shared AI agent systems using dual Continuous Integrity annotations: appropriateness and visibility. The research found that current system designs fail to adequately prevent inappropriate information from leaking across users, highlighting a critical gap in existing privacy benchmarks.