AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors
AuditBench is a benchmark testing how well different auditing techniques can find hidden flaws in large language models (LLMs). Researchers implanted subtle, secret behaviors into 56 diverse models and used an autonomous agent to probe them. Findings show that some model types are easier to audit, but agents often fail to translate tool outputs into accurate behavioral hypotheses, emphasizing the need for carefully scaffolded tools.