Circuit Oracle: Automating Attribution Graph Analysis via Natural-Language Queries
cs.CL
Submitted: 2026-10-30
Updated: 2026-10-30
Comments: Extended version of a paper presented at the Mechanistic Interpretability and Compositional Learning workshops at ICML 2026
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Attribution graphs, an emerging tool in mechanistic interpretability, use transcoders to decompose language model computations into sparse interpretable features connected by causal edges.
Terminology
Abstract
Attribution graphs, an emerging tool in mechanistic interpretability, use transcoders to decompose language model computations into sparse interpretable features connected by causal edges. However, turning a graph into a safety-relevant insight requires hours of manual analysis by experts. We introduce Circuit Oracle, an off-the-shelf LLM equipped with a multi-agent harness and task-specific authored skills, which answers natural-language questions about a target model by reading the attribution graph through tool calls. We evaluate it on three safety-relevant proxy tasks: detecting spurious features in probe circuits, eliciting hidden knowledge from taboo-finetuned models, and suppression jailbreaking via feature-level interventions. Beyond task scores, the oracle produces pruned subgraphs and context-specific feature explanations that we assess qualitatively. Results are mixed. On spurious-feature detection, Circuit Oracle reaches 78.2 plus or minus 2.4% accuracy, outperforming single-layer feature-ranking baselines. On secret elicitation, trained self-explainers like Activation Oracles remain the stronger baseline. On refusal suppression over 50 prompts, its single committed intervention is slightly below a diff-in-means baseline, and only exceeds it when allowed the best of five attempts. Results are stable across five runs and across model choices for the agents. We also release the pruned subgraphs and context-specific feature readings that revise existing global labels, and invite future work to develop evaluations that can measure their usefulness directly.
Sources
- Refusal in Language Models Is Mediated by a Single Direction
- ADAG: Automatically Describing Attribution Graphs
- Towards eliciting latent knowledge from LLMs with mechanistic interpretability
- Eliciting Secret Knowledge from Language Models
- Transcoders Find Interpretable LLM Feature Circuits
- Gemma: Open Models Based on Gemini Research and Technology
- Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition
- Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
- Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
- AtP*: An efficient and scalable method for localizing LLM behaviour to components
- Auditing language models for hidden objectives
- Automatically Interpreting Millions of Features in Large Language Models
- Auditing Games for Sandbagging
- AI Sandbagging: Language Models can Strategically Underperform on Evaluations
- Representation Engineering: A Top-Down Approach to AI Transparency
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data