Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery
cs.CL
Submitted: 2026-06-04
Updated: 2026-08-26
Comments: 101 pages, 44 figures
Journal ref: Transactions on Machine Learning Research, 2026
Code: https://github.com/UKPLab/arxiv2026-phantom-specialization
License: http://creativecommons.org/licenses/by/4.0/
The gist: Circuit discovery methods identify subgraphs that explain model behaviors, and structural differences between discovered circuits are commonly interpreted as evidence of distinct mechanisms.
Terminology
Abstract
Circuit discovery methods identify subgraphs that explain model behaviors, and structural differences between discovered circuits are commonly interpreted as evidence of distinct mechanisms. We test this assumption by varying input-token frequency while holding the task fixed. The discovered circuits appear specialized by frequency when compared structurally, but functional and representational analyses show no reliable evidence of corresponding differences. We term this mismatch phantom specialization. Using the Literal Sequence Copying task across four frequency bands plus a frequency-weighted control, we extract 75 circuits from five Pythia models (70M-1.4B). We find that structurally distinct circuits implement the same computation: band-specific edges transfer broadly across bands, a core shared across most bands recovers at least 99% of circuit performance in models above 70M, and causal interchange interventions confirm that internal representations are interchangeable across frequency bands. A smaller subject-verb agreement replication shows the same pattern: circuits differ structurally but transfer broadly across bands, and the core shared by most bands recovers nearly all circuit accuracy. Repeated extractions within the same band further suggest that discovery algorithms sample from an equivalence class of valid subgraphs rather than recovering a unique mechanism. Standard evaluation practice obscures this pattern: source-level evaluation inflates apparent faithfulness, while edge-level evaluation reveals the many-to-one mapping from structure to function. We do not test specialization at the level of token positions or features inside a component. Our results show that structural differences between circuits are not sufficient evidence for distinct mechanisms, and that exposing this requires edge-level evaluation and cross-condition transfer tests.
Sources
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- An Interpretability Illusion for BERT
- Finding Interpretable Prompt-Specific Circuits in Language Models
- The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- Localizing Model Behavior with Path Patching
- How to use and interpret activation patching
- AtP*: An efficient and scalable method for localizing LLM behaviour to components
- Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla
- Distributed Specialization: Rare-Token Neurons in Large Language Models
- Repetitions are not all alike: distinct mechanisms sustain repetition in language models
- The Hydra Effect: Emergent Self-repair in Language Model Computations
- Mechanistic Interpretability as Statistical Estimation: A Variance Analysis
- Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models
- The Fair Language Model Paradox
- A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
- Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale
- Gemma: Open Models Based on Gemini Research and Technology
- Gemma 2: Improving Open Language Models at a Practical Size
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Adversarial Circuit Evaluation
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering