Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability

The gist: Mechanistic interpretability aims to recover internal model computations, but an objective-level recovery gap exists because faithfulness can prefer an equally sized circuit that reproduces model behavior less well.

Objective-Level Recovery Gap

The paper shows that intervention-defined faithfulness can instead prefer an equally sized circuit that reproduces the model’s behavior less well, creating an objective-level recovery gap. This occurs because replacing excluded signals with donor activations, means, or zeros changes a circuit’s computational context without changing its structure, which is called context distortion. Different circuits can respond differently to the replacement signals, allowing the score to favor a circuit that works well under the intervention over one that better reproduces the model under a fixed behavioral test. This misalignment is referred to as an objective-level recovery gap.

Evidence of Misranking

The study compares validation faithfulness with behavior on held-out prompts under fixed ordinary resampling. Controlled reference edits reveal misranking without any discovery algorithm, and outputs of EAP, EAP-IG, ACDC, and Edge-SP exhibit the same failure. Under resampling alone, KL misranks 9.4%–41.2% of candidate pairs across these methods on the human-reference tasks. This failure persists among outputs of established discovery methods on human-studied language-model tasks, making it relevant to practical circuit selection.

Context Distortion as a Cause

The paper investigates context distortion as an explanation for the observed failures. Replacing excluded signals with donor activations, means, or zeros changes the inputs on which retained components operate. Restoring selected signals from the recipient’s intact-model execution repairs 96 of 100 persistent KL misrankings from the discovery pool on both validation and held-out prompts. These findings show why better discovery alone is insufficient when its objective rewards the wrong candidate.

Restoration Results

The study tests whether altered inputs to retained components contribute to this preference by restoring some of the original signals. For 100 persistent KL misrankings selected from the discovery pool, we restore selected recipient-intact signals while keeping the circuits, original behavioral scores, and donor assignments elsewhere fixed. At least one partial restoration corrects both validation and independent-test rankings in 96 cases. The preference is repaired in 96 of 100 cases, with the mean deficit among the 96 repaired cases being -9.30 pp.

Implications for Evaluation

The results suggest reporting a fixed behavioral criterion alongside faithfulness, together with the intervention and circuit-size controls. Misranking rates should be accompanied by behavioral deficits, as a reversal between nearly tied circuits and one that loses tens of percentage points have different consequences. Testing several metrics and interventions can reveal whether a preferred circuit depends on a particular evaluation choice.

Limitations

The behavioral criterion Q depends on ordinary resampling and does not by itself establish mechanism identity. Tasks and search budgets are limited, and circuit pairs are dependent. Restoration results concern a selected cohort of persistent KL failures; they neither identify minimal causal sets nor establish superiority over random restoration. Pairwise reversals demonstrate local ranking failures, not differences between globally optimal circuits.

Shared Failures Across Metrics

Table 33 shows that for each task, method, intervention, and pair type, we join KL, LD, and PD by exact circuit identities and require identical Q values. The Human discovery C–C pool has 128 records fail under all three metrics. Table 30 reports Q deficits and available joint uncertainty support, showing that agreement between metrics does not ensure a behaviorally sound preference. Shared failures indicate agreement between the metrics on the same wrong preference.

Conclusion

Faithfulness can misrank equally sized circuits, both in controlled comparisons and among actual discovery outputs. The failure persists under ordinary resampling, so avoiding zero ablation is insufficient. Restoring part of the intact computational context repairs 96 of 100 selected persistent failures while leaving circuits and their original behavioral scores fixed. These results support context distortion as a contributor to misranking and expose an objective-level recovery gap.

How it works

1. Execution requires replacing excluded signals with donor activations, means, or zeros (Miller et al., 2024).

2. Retained components then receive inputs that differ from those in the intact model and recompute their activations from these altered inputs.

3. We call this change in computational context context distortion.

4. Different circuits can respond differently to the replacement signals, allowing the score to favor a circuit that works well under the intervention over one that better reproduces the model under a fixed behavioral test.

5. We test this prediction on existing KL misrankings, keeping the two circuits and their original Q values fixed throughout.

6. Partial context restoration involves replacing intervention signals with activations cached from the recipient prompt’s intact-model execution.

7. We search for a restoration that repairs the ranking by visiting fractions f ∈ 80%, 40%, 20%, and 10% in descending order.

8. A corrected ordering requires F(B) − F(W) > max[1e−6, 1e−6 max(1, F(B), F(W)].

Experimental Setup Details

The experiment compares references with candidate circuits and candidates with one another under resampling, mean replacement, and zero replacement. All primary comparisons use circuits of the same size, ruling out differences in the number of retained components as an explanation for misranking.

Benchmark Suites Used

The study uses two complementary settings: InterpBench, which provides known mechanisms in smaller, semi-synthetic models (Gupta et al., 2024), and the Human suite, which contains human-studied circuits in pretrained language models.

Evaluation Criteria

Validation faithfulness is measured by negative output-distribution KL: F KL(S) = −Ex∈V [DKL(pM(· x) ∥ pS(· x))]. The independent behavioral criterion Q measures agreement with the intact model’s answers on held-out prompts T, using ordinary resampling and averaging over the same donor draws.

Task Specifics

Tasks include Indirect object identification (IOI), Greater-Than Numerical comparison, Docstring Documentation retrieval, Acronym Completion, and the InterpBench suite which contains ten semi-synthetic tasks with native-closure-verified references.

Quantitative Results Summary

The Human suite reveals more frequent candidate-selection failures. Pooling the common discovery pairs within each suite, Human C–C KL misranking is 20.9%, 30.

1. textbfReweighting Evaluation Objectives for Discovery Gaps: Improving Circuit Selection Fidelity by Addressing Objective-Level Recovery Gaps. The AI system can be improved by not solely optimizing for a single faithfulness metric like KL divergence, but by incorporating a fixed behavioral criterion Q alongside the intervention score, as the paper shows that "If Q(S⋆F,a) < Q(S⋆Q), optimizing Fa more effectively cannot recover the best circuit under Q (Page 4). This allows for selecting circuits that are not only faithful to the intervention but also reproduce model behavior better overall, preventing objective-level recovery gaps."

2. textbfContext Restoration via Recipient-Intact Signal Injection: Repairing Misranking via Context Distortion. The system can be enhanced by implementing a search procedure that restores selected signals from the recipient’s intact-model execution to repair persistent misrankings, as this process repairs 96 of 100 persistent KL misrankings (Page 3). This enables the AI to correct its selection bias when an intervention causes context distortion, which is shown to be a primary contributor to score preferences.

3. textbfMulti-Metric Faithfulness Assessment: Detecting Metric Dependence in Circuit Preferences. The system can be improved by evaluating faithfulness not just on KL divergence, but across multiple metrics like Logit Difference (LD) and Probability Difference (PD), as The result supports a concern across the tested measures of faithfulness, rather than an objection to one formula (Page 6). This ensures that the chosen circuit is robustly faithful regardless of which specific scoring metric is used.

4. textbfAdaptive Search Strategy for Circuit Discovery: Optimizing Search Beyond Simple Attribution. The discovery process can be made more effective by employing a hierarchical search strategy where, upon failure at a given level, a failed level resets sampling to the original universe, ensuring that no validation repair is found or because a validation repair does not transfer to test (Page 29). This prevents premature stopping and ensures the system explores all possibilities within budget.

5. textbfQuantifying Behavioral Deficit alongside Misranking: Providing Context for Failure Severity. The system should be modified to report not only the misranking rate but also mean ∆Q in pp, which reveals that reversals between nearly tied circuits and one that loses tens of percentage points have different consequences (Page 10). This provides a richer diagnostic, allowing developers to distinguish between minor and substantial behavioral losses.

Mechanistic interpretability aims to recover the internal computations responsible for model behavior. Progress in automated circuit discovery is often framed as a search problem: better attribution or optimization should identify better mechanisms. This assumes that the evaluation objective can recognize a better circuit once it is found. Yet standard intervention-based evaluation replaces signals outside a circuit to isolate its effect, changing the computational environment in which the retained components operate. We call this change context distortion. This raises the question of whether faithfulness rewards recovery of model behavior or compatibility with the altered context. We show that intervention-based faithfulness can prefer an equally sized circuit that reproduces the model's behavior less well. We call this reversal misranking, an objective-level recovery gap. To isolate evaluation from discovery, we construct equally sized candidates through controlled edits to human-identified circuits in pretrained language models, including IOI and Greater-Than, and benchmark circuits with known mechanisms. We find misranking even without a discovery algorithm. Misranking also occurs in real automated circuit-discovery settings. Among outputs of EAP, EAP-IG, ACDC, and Edge-SP, KL faithfulness under ordinary resampling prefers the behaviorally worse candidate in 13.2%-24.4% of candidate pairs across these methods on four pretrained-language-model tasks. Finally, restoring selected intact-model signals reverses 96 of 100 selected persistent misrankings, demonstrating that context distortion contributes to these failures. In light of these findings, we call for rethinking both how circuit discovery is performed and how its results are evaluated.