Causal Attribution via Activation Patching

arXiv:2603.13652 · cs.CV · Submitted 2026-03-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Causal Attribution via Activation Patching".

Jane: Here's the gist of the paper. Attribution methods for Vision Transformers (ViTs) aim to identify image regions that influence model predictions, but producing faithful and well-localized attributions remains challenging The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations We propose Causal Attribution via Activation Patching (CAAP), which estimates the contribution of individual image patches to the ViT’s prediction by directly intervening on internal activations rather than using learned masks or synthetic perturbation patterns The resulting attribution map reflects the causal contribution of patch-associated internal representations on the model’s prediction.

Tom: Pruning redundant reasoning tokens Existing attribution methods face several limitations, with gradient-based, relevance-propagation, and attention-based methods relying on local approximations The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations.

Jane: Gradient-based and relevance-propagation methods provide efficient class-specific signals, but they measure local sensitivity or propagate relevance through the computation graph rather than directly testing whether a contextualized patch representation is sufficient to support the prediction.

Tom: Attention-based methods expose token-mixing patterns, but attention weights or attention flow are indirect proxies for feature contribution, especially after information from different tokens has been repeatedly mixed across layers.

Title and authors: Tom: So we're talking about this paper, "Causal Attribution via Activation Patching," and what it does to how we figure out which parts of an image matter for a Vision Transformer.

Jane: It seems like the main problem they're tackling is that existing methods often give us indirect hints instead of actually testing if a specific part of the image truly drives the model's prediction.

Lu: Yeah, they point out that class-relevant evidence isn't just about what a single patch looks like in isolation; it’s about how those patches interact across many layers through self attention, which makes methods relying only on input changes or attention weights kind of an approximation.

Meng: So, if I understand this right, traditional methods are giving us some local sensitivity or looking at where the tokens are paying attention, but they aren't really measuring the direct causal effect of that specific patch representation.

Lalam: Exactly. The core idea here is to move the intervention away from the input pixels or even just the attention weights and right into the internal activations of a specific image patch itself.

Tom: And their proposed solution, Causal Attribution via Activation Patching, essentially estimates that causal contribution by directly intervening on those internal activations instead of using learned masks or synthetic patterns.

Jane: That sounds like they are taking those source-image activations for a chosen patch and injecting them into a neutral target context over several layers to see what happens to the final prediction score.

Lu: They construct this patched context, which we can call c star of pi, by replacing the original activations with those from the source image at that specific patch position across all layers L in the model.

Meng: So, they are essentially creating a modified version of the target context where only one patch's internal representation is swapped in, and then running a full forward pass to see how much that single patch influences the final class score.

Tom: The attribution score for that specific patch pi with respect to a target class y is defined as the prediction score you get from that patched context when predicting y, which they call A of pi equals f of c star of pi at y.

Jane: And they argue that because these patched activations come directly from the original context, it avoids introducing inputs that are completely out-of-distribution in a way input perturbation methods can.

Lu: That's an important distinction because it means the patched activations already encode those interactions between patches that we were worried about ignoring when using simpler methods.

Tom: The paper shows that this causal intervention serves as a principled measure of patch influence because it captures the semantic evidence after the initial representation formation, which is a big step forward.

Title and authors: Jane: But they also have to be careful about where they stop patching; if you patch too many layers, you risk losing spatial specificity because of late-layer global mixing.

Lu: They actually looked at attention patterns and found that in the middle layers, where intra-object grouping is happening, the information enrichment from other patches increases while after the middle layers it decreases.

Meng: So they suggest a reasonable choice for that layer range L is to include the early and intermediate layers up to around two thirds of L because that's when you still have strong object-level information but it's not getting too mixed with background stuff.

Tom: And on the practical side, they tackle efficiency by suggesting two things: parallel computation of the classification tokens, or CLS tokens, and precomputation of blank context attention.

Jane: That precomputation lets them reuse activations from a blank image xzero across samples and then compute all patch scores in parallel during attribution time.

Lu: They use a dedicated classification token, CLSi, with the same positional encoding as the original CLS token to manage this intervention process efficiently.

Meng: From an engineering standpoint, that parallelization is huge because it means for every single image we analyze, we only need one forward pass for the source image and one pass using those precomputed contexts.

Tom: And the cost per sample drops from something like N cubed operations in a naive implementation to approximately N operations when you use that blank context precomputation.

Jane: So, the main gain they claim is that this design lets them isolate the causal contribution of class-relevant patch representations without relying on indirect evidence like attention weights or approximate gradient signals.

Lu: They also mentioned they considered a padded variant where the selection operator S of pi includes not just the patch pi but its immediate spatial neighbors to better capture feature spread across those tokens.

Meng: That makes sense for locality, but how much extra computation does adding those neighbors cost in practice?

Tom: Well, they found that faithfulness is largely insensitive to the choice of the target blank patches or even the exact selection operator, as long as you use a padded version instead of just a single patch.

Jane: That's reassuring because it means their results aren't overly dependent on those specific implementation details we might change later.

Lu: They also tested different layer depths and found that the performance, especially the insertion score, stayed stable across a broad range of intermediate layers, not just one exact cutoff point.

Meng: That stability is good for deployment because it means we don't have to be super picky about tuning that layer depth parameter when we move to production models.

Title and authors: Tom: The results on CNN architectures also show that even with these transformer approaches, CAAP still achieves the best overall results in most cases compared to other intervention-based methods.

Jane: So, if you look at the metrics they report, like Deletion AUC and Insertion AUC, they consistently get lower Deletion AUC and higher Insertion AUC across multiple ViT backbones on ImageNet-1K <ref:2603.13652#pg1>.

Lu: And they found that this difference in insertion versus deletion scores is notably larger compared to prior methods, which suggests the regions ranked by their attribution maps actually reflect the model’s true decision-making process.

Meng: That's what I care about—if the map is accurate, it means we can actually trust it for debugging or auditing.

Tom: They also reported improvements in localization metrics like AUPR1, AUPR0, and Pointing Game on most models because the patch-level interventions align better with actual object regions rather than just highlighting a few very discriminative patches.

Jane: It seems they're producing more compact and coherent attribution maps that line up closely with the target object, which is much better than having diffuse attention signals.

Lu: They also found that for convolutional architectures like ResNet-eighteen or VGG-sixteen while their overall performance might be lower than transformer results, CAAP still achieves the best overall results in most cases when comparing intervention methods to gradient or relevance propagation.

Meng: That's interesting because I was worried about distribution shifts on CNNs, but they still beat those other local sensitivity measures.

Tom: The main point is that they shift the intervention site from external variables or early model states into internal representations that are closer to the final decision computation, which is a really helpful perspective.

Jane: So, to wrap this up, this paper on Causal Attribution via Activation Patching proposes a framework based on activation patching where you inject source activations into a target context to see the causal effect on the prediction score for each patch individually.

Lu: It moves us past relying on indirect evidence like attention weights or input masks and directly tests the predictive effect of contextualized patch representations within the model’s layers.

Meng: And they make it practical by suggesting parallel computations using CLS tokens and precomputed blank context to keep the cost manageable for real-world use.

Tom: Ultimately, they show that this method consistently achieves lower Deletion AUC and higher Insertion AUC across multiple ViT backbones on ImageNet-1K, leading to more faithful and localized attributions <ref:2603.13652#pg1>.

Jane: It gives us a new tool for interpretability that moves from looking at inputs or attention patterns to actually probing the internal representations that the model is using for classification.

The paper's summary: Tom: So, to recap, this paper is proposing a way to find out which part of an image matters for an AI model's decision by directly poking at the internal representations instead of using some kind of approximation or mask.

Jane: Right, and they do that by inserting the actual activations from a specific patch into a neutral context over several layers and seeing how that changes the final prediction score.

Lu: The big idea here is moving past looking at things like attention weights or simple input changes, because those are just indirect clues about what's important in the model's deep layers.

Meng: So, instead of guessing where a patch is important based on some learned score, they are testing the actual causal effect of that patch's internal signal on the final outcome.

Tom: Exactly. They call it Causal Attribution via Activation Patching, and they argue this lets them capture the true causal contribution of those representations as they form through self-attention across the layers.

Jane: It’s a big shift because it tries to connect what happens deep inside the neural network—those patches interacting across many layers—to an interpretable map we can actually see on an image.

Lu: They’re careful about how many layers they patch, though. They found that patching too deep into the model causes the spatial information to get washed out by things from other parts of the scene, which is a problem in itself.

Tom: So they settled on patching up to around two thirds of the layers because that's where you still have strong object-level information while it's not getting too mixed with background stuff.

Jane: That sounds like a practical compromise, balancing capturing the object's identity with keeping it spatially specific.

Lu: And they made things practical by suggesting ways to speed this up massively, like using precomputed blank context and parallel classification tokens to make the whole process much faster for big Vision Transformers.

Meng: Because I’m thinking about deployment, if an attribution method takes too long to run on a large model, it just isn't going into production. Speed matters when you’re running these kinds of analyses on real traffic data.

Tom: Yeah, and they found that with that precomputation trick, the cost per sample drops from something like N cubed operations down to roughly N operations for an image with N tokens.

Jane: That’s a huge win for scaling this kind of analysis because it makes it feasible to run these kinds of detailed causal tests on much larger models than we could before.

Lu: The results they're showing, especially on ImageNet-1K, show that this method consistently gets lower Deletion AUC and higher Insertion AUC compared to existing methods <ref:2603.13652#pg1>.

Tom: And what that means is that the regions they rank as important by their attribution maps actually reflect the model’s true decision-making process more accurately.

Jane: That's a big deal for debugging, because if the map is accurate, you can actually trust it to figure out why an AI made a certain choice.

Lu: They also found that these localized regions are generally more compact and coherent, which means the attribution maps aren't just scattered noise; they’re actually pointing to meaningful object areas.

Meng: So, this isn't just a theoretical improvement in accuracy metrics; it’s producing maps that are useful for actual debugging workflows where you need to pinpoint a specific object or region.

Tom: And they even checked convolutional networks like ResNet and VGG, and CAAP still achieved the best overall results in most cases when compared to other intervention methods on those architectures.

Jane: It seems like this approach is robust across different model types, which makes it more of a general tool for interpreting Vision Transformers than something specific to just one architecture.

Lu: The authors also pointed out that they don't need to be super picky about the exact number of layers they patch; the performance stayed stable across a broad range of intermediate layers.

Tom: That stability is good for us because it means when we move to new, bigger models, we don't have to redesign the entire intervention strategy from scratch just because the layer count changes a little.

Jane: It gives researchers a lot more flexibility in choosing where they want to look inside the model's thinking process.

Lu: So, what this paper really brings to the table is a principled causal framework for attribution that moves us away from indirect proxies and toward directly testing how internal representations affect predictions.

The paper's improvements: Tom: So, after showing us their core method, they also suggest some practical improvements to make it actually work in the real world.

Jane: What kind of improvements are we talking about now? Are they just tweaking the math, or are they changing how it’s actually run?

Lu: They focus a lot on making sure the selection operator S of pi doesn't just look at one single patch, but includes its immediate spatial neighbors.

Tom: So they broaden the scope of what gets patched; instead of just one token, they consider that whole little area to capture more feature spread across the image.

Jane: That makes sense because a single patch can only tell you about itself in isolation, so including neighbors helps get a better picture of the local context.

Lu: And they also looked at where they should stop patching layers; instead of picking one fixed number, they found that performance stays stable across a whole range of intermediate layers.

Tom: That’s good for deployment because it means we don't have to be super precise about that layer cutoff when we move to new models later on.

Jane: And then they tackled the speed issue again by proposing precomputing blank context attention, which lets them reuse activations from a blank image across different samples.

Lu: That’s the efficiency boost; it turns what could be a very slow process into something that scales much better, with the cost per sample dropping to roughly N operations.

Tom: So they are basically trying to make this method fast enough for actual use on large models, which is a huge step for any practical application.

Jane: And they also mentioned that when comparing the different types of blank context or selection operators, the faithfulness of their results doesn't really change much, as long as you use that padded version with neighbors.

Lu: They also pointed out that when looking at how information flows through layers, the most important stuff is actually happening in those early and middle layers where things are still grouping objects together.

Tom: So they’re confirming that focusing on those specific layer ranges is the right move because it aligns with how the model actually builds its understanding of objects.

Jane: It seems like this paper isn't just about finding a better score; it’s about creating a more principled and efficient way to probe the internal workings of these massive models.

Lu: They are moving us toward methods that test causal contributions directly, rather than relying on those indirect signals from attention weights or input perturbations.

Conclusion: Tom: So we’re wrapping up this paper on Causal Attribution via Activation Patching by summarizing what it actually means for us in practical terms and saying goodbye to it.

Jane: Right, so at its heart, this is a method that moves past just looking at input changes or attention patterns to directly testing the causal effect of an internal patch representation on the final prediction score.

Lu: The authors have shown that by intervening on these activations across several layers and using those clever parallelization tricks with blank context, we can get very faithful and localized attribution maps.

Meng: For us in production, that means we get a much more reliable signal for debugging what’s actually causing the model to make a mistake, instead of just guessing based on where it looked.

Lalam: I’m confident in saying that this approach provides a principled way to measure patch influence by directly testing the predictive effect of those representations.

Tom: It really does shift us from looking at external evidence or early model variables to probing internal representations that are closer to the final decision computation.

Jane: And they found it works across different Vision Transformer backbones, even showing better results than other intervention methods on ImageNet-1K, especially with deletion and insertion metrics <ref:2603.13652#pg1>.

Lu: The result is maps that are more compact and coherent, which means they aren't just diffuse signals; they align closely with the actual object regions we see.

Meng: That’s what I care about—if the map is accurate, it means we can actually trust it for debugging or auditing when dealing with complex vision systems in sensitive areas.

Tom: So, this paper on Causal Attribution via Activation Patching gives us a new tool for interpretability that lets us understand *why* an AI makes a decision by looking inside the model’s layers.

Jane: It moves us past relying on indirect evidence and shows how to use causal intervention to get cleaner, more semantically meaningful explanations.

Lu: The authors also pointed out the limitations: they said they didn't test every single layer depth or every possible selection operator, but even with those caveats, the method holds up well.

Meng: So we can use this as a solid starting point for our next experiments instead of trying to tweak every single parameter just to get a tiny bit better performance on one metric.

Tom: Exactly. It’s about building an attribution framework that is both accurate and efficient enough for real-world engineering challenges.

Jane: This work opens up a new direction for researchers who want to understand the causal pathway of information flow within these large vision models.

Sharif University of Technology

cs.CV

Submitted: 2026-03-13

Updated: 2026-11-10

Comments: Accepted at NeurIPS 2026

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 92/100

The gist: Attribution methods for Vision Transformers (ViTs) aim to identify image regions that influence model predictions…

Key concepts

Activation Patching
CAAP works by taking the actual internal feature activations from a source image's patches and inserting them into a neutral target context over several layers of the ViT. This simulates what happens when a specific patch's information is present, allowing researchers to see how that specific patch representation affects the final prediction score.
Causal Attribution
This approach aims to establish a direct causal link between an image patch and the model's output. Unlike indirect methods like attention weights, CAAP tests the predictive effect of the patch's internal features after they have been processed through multiple layers, providing a principled measure of influence.
Pruning Redundant Reasoning Tokens
The method considers spatial feature spread by including immediate neighbors when patching a single patch. This ensures that the intervention captures not just the isolated patch but also its surrounding context, leading to more faithful and localized measures of contribution.

Terminology

Summary

Attribution methods for Vision Transformers (ViTs) aim to identify image regions that influence model predictions, but producing faithful and well-localized attributions remains challenging The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations We propose Causal Attribution via Activation Patching (CAAP), which estimates the contribution of individual image patches to the ViT’s prediction by directly intervening on internal activations rather than using learned masks or synthetic perturbation patterns The resulting attribution map reflects the causal contribution of patch-associated internal representations on the model’s prediction. p. 0

Where the paper says it

The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations. We propose Causal Attribution via Activation Patching (CAAP), which estimates the contribution of individual image patches to the ViT’s prediction by directly intervening on internal activations rather than using learned masks or synthetic perturbation patterns.

Pruning redundant reasoning tokens

Existing attribution methods face several limitations, with gradient-based, relevance-propagation, and attention-based methods relying on local approximations The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations. Gradient-based and relevance-propagation methods provide efficient class-specific signals, but they measure local sensitivity or propagate relevance through the computation graph rather than directly testing whether a contextualized patch representation is sufficient to support the prediction. p. 1

Where the paper says it

This creates limitations for several attribution families. Gradient-based and relevance-propagation methods provide efficient class-specific signals, but they measure local sensitivity or propagate relevance through the computation graph rather than directly testing whether a contextualized patch representation is sufficient to support the prediction [7–10, 12].

Attention-based methods expose token-mixing patterns, but attention weights or attention flow are indirect proxies for feature contribution, especially after information from different tokens has been repeatedly mixed across layers. p. 1
Where the paper says it

Gradient-based and relevance-propagation methods provide efficient class-specific signals, but they measure local sensitivity or propagate relevance through the computation graph rather than directly testing whether a contextualized patch representation is sufficient to support the prediction [7–10, 12]. Attention-based methods expose token-mixing patterns, but attention weights or attention flow are indirect proxies for feature contribution, especially after information from different tokens has been repeatedly mixed across layers [13–15].

Perturbation and masking methods measure causal output changes under interventions, but input-space perturbations can introduce distribution shift. p. 1
Where the paper says it

Attention-based methods expose token-mixing patterns, but attention weights or attention flow are indirect proxies for feature contribution, especially after information from different tokens has been repeatedly mixed across layers [13–15]. Perturbation and masking methods measure causal output changes under interventions, but input-space perturbations can introduce distribution shift [17–20].

Input-based deletion does not account for redundancy across patches: when multiple patches encode similar class evidence, removing each patch from a group of redundant patches may cause little change in the output, leading to underestimation of the group’s importance. p. 1
Where the paper says it

Perturbation and masking methods measure causal output changes under interventions, but input-space perturbations can introduce distribution shift [17–20]. Input-based deletion does not account for redundancy across patches: when multiple patches encode similar class evidence, removing each patch from a group of redundant patches may cause little change in the output, leading to underestimation of the group’s importance.

How the training signal is built

For each patch, CAAP inserts the corresponding source-image activations into a neutral target context over an intermediate range of layers and uses the resulting target-class score as the attribution signal The patched context c∗(pi) is constructed by replacing activations as h l(c∗(pi))S(pi) = h l(x)S(pi), hl l(c∗(pi))¬S(pi) = h l(x0)¬S(pi), ∀l ∈ L The attribution score for patch pi with respect to class y is defined as the prediction score obtained from the patched context, A(pi) = f(c∗(pi))y. An important property of activation patching, compared to input-based perturbation methods, is that the patched activations are taken directly from the original context, so they already encode interactions between patches and avoid introducing OOD inputs in the source context. p. 2

Where the paper says it

Repeating this procedure for all patches yields a patch-level attribution map over the input image. An important property of activation patching, compared to input-based perturbation methods, is that the patched activations are taken directly from the original context, so they already encode interactions between patches and avoid introducing OOD inputs in the source context.

Where the gains come from

The causal intervention serves as a principled measure of patch influence by capturing semantic evidence after initial representation formation, while avoiding late-layer global mixing that can reduce spatial specificity CAAP consistently outperforms existing methods in various settings and produces more faithful and localized attributions. Empirical results across multiple Vision Transformer backbones show that our method improves over existing approaches on most evaluated settings, achieving state-of-the-art results on deletion, insertion, and foreground and background localization metrics. p. 1

Where the paper says it

The procedure can be efficiently parallelized, enabling practical patch-level attribution for large ViT backbones. Empirical results across multiple Vision Transformer backbones show that our method improves over existing approaches on most evaluated settings, achieving state-of-the-art results on deletion, insertion, and foreground and background localization metrics.

CAAP achieves the best overall performance across most backbones. p. 5
Where the paper says it

Localization results are reported in Table 2. CAAP achieves the best overall performance across most backbones.

In the naive implementation, the patch selection operator S(pi) corresponds exactly to the patch pi However, patching a single token often does not transfer sufficient predictive information from the source context To account for spatial feature spread across neighboring tokens, we consider a padded variant in which S(pi) includes the patch pi and its immediate spatial neighbors This choice preserves both contextual information and spatial locality, making the patch-level intervention a more faithful measure of localized contribution.

ViTs build patch representations through repeated self-attention, causing each patch token to encode information aggregated from other patches In a single attention head, the attention update for patch i can be written as z l+1 i = X N / N j=1 α l ijv l j, (4) where v l j are value vectors derived from the token representations and α l ij are attention weights defined by α l ij = exp ⟨q li, klj⟩/√d m=1 exp ⟨q li, klm⟩/√d. We empirically show that in the early layers, where intra-object grouping occurs, the quantity E j∈O[α l ij] − E j /∈O[α l ij] increases, while after the middle layers it decreases. p. 4

Where the paper says it

The attention update can be decomposed as z˜ l+1 i = X j∈O α l ijv l j + X j /∈O α l ijv l j . (6) We empirically show that in the early layers, where intra-object grouping occurs, the quantity Ej∈O[α l ij ] − Ej /∈O[α l ij ] increases, while after the middle layers it decreases.

This enrichment of intra-object grouping supports the motivation for layer-wise activation patching rather than input-level insertion. p. 4
Where the paper says it

(6) We empirically show that in the early layers, where intra-object grouping occurs, the quantity Ej∈O[α l ij ] − Ej /∈O[α l ij ] increases, while after the middle layers it decreases. This enrichment of intra-object grouping supports the motivation for layer-wise activation patching rather than input-level insertion.

We next propose two practical techniques to further accelerate the intervention process: parallel computation of CLS tokens and precomputation of blank-context attention This parallel construction of CLS tokens yields the same attribution map (no approximation) while computing all patch scores in parallel. The total cost per sample is one forward pass for the source image and one forward pass with multiple CLS tokens. p. 4

Where the paper says it

This parallel construction of CLS tokens yields the same attribution map (no approximation) while computing all patch scores in parallel. The total cost per sample is one forward pass for the source image and one forward pass with multiple CLS tokens.

We next propose two practical techniques to further accelerate the intervention process: parallel computation of CLS tokens and precomputation of blank-context attention For the blank image x0, we perform one forward pass and cache its activations h l(x0), l∈L, reused across samples. We then build a patched computation containing both activation sets. p. 4

Where the paper says it

For each source image x, we run a standard forward pass and cache {h l (x)}l∈L. We then build a patched computation containing both activation sets.

For each patch pi, we introduce a dedicated classification token CLSi with the same positional encoding as the original CLS The attention mask ensures that CLSi attends to source activations at S(pi) and to target activations at ¬S(pi). p. 4
Where the paper says it

For each patch pi , we introduce a dedicated classification token CLSi with the same positional encoding as the original CLS. The attention mask ensures that CLSi attends to source activations at S(pi) and to target activations at ¬S(pi).

We therefore precompute two quantities for each layer and attention head using the blank context. First, the normalization contribution of blank tokens, Z l 0 = X j /∈S(pi) exp(⟨q c′CLS, kl 0 j⟩/√d), (8) which is reused as a fixed term in the softmax denominator. p. 1

Where the paper says it

CLS Selected Token Activations Source Context Classification Head cat Softmax Probability Repeat for each patch to form attribution map CLS Target Context CLS Patched Activations Patched Context Forward Pass 𝒍𝒆 Forward Pass Figure 1: Overview of CAAP. Given a source image (top) and a blank target image (middle), we extract internal activations from the source image corresponding to a selected patch and a specified range of layers.

Second, a prototype value vector for the blank tokens, v¯ l 0 = 1 / B X j∈B v l 0, (9) where B denotes the set of blank tokens. p. 1
Where the paper says it

Insertion into a blank or blurred image overlooks interactions among patches and may fail to reproduce the functional role a patch plays in the full image context. More ViT-specific perturbation or optimization methods operate over tokens, attention mechanisms, patch embeddings, or attribution metrics [21–23], but they still do not directly evaluate the class evidence carried by the original internal patch representation within a controlled target context.

During attribution, we reuse Z l 0 and v¯ l 0 and compute only the terms involving the source-patched tokens dynamically. p. 4
Where the paper says it

To avoid recomputing these vectors from hidden states during activation patching, we reuse cached key and value projections associated with the intervened hidden states, rather than recomputing them from residual streams. Only the query vectors of the additional CLS tokens are computed dynamically in the patched context.

This design allows CAAP to better isolate the causal contribution [25, 26] of class-relevant patch representations to the final prediction. It derives patch scores directly from information already encoded in contextualized token representations, rather than relying on indirect evidence such as attention weights, approximate gradient signals, input-space masks, or metric-optimized surrogate maps. p. 1

Where the paper says it

Causal intervention offers a practical advantage for CAAP over existing attribution techniques. It derives patch scores directly from information already encoded in contextualized token representations, rather than relying on indirect evidence such as attention weights, approximate gradient signals, input-space masks, or metric-optimized surrogate maps.

The procedure can be efficiently parallelized, enabling practical patch-level attribution for large ViT backbones. p. 1
Where the paper says it

It derives patch scores directly from information already encoded in contextualized token representations, rather than relying on indirect evidence such as attention weights, approximate gradient signals, input-space masks, or metric-optimized surrogate maps. The procedure can be efficiently parallelized, enabling practical patch-level attribution for large ViT backbones.

We next propose two practical techniques to further accelerate the intervention process: parallel computation of CLS tokens and precomputation of blank-context attention This approximation substantially reduces computational complexity. A naive implementation requires approximately O(N 3) operations for an image with N tokens, due to N patch interventions combined with N × N attention interactions With precomputation, the contribution of blank tokens becomes a constant term (O(1)), and only the N patch interventions are evaluated dynamically, reducing the overall cost to approximately O(N). p. 4

Where the paper says it

We next propose two practical techniques to further accelerate the intervention process: parallel computation of CLS tokens and precomputation of blank-context attention. The patch-wise formulation requires one forward pass per patch, which scales linearly with the number of tokens in the image.

In addition to gradient-based attribution, attention-based attribution, and perturbation-based attribution and masking in ViTs, CAAP measures the effect of patch-associated representations on the model output through an explicit activation intervention. It complements masking- and sampling-based perturbation methods by moving the intervention from the input or token level to the representation level It is also distinct from metric-driven approaches that optimize attribution maps to fit evaluation criteria. p. 1

Where the paper says it

It derives patch scores directly from information already encoded in contextualized token representations, rather than relying on indirect evidence such as attention weights, approximate gradient signals, input-space masks, or metric-optimized surrogate maps. The procedure can be efficiently parallelized, enabling practical patch-level attribution for large ViT backbones.

The key distinction is that CAAP directly tests the predictive effect of intermediate residual representations associated with image patches. p. 0
Where the paper says it

The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations. We propose Causal Attribution via Activation Patching (CAAP), which estimates the contribution of individual image patches to the ViT’s prediction by directly intervening on internal activations rather than using learned masks or synthetic perturbation patterns.

p. 0
Where the paper says it

The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations. We propose Causal Attribution via Activation Patching (CAAP), which estimates the contribution of individual image patches to the ViT’s prediction by directly intervening on internal activations rather than using learned masks or synthetic perturbation patterns.

CAAP consistently achieves the lowest Deletion AUC and the highest or competitive Insertion AUC on all backbones. In particular, CAAP improves the Insertion−Deletion with a notable margin compared to prior methods, indicating that the regions ranked as important by our attribution maps more accurately reflect the model’s true decision-making process Localization metrics such as AUPR1, AUPR0, and Pointing Game also improve, indicating that patch-level interventions better align with object regions rather than highlighting only a few highly discriminative patches. p. 5

Where the paper says it

In particular, CAAP improves the Insertion−Deletion with a notable margin compared to prior methods, indicating that the regions ranked as important by our attribution maps more accurately reflect the model’s true decision-making process. Localization.

CAAP produces more compact, coherent, and semantically meaningful attributions that align closely with the target object. p. 6
Where the paper says it

As illustrated, baseline methods often produce diffuse or fragmented attribution maps that extend beyond the target object into the background, or highlight only a small subset of discriminative patches. In contrast, CAAP tends to generate more compact, coherent, and semantically meaningful attributions that align closely with the target object.

We ablate three design choices in CAAP on ImageNet: (i) the type of the target’s blank patches, (ii) the type of the selection operator, and (iii) the choice of intervened layers Fig. 4a shows that faithfulness is largely insensitive to this choice, as Insertion, Deletion, and Insertion−Deletion remain close across blank types. p. 7

Where the paper says it

Gaussian noise to the target and then applies a blur filter. 4a shows that faithfulness is largely insensitive to this choice, as Insertion, Deletion, and Insertion−Deletion remain close across blank types.

Fig. 4b compares four neighborhood variants The no-padding variant performs the worst, while all padded alternatives improve both faithfulness and localization. p. 7
Where the paper says it

4b compares four neighborhood variants. The no-padding variant performs the worst, while all padded alternatives improve both faithfulness and localization.

To analyze the effect of layer depth, we first examine attention patterns that govern information flow (Section 3.2) Fig. 5a shows that E j∈O[α l ij] − E j /∈O[α l ij] peaks in the middle layers. p. 7

Where the paper says it

To analyze the effect of layer depth, we first examine attention patterns that govern information flow (Section 3. 5a shows that Ei [Ej∈O[α l ij ] − Ej /∈O[α l ij ]] peaks in the middle layers.

In later layers, intra-object attention decreases while object-to-background attention increases, reflecting growing extra-object contextualization and reduced spatial specificity. p. 8
Where the paper says it

Around the middle layers, this gap reaches its peak. In later layers, intra-object attention decreases while object-to-background attention increases, reflecting growing extra-object contextualization and reduced spatial specificity.

Therefore, a reasonable choice of L is to include the early and intermediate layers, up to the point where background or other non-class-related features begin to mix into the patch representations

The performance (particularly Insertion AUC) remains stable across a broad range of intermediate layers, indicating that the method is not highly sensitive to the exact cutoff choice (results on other backbones are provided in Appendix D.1). This choice balances intra-object grouping and extra-object contextualization and performs near-optimally across models. p. 7

Where the paper says it

2, we fix le = ⌈ 2 3 L⌉ as the default setting for all architectures. This choice balances intra-object grouping and extra-object contextualization and performs near-optimally across models.

We further evaluate the intervention strategies on convolutional architectures, including ResNet-18, ResNet-50 [41], DenseNet-161 [42], and VGG-16 [43]. Compared with the transformer results reported earlier, the overall performance of intervention-based methods is lower across both faithfulness and localization metrics One possible explanation is that transformer architectures are generally more robust to distribution shifts than convolutional networks. Within the CNN architectures, CAAP still achieves the best overall results in most cases. p. 5

Where the paper says it

CAAP achieves the best overall performance across most backbones. In particular, the higher PG and AUPR1 values indicate that the regions identified as most relevant are consistently located within the annotated foreground areas.

In contrast to gradient- and relevance-propagation methods, which trace local sensitivity or backward relevance through the computation graph, CAAP directly intervenes on the forward activations of contextualized patch tokens It complements masking- and sampling-based perturbation methods by moving the intervention from the input or token level to the representation level CAAP can be viewed as an interventional attribution approach that assigns relevance by replacing internal activations and measuring the resulting target-class score.

CAAP shifts the intervention site for ViT attribution from external or early model variables to internal representations that are closer to the model’s final decision computation It complements attention-derived explanations by treating attention as part of the computation that forms the intervened states rather than as the explanatory signal itself CAAP derives each score from an explicit intervention within the classifier itself, without learning an external explainer, mask generator, or metric-specific surrogate. The key distinction is that CAAP directly tests the predictive effect of intermediate residual representations associated with image patches.

The overall, CAAP consistently outperforms both input-based baselines As shown in Table 13, CAAP achieves substantially lower Deletion AUC and higher Insertion AUC, leading to much larger Insertion−Deletion scores across all architectures. Localization metrics in Table 14 show the same pattern CAAP yields higher AUPR1, AUPR0, and PG on most models, indicating better alignment with object regions. p. 8

Where the paper says it

Empirically, CAAP improves faithfulness across multiple backbones, including CLIP, DINOv2, ViT, and DeiT3, by achieving lower Deletion AUC and higher Insertion AUC. Localization metrics such as AUPR1, AUPR0, and Pointing Game also improve, indicating that patch-level interventions better align with object regions rather than highlighting only a few highly discriminative patches.

The results suggest that manipulating pixels alone does not reconstruct the internal evidence used by the model In contrast, intervening on activations across layers preserves contextualized patch representations formed through self-attention, which leads to more faithful and spatially consistent attribution.

This work contributes to interpretability and attribution methods for Vision Transformers by proposing a causal intervention framework based on activation patching Improved attribution quality may help researchers better understand model behavior, identify reliance on spurious correlations, and support debugging and auditing of vision systems in sensitive applications. The paper introduces a new attribution method but does not release new datasets or generative models. p. 2

Where the paper says it

reliable and effective approach to spatial attribution in ViTs, offering a new perspective based on mechanistic interpretability tools. 2 Preliminaries 2.

The paper is 2603. p. 0
Where the paper says it

∗Equal contribution. arXiv:2603.

13652

All experiments use publicly available datasets and pretrained models, including ImageNet-1K, ImageNet-S, Oxford-IIIT-Pet, Food-101, CLIP, DINOv2, DeiT3, and ViT models The paper introduces a new attribution method but does not release new datasets or generative models.

The paper is 2603.13652 The paper introduces a new attribution method but does not release new datasets or generative models. p. 2

Where the paper says it

reliable and effective approach to spatial attribution in ViTs, offering a new perspective based on mechanistic interpretability tools. 2 Preliminaries 2.

p. 2
Where the paper says it

reliable and effective approach to spatial attribution in ViTs, offering a new perspective based on mechanistic interpretability tools. 2 Preliminaries 2.

p. 2
Where the paper says it

reliable and effective approach to spatial attribution in ViTs, offering a new perspective based on mechanistic interpretability tools. 2 Preliminaries 2.

The paper is 2603.13652 The paper introduces a new attribution method but does not release new datasets or generative models.

The paper is 2603.13652 The paper introduces a new attribution method but does not release new datasets or generative models.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Causal Attribution via Activation Patching (CAAP) directly intervenes on internal activations rather than using learned masks or synthetic perturbation patterns to estimate patch contribution. This allows it to capture the causal contribution of patch-associated internal representations on the model’s prediction by inserting source-image activations into a neutral target context.

  2. CAAP is designed to avoid late-layer global mixing that reduces spatial specificity by applying activation patching up to intermediate layers, specifically selecting layers up to 2/3 L, where patch representations contain strong object-level information while preserving spatial specificity.

  3. The method can be parallelized by computing all patch attribution scores in parallel within a single forward pass using dedicated classification tokens (CLS) with attention masks, reducing the total cost per sample to approximately O(N).

  4. CAAP provides more faithful and localized attributions than existing methods, consistently achieving lower Deletion AUC and higher Insertion AUC across multiple ViT backbones on ImageNet-1K.

  5. The method produces more compact and coherent attribution maps, evidenced by consistently lower entropy and higher Gini scores compared to baselines. This indicates that the attribution is more compact and coherent, better aligning with object regions rather than being diffuse.

Abstract

Attribution methods for Vision Transformers (ViTs) aim to identify image regions that influence model predictions, but producing faithful and well-localized attributions remains challenging. Existing attribution methods face several limitations, with gradient-based, relevance-propagation, and attention-based methods relying on local approximations, while perturbation- or optimization-based methods intervene on inputs, tokens, or surrogates rather than internal patch representations. The key challenge is that class-relevant evidence is formed through interactions between patch tokens across layers; methods that operate only on input changes, attention weights, or backward relevance signals may therefore provide indirect proxies for patch importance rather than directly testing the predictive effect of contextualized patch representations. We propose Causal Attribution via Activation Patching (CAAP), which estimates the contribution of individual image patches to the ViT's prediction by directly intervening on internal activations rather than using learned masks or synthetic perturbation patterns. For each patch, CAAP inserts the corresponding source-image activations into a neutral target context over an intermediate range of layers and uses the resulting target-class score as the attribution signal. The resulting attribution map reflects the causal contribution of patch-associated internal representations on the model's prediction. The causal intervention serves as a principled measure of patch influence by capturing semantic evidence after initial representation formation, while avoiding late-layer global mixing that can reduce spatial specificity. Across multiple ViT backbones and standard metrics, CAAP consistently outperforms existing methods in various settings and produces more faithful and localized attributions.

Sources

Related papers