Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design

summary

The gist

Few-Shot Semantic Segmentation (FSS) models achieve strong performance in segmenting novel classes with minimal labeled examples…

In short

This work introduces Affinity Explainer (AffEx), a method to interpret matching-based few-shot segmentation models by analyzing their internal matching scores. AffEx extracts attribution maps showing which pixels in support images most influence the query's segmentation prediction. This provides structured, coherent attention patterns, enabling better model understanding and diagnostics for few-shot systems.

Key concepts

Matching-Based FSS Models
These are segmentation models that use a matching mechanism to relate a new query image to a set of few labeled support images. The core idea is that the model computes similarity scores between the query features and the support features to make predictions.
Affinity Explainer (AffEx)
AffEx is an approach that interprets these models by examining how matching scores contribute to the final segmentation. It derives spatial attribution maps from these scores, highlighting which parts of the support images are most relevant for a specific query pixel's prediction.
Attribution Maps
These are spatial heatmaps generated by AffEx that quantify the contribution of individual pixels in a support image to the model's decision for a specific location in the query image. They reveal which parts of the support images are most influential.
DAUC (Deletion Area Under the Curve)
DAUC is an evaluation metric used to check if attribution maps are accurate. If an explanation is good, removing the most relevant pixels from a support set should cause a measurable drop in the model's segmentation confidence, indicating reliable attribution.

Terminology used across episodes

This episode discusses

The paper

Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design · Read on arXiv

Pasquale De Marinis, Uzay Kaymak, Rogier Brussee, Gennaro Vessio, Giovanna Castellano

Department of Computer Science, University of Bari Aldo Moro · Jheronimus Academy of Data Science (JADS) · Eindhoven University of Technology (TU/e)

Few-Shot Semantic Segmentation (FSS) models achieve strong performance in segmenting novel classes with minimal labeled examples, yet their decision-making processes remain largely opaque. Explainable AI has matured on standard computer vision tasks, but not on FSS, where an explanation would tell a practitioner which support pixels the prediction rested on and which examples to provide next. We argue that matching-based FSS models are interpretable by design: their query-support similarity scores, once normalized and processed, attribute the prediction to individual support pixels. Our approach, Affinity Explainer (AffEx), makes this intrinsic interpretability explicit by extracting attribution maps directly from matching scores at multiple feature levels, without requiring gradients or external perturbations. We extend the standard causal metrics for evaluating explanations to the FSS domain and propose additional metrics to better capture the practical utility of explanations in few-shot scenarios. Experiments on two FSS benchmarks, covering seven architectures that range from classical matching-based models to recent pipelines built on vision foundation models (DINOv2/v3, SAM, SAM2) and a prototype-based head, show that AffEx outperforms adapted standard attribution methods at nearly every evaluation point, and that the gain comes from the matching mechanism itself. On pipelines whose segmentation stage transforms the matching output, the same evaluation doubles as a diagnostic of where the decision is made, and on the prototype-based head AffEx subsumes the prototype-style read-out as a special case. Qualitatively, the attribution maps expose how each architecture resolves the support set, and we show how they guide model diagnosis. Code is available at https://github.com/pasqualedem/AffinityExplainer.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design".

Jane: Here's the gist of the paper. Few-Shot Semantic Segmentation (FSS) models achieve strong performance in segmenting novel classes with minimal labeled examples, yet their decision-making processes remain largely opaque Abstract</ref:2511.18163 The Core Problem and Proposed Solution This paper introduces the first dedicated method for interpreting matching-based FSS models by leveraging their inherent structural properties The Affinity Explainer approach extracts attribution maps that highlight which pixels in support images contribute most to query segmentation predictions, using matching scores computed between support and query features at multiple feature levels This work establishes the foundation for interpretable FSS research, enabling better model understanding and diagnostic for more reliable few-shot segmentation systems The AffEx framework comprises three variants that directly build on the core matching mechanisms underlying these models Starting from the raw similarity scores computed between query and support features, AffEx derives contribution maps, which are then aggregated via feature ablation to produce attribution heatmaps over the support set The three variants—Unmasked, Masked, and Signed AffEx—differ in how the support masks are incorporated into the attribution process The process involves defining an attribution function A: (Iq,(x, y), S) 7→ R K×Hs×Ws (8) which returns, for a given query pixel, a spatial attribution map for each support image The attribution value at pixel (x, y) of support image Ii is denoted αi,x,y =

A(Iq,(x, y), S): i,x,y (9) and quantifies the contribution of that pixel to the segmentation prediction at (x, y) in the query The attribution function for an element (u, v) in the query feature map can be defined as A(Pj q,(u, v), Pj S) = softmax σ j u,v ∈ R K×Hj×Wj (11) where Pj q and Pj S are the mapped feature maps for the query and support images at layer j, respectively The weights wj are calculated based on the importance of each layer in the matching process through feature ablation Evaluation Framework To evaluate the interpretability of the segmentation outputs, we extend the concepts of Insertion Area Under the Curve (IAUC) and Deletion Area Under the Curve (DAUC) to the FSS domain These metrics are designed to quantify the decision perturbation in the model when changing the image according to the explanation The logic behind DAUC is that if the attribution map is accurate, perturbing the support set by removing the most relevant pixels should decrease the model’s confidence in the segmentation prediction Furthermore, we propose two additional metrics: IAUCConf@p represents the IAUCConf calculated by considering only a percentage p of the most relevant pixels and mIoULoss@p quantifies the accuracy loss incurred by the models when using the most relevant pixels, as determined by the explanation method, compared to the complete support set Experimental Results and Analysis Comprehensive experiments on FSS benchmark datasets demonstrate that our Affinity Explainer significantly outperforms adapted standard attribution methods Qualitative analysis reveals that our explanations provide structured, coherent attention patterns that align with model architectures and enable effective model diagnosis Table 3 shows the relative error on the 95% percentile of the Insertion Area Under the Curve (IAUC) and Deletion Area Under the Curve (DAUC) metrics with respect to the ground truth value obtained with 1000 steps The lower the relative error, the more stable the metric is with respect to the number of steps In Table 2, mIoULoss@p results show that Unmasked AffEx outperforms all other methods Computational Efficiency and Future Directions AffEx runs substantially faster than gradient-based baselines (e.g., Integrated Gradients) and uses moderate memory, achieving a favorable trade-off between memory efficiency and computational latency The best results are highlighted in bold, except for IAUC and DAUC, as their values are not directly comparable alone Future directions include incorporating complementary XAI techniques—such as Grad-CAM

31: applied directly to the feature extractor—to capture these contributions more faithfully and combine them with AffEx’s attribution maps Another promising direction is to extend our method to prototype-based FSS models

5, 7, 46, 50, 54: which do not explicitly compute similarity maps between query and support features Page 1 --- Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design Page 2 --- Match Seg Matching scores Segmentation Attribution Maps Feature Ablation IAUC DAUC mIoULoss@p Matching-based Model Image Encoder Support Masks Support Images Query Image Softmax Masking Contributions Figure 1.

Tom: Illustration of the proposed framework with two examples and one class.

Jane: The Affinity Explainer method can be applied to any matching-based FSS model adhering to the illustrated structure It computes attribution maps using the matching scores between query and support images Attribution is restricted to a region R—typically defined by the ground truth mask, segmentation output, or userspecified input These maps are then softmax-normalized, weighted via feature ablation, and aggregated to yield the final attribution map The resulting maps highlight the support image regions most influential in segmenting the query, thereby enabling model interpretability Page 3 --- Matching-based FSS Models To interpret matching-based FSS models, we propose Affinity Explainer (AffEx)—a method that leverages the inherent structure of matching operations The key idea is to analyze the contributions of individual support images to the final segmentation mask by examining the matching scores and their corresponding feature maps This relies on the assumption that the matching scores reflect the relevance of each support image to the query image Page 4 --- Matching-based FSS Models The overall pipeline corresponding to this formulation is illustrated in Fig.

Tom: 1 Given an image I, the feature extractor produces a feature map Fθ(I) ∈ R C×H×W where C is the number of channels and H and W are the height and width of the feature map, respectively However, several works have shown that using mid-level features can also improve the performance of FSS models Therefore, we define the feature extractor as a function that returns a set of feature maps: Fθ(I) = F 1 θ (I), F2 θ (I),..., F n θ (I) (2) where Fjθ(I) ∈ R Cj×Hj×Wj denotes the j-th feature map with Cj channels and spatial dimensions Hj × Wj The matching module can be decomposed into three components: (i) an input mapping module Mapin θ, which transforms the features and masks into a common relational space (including normalization, reshaping, or non-linear projections such as those used in attention mechanisms); (ii) a matching mechanism MMθ (e.g., cross-attention or Euclidean distance) that computes the matching scores between support and query representations; and (iii) an output mapping module Mapout θ that aggregates the matching results to produce a single feature map Page 5 --- Matching-based FSS Models Pq = Mapin θ (Fθ(Iq)) (4) Σ = n σ j = MMθ(Pj q, Pj S) on j=1 (5) Fout = Mapout θ (Σ), (6) The segmentation head then processes the output feature map to produce the final segmentation mask: Mˆ q = Segθ(Fout).

Paper summary: Tom: So, to wrap up the "Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design" paper, the main point is that they introduced Affinity Explainer to interpret matching-based few-shot models by analyzing their matching scores.

Jane: They claim this gives us attribution maps that highlight which pixels in the support images are most influential for a query segmentation prediction, and they evaluate it using metrics like IAUC and DAUC to show how stable those explanations are.

Lu: The authors established this as the first dedicated method for interpreting matching-based FSS models by leveraging their inherent structural properties.

Meng: It means that instead of just trusting the final segmentation mask, we can use these maps to diagnose *why* the model chose that prediction, which is vital when we deploy these systems.

Tom: The implication is that we can get a better understanding of how these models are actually learning from those few examples, even if the models themselves are mostly opaque.

Jane: It's about moving past just achieving high accuracy and starting to build more trustworthy AI systems in data-scarce environments by understanding the decision process behind them.

Conclusion: Tom: So, we’re wrapping up this look at "Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design." Basically, this paper shows how to take those complex models that are trained with a few examples and actually figure out what they're looking at.

Jane: It seems like the authors are focusing on these matching operations within those few-shot models because that’s where the magic happens when you don't have tons of labeled data.

Lu: Exactly, they’re taking those raw matching scores—the comparison between the query image and all those support images—and turning them into actual maps showing which parts of the support images were most important for the final segmentation decision.

Meng: So, it's not just giving you a box saying "this is what it saw," but actually pointing to pixel by pixel in those reference pictures. That's a practical difference for deployment because you can debug *why* it failed on a specific object.

Tom: Right, and they use these attribution maps—they call them AffEx—and they test their stability with metrics like IAUC and DAUC to make sure the explanation isn't just random noise.

Jane: The paper’s authors are making a big deal out of this because it moves interpretation from some abstract idea into a concrete, measurable framework for these specific types of models.

Lu: It sets a foundation here for anyone trying to understand why matching-based methods work so well in low-data settings, because now there's a systematic way to see the contribution.

Meng: For me, if we can reliably tell which support image pixels drove the decision, that’s huge when we need to check for bias or weird artifacts in our small training sets.

Tom: It’s definitely about making sure these few-shot systems aren't just guessing, but actually focusing on the right evidence.

Jane: It leaves us wondering how much more of these matching-based models we can trust once we have this kind of visual evidence to check their logic against.

More episodes

← Home