OT-BLS: Optimal-Transport-based Backbone Layer Selection for Frozen DINOv3 in Cross-Domain Few-Shot Segmentation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "OT-BLS: Optimal-Transport-based Backbone Layer Selection for Frozen DINOv3 in Cross-Domain Few-Shot Segmentation".
Jane: Here's the gist of the paper. Cross-domain few-shot segmentation (CD-FSS) aims to segment a target class in query images from a previously unseen domain using only a few annotated support examples The optimal DINOv3 layer for mask prediction varies across support–query episodes The Problem Addressed Cross-domain few-shot segmentation (CD-FSS) is challenging because domain differences make support–query correspondence ambiguous Existing layer-selection criteria do not explicitly evaluate whether the predicted query foreground and background labels agree with the corresponding support labels This limitation can cause a criterion to select a layer that performs worse than the simple last-layer baseline, which always uses the last-layer features.
Tom: The Proposed Solution We propose Optimal-Transport-Based Backbone Layer Selection (OT-BLS) which scores how well the predicted query foreground and background correspond to the matching support foreground and background, and how clearly the predicted foreground is separated from the background.
Jane: This score reflects how well the predicted foreground and background of the query correspond to those of the support.
Tom: The core insight is that an effective layerselection criterion can be derived from the optimal-transport correspondence between query and support patch-token features.
Paper summary: Tom: So, to wrap up this paper, "OT-BLS: Optimal-Transport-based Backbone Layer Selection for Frozen DINOv3 in Cross-Domain Few-Shot Segmentation," it’s proposing a novel layer selection method for frozen vision models in cross-domain few-shot segmentation.
Jane: The main contribution is using optimal transport correspondence between query and support patch tokens to derive a score that measures label agreement between the predicted query foreground and background and the corresponding support labels.
Lu: It addresses the problem where existing layer selection criteria don't explicitly evaluate whether the predicted query labels agree with what they expect from the support examples. This is important because domain differences make that correspondence ambiguous, as mentioned in introduction <ref:2611.15187#pg1>.
Meng: In simple terms, it means we are using a mathematical way to check if the predicted foreground and background of our query mask actually look like the foreground and background we saw in the support images.
Tom: And even though they found that it doesn't work perfectly on every single setting—like those DeepGlobe or ISIC one-shot cases—it still outperforms the last-layer baseline in twelve out of sixteen settings and improves performance metrics on eight benchmark datasets.
Jane: The implication for us is that transport-based support query label agreement is a more reliable signal for layer selection than the proxy-based criteria used in prior work, which is what this paper claims <ref:2611.15187#pg3>.
Lu: It suggests that building a layer selection criterion based on optimal transport correspondence between features is a solid direction for trainingfree segmentation methods.
Lalam: I'm pretty sure this paper offers a more grounded, evidence-based way to select the best backbone layer when you're dealing with cross-domain few-shot segmentation.
Conclusion: Tom: So, we’ve been looking at this paper titled "OT-BLS: Optimal-Transport-based Backbone Layer Selection for Frozen DINOv3 in Cross-Domain Few-Shot Segmentation." It’s basically about finding the right layer in a frozen vision model for cross domain few shot segmentation.
Jane: Right, so what’s the core idea here? It seems like they are trying to solve this problem where the best layer isn't always obvious because we don're dealing with different image domains. What did they actually propose?
Lu: They propose Optimal Transport Based Backbone Layer Selection, which is a scoring method. It scores layers by looking at how well the predicted query foreground and background match what we saw in the support examples.
Meng: Matching foreground and background—that sounds like it’s checking for label consistency across different image styles. But how do they actually do that mathematically? What’s the mechanism?
Tom: They use optimal transport correspondence between patch tokens from the query and support images. They compute a score by balancing precision for both foreground and background labels based on this transported mass.
Jane: So, it’s not just looking at one thing; it’s weighing how well the model predicts both what’s in the foreground versus what’s in the background relative to our known support data. That sounds like a more complete check than just one simple metric.
Lu: Exactly. They define this score by combining precision for both foreground and background agreement, controlled by a parameter alpha, which lets you balance how much you care about each label type. It’s an aggregation of the transported mass between query and support features.
Meng: Alright, so if I’m deploying a model to production, I want to know what this means for my pipeline. Does it make inference slower? Does it require more complex setup?
Tom: The authors found that OT-BLS outperforms the last layer baseline in twelve out of sixteen dataset shot settings. And they improved the average mean intersection over union, mIoU, from five point seven two to five point seven six on eight benchmark datasets.
Jane: That’s a solid lift in performance for a method that supposedly runs faster than some existing criteria. So what’s the big picture implication here for someone just trying to segment something new?
Lu: The paper suggests that this transport-based label agreement is a more reliable signal than the proxy based criteria they compared it against. It implies that using feature correspondence, rather than just looking at simple confidence or reconstruction metrics, leads to better layer selection.
Meng: I’m still watching those results on DeepGlobe and ISIC one shot cases because if it fails there, it’s not ready for real traffic deployment. So where does this leave the research?
Tom: It shows that transport based support query label agreement is a more reliable signal than proxy based criteria. The final score, that s value they calculate, is also lower when the masks are wrong, which is good because it penalizes bad choices.
Jane: It sounds like the authors are confident this mechanism provides a better foundation for trainingfree segmentation methods moving forward. What's next for this research?
Lu: The work shows that transport based support query label agreement is a more reliable layer selection signal than the proxy based criteria used in prior work. This points toward using feature correspondence as a primary signal instead of just indirect proxies like prediction confidence.
NTT Docomo Business, Inc.
cs.CV
Submitted: 2026-11-11
Updated: 2026-11-11
Comments: Accepted for publication at the 2026 IEEE International Conference on Visual Communications and Image Processing (VCIP 2026)
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 83/100
The gist: Cross-domain few-shot segmentation (CD-FSS) aims to segment a target class in query images from a previously unseen domain using only a few annotated support examples The optimal DINOv3 layer for…
Key concepts
- Cross-domain few-shot segmentation (CD-FSS)
- This task involves segmenting a target class in images from a new domain using only a few labeled examples from that domain. It is hard because the differences between the source and target domains make it difficult to reliably match which support example corresponds to which query image.
- Optimal Transport (OT) Algorithm
- This mathematical tool is used to find the best way to map or 'transport' features from the query images onto features from the support images. It computes a plan that minimizes the difference between these two sets of features, helping determine correspondences between patches in different images.
- OT-BLS Score (s(l))
- This score measures how well a specific DINOv3 layer predicts the foreground and background labels for both the query and support. It balances agreement on foreground matches against agreement on background matches, providing a reliable signal to select the best backbone layer for segmentation.
Terminology
Summary
Cross-domain few-shot segmentation (CD-FSS) aims to segment a target class in query images from a previously unseen domain using only a few annotated support examples The optimal DINOv3 layer for mask prediction varies across support–query episodes
The Problem Addressed
Cross-domain few-shot segmentation (CD-FSS) is challenging because domain differences make support–query correspondence ambiguous Existing layer-selection criteria do not explicitly evaluate whether the predicted query foreground and background labels agree with the corresponding support labels This limitation can cause a criterion to select a layer that performs worse than the simple last-layer baseline, which always uses the last-layer features.
The Proposed Solution
We propose Optimal-Transport-Based Backbone Layer Selection (OT-BLS) which scores how well the predicted query foreground and background correspond to the matching support foreground and background, and how clearly the predicted foreground is separated from the background. This score reflects how well the predicted foreground and background of the query correspond to those of the support. p. 0 The core insight is that an effective layerselection criterion can be derived from the optimal-transport correspondence between query and support patch-token features. p. 0
How the Training Signal is Built
The OT-BLS score s(l) is defined as s(l) = α precision(l) fg + (1 − α) precision(l) bg, where α ∈ [0, 1] controls the balance between foreground and background label agreement. Precision scores are defined based on transported mass: precision(l) fg = TP(l) fg / (TP(l) fg + FP(l) fg), where TP(l) fg is the amount of the predicted query foreground matched to the support foreground. p. 1 p. 1 p. 1 The transport plan P∗ is computed using an entropy-regularized optimal transport algorithm on patch-token features. p. 0 p. 0 p. 0
Where the Gains Come From
Experiments on eight benchmark datasets demonstrate that OT-BLS outperforms the last-layer baseline in 12 of 16 dataset/shot settings. Furthermore, OT-BLS improves the average mIoU from 55. p. 072 to 57.62 and achieves a 2. p. 0 p. 099-point improvement over the bestperforming existing layer-selection criterion while running 3. p. 0 p. 07–14.6× faster. p. 0 OT-BLS provides the best selection performance with a modest runtime cost. p. 3
Conclusion
We have proposed OT-BLS, an optimal-transport-based layer-selection criterion for CD-FSS with frozen DINOv3. These results indicate that transportbased support–query label agreement is a more reliable layerselection signal than the proxy-based criteria used in prior work. p. 4 The final score s is also lower for both erroneous masks. p. 3 p. 3
REFERENCES
[26] Natsuo Okamoto, Kazuki Kobayashi, Yoshio Kato, Shuhei Tarashima NTT DOCOMO BUSINESS, Inc., Tokyo, Japan natsuo. p. 0okamoto@ntt. p. 0com kzk.kobayashi@ntt. p. 0com yoshio. p. 0kato@ntt.com tarashima@acm. p. 0org OT-BLS: Optimal-Transport-based Backbone Layer Selection for Frozen DINOv3 in Cross-Domain Few-Shot Segmentation Abstract—Cross-domain few-shot segmentation (CD-FSS) aims to segment a target class in images from a previously unseen domain using only a few annotated support examples Trainingfree CD-FSS methods, which typically leverage vision foundation models such as DINOv3, achieve competitive performance without parameter updates However, the optimal DINOv3 layer for mask prediction varies across support–query episodes Although several layer-selection criteria have been proposed, they do not explicitly evaluate whether the predicted query foreground and background labels agree with the corresponding support labels To address this limitation, we propose a novel layerselection criterion, Optimal-Transport-Based Backbone Layer Selection (OT-BLS) Our key insight is that an effective layerselection criterion can be derived from the optimal-transport correspondence between query and support patch-token features Based on this insight, we first estimate the transport plan using a well-established optimal transport algorithm and then compute a layer-selection score by aggregating the transported mass according to the foreground–background partition defined by each candidate query mask Experiments on eight benchmark datasets demonstrate the effectiveness of OT-BLS It outperforms the conventional last-layer baseline in 12 of 16 dataset/shot settings Furthermore, OT-BLS improves the average mIoU from 55. p. 072 to 57.62 and achieves a 2.99-point improvement over the bestperforming existing layer-selection criterion while running 3.7–14.6× faster Index Terms—Cross-domain few-shot segmentation, trainingfree segmentation, DINOv3, optimal transport, layer selection I. p. 0 INTRODUCTION Cross-domain few-shot segmentation (CD-FSS) aims to segment a target class in query images given only a few annotated support examples of that class The support and query images come from a target domain that differs from the domain used to train the model It is especially useful in domains such as medical, underwater, and satellite imagery, where dense pixel-level annotation is costly CD-FSS is more challenging than ordinary few-shot segmentation (FSS) because domain differences make support–query correspondence ambiguous [12] In addition to CD-FSS methods that assume additional training or domain adaptation [13], [5], [11], [12], trainingfree methods based on pre-trained vision foundation models have recently attracted attention In this setting, the backbone is kept frozen, and no parameters are fine-tuned for the target domain or for each support–query episode Keeping the backbone frozen also preserves the generalization of the pre-trained features, which fine-tuning on a few target examples can weaken through overfitting Zakir et al. p. 0 [15] reported that frozen DINOv3 [14] can be competitive with CD-FSS methods that require additional training Interestingly, they also showed that the last layer is not always the best choice for query mask prediction Existing criteria [15] score candidate layers using indirect proxies such as prediction confidence, feature separability, mask reconstruction consistency, or feature-statistic similarity These proxies do not directly measure whether the foreground predicted on the query agrees with the support foreground and is clearly separated from the background This limitation can cause a criterion to select a layer that performs worse than the simple last-layer baseline, which always uses the last-layer features. p. 0
II. OPTIMAL-TRANSPORT-BASED BACKBONE LAYER SELECTION (OT-BLS) Given a query image and support images, we use the predictor of Zakir et al. p. 0 p. 0 [15] to generate one candidate mask for each DINOv3 layer Our goal is to select the best mask from these candidates OT-BLS only selects among these candidates and does not alter the predictor Layer selection can be written as selecting the candidate layer with the largest scalar score s(l) from the search set L, l∗ = arg max l∈L s(l). p. 1
A. Selection Criterion We now define the OT-BLS score s(l) used in Eq. p. 1 p. 1 (1). For a candidate layer l ∈ L, the score of the candidate mask mˆ(l)q is s(l) = α precision(l) fg + (1 − α) precision(l) bg. p. 1 Precision scores are defined based on transported mass: precision(l) fg = TP(l) fg / (TP(l) fg + FP(l) fg), where TP(l) fg is the amount of the predicted query foreground matched to the support foreground. The transport plan P∗ is computed using an entropy-regularized optimal transport algorithm on patch-token features.
B. Computing the Selection Criterion Computing TP(l) fg and FP(l) fg requires correspondences between query and support tokens We regard a query token and a support token as more likely to correspond when their features are more similar OT establishes a soft setlevel correspondence from such similarities rather than relying on independent nearest-neighbor matches, and can therefore compare feature sets with different numbers of tokens and region sizes. p. 1 We therefore adopt OT and use the resulting plan as a patch-level correspondence between the query and the concatenated collection of support tokens
III. EXPERIMENTS A. p. 0 Experimental Setup We follow the experimental setting of Zakir et al. p. 3 [15] The backbone is a frozen DINOv3 ViT-B with an input resolution of 512 × 512 For ViT-B with L = 12, the search set is L = (7,. p. 3.., 12) Mean intersection over union (mIoU) is the evaluation metric for CD-FSS
B. Comparison Methods To assess layer-selection performance within a fixed prediction pipeline, we compare OT-BLS with six criteria from Zakir et al. p. 3 [15], all of which select from the same candidate masks in each episode These are Entropy, Regularized Patch-Norm Ratio, Query Prediction Fisher Information, Support Prediction mIoU, Reverse mIoU Consistency, and Gram Matrix Distance. p. 3 We report the last-layer baseline
C. Results and Discussion Table I reports the 1-shot and 5-shot results OT-BLS outperformance the last-layer baseline in 12 of 16 dataset/shot settings It falls below the baseline only on DeepGlobe (1- and 5-shot), ISIC 1-shot, and Kvasir-SEG 5-shot Compared with existing criteria, OT-BLS achieves the best mIoU in 11 of 16 settings and the best overall average rank. p. 3 Table II shows that OT-BLS ranks third in runtime and is faster than Support Prediction mIoU, the best-performing existing criterion, by 3. p. 37× in 1-shot and 14. p. 06× in 5-shot. p. 0
IV. CONCLUSION We have proposed OT-BLS, an optimal-transport-based layer-selection criterion for CD-FSS with frozen DINOv3 These results indicate that transportbased support–query label agreement is a more reliable layerselection signal than the proxy-based criteria used in prior work. p. 4 The final score s is also lower for both erroneous masks.
II. OPTIMAL-TRANSPORT-BASED BACKBONE LAYER SELECTION (OT-BLS) Given a query image and support images, we use the predictor of Zakir et al. [15] to generate one candidate mask for each DINOv3 layer Our goal is to select the best mask from these candidates OT-BLS only selects among these candidates and does not alter the predictor Layer selection can be written as selecting the candidate layer with the largest scalar score s(l) from the search set L, l∗ = arg max l∈L s(l)
A. Selection Criterion We now define the OT-BLS score s(l) used in Eq. (1). For a candidate layer l ∈ L, the score of the candidate mask mˆ(l)q is s(l) = α precision(l) fg + (− α) precision(l) bg. p. 1 Precision scores are defined based on transported mass: precision(l) fg = TP(l) fg / (TP(l) fg + FP(l) fg), where TP(l) fg is the amount of the predicted query foreground matched to the support foreground. The transport plan P∗ is computed using an entropy-regularized optimal transport algorithm on patch-token features.
B.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Optimal-Transport-Based Backbone Layer Selection (OT-BLS) can be integrated into a trainingfree DINOv3 segmentation pipeline to select a candidate layer that best aligns predicted query labels with support labels based on optimal transport correspondence between query and support patch-token features.
-
The improved AI system can achieve better performance in cross-domain few-shot segmentation by selecting the most appropriate transformer block, as OT-BLS
scores how well the predicted query foreground and background correspond to the matching support foreground and background, and how clearly the predicted foreground is separated from the background.
-
The system will significantly reduce computational overhead compared to prior criteria because it utilizes a shared optimal transport plan computed once on last-layer features, allowing it to run
3.7–14.6× faster
than existing layer-selection criteria while improving average mIoU from 55.72 to 57.62 on benchmark datasets. -
The selection criterion can be balanced using a weighting parameter alpha, which allows the system to adapt its focus based on domain characteristics, as
the appropriate balance between foreground and background agreement depends on the target domain.
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs