Mind the Gap: Bridging Prior Shift in Realistic Few-Shot Crop-Type Classification

arXiv:2511.16218 · cs.LG · Submitted 2025-11-20 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Mind the Gap".

Jane: The gist:

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper today, "Mind the Gap: Bridging Prior Shift in Realistic Few-Shot Crop-Type Classification." Essentially, they are tackling how to make few-shot learning work better when the training and testing data look different in the real world.

Jane: That’s right. The core idea here is that real agricultural data is often skewed, with some crop types being way more common than others, which messes up standard training methods when you only have a few samples to learn from.

Lu: They propose this method called Dirichlet Prior Augmentation, or DirPA. It proactively simulates the unknown label distribution skew of the target domain during model training.

Meng: So they aren't waiting for test data to see the imbalance; they are injecting that knowledge into the training itself using a Dirichlet distribution to sample class distributions.

Tom: Exactly. They model these real-world distributions as Dirichlet-distributed random variables, which effectively performs a prior augmentation specifically during few-shot learning scenarios.

Jane: The authors claim this approach achieves two main things: "Prior-agnostic representation learning" and "Enhanced regularization and robustness." It seems like they’re trying to make the model less dependent on the exact prior it sees during training.

Lu: They do this by sampling a pseudo-prior vector (s) from the Dirichlet distribution Dir(alpha times one) <ref:2511.16218#pg1>. The parameter alpha, alpha, controls how imbalanced those sampled distributions are; a higher alpha means closer to uniform sampling.

Meng: Then they use a scaling factor tau, tau, to adjust the logits z i via the formula z'i from z i + tau ((s)). This adjustment creates these adjusted logits z'i, and then they compute the predictive probability distribution using the softmax function.

Tom: So what they’re doing is forcing the model to learn a feature representation that works well under a variety of different priors, meaning it doesn't need any prior estimation when it actually has to predict on new data.

Jane: It sounds like this whole setup is built into their end-to-end model composition, which has a Transformer encoder backbone and then a single linear layer head mapping features to class logits.

Lu: They trained this end-to-end model for up to two hundred epochs with a batch size of sixteen optimizing hyperparameters like the learning rate beta, the focal focusing parameter gamma, and those Dirichlet parameters alpha and tau on a fixed set of validation data points.

Meng: They tested this on the Estonia dataset, which is time-series data combining parcel references with Sentinel-two satellite imagery, and that dataset has a strong class imbalance where grassland grass is forty-six percent of all samples.

Tom: And the results show that for cross-entropy loss, adding this prior augmentation achieved higher overall accuracy and kappa scores across all few-shot tasks from one shot up to two hundred shots.

Jane: They noted the biggest gains were actually seen in the low-shot regime, specifically when k is less than or equal to twenty shots <ref:2511.16218#pg3>. That’s significant because that’s where few-shot learning is usually most fragile.

Lu: For their focal loss metric, FL, DirPA yielded higher scores in both metrics for nearly all few-shot configurations except for the very one shot and the two hundred shot settings.

Meng: When they fine-tuned a model pretrained on Latvian data, DirPA improved both metrics across every few-shot regime. And they found that FL DirPA obtained the highest accuracy among all models in most of those few-shot settings.

Tom: Across both ways of starting the model, whether randomly initialized or pretrained, the Dirichlet prior consistently improves both accuracy and Cohen’s kappa scores, with the biggest relative advantage appearing in those low-shot regimes again.

Jane: So for someone just listening to this show, it means that if your training data is imbalanced—like in agriculture—you don't have to worry so much about the test data distribution shifting because you’ve already baked the expected skew into the learning process.

Lu: The conclusion of "Mind the Gap: Bridging Prior Shift in Realistic Few-Shot Crop-Type Classification" is that DirPA successfully bridges that gap by augmenting balanced few-shot training data with those dynamically sampled pseudo-priors from the Dirichlet distribution.

Meng: They argue this process acts as a robust regularizer, improving generalization and stability of the model when it encounters imbalanced test data.

Tom: So, to wrap up on what we’ve heard about "Mind the Gap," this method is applied directly during training and doesn't require any knowledge about the final test distribution when you use it.

Jane: The paper demonstrates that DirPA improved overall accuracy and Cohen’s kappa across various few-shot regimes, confirming its benefits even under label imbalance.

Lu: Their future work looks at applying pseudo-priors sampled from an asymmetric Dirichlet distribution while still assuming an unknown but imbalanced test prior. That’s where they think the next step is going.

Meng: For practical application, this method seems applicable to any few-shot learning task that suffers from a discrepancy between the training and test label distributions.

Tom: It really shows how modeling those latent distribution shifts proactively during training can stabilize models in these tricky scenarios.

Conclusion: Tom: So we're wrapping up our look at "Mind the Gap," which is about how to handle that gap between training data and test data when you’re doing few-shot classification, especially in farming.

Jane: Right, it’s about Dirichlet Prior Augmentation. Basically, the authors are showing you a way to inject realistic knowledge about label imbalance directly into the training process using these prior samples.

Lu: The core idea is that instead of just feeding the model balanced data and hoping for the best, they proactively simulate what an imbalanced real-world distribution might look like during training.

Meng: So, it’s not just adjusting parameters after training; it’s changing how the model learns features from the start to be more robust against those prior shifts.

Tom: And the results show that this method improves both accuracy and stability across all few-shot settings, especially when you have very few shots to learn from.

Jane: That means for someone just listening, it’s like giving the model a head start by teaching it how to handle real-world skew before it even sees the test data.

Lu: The implication is that you don't necessarily need perfect balance in your small training sets if you use techniques that simulate the uncertainty of a skewed distribution.

Meng: From an engineering side, this is neat because it means we might not have to spend so much time trying to manually balance every single tiny dataset.

Tom: But remember, they specifically test this on real Estonian crop data with strong class imbalance, so the proof is in the practical application there.

Jane: The paper shows that for few-shot learning, this prior augmentation helps pull the performance up across almost all scenarios tested.

Lu: They’re also looking ahead to applying these pseudo-priors from asymmetric Dirichlet distributions when dealing with an even more unknown test prior structure.

Tom: That asymmetry is where they think the next big step in this research lies, so we'll be talking about that next.

Technical University of Munich (TUM) · Munich Data Science Institute (MDSI) · ELLIS Unit Jena, University of Jena

cs.LG

Submitted: 2025-11-20

Updated: 2025-11-20

Comments: 7 pages, 4 figures

DOI: 10.5194/isprs-annals-XI-3-2026-787-2026

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 78/100

The gist: The gist: Dirichlet Prior Augmentation (DirPA) is a novel method that simulates an unknown label distribution skew of the target domain proactively during model training to enhance robustness against

Key concepts

Class Imbalance: Real-world agricultural data often has severe class imbalance, meaning some crop types appear much more frequently than others. Standard training methods fail because they use artificially balanced datasets, causing the model to perform poorly on unseen, imbalanced test data.
Dirichlet Distribution: This is a probability distribution used here to sample a wide range of possible class distributions. By sampling from this distribution, DirPA simulates different unknown label biases encountered in the real world during training.
Prior Augmentation: This technique proactively introduces knowledge about the expected class distribution into the model during training. Instead of just learning patterns from limited data, DirPA helps the model learn features that are stable even when the true underlying class proportions change.
Few-Shot Learning (FSL): This involves training a model to perform well on tasks with very few examples per class. The paper addresses the problem where artificially balanced FSL training sets do not accurately reflect the skewed distributions found in actual crop classification tasks.

Terminology

Few-Shot Learning — 2× in this paper · explained in 4 episodes
Generalization — 1× in this paper · explained in 13 episodes

Summary

The gist: Dirichlet Prior Augmentation (DirPA) is a novel method that simulates an unknown label distribution skew of the target domain proactively during model training to enhance robustness against prior shift in realistic few-shot crop-type classification.

Introduction and Motivation

Real-world agricultural distributions often suffer from severe class imbalance, typically following a long-tailed distribution Labeled datasets for crop-type classification are inherently scarce and remain costly to obtain When working with such limited data, training sets are frequently constructed to be artificially balanced—in particular in the case of few-shot learning—failing to reflect real-world conditions This mismatch induces a shift between training and test label distributions, degrading real-world generalization To address this, the authors propose Dirichlet Prior Augmentation (DirPA), a novel method that simulates an unknown label distribution skew of the target domain proactively during model training Specifically, they model the real-world distribution as Dirichlet-distributed random variables, effectively performing a prior augmentation during few-shot learning The main contributions include Prior-agnostic representation learning and Enhanced regularization and robustness

Methodology of Dirichlet Prior Augmentation (DirPA)

DirPA is designed to simulate prior distribution shifts proactively during the training process The core mechanism involves leveraging the Dirichlet distribution to sample a vast range of class distributions The process consists of several steps applied at each training step s = 1,..., S

** Sample pseudo-prior: They sample a pseudo-prior vector π˜(s) ∼ Dir(α · 1) from the symmetric Dirichlet distribution Dir(α · 1) The parameter α ∈ R+ controls the degree of imbalance, with α 1 sampling distributions closer to uniform The adjustment is performed using a scaling factor τ ∈ R+ to adjust the logits zi via z′i ← zi + τ log(π˜(s)) (4) This results in the adjusted logits z′i, and subsequently, the predictive probability distribution pˆi = σ(z′i) is computed by applying the Softmax function The full pseudo-code is outlined in Algorithm 1 This approach forces the model to learn a feature representation that works under varied priors, eliminating the need for any inference-time prior estimation The complete end-to-end model is given by the composition fθ = f head θhead ◦ f backbone θbackbone The backbone is a Transformer encoder architecture, and the head is a single linear layer mapping features to class logits The maximum sequence length Tmax is set to 366 days The model is trained end-to-end for up to 200 epochs with a batch size of 16 Hyperparameters, such as the learning rate β, the focal focusing parameter γ, and the Dirichlet parameters α and τ, are optimized on the 1000 fixed validation data points The experiments are repeated five times with a different random seed r ∈ R The model is trained on the entire Estonian training data to establish a baseline for the task’s complexity in a standard (non-few-shot) setting The two loss functions used are cross-entropy loss (CE) and FL (without a class-imbalance factor), with FL allowing for analysis of the impact of DirPA in combination with a class-agnostic and difficulty-aware loss function The Dirichlet distribution is defined by its probability density function p(x α) = 1/B(α) **

YK c=1 x c / αc−1 c, (b)

Experimental Setup and Results

The study uses the Estonia data from the EUROCROPSML dataset for training and evaluation This dataset is a time-series dataset that combines parcel reference data and multi-class hierarchical crop and agriculture taxonomy (HCAT; Schneider et al., 2023, 2021) labels from EUROCROPS (Schneider et al., 2023) with Sentinel-2 L1C optical satellite observations captured during the year 2021 The dataset exhibits a strong class imbalance with grassland grass being the most frequent one among the 129 crop types, representing 46 % of all samples The total dataset comprising 175 906 samples is divided into training, validation, and test sets The results for the randomly initialized model show that DirPA improved or matched the baseline across all few-shot settings For CE, adding prior augmentation achieved higher overall accuracy and kappa scores across all fewshot tasks from 1- to 200-shot, with the largest gains observed in the low-shot regime (k ≤ 20) For FL, DirPA yielded higher scores in both metrics for nearly all few-shot configurations, except for 1 and 200 samples When fine-tuning the model pretrained on Latvian data, DirPA improved both metrics in all few-shot regimes FL DirPA obtained the highest accuracy among all models in most few-shot settings, while CE DirPA was slightly better at 5, 200, or 500 shots Across both initialization regimes, the Dirichlet prior consistently improves both accuracy and Cohen’s kappa The largest relative advantage appears in the low-shot regimes

Conclusion

DirPA successfully bridges the gap between training and test priors by augmenting the balanced few-shot training data with dynamically sampled pseudo-priors from the Dirichlet distribution This process acts as a robust regularizer, improving generalization and stability of the model across imbalanced test data It is applied directly during the training process and does not require any knowledge about the final test distribution The study demonstrated that DirPA improved overall accuracy and Cohen’s kappa across various few-shot regimes, confirming its benefits under label imbalance Future work will investigate applying pseudo-priors sampled from an asymmetric Dirichlet distribution while still assuming an unknown but imbalanced test prior The method can be applied to any few-shot learning task that suffers from a discrepancy between the training and test label distributions

Algorithm Benchmark Task Summary

The benchmark task involves evaluating performance across various few-shot scenarios (k-shot) for both CE and FL loss functions, comparing randomly initialized and pretrained networks The results show that for the randomly initialized network, DirPA improved or matched the baseline across all few-shot settings For the pretrained network, DirPA improved both metrics in all few-shot regimes The largest relative advantage appears in the low-shot regimes The performance of all variants converges to nearly identical results when the number of available samples increases, as the training prior converges towards the empirical prior of the full Estonia data<ref:2511.

Improvements for AI systems

This improvement involves implementing Dirichlet Prior Augmentation (DirPA) during training to make models invariant to the class prior p(y), which addresses distributional shifts by sampling a vast range of class distributions proactively during training. This allows the AI system to achieve superior robustness against prior shift during inference without any knowledge of the actual test skew.
The improved system will exhibit enhanced regularization and stability, as DirPA is demonstrated to act as an effective regularizer, stabilizing the training process and improving robustness on severely imbalanced target domains, in particular in low-shot regimes. This means the AI can perform reliably even when generalizing from very few labeled samples.
The system will possess superior performance on imbalanced datasets compared to methods that use balanced training sets, as DirPA forces the model to learn a feature representation that is fundamentally prioragnostic, eliminating the need for any inference-time prior estimation.
For low-shot regimes (k ≤ 20), the system will show significant gains in overall accuracy and Cohen’s kappa when using CE or FL loss functions, as DirPA achieves the largest gains observed in the low-shot regime.
The improved system will exhibit better performance with larger few-shot settings, as the results show that for high shot counts, all methods converge to nearly identical results, indicating stability in large data scenarios.

Abstract

Real-world agricultural distributions often suffer from severe class imbalance, typically following a long-tailed distribution. Labeled datasets for crop-type classification are inherently scarce and remain costly to obtain. When working with such limited data, training sets are frequently constructed to be artificially balanced -- in particular in the case of few-shot learning -- failing to reflect real-world conditions. This mismatch induces a shift between training and test label distributions, degrading real-world generalization. To address this, we propose Dirichlet Prior Augmentation (DirPA), a novel method that simulates an unknown label distribution skew of the target domain proactively during model training. Specifically, we model the real-world distribution as Dirichlet-distributed random variables, effectively performing a prior augmentation during few-shot learning. Our experiments show that DirPA successfully shifts the decision boundary and stabilizes the training process by acting as a dynamic feature regularizer.