MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training".
Jane: Here's the gist of the paper. Recent research in geospatial machine learning shows that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks even when labeled data is limited.
Tom: However, existing benchmark datasets often suffer from having few data modalities and poor global representation, which hinders the evaluation of multimodal pretrained models at a global scale.
Jane: To address this gap, the authors introduce MMEarth-Bench, a new multimodal benchmark dataset comprising five environmental monitoring tasks across 12 aligned Earth observation (EO) modalities.
Tom: This dataset provides globally distributed data and includes both random and geographic test splits for robust benchmarking.
Paper summary: Tom: So wrapping up this paper, MMEarth-Bench gives us a complete set of environmental monitoring tasks and data types to really test how well pretrained models perform when they have limited labeled data or need to adapt to new geographic areas.
Jane: And the authors are demonstrating that while self-supervised multimodal pretraining helps in those low-shot regimes and under geographic shifts, the TTTMMR approach gives us a way to adapt those models efficiently at test time by leveraging all available modalities as reconstruction tasks.
Lu: The study suggests that TTTMMR-Geo is the best method for most models, although they also pointed out that it isn't perfect, and some specific models still show limitations when using this adaptation technique.
Meng: From an engineering view, this means we have a standardized way to improve model adaptation without having to retrain the whole thing from scratch every time you move to a new region.
Lalam: It confirms that leveraging all modalities at test time as reconstruction tasks is a solid strategy for boosting performance in these complex geospatial scenarios.
Conclusion: Tom: So we've seen how they built this whole new testing ground called MMEarth-Bench, and now we need to talk about what that title means for us.
Jane: It’s titled "MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training." That sounds like a lot of jargon for something that’s actually about how models work when they have to change their minds.
Tom: Exactly. The core idea here is taking these models that are already trained on tons of data and figuring out how to make them perform well, even when they move to a completely new geographic area or see a different type of data they weren't trained on.
Lu: What the authors are really pushing here is that self-supervised multimodal pretraining helps with those low-shot scenarios and shifting geography, but it's not perfect across all tasks.
Jane: They found that while this pretraining is good, geographic generalization still struggles on some of the tasks they tested. It stops working reliably for about four out of five monitoring goals they looked at.
Tom: And the big takeaway from their work is introducing TestTime Training with MultiModal Reconstruction, or TTT-MMR, as a way to fix that adaptation problem at test time without retraining the whole thing.
Jane: So you’re telling us they use all the different data types available—all twelve of them—as extra training tasks during the test phase to help the model adjust quickly.
Lu: Right. They normalize those reconstruction losses across all modalities so every single one contributes fairly to that adaptation signal, no matter if it was a big or small dataset originally.
Tom: I’m seeing results where TTT-MMR and a version called Geographic Batching actually beat training the model from scratch in many cases, which is huge for saving time on new deployments.
Jane: That means for someone deploying an AI system into a remote part of the world, they don't have to wait months to collect enough labeled data just to get it working.
Tom: It shows that we can leverage existing knowledge across different sensors—RGB, multispectral, whatever—to make models more robust when the real-world conditions change.
Lu: The authors highlight that this method works best for most models except for a couple of specific ones, which tells you exactly where the current limitations are in the technology itself.
Jane: It’s a solid step forward, showing that we can bridge the gap between training models on clean data and making them useful in messy, real-world environments.
Tom: And this opens up a lot of discussion about how we should be designing those next generation benchmarks to really test these new adaptation techniques.
Harvard University · University of Copenhagen
cs.CV
Submitted: 2026-02-06
Updated: 2026-11-11
Comments: Published at ECCV 2026, project page: https://lgordon99.github.io/mmearth-bench/
DOI: 10.1007/978-3-032-37252-9_19
Project page: https://lgordon99.github.io/mmearth-bench-app
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 99/100
The gist: Recent research in geospatial machine learning shows that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks even when labeled data is…
Key concepts
- MMEarth-Bench
- This is a new multimodal benchmark dataset comprising five environmental monitoring tasks across 12 aligned Earth observation (EO) modalities. It provides globally distributed data and includes both random and geographic test splits to robustly evaluate how well models generalize across different types of environmental data.
- Multimodal Pretraining
- This involves training models using self-supervised learning on diverse Earth observation data, such as RGB or multispectral images. The goal is to teach the model general features from many different sources so it can perform well on new, related tasks even when labeled training data is scarce.
- TestTime Training with MultiModal Reconstruction (TTT-MMR)
- This method adapts a pretrained model at test time by treating all available modalities as auxiliary self-supervised tasks. It uses modality reconstruction losses as the adaptation signal, normalizing gradients across modalities to ensure every input type contributes equally to the model's learning process.
- Geographic Batching (TTT-MMR-Geo)
- This variant of TTTMMR introduces a trade-off between regularization and specialization during test-time adaptation. It is particularly effective at improving performance on rare events or long-tail distributions, making it robust to sample rarity in regression tasks.
Terminology
Summary
Recent research in geospatial machine learning shows that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks even when labeled data is limited. p. 0 dk
Abstract. Recent research in geospatial machine learning demonstrates
that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks with limited labeled
data. Recent research in geospatial machine learning demonstrates
that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks with limited labeled
data. However, most benchmark datasets have few data modalities and
poor global representation, limiting the ability to evaluate multimodal
pretrained models at global scales. MMEarth-Bench dataset: We introduce five multimodal environmental
monitoring tasks for benchmarking pretrained models. Each task consists
of 12 aligned modalities and provides globally distributed data with both
random and geographic test splits. MMEarth-Bench dataset: We introduce five multimodal environmental
monitoring tasks for benchmarking pretrained models. Each task consists
of 12 aligned modalities and provides globally distributed data with both
random and geographic test splits.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The study benchmarks eight recent pretrained models, including those with RGB, multispectral, and multimodal pretraining strategies. p. 2 Benchmarking: We benchmark 8 recent pretrained models covering a range
of architectures and including both uni- and multimodal models. We find
that all models still struggle to generalize geographically and that training
a multimodal model from scratch is competitive with global pretraining. In order to fill this gap, we introduce
MMEarth-Bench, a collection of five new environmental tasks with 12
modalities, globally distributed data, and both random and geographic
test splits. We benchmark a diverse set of pretrained models and find
that while (multimodal) pretraining tends to improve model robustness
in limited data settings, geographic generalization abilities remain poor. We benchmark a diverse set of pretrained models and find
that while (multimodal) pretraining tends to improve model robustness
in limited data settings, geographic generalization abilities remain poor. Moreover, a simple randomly initialized multimodal model is competitive given enough labeled data.Where the paper says it
Where the paper says it
Where the paper says it
To overcome the limitation where models can only utilize modalities they were pretrained on, they propose TestTime Training with MultiModal Reconstruction (TTT-MMR). p. 2 Complementary to using multimodal data
for self-supervised pretraining, we propose using the modalities as reconstruction
tasks in the regime of test-time adaptation, which uses unlabeled test data to
adapt the model at test time. We formulate our proposed method for TestTime Training with MultiModal Reconstruction (TTT-MMR), along with a
second variant using geographic batching (TTT-MMR-Geo), which leads to
a good trade-off between specialization and regularization, a crucial aspect in
TTT [25, 39]. To address this inefficiency, we introduce
a method for using all available modalities at test time, whether or not they are
compatible with the pretrained model. Complementary to using multimodal data
for self-supervised pretraining, we propose using the modalities as reconstruction
tasks in the regime of test-time adaptation, which uses unlabeled test data to
adapt the model at test time. Then at test time the reconstruction losses
serve as an adaptation signal for the encoder (see Fig. TTT-MMR normalizes the gradients of the task modality reconstruction losses by modality and
then averages them across modalities (excluding missing modalities) so that each
modality contributes equally to the adaptation signal, regardless of its original
scale.Where the paper says it
Where the paper says it
Where the paper says it
A variant, Geographic Batching (TTT-MMR-Geo), is proposed to further improve performance by introducing a trade-off between regularization and specialization during test-time adaptation. p. 2 Complementary to using multimodal data
for self-supervised pretraining, we propose using the modalities as reconstruction
tasks in the regime of test-time adaptation, which uses unlabeled test data to
adapt the model at test time. We formulate our proposed method for TestTime Training with MultiModal Reconstruction (TTT-MMR), along with a
second variant using geographic batching (TTT-MMR-Geo), which leads to
a good trade-off between specialization and regularization, a crucial aspect in
TTT [25, 39]. To solve this problem, we propose using all the modalities available at
test time as auxiliary tasks for test-time adaptation. Our model-agnostic
method for test-time training with multimodal reconstruction (TTTMMR) can improve performance across all models and tasks on both
test splits. Our model-agnostic
method for test-time training with multimodal reconstruction (TTTMMR) can improve performance across all models and tasks on both
test splits. Furthermore, geographic batching leads to a good trade-off between regularization and specialization during TTT, which is especially
beneficial for long-tail distributions.Where the paper says it
Where the paper says it
Where the paper says it
The results indicate that while all benchmarked models benefit from the proposed methods, TTTMMR-Geo is the best method for most models except SatlasNet, Galileo, and ConvNeXtV2A-MM. p. 9 Table 4: Pretrained models benchmarked. Method Architecture Pretraining dataset Input modalities
Scale-MAE [50] ViT-L FMoW-RGB [12] RGB
DINOv3 Web [55] ViT-L/16 distilled LVD-1689M [55] RGB
DINOv3 Sat [55] ViT-L/16 distilled SAT-493M [55] RGB
SatlasNet [5] Swin-v2-B SatlasPretrain [5] S2
MPMAE [42] ConvNeXt V2-A MMEarth [42] S2
TerraMind [29] TerraMindv1-B TerraMesh [8] S2, S1, DEM, RGB
Copernicus-FM [62] ViT-B Copernicus-Pretrain [62] S2, S1, DEM, Geolocation, Date
Galileo [60] ViT-B Galileo dataset [60] S2, S1, NDVI, Temperature, Precipitation,
DEM, Dynamic World, Geolocation, Month
(ConvNeXtV2A-MM) with all modalities as input. As we do not
use test labels, we treat these modalities as proxies for our task of interest. While
we know that canopy height is correlated with biomass [17, 32], for example, it
is generally nontrivial to preselect the best proxies for a given task. Our model-agnostic
method for test-time training with multimodal reconstruction (TTTMMR) can improve performance across all models and tasks on both
test splits. Furthermore, geographic batching leads to a good trade-off between regularization and specialization during TTT, which is especially
beneficial for long-tail distributions. MMEarth-Bench 7
Table 2: MMEarth-Bench tasks. Task # Tiles Unit Scale Type License
Biomass 18,393 Mg/ha Pixel-level Regression CC BY
Soil Nitrogen 5,643 g/kg Tile-level Regression CC BY-NC
Soil Organic Carbon 7,982 g/kg Tile-level Regression CC BY-NC
Soil pH 8,508 Unitless Tile-level Regression CC BY-NC
Species 36,410 Occurrence Tile-level Multi-label classification Terms of Use
Table 3: MMEarth-Bench modalities.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
In conclusion, MMEarth-Bench serves as a comprehensive multimodal benchmark dataset. p. 1 While existing EO benchmarks [9, 22, 31,38, 62] have helped to develop
and evaluate pretrained geospatial models, they limit the evaluation of progress
towards overcoming these three grand challenges, as most of their downstream
tasks are i) regional rather than global, ii) not evaluated on geographic splits
with an entire held-out region, and iii) only provide few modalities per task. In this work we introduce MMEarth-Bench, a multimodal EO benchmark
dataset containing five global tasks designed to evaluate pretrained geospatial
models in these three challenging yet crucial settings. To solve this problem, we propose using all the modalities available at
test time as auxiliary tasks for test-time adaptation. Our model-agnostic
method for test-time training with multimodal reconstruction (TTTMMR) can improve performance across all models and tasks on both
test splits.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Multimodal adaptation signal using all available modalities at test time (TTT-MMR) can be used to improve performance across all models and tasks on both test splits, as it
improves performance across all models and tasks on both test splits.
-
Geographic batching in the TTT process (TTT-MMR-Geo) leads to a good trade-off between regularization and specialization during adaptation, which is
especially beneficial for long-tail distributions.
-
The proposed method allows for model adaptation when labels are limited and geographically biased by
repurposing multimodal pretext tasks for testtime adaptation, using modality reconstruction losses as a test-time adaptation signal.
-
The system can be designed to use all modalities available at test time as auxiliary tasks for inference, addressing the inefficiency where models
can currently only make use of the modalities on which they were pretrained.
-
Geographic batching provides
unique robustness to rare samples
for regression tasks, as it allowsmore specialization to the geographic domain defined by the batch,
which is crucial when dealing with long-tail distributions.
Abstract
Recent research in geospatial machine learning demonstrates that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks with limited labeled data. However, most benchmark datasets have few data modalities and poor global representation, limiting the ability to evaluate multimodal pretrained models at global scales. In order to fill this gap, we introduce MMEarth-Bench, a collection of five new environmental tasks with 12 modalities, globally distributed data, and both random and geographic test splits. We benchmark a diverse set of pretrained models and find that while (multimodal) pretraining tends to improve model robustness in limited data settings, geographic generalization abilities remain poor. Moreover, a simple randomly initialized multimodal model is competitive given enough labeled data. Although data is abundant, models can currently only make use of the modalities on which they were pretrained. To solve this problem, we propose using all the modalities available at test time as auxiliary tasks for test-time adaptation. Our model-agnostic method for test-time training with multimodal reconstruction (TTT-MMR) can improve performance across all models and tasks on both test splits. Furthermore, geographic batching leads to a good trade-off between regularization and specialization during TTT, which can be beneficial for long-tail distributions. Our dataset, code, and visualization tool are linked on the project page.
Sources
- TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data
- PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Models
- Using Multiple Input Modalities Can Improve Data-Efficiency and O.O.D. Generalization for ML with Satellite Imagery
- DINOv3
- Unsupervised Domain Adaptation through Self-Supervision
- Towards a Unified Copernicus Foundation Model for Earth Vision
- Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation
Related papers
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs
- From Formulas to Protocols: A Compositional View of Binary Target Segmentation Metrics
- Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment