Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework

The gist The CORE dataset and PLANET framework establish a million-scale benchmark for global cross-modal geo-localization by integrating diverse geographic imagery with a physical-law-aware network that aligns textual descriptions with intrinsic physical signatures.

CORE Dataset Introduction

The CORE dataset is introduced as the first million-scale dataset dedicated to global CMGL, comprising 1,034,786 cross-view images sampled from 225 distinct geographic regions across six continents The dataset offers an unprecedented variety of perspectives in varying environmental conditions and urban layouts It is characterized by five key dimensions that facilitate robust global-scale geo-localization: Geographical Diversity, Scale Diversity, Scene Diversity, Physical Attribute Profiling, and Fine-grained Tokens

** Geographical Diversity: The CORE dataset spans 225 distinct regions across six continents, encompassing a broad spectrum from high-density urban centers to sprawling suburban landscapes This extensive sampling strategy is designed to mitigate overfitting to specific architectural styles or regional surface textures **

** Scale Diversity: By integrating multi-source streetlevel and high-resolution satellite imagery, CORE establishes a hierarchical framework supporting localization at city, continental, and global scales **

** Scene Diversity: CORE incorporates a highly heterogeneous scene distribution spanning multiple continents, achieving multidimensional coverage from dense urban centers to expansive suburban areas **

** Physical Attribute Profiling: A defining merit of the CORE dataset lies in its emphasis on intrinsic physical signature characterization, which prioritizes immutable physical realities over transient phenomena **

** Fine-grained Tokens: The token lengths in CORE primarily range from 125 to 175, peaking at 140, ensuring semantic depth while enhancing training robustness **

PLANET Framework Architecture

The PLANET framework is proposed as a physical-law-aware network for cross-modal geo-localization, introducing a novel contrastive learning paradigm to guide textual representations in capturing the intrinsic physical signatures of satellite imagery The overall architecture consists of three integral parts: an intrinsic physical signature mining module, a general dual-stream vision-language architecture, and a consistency contrastive learning paradigm

** Intrinsic Physical Signature Mining: This module explicitly decouples the statistical distribution patterns of satellite imagery to mine essential physical signatures along three orthogonal dimensions: spectral chromaticity, geometric structure, and surface texture **

** Spectral Chromaticity Mining: Spectral distribution serves as the most intuitive physical attribute for distinguishing land cover types, utilizing statistical color histograms as supervision targets to capture macrolevel spectral priors **

** Geometric Structure Mining: This involves leveraging the histogram of gradient orientations within a masked set Ω to construct the geometric structure signature, which encodes the dominant directionality of the scene **

** Surface Texture Mining: This uses the histogram of Laplacian energy distribution after logarithmic transformation to characterize high-frequency details, enabling explicit differentiation between sparse strong edges and dense weak textures **

Dual-Stream Encoding and Projection

PLANET operates on a dual-stream backbone consisting of an image encoder ΦiMg and a text encoder ΦtXt to extract representations from the satellite image I and the description T The visual branch maps the satellite image into a normalized global embedding vector, while the textual branch extracts both a global semantic embedding and sequence features

** Global Dual-Stream Encoding & Contrastive Alignment: Standard InfoNCE loss is applied to these global features to ensure general semantic consistency between modalities, formulating the objective as itc = −1/N∑i=1 log exp(sim(vi, ti)∕τ)∑j=1 exp(sim(vi, tj)∕τ) **

** Physical Semantic Projection via Learnable Queries: To disentangle intrinsic physical properties from the linguistic stream, learnable physical queries (Qcolor, Qstruct, Qtex) are introduced to project high-level textual features into fine-grained physical attribute descriptors t̂k **

** Projection Mechanism: The projected physical descriptors t̂k are computed using a scaled dotproduct attention mechanism, where the query vector attends to tokens relevant to the specific physical attribute, effectively deconstructing holistic semantics into a fine-grained form that is dynamically queryable **

Consistency Contrastive Learning

The core of PLANET is a Consistency Contrastive Learning paradigm that utilizes the intrinsic physical signatures mined from satellite imagery as direct supervision signals This mechanism enforces the alignment between the projected textual descriptors and the deterministic visual statistics of the image

** Intrinsic Physical Supervision: The three orthogonal physical signatures—fcolor, fstruct, and ftex—are treated directly as unlearnable ground truth anchors for their respective physical dimensions **

** Fine-grained Consistency Objective: The model is optimized via a physical consistency loss phy = ∑k−1N∑i=1 log exp(sim(t̂k,i,fk,i)∕τp)∑j=1 exp(sim(t̂k,i,fk,j)∕τp), which forces the text encoder to learn interpretably by mapping linguistic cues to their corresponding physical manifestations **

** Joint Optimization Objective: The total loss function is formulated as total = itc + λphy, where λ is a hyper-parameter balancing global semantic understanding and physical grounding **

Experimental Results and Analysis

Experiments on the CORE dataset demonstrate that PLANET significantly outperforms established methods across various regions of the CORE dataset and other datasets The quantitative results indicate that PLANET achieves state-of-the-art performance on this benchmark

** Comparison With Baselines: In highly challenging global benchmarks, PLANET achieves 55.84% on R@1 and 59.66% on L@150, representing significant improvements of 3.92% and 3.78% in retrieval accuracy and localization accuracy, respectively, compared to the strong baseline CrossText2Loc **

** Ablation Studies: The introduction of any single physical constraint (fcolor, fstruct, or ftex) in the baseline consistently enhances performance across all continental subsets **

** Cross-Subset Generalization Experiments: When tested on Subset 4 (African and South American regions), PLANET achieves absolute performance improvements of 5.19%, 5.74%, and 6.21% in R@1, fully demonstrating its outstanding robustness in capturing domain-invariant physical laws **

The results confirm that the highly compact latent space learned by PLANET stems not from overfitting to domain-specific statistics, but from its successful capture of globally applicable, domain-invariant physical laws This robust performance across diverse geographic environments validates the effectiveness of the proposed framework for worldwide CMGL.

Conclusion

The paper proposes CORE as a million-scale dataset and PLANET as a physical-law-aware consistency contrastive learning paradigm that explicitly aligns intrinsic physical signatures between imagery and text This benchmark facilitates the development of new algorithms for global CMGL, demonstrating robust localization capabilities across highly heterogeneous global scenarios. The work establishes a new benchmark for robust, global-scale geo-localization.

The dataset and source code will be released at https://github.

1. This system can perform robust, global-scale geo-localization by leveraging CORE, the first million-scale dataset dedicated to global CMGL, which offers unprecedented variety of perspectives in varying environmental conditions and urban layouts. The model achieves this through the PLANET framework, which employs a physical-law-aware network that utilizes a novel contrastive learning paradigm to guide textual representations in capturing the intrinsic physical signatures of satellite imagery.

2. The improved AI system can establish fine-grained semantic alignment by explicitly aligning the projected descriptors t̂k with the deterministic visual statistics of the image, ensuring that specific linguistic cues (e.g., “red roofs”, “grid layout”) are correctly mapped to their corresponding physical manifestations without semantic drift.

3. The model's localization accuracy is enhanced by enforcing a physical consistency loss phy which forces the text encoder to learn interpretably, ensuring that specific attributes from the broader scene context, which leads to incorrect retrieval results are accurately retrieved by perceiving key physical attribute terms.

4. The system gains superior domain-invariant feature extraction capabilities by utilizing intrinsic physical signature mining, which explicitly decouples the statistical distribution patterns of satellite imagery to mine essential physical signatures along three orthogonal dimensions: spectral chromaticity, geometric structure, and surface texture.

5. The model's robustness against geographic shifts is significantly improved; experiments show that PLANET achieves performance improvements of 5.19%, 5.74%, and 6.21% when migrating from training subsets to the Subset 4, validating its ability to capture globally applicable, domain-invariant physical laws.