Verifiably grounded machine interpretation of lunar geology

This paper presents a step toward an automated machine intelligence geologist by embedding the methodology of geologic knowledge discovery and inference into a multimodal vision-language architecture. The work focuses on the stratigraphy of lunar basaltic mare volcanism, training a model to generate verifiably grounded geologic interpretations directly from co-registered topographic, spectral, and geologic maps.

The system, called SelenoVLMage, is built in two stages. Stage A pretrains a multimodal masked autoencoder over tokenized co-registered lunar mosaics (WAC imagery, GLD100 elevation, Clementine 415/750 ratio, and geologic map). Stage B freezes this encoder as a vision tower and bridges it into a LoRA-adapted 7B language model (Qwen2.5-7B) through a Perceiver resampler and zero-initialized gated cross-attention. The supervision targets are distilled prose that open with deterministic raster-derived facts (albedo, relief, roughness, and Clementine ratio), followed on mare tiles by one age sentence, and then the distilled interpretive paragraph.

The paper demonstrates that the design of the supervision targets decides whether the system is verifiably grounded. The governing rule is that a target must not be reproducible without reading the tile. The results show that the model produces text that depends on which tile it sees, with a tile-swap flip rate of 0.520 and a vision-removal flip rate of 0.631. The model names the mapped geologic unit on 85.8% of held-out tiles, and stated-fact fidelity reaches 0.458 in aggregate (0.527 on mare, 0.399 on highland tiles). Mean albedo is stated within one quantization level on 51.6% of tiles and the Clementine 415/750 ratio on 36.9%, whereas relief and roughness are harder (50% and 44.6%).

Two observations locate this grounding in the target design rather than in the architecture. First, fitting the targets competes with reading the tile over training, with causal grounding rising over training while alignment with target content peaks in the first epoch and decays. Second, coarsening the injected facts lifts content metrics at the cost of causal grounding, with the paired tile-swap flip dropping by 0.093 ± 0.024 when terrain-adaptive metre bins are used.

The age behavior follows the stratigraphic model. SelenoVLMage provides an age for 73% of mare tiles (n = 230), while remaining nearly silent on highland terrain, with a false-age rate of 1.5% (n = 270). A stated epoch is correct in 65.9% of cases and adjacent in the remainder. However, the stated age values regress to the global mare prior under every target design. Against centroid ground truth, the served model marginally surpasses the constant baseline (MAE 0.425 vs. 0.43 Ga, Spearman ρ = 0.295), but the difference of −0.005 Ga falls well within the cluster-bootstrap interval of [−0.022, +0.014] Ga. A leave-one-region-out control eliminates the apparent skill, showing the nominal skill reflects a memorized regional age field rather than age values inferred from imagery.

The failure is not attributable to missing visual information. A ridge probe on the frozen encoder latent, under a leave-unit-out evaluation, recovers age with an MAE of 0.429 Ga against 0.449 Ga for the constant prior, with ρ = 0.342 and a slope of 0.143. The probe on the frozen resampler output of the served bridge then localizes the loss, recovering the signal at the level of the encoder probe (MAE 0.428 Ga, ρ = 0.306, slope 0.126). The failure thus lies in extraction through the language pathway during joint training, rather than in representation itself.

Integrating an open-book retrieval mechanism resolves this, enabling the model to faithfully cite published chronologies. The stated age follows the injected record causally (perturbation slope 0.711), visually gated emission persists, and deployed age accuracy is bounded by the retrieval oracle rather than by the model. On mare tiles with an injected record and a stated age (n = 167), the stated value matches the retrieved one within 0.05 Ga on 91.6% of tiles (mean absolute difference 0.097 Ga). In the non-circular polygon-GT set, open book performs comparably to the constant prior overall (MAE 0.416 versus 0.414 Ga), reflecting imperfect retrieval rather than model inference. Where the injected record matches the polygon age (n = 73), the stated age is nearly exact (MAE 0.087 Ga), while mismatches are transferred equally faithfully (MAE 1.38 Ga).

The paper concludes that tile-grounded geological prose is achievable with a frozen, self-supervised vision tower and a standard bridge, provided the supervision targets cannot be reproduced without reading the tile. The measurements delineate what a machine geologist reads from the site and what it must cite from the literature: site evidence enters through vision and is verifiable against the rasters, the published chronology enters through retrieval and is citable, and the model's weights carry behaviour, style, and the mare gate, but should not be trusted as a store of numeric ages. The paper recommends that numeric chronology should not be generated but estimated, by decoupled heads reading the same frozen representations and aggregating over mapped units, and then cited by the language model like any other retrieved fact.

Improvements to AI Systems:

1. Grounding-By-Construction Training Targets: Replace open-ended generative supervision with targets that are provably unanswerable without the input. Each training example must include deterministic, raster-derived facts (e.g., albedo, relief, roughness) that force the model to read the specific tile. This eliminates hallucinated generic outputs and makes causal grounding measurable (e.g., via tile-swap flip rate).

2. Two-Pathway Knowledge Architecture: Separate perceptual inference (vision → language) from factual retrieval (external database → language). Use a frozen vision encoder for site evidence, and an explicit open-book retrieval mechanism for numeric chronologies or other external facts. The language model should cite retrieved records, not generate them from weights.

3. Decoupled Numeric Estimation Heads: For tasks requiring numeric values (e.g., age, depth, temperature), do not generate numbers via the language model. Instead, train a lightweight regression head on the frozen vision encoder’s latent space, aggregate over mapped units, and then feed the resulting estimate as a retrieved fact into the language model for phrasing and citation.

4. Causal Grounding Metrics as Primary Evaluation: Adopt tile-swap flip rate and vision-removal flip rate as first-class metrics during training. Monitor these alongside content fidelity; if content fidelity rises while causal grounding falls, adjust target design (e.g., coarsen injected facts) to rebalance.

5. Region-Aware Generalization Controls: Implement leave-one-region-out validation during training to detect memorization of regional priors. If performance collapses under this control, the model is memorizing spatial fields—switch to retrieval-augmented generation for that variable.

6. Probe-Based Failure Localization: After training, run ridge probes on intermediate representations (encoder, resampler, language input) to identify exactly where information is lost. If the encoder retains the signal but the language pathway drops it, insert a decoupled head or adjust the bridge (e.g., increase cross-attention capacity).

---

What the Improved AI System Can Do:

• Generate geologically grounded prose for any planetary surface tile, where every stated fact (albedo, relief, roughness, spectral ratio) is verifiably derived from the input raster, with a tile-swap flip rate above 0.5 and vision-removal flip rate above 0.6.

• Correctly name mapped geologic units on >85% of unseen tiles, while remaining silent on out-of-domain terrain (false-positive rate <2%).

• Estimate numeric ages with decoupled heads that match retrieval-oracle accuracy (MAE <0.1 Ga when the oracle is correct), and explicitly cite the source record—never inventing a numeric value from memory.

• Distinguish between site evidence and literature knowledge: it will read topography/spectra for physical properties, but query an external database for chronologies, ensuring that all numeric claims are traceable to a citable source.

• Fail gracefully: when retrieval is imperfect, it will match the constant prior baseline (MAE 0.42 Ga) rather than hallucinate a confident wrong answer, and it will flag uncertainty when the retrieved record conflicts with polygon ground truth.

• Transfer to other domains (e.g., medical imaging, remote sensing, materials science) where the same principle applies: perceptual facts come from the input, numeric facts come from retrieval, and the language model only orchestrates style and reasoning.