Geometry-Aware Transformer for Five-Dimensional Seismic Noise Attenuation and First-Arrival Picking

arXiv:2608.25668 · physics.geo-ph · Submitted 2026-08-26 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Geometry-Aware Transformer for Five-Dimensional Seismic Noise Attenuation and First-Arrival Picking".

Mira: Here's the gist of the paper. GeoFormer proposes a Geometry-Aware Transformer architecture specifically designed for prestack seismic data because it explicitly incorporates acquisition geometry into the network architecture to improve first-arrival picking accuracy.

Kai: Core Architecture GeoFormer is designed to handle the 5D structure of prestack seismic data by tokenizing each trace with its waveform and four source–receiver coordinates.

Mira: Each token represents a single seismic trace, combining the waveform with its source–receiver coordinates, which constitute the 5D trace representation.

Kai: The model processes these inputs through dedicated pathways that project the waveform to a per-trace token embedding and process source–receiver coordinates via geometry-aware positional encoding.

Paper summary: Kai: The paper introduces GeoFormer as this Geometry-Aware Transformer architecture tailored for prestack seismic data specifically to improve first-arrival picking accuracy.

Mira: They argue that standard Vision Transformers struggle here because they treat the data like a 2D image, completely ignoring the acquisition geometry inherent in seismic surveys <ref:2608.25668#pg1>.

Lev: So they are saying that treating every trace as a token with its waveform and its coordinates is the right starting point for this specific type of data.

Kai: Right, and to make sure the model actually uses that geometry effectively, they implement three different ways to inject it into the model's layers.

Mira: First, at the token level, there's something called GeomMLP which replaces a standard classification token with a geometric representation based on offset and elevation.

Lev: So that means when the model looks at a trace, it gets this extra spatial information right from the start of its processing path.

Kai: Then they have normalization level injection, where they use something called GeomAdaLN to modulate the features based on that geometry.

Mira: And finally, at the attention level, there's GeomAttnBias which injects a parameter-free physical prior suggesting that traces close to each other in space should pay more attention to each other.

Lev: That sounds like it’s trying to mimic how physical proximity affects signal propagation, which is crucial for noise attenuation.

Kai: The results they show are pretty strong across four different field datasets when comparing GeoFormer against models like Vision Transformer and two task-specific baselines.

Mira: They specifically report that GeoFormer achieves the lowest root mean square error on all those datasets, reducing it by nearly an order of magnitude compared to Vision Transformer on Brunswick, Halfmile, and Dongbei.

Lev: That massive reduction in error is what we're looking for when you think about running this on real hardware; a lower error means less noise and better fidelity.

Kai: And they also produced a cleaner first-arrival mask that stays continuous across the three dimensional acquisition space when compared to other methods like Attention UNet and HU-Net on the Dongbei dataset.

Mira: That continuity is important because it means the picking isn't just right for one specific spot, but makes sense across the whole survey area.

Lev: So what's the catch or what are they saying this method can't do? I need to know where it stops working.

Conclusion: Kai: So, to wrap up the "Geometry-Aware Transformer for Five-Dimensional Seismic Noise Attenuation and First-Arrival Picking," the core contribution is embedding acquisition geometry directly into the network architecture instead of treating seismic data just like a standard 2D image <ref:2608.25668#pg1>.

Mira: It shows that explicitly incorporating physical constraints like offset and relative elevation into the tokenization and processing layers actually helps when you're trying to pick first arrivals in prestack seismic data.

Lev: What does this mean practically for someone trying to build an error correction tool? It suggests that if you ignore the geometry, you lose a lot of accuracy, as shown by their ablation studies where removing all three geometry modules increased the error from one point eight four on Halfmile to one point zero three zero.

Kai: That loss is pretty significant because it validates that these geometric context injections are necessary for performance and not just cosmetic additions to the model.

Mira: And they've proven that this approach works better than models like Vision Transformer because those visual models don't encode the physical constraints of first-arrival moveout or the relationships between traces.

Lev: So, for a listener who only cares about the results, what’s the simple version? It means using geometry awareness gives you a much cleaner picking result and significantly reduces your error on these specific seismic tasks compared to standard Transformer setups.

Kai: Exactly. They've shown that by treating each trace with its full physical context—the waveform plus where it came from—you get a model that performs better than what you’d expect from a general vision model.

Harbin Institute of Technology · Peking University

physics.geo-ph

Submitted: 2026-08-26

Updated: 2026-11-10

Comments: Added seismic noise-attenuation experiments and cross-site transfer experiments with geometry-only fine-tuning. Removed the SAM comparison.Corrected the Dongbei picking error metrics and updated the corresponding comparisons and conclusions

Code: https://github.com/ZDDWLIG/GeoFormer

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 99/100

The gist: GeoFormer proposes a Geometry-Aware Transformer architecture specifically designed for prestack seismic data because it explicitly incorporates acquisition geometry into the network architecture to…

Key concepts

5D Trace Representation
Each seismic trace is treated as a 5D token combining its raw waveform data with four source–receiver coordinates. This structure allows the model to process the entire physical context of a single trace simultaneously, moving beyond simple 2D image processing.
Geometry Injection Mechanisms
The architecture uses three distinct methods—GeomMLP, GeomAdaLN, and GeomAttnBias—to inject geometric information (offset and elevation) into the model. These mechanisms adapt the network's internal representations to account for how physical location affects signal characteristics.
First-Arrival Picking
This is the task of accurately locating the precise moment a seismic wave first arrives at a receiver. GeoFormer uses its geometry-aware features to better distinguish true first arrivals from noise, leading to more accurate picking and cleaner results across various datasets.

Terminology

Summary

GeoFormer proposes a Geometry-Aware Transformer architecture specifically designed for prestack seismic data because it explicitly incorporates acquisition geometry into the network architecture to improve first-arrival picking accuracy. p. 1

Core Architecture

GeoFormer is designed to handle the 5D structure of prestack seismic data by tokenizing each trace with its waveform and four source–receiver coordinates. Each token represents a single seismic trace, combining the waveform with its source–receiver coordinates, which constitute the 5D trace representation. p. 7 The model processes these inputs through dedicated pathways that project the waveform to a per-trace token embedding and process source–receiver coordinates via geometry-aware positional encoding. p. 6 This results in a geometry-aware per-trace representation where each token carries both waveform and physical acquisition context. p. 6

Geometry Injection Mechanisms

The architecture introduces three complementary geometry injection mechanisms to exploit the geometric attributes of offset and relative elevation. These mechanisms operate at different levels of the Transformer pipeline, corresponding to distinct physical questions about a trace's representation.

  1. Token level (GeomMLP) replaces the classification token with a per-trace geometric representation derived from offset and elevation, providing a spatial prior when waveform is degraded by noise. p. 1

  2. Normalization level (GeomAdaLN) replaces uniform layer normalization with geometry-conditioned feature modulation, enabling the feature distribution of each trace to adapt to its spatial position. p. 1

  3. Attention level (GeomAttnBias) injects a parameter-free physical prior that geometrically proximate traces should attend more to each other, allowing neighboring traces with similar propagation characteristics to reinforce the prediction when one trace is corrupted by noise. p. 4

Performance Validation

GeoFormer was validated on first-arrival picking across four field datasets, demonstrating superior performance compared to Vision Transformer and two task-specific baselines. Experiments on these datasets show that GeoFormer outperforms Vision Transformer and two task-specific baselines. p. 1 Specifically, GeoFormer achieves the lowest RMSE on all datasets, reducing it by nearly an order of magnitude compared to Vision Transformer on Brunswick, Halfmile, and Dongbei. p. 5 Furthermore, GeoFormer produces a cleaner first-arrival mask that remains continuous across the 3D acquisition space when compared to other methods like Attention UNet and HU-Net on the Dongbei dataset.

Contribution Quantification

Ablation studies confirm that GeomMLP, GeomAdaLN, and GeomAttnBias each contribute to GeoFormer’s picking accuracy. Removing all three geometry modules effectively reverts GeoFormer to a standard trace-level Transformer without geometry awareness, causing the RMSE to increase from 1. p. 184 to 10.30 on the Halfmile dataset. The progressive degradation validates the contribution of each design choice, showing that GeomMLP removal produces the largest degradation. p. 5

Broader Applicability

The design principle of GeoFormer, embedding physically meaningful acquisition geometry directly into the Transformer architecture, is applicable to other prestack seismic processing tasks where acquisition geometry constrains the expected output. Examples include velocity analysis, ground-roll suppression, and interpolation where per-trace geometric attributes carry physical information that is rarely incorporated explicitly into existing deep learning models. The comparison with Segment Anything Model (SAM) suggests that general-purpose vision models struggle with seismic first-arrival picking because they do not encode the physical constraints of first-arrival moveout or the geometric relationships between traces. p. 2 GeoFormer's geometry-aware mechanisms help preserve the continuity of the first-arrival curve even under strong noise, a limitation observed in SAM. The model also directly outputs per-trace first-arrival probabilities without manual guidance, unlike SAM which requires user-provided prompts. p. 3

Conclusion

GeoFormer successfully embeds acquisition geometry directly into the network architecture to address the limitations of treating seismic data as 2D images. It outperforms a standard Vision Transformer achieving a 7–9× reduction in RMSE and surpasses task-specific methods including Attention UNet and HU-Net. p. 1 The effectiveness of this geometry-aware design is confirmed by ablation studies, showing that removing all three mechanisms significantly increases the error, validating the necessity of incorporating physical context into the Transformer architecture. p. 5 This approach provides a robust solution for first-arrival picking and suggests a direction for combining large-scale pre-training with geophysical constraints. p. 2

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Replace standard Vision Transformer tokenization with GeoFormer's trace-level tokenization that combines its waveform with its source-receiver coordinates to preserve 5D structure rather than ignoring acquisition geometry. This allows the system to explicitly encode offset and relative elevation, which are critical for first-arrival traveltime prediction.

  2. Implement GeomMLP at the token level to replace the classification token with a per-trace geometric representation derived from offset and elevation. This provides every trace a physically meaningful representation before attention begins, constraining predictions even when waveforms are degraded by noise.

  3. Introduce GeomAdaLN at the normalization level, replacing uniform layer normalization with geometry-conditioned feature modulation that predicts channelwise scale and shift parameters from per-trace geometric attributes. This allows the model to dynamically adapt feature distributions based on a trace's spatial position, accounting for systematic variations in signal-to-noise ratio across offsets.

  4. Integrate GeomAttnBias at the attention level to inject a parameter-free physical prior that geometrically proximate traces should attend more to each other. This mechanism ensures that geometrically close neighbors can reinforce its prediction through stronger attention weights when a trace is corrupted by noise, leading to more robust picking than waveform-only methods.

  5. The improved system can maintain robust picking accuracy under strong noise because the trace-level tokens encode geometric attributes that provide a physical prior independent of waveform quality, which is superior to existing methods where geometry is treated as a post-processing constraint.

Related papers