Geometry-Aware Transformer for Five-Dimensional Seismic Noise Attenuation and First-Arrival Picking

summary

The gist

GeoFormer proposes a Geometry-Aware Transformer architecture specifically designed for prestack seismic data because it explicitly incorporates acquisition geometry into the network architecture to…

In short

GeoFormer is a Geometry-Aware Transformer designed for prestack seismic data to improve first-arrival picking accuracy. It incorporates acquisition geometry directly into the network by injecting spatial context at token, normalization, and attention levels. Validation shows GeoFormer significantly outperforms Vision Transformers and other baselines by achieving lower RMSE and producing cleaner first-arrival masks.

Key concepts

5D Trace Representation
Each seismic trace is treated as a 5D token combining its raw waveform data with four source–receiver coordinates. This structure allows the model to process the entire physical context of a single trace simultaneously, moving beyond simple 2D image processing.
Geometry Injection Mechanisms
The architecture uses three distinct methods—GeomMLP, GeomAdaLN, and GeomAttnBias—to inject geometric information (offset and elevation) into the model. These mechanisms adapt the network's internal representations to account for how physical location affects signal characteristics.
First-Arrival Picking
This is the task of accurately locating the precise moment a seismic wave first arrives at a receiver. GeoFormer uses its geometry-aware features to better distinguish true first arrivals from noise, leading to more accurate picking and cleaner results across various datasets.

Terminology used across episodes

This episode discusses

The paper

Geometry-Aware Transformer for Five-Dimensional Seismic Noise Attenuation and First-Arrival Picking · Read on arXiv

Harbin Institute of Technology · Peking University

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Geometry-Aware Transformer for Five-Dimensional Seismic Noise Attenuation and First-Arrival Picking".

Mira: Here's the gist of the paper. GeoFormer proposes a Geometry-Aware Transformer architecture specifically designed for prestack seismic data because it explicitly incorporates acquisition geometry into the network architecture to improve first-arrival picking accuracy.

Kai: Core Architecture GeoFormer is designed to handle the 5D structure of prestack seismic data by tokenizing each trace with its waveform and four source–receiver coordinates.

Mira: Each token represents a single seismic trace, combining the waveform with its source–receiver coordinates, which constitute the 5D trace representation.

Kai: The model processes these inputs through dedicated pathways that project the waveform to a per-trace token embedding and process source–receiver coordinates via geometry-aware positional encoding.

Paper summary: Kai: The paper introduces GeoFormer as this Geometry-Aware Transformer architecture tailored for prestack seismic data specifically to improve first-arrival picking accuracy.

Mira: They argue that standard Vision Transformers struggle here because they treat the data like a 2D image, completely ignoring the acquisition geometry inherent in seismic surveys <ref:2608.25668#pg1>.

Lev: So they are saying that treating every trace as a token with its waveform and its coordinates is the right starting point for this specific type of data.

Kai: Right, and to make sure the model actually uses that geometry effectively, they implement three different ways to inject it into the model's layers.

Mira: First, at the token level, there's something called GeomMLP which replaces a standard classification token with a geometric representation based on offset and elevation.

Lev: So that means when the model looks at a trace, it gets this extra spatial information right from the start of its processing path.

Kai: Then they have normalization level injection, where they use something called GeomAdaLN to modulate the features based on that geometry.

Mira: And finally, at the attention level, there's GeomAttnBias which injects a parameter-free physical prior suggesting that traces close to each other in space should pay more attention to each other.

Lev: That sounds like it’s trying to mimic how physical proximity affects signal propagation, which is crucial for noise attenuation.

Kai: The results they show are pretty strong across four different field datasets when comparing GeoFormer against models like Vision Transformer and two task-specific baselines.

Mira: They specifically report that GeoFormer achieves the lowest root mean square error on all those datasets, reducing it by nearly an order of magnitude compared to Vision Transformer on Brunswick, Halfmile, and Dongbei.

Lev: That massive reduction in error is what we're looking for when you think about running this on real hardware; a lower error means less noise and better fidelity.

Kai: And they also produced a cleaner first-arrival mask that stays continuous across the three dimensional acquisition space when compared to other methods like Attention UNet and HU-Net on the Dongbei dataset.

Mira: That continuity is important because it means the picking isn't just right for one specific spot, but makes sense across the whole survey area.

Lev: So what's the catch or what are they saying this method can't do? I need to know where it stops working.

Conclusion: Kai: So, to wrap up the "Geometry-Aware Transformer for Five-Dimensional Seismic Noise Attenuation and First-Arrival Picking," the core contribution is embedding acquisition geometry directly into the network architecture instead of treating seismic data just like a standard 2D image <ref:2608.25668#pg1>.

Mira: It shows that explicitly incorporating physical constraints like offset and relative elevation into the tokenization and processing layers actually helps when you're trying to pick first arrivals in prestack seismic data.

Lev: What does this mean practically for someone trying to build an error correction tool? It suggests that if you ignore the geometry, you lose a lot of accuracy, as shown by their ablation studies where removing all three geometry modules increased the error from one point eight four on Halfmile to one point zero three zero.

Kai: That loss is pretty significant because it validates that these geometric context injections are necessary for performance and not just cosmetic additions to the model.

Mira: And they've proven that this approach works better than models like Vision Transformer because those visual models don't encode the physical constraints of first-arrival moveout or the relationships between traces.

Lev: So, for a listener who only cares about the results, what’s the simple version? It means using geometry awareness gives you a much cleaner picking result and significantly reduces your error on these specific seismic tasks compared to standard Transformer setups.

Kai: Exactly. They've shown that by treating each trace with its full physical context—the waveform plus where it came from—you get a model that performs better than what you’d expect from a general vision model.

More episodes

← Home