2.5D Transformer: An Efficient 3D Seismic Interpolation Method without Full 3D Training

summary

The gist

Transformer has emerged as a powerful deep-learning technique for two-dimensional (2D) seismic data interpolation…

In short

The T-2.5D network addresses 3D seismic data interpolation by using a Transformer architecture adapted via cross-dimensional transfer learning (TL). It combines 2D Transformer encoders with 3D dimension adapters to learn spatial correlation across seismic lines. This two-stage training process allows the model to achieve performance comparable to full 3D Transformers while requiring significantly less memory and training time.

Key concepts

Transformer
A deep learning technique known for its global modeling ability, which is powerful for understanding long-range dependencies in data. In this context, it is used as the core encoder to process 2D seismic data patches effectively.
Cross-Dimensional Transfer Learning (TL)
A two-stage training strategy where a model trained on one dimension (2D) is adapted to another (3D). The first stage pre-trains the 2D encoders, and the second stage fine-tunes the specific adapters for the 3D data, avoiding costly full 3D training.
Seismic Dimension Adapters (SDAs)
These are specialized modules placed before each Transformer encoder in T-2.5D. They are composed of linear layers and a depth-wise 3D Convolution to specifically learn the spatial correlation information that exists across seismic lines in the 3D data.

Terminology used across episodes

This episode discusses

The paper

2.5D Transformer: An Efficient 3D Seismic Interpolation Method without Full 3D Training · Read on arXiv

Changxin Wei, Xintong Dong, Xinyang Wang

State Key Laboratory of Deep Earth Exploration and Imaging, College of Instrumentation and Electrical Engineering, Jilin University · College of Electric Engineering, Department of Communication Engineering, Key Laboratory of Modern Power System Simulation and Control and Renewable Energy Technology (Ministry of Education)

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "2.5D Transformer: An Efficient 3D Seismic Interpolation Method without Full 3D Training".

Mira: Here's the gist of the paper. Transformer has emerged as a powerful deep-learning technique for two-dimensional (2D) seismic data interpolation, owing to its global modeling ability To achieve Transformer-based three-dimensional (3D) seismic interpolation, we propose a 2.5-dimensional Transformer network (T-2.5D) that adopts a cross-dimensional transfer learning (TL) strategy, so as to adapt the 2D Transformer encoders to 3D seismic data The proposed T-2.5D is mainly composed of 2D Transformer encoders and 3D seismic dimension adapters (SDAs) Each 3D SDA is placed before a Transformer encoder to learn spatial correlation information across seismic lines The proposed cross-dimensional TL strategy comprises two stages: 2D pre-training and 3D fine-tuning In the first stage, we optimize the 2D Transformer encoders using a large amount of 2D data patches In the second stage, we freeze the 2D Transformer encoders and fine-tune the 3D SDAs using limited 3D data volumes Experimental results demonstrate that the proposed method achieves comparable performance to that of full 3D Transformer at a significantly low cost The T-2.5D network is designed as a hybrid of 2D and 3D modules, incorporating four 2D Transformer encoders and four 3D seismic dimension adapters (SDAs) The T-2.5D is implemented by permuting the input tensor to be compatible with the input requirements of the 2D network components The 3D SDA is composed of two linear layers and a depth-wise 3D Conv (DWConv3d) This structure allows for learning spatial correlation information across seismic lines Cross-Dimensional Transfer Learning Strategy The cross-dimensional TL strategy involves two distinct stages to optimize the T-2.5D parameters, denoted as θ2.5D, which is composed of θ′2D and θ′3D The first stage focuses on optimizing the four 2D Transformer encoders using a large number of 2D patches to generate a 2D pre-trained model In the second stage, the trainable parameters of the four 3D SDAs are optimized using a small amount of 3D volumes, where spatial information across seismic lines is learned This workflow is detailed in Algorithm 1 Performance and Efficiency Gains Extensive experiments show that the T-2.5D achieves comparable interpolation performance to 3D Transformer at a significantly lower computational cost The total training time of T-2.5D is composed of the pre-training time of the first stage and the training time of the second stage T-2.5D requires only about 1/12 the memory of T-3D with a training time reduced to approximately 1/20 The inference time of T-2.5D is even less than that of T-2D This reduction is explained by the computational complexity of the self-attention mechanism being reduced to 1/I (where I=16 in our implementation) of that of T-3D Generalization and Robustness The generalization ability of T-2.5D is demonstrated by utilizing the Opunake dataset for further TL of T-2.5D and T-3D The comparison on the Stratton dataset with real missing data shows that the interpolated result in Fig.

Kai: 17b demonstrates a good continuity of the reconstructed events In summary, T-2.5D has great potential for effectively interpolating missing traces in 3D seismic data from different survey areas, exhibiting great generalization ability The proposed T-2.5D is qualified to replace time-consuming, memory-intensive full 3D training In this paper, we propose a Transformer-based network, named T-2.5D, to accomplish 3D seismic data interpolation tasks Specifically, a cross-dimensional TL training strategy is utilized to optimize T-2.5D This strategy consists of two stages The first stage is the 2D pre-training stage, which generates a pre-trained model by optimizing the Transformer encoders in T-2.5D using a large amount of 2D data patches The second stage is the 3D fine-tuning stage, in which the 3D SDAs are fine-tuned using a small number of 3D data volumes Extensive experiments demonstrate that the T-2.5D with this cross-dimensional strategy exhibits comparable or even better performance to T-3D, while requiring only small amounts of memory and training time In summary, the proposed T-2.

Title and authors: Kai: So we're looking at this paper today called "2 point 5D Transformer: An Efficient three dee Seismic Interpolation Method without Full three dee Training" <ref:2511.10033#pg1,2.5D Transformer: An Efficient 3D Seismic Interpolation Method without Full 3D>. It’s about taking something that works really well for two-dimensional seismic data and trying to make it work for three-dimensional seismic data by not doing the heavy lifting of full three-dimensional training.

Mira: That sounds like a smart approach, especially since the core Transformer method gets computationally expensive when you move to higher dimensions. We're hoping this T2 point 5D network can bridge that gap without needing massive amounts of data or training time to get it right <ref:2511.10033#pg1>.

Lev: From an engineering standpoint, the key thing here is adapting those two-dimensional Transformer encoders to handle the three-dimensional structure, which sounds complicated when you think about how spatial correlation works across different seismic lines.

Kai: Right, so what they propose is a two point five-dimensional Transformer network called T2 point 5D that uses a cross-dimensional transfer learning strategy to do exactly that adaptation for three-dimensional seismic data <ref:2511.10033#pg1,a 2.5-dimensional Transformer network>.

Mira: And the architecture relies on these four two-dimensional Transformer encoders paired with four three-dimensional seismic dimension adapters or SDAs, which are placed before each encoder to learn spatial correlation information across those lines.

Lev: I'm curious about how they've structured that adaptation; does it mean we can just plug in a new layer for each dimension, or is there a more integrated way they handle the dimensional shift?

Kai: They use this cross-dimensional transfer learning strategy which has two stages: first, you pre-train those four two-dimensional Transformer encoders using a large amount of two-dimensional data patches.

Mira: And then in the second stage, you freeze those original encoders and fine-tune only the four three-dimensional SDAs using a small amount of three-dimensional data volumes.

Lev: So, essentially they are doing a lot of heavy training on the 2D side first and then just tweaking the new parts for each dimension with limited data <ref:2511.10033#pg1>? That sounds like it might be feasible for real hardware constraints.

Kai: Exactly, and experimental results show that this method achieves comparable performance to a full three-dimensional Transformer at a significantly lower cost than that full model.

Mira: And they quantify that cost reduction quite clearly: the total training time for T2 point 5D is only about one twelfth the memory of a full three-dimensional Transformer, and the training time is reduced to approximately one twentieth <ref:2511.10033#pg1>.

Lev: If we look at inference, they say it's even faster than a two-dimensional Transformer, which they explain by saying the computational complexity of the self-attention mechanism is reduced to about one twelfth of what it is in a full three-dimensional Transformer.

Kai: That means for someone running this on real hardware, the speed increase in inference is pretty substantial when you're dealing with large datasets.

Mira: But we also need to consider how robust this model is when it actually has to interpolate messy, real-world data, not just clean training examples.

Lev: That brings us to generalization; they test it on the Opunake dataset for further transfer learning between T2 point 5D and a full Tthree dee network <ref:2511.10033#pg1>.

Kai: And they showed good continuity in their results when comparing the interpolated images from this method to real missing data shown on the Stratton dataset, which suggests it handles complex structural models well, like an overthrust model.

Mira: They also found that T2 point 5D performs better than both traditional methods like POCS and even a standard two-dimensional Transformer on datasets with regularly missing traces at high rates, specifically showing performance comparable to Tthree dee even under sparse acquisition scenarios <ref:2511.10033#pg1>.

Lev: That's important for us because it means we can use this method for interpolating data from different survey areas without needing a full three-dimensional training run every single time.

Kai: So, to wrap up on the overall implication, the T2 point 5D network with this cross-dimensional strategy has strong potential for effectively interpolating missing traces in three-dimensional seismic data from different survey areas while requiring much less memory and training time than a full three-dimensional training <ref:2511.10033#pg1>.

Mira: It really comes down to using that two-stage process of pre-training the encoders on 2D data and then fine-tuning the dimension adapters on limited three dee data to get this level of performance efficiently <ref:2511.10033#pg1>.

Lev: For me, what this means for us is that we can potentially replace those time-consuming, memory-intensive full three-dimensional training runs with something much more manageable while still getting high quality results.

Kai: So the T2 point 5D Transformer is designed to be a practical way to tackle three dee seismic interpolation by intelligently borrowing knowledge from 2D models through that transfer learning approach we just talked about <ref:2511.10033#pg1>.

Mira: It’s a solid piece of research for improving the efficiency of these types of deep learning methods when scaling up from two dimensions to three dimensions in geophysical imaging tasks.

Lev: We'll keep an eye on how they use this cross-dimensional TL strategy when applying it to even more complex or irregularly sampled three dee data structures in our own error correction systems <ref:2511.10033#pg1>.

The paper's summary: Kai: So, to recap, this paper proposes T2 point 5D Transformer for three dimensional seismic interpolation using a cross dimensional transfer learning strategy instead of needing full three dimensional training from scratch.

Mira: Right, it boils down to taking two four dimensional transformers and attaching these specialized dimension adapters that learn the spatial relationships between those different dimensions. It’s a way to borrow the 2D knowledge and adapt it intelligently for the three dee task without needing all that brute force training time <ref:2511.10033#pg1>.

Lev: From where I sit with building real hardware, what I find most interesting is how they split the learning process across two stages—pre-training on lots of two dimensional data patches and then fine tuning only those dimension adapters with a small amount of three dimensional volumes. It sounds like a very smart way to manage the computational overhead.

Kai: Exactly, because that splits the work so cleanly. The result they show is that this method gets interpolation performance comparable to a full three dimensional transformer but with much less training time and memory usage on the hardware side.

Mira: And look at these numbers—the total training time for T2 point 5D is about one twelfth the memory of a full three dimensional transformer, and the actual wall clock training time is reduced to about one twentieth of what you’d spend on the full model. That’s a significant reduction in resource consumption for a method that achieves similar accuracy.

Lev: For running this on real hardware, that reduction in training time is pretty substantial. And the inference speed, they say it's even faster than a two dimensional transformer because the self attention complexity drops to about one twelfth of what it is in the full three dimensional transformer. That means we can get fast results on large scale datasets.

Kai: And beyond just speed and memory, they show good generalization ability. They test this method using the Opunake dataset for further transfer learning between T2 point 5D and a full three dimensional network, which suggests it’s not just memorizing the training data but actually learning robust patterns across different scenarios.

Mira: Plus, they used real missing data from the Stratton dataset to show good continuity in their reconstructed events, even handling complex structural models like an overthrust model well. That means this technique has potential for interpolating traces from totally different survey areas with less retraining.

Lev: So it’s not just a nice theoretical improvement; it’s practically useful for geophysical data processing where datasets are often sparse and acquisition is expensive. It makes the full three dimensional training route much more accessible than it used to be.

Kai: The implication here is that we can use this T2 point 5D approach to tackle missing traces in three dimensional seismic data without needing the massive, time consuming, memory intensive full three dimensional training runs that were previously required.

Mira: It’s about intelligently borrowing knowledge from two dimensions and adapting it for three dimensions through that cross dimensional transfer learning strategy. It shows how structured network designs can bypass some of the traditional limitations in scaling up deep learning models to higher dimensions.

Lev: We should probably think about how this modular approach with separate dimension adapters could be applied to other complex physical systems where you need to model interactions across multiple spatial dimensions simultaneously. That's a direction we can explore next when we talk about efficiency in modeling those things.

The paper's improvements: Kai: So, to wrap up on the improvements, the authors suggest using this T2 point 5D network for generalization testing across different seismic environments using datasets like Opunake and real missing data from Stratton.

Mira: Right, they are showing that this cross dimensional transfer learning strategy actually works when you apply it to external survey areas or real world missing data. That’s a big deal because it means the method has some robustness outside of just the specific training patches they used initially.

Lev: What I see as an improvement for us is how they've framed this modular structure with those separate dimension adapters, which suggests we could use that same concept to model interactions across multiple spatial dimensions in other physical systems. It’s a more flexible way to handle complexity.

Kai: They also demonstrate that T2 point 5D performs better than traditional methods like POCS and even a standard two dimensional transformer on datasets where traces are missing at high rates, like sixty seven percent or eighty percent of the time. That means it maintains comparable interpolation capabilities to a full three dimensional model even when the data acquisition is very sparse.

Mira: And they also point out that the inference time drops significantly, down to about one ninth of what a full three dimensional transformer takes, which makes it capable of interpolating seismic data rapidly on large scale datasets. That speed reduction is something we need to focus on for real world applications.

Lev: The caveat I see is that they are still relying on the initial two dimensional pre-training stage, so if you start with totally uninformative 2D data, the whole transfer learning chain might suffer, though they show it works well in practice <ref:2511.10033#pg1>.

Kai: They also mention that this method is qualified to replace those time consuming memory intensive full three dimensional training runs because it achieves comparable or even better performance while requiring only small amounts of memory and training time.

Mira: It really shows how we can move away from the brute force approach of just dumping everything into a massive three dimensional model when we can intelligently structure the network using these dimension adapters. It's about efficiency in modeling higher dimensional physical data.

Lev: So, the main implication is that we get high quality three dee interpolation capability without needing to dedicate the massive computational resources and time that full three dimensional training demands. It makes high resolution seismic reconstruction more feasible for many researchers.

Conclusion: Kai: So we’re wrapping up on T2 point 5D Transformer for three dimensional seismic interpolation without full three dimensional training, which is a network that uses cross dimensional transfer learning to adapt two dimensional encoders to three dimensions.

Mira: Right, essentially it’s a method that cleverly borrows knowledge from two dimension models and adapts it for the three dee task using a pre-training and fine tuning workflow. It’s about making higher dimensional modeling more resource efficient than just training everything from scratch.

Lev: I think what this means is that we can tackle complex geophysical problems where you need three dimensional data without needing a massive amount of compute time or memory for the full three dimensional approach. The overhead lands much lower on the hardware side, which is huge for real world deployment.

Kai: Exactly, and they show it performs comparably to a full three dimensional transformer while requiring significantly less training time and memory usage. That’s a practical result for anyone working with large seismic datasets.

Mira: It highlights how structured network designs can bypass some of the traditional limitations in scaling up deep learning models to higher dimensions when you use transfer learning in this specific way. It’s about intelligent knowledge borrowing rather than brute force training across the entire high dimensional space.

Lev: I just want to say that the modularity of those dimension adapters is something we can look at for other systems, because it suggests a more flexible way to handle interactions across multiple spatial dimensions simultaneously on real hardware.

Kai: So in summary, this paper on T2 point 5D Transformer shows a practical way to get high quality three dimensional seismic interpolation by using cross dimensional transfer learning with manageable training costs.

Mira: It’s a solid piece of research for improving the efficiency of these types of deep learning methods when scaling up from two dimensions to three dimensions in geophysical imaging tasks.

Lev: We'll keep an eye on how they use this cross dimensional TL strategy when applying it to even more complex or irregularly sampled three dimensional data structures in our own error correction systems.

More episodes

← Home