GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems

arXiv:2607.01128 · cs.LG, cs.NA, math.NA · Submitted 2026-07-01 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems".

Jane: Here's the gist of the paper. Operator learning for partial differential equations (PDEs) on arbitrary geometries builds fast neural surrogates for large-scale simulation The Geometry-Adaptive Integral Autoencoder (GAIA), an operator learning model that encodes the domain boundary and the interior field distribution into geometry tokens, and conditions integral transform layers on these tokens via cross-attention, allowing the kernel to adapt locally to geometric features.

Tom: Model Architecture and Unification GAIA introduces a unified model for both forward and inverse problems on arbitrary domains.

Jane: It builds on IAE-Net

10: and introduces a dual-pathway tokenization that encodes both the domain boundary and the interior field distribution.

Tom: The architecture uses two complementary token pathways, specifically boundary feature tokens and state slice tokens, which are concatenated into a unified T.

Paper summary: Tom: So, we're talking about this paper now called "GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems". It sounds like they've built something to make these neural surrogates work for things that aren't just simple forward problems where the input and output live on the exact same grid.

Jane: Right, so the main idea here seems to be tackling boundary value problems and inverse problems, where the input and output can be on different shapes or grids. The paper claims they built a model that encodes both the domain shape—the boundary—and how the field is distributed inside it into geometry tokens, and then uses those tokens to condition integral transform layers with cross-attention so the kernel can adjust locally to geometric features.

Lu: I think what's really interesting is that it tries to unify a single architecture for both forward problems and inverse problems on arbitrary domains in just one pass without needing any retraining or iterative optimization, which is a big deal because most existing geometry-adaptive operators are mostly for forward modeling six <ref:2607.01128#pg2>.

Meng: But unification doesn't always mean it works well at scale. I wonder what the actual cost is to run this thing, especially if we need to deploy it on real traffic where the geometries are really messy.

Lalam: From my view, its success hinges on how those geometry tokens are built; they use two pathways—boundary feature tokens and state slice tokens—which then feed into a unified token set that gets refined by transformer layers.

Jane: That makes sense, so they aren't just looking at the shape; they're also encoding the actual field distribution inside, which is what those slice tokens handle. Tom, what do you see as the biggest immediate win in this paper?

Tom: From my side, I'm focused on the mechanics of how it learns. They use a dual-pathway tokenization to capture both the boundary and the interior state, and they condition integral transform layers using cross-attention with those geometry tokens <ref:2607.01128#pg3>. That cross-attention mechanism seems key because it allows the kernel itself to adapt based on what it sees in the geometry tokens, which is a smart way to handle varying shapes.

Lu: The idea of encoding boundary features using a PointNet style encoder from boundary coordinates alone, and then combining that with soft clustering aggregation for the slice tokenization, seems like they're covering the geometric input very thoroughly <ref:2607.01128#pg3>. It’s not just one way to see the shape; it’s two complementary views.

Paper summary: Meng: That sounds complex to implement robustly at scale, Lu. What about performance when we move beyond textbook examples and look at real-world data where the noise is high? Does this architecture hold up?

Lalam: The paper mentions that GAIA sets new state-of-the-art results on every inverse and BVP task they tested <ref:2607.01128#pg2>. Specifically, on an electroinductively coupled tomography benchmark, GAIA reduces the median relative L two error by twenty-seven percent compared to the next best method.

Jane: Twenty-seven percent reduction on that benchmark is significant because it shows tangible improvement over current methods for those hard inverse problems. Tom, can you elaborate on how this compares to what we know about other geometry-adaptive approaches like GAOT or GINO?

Tom: Well, the paper notes that while other methods exist, they often require iterative optimization loops when applied to inverse problems with different point sets <ref:2607.01128#pg2>. GAIA avoids that by being a single pass architecture. The authors also say it's competitive with specialized solvers on some forward problems while keeping stable accuracy where transformer baselines tend to degrade as point resolution gets finer.

Lu: I'm interested in the robustness part. They show graceful degradation under test-time measurement noise across all three inverse benchmarks <ref:2607.01128#pg3>. This is consistent with what we expect from an integral-transform architecture because the noise gets averaged out by the integration kernel instead of being amplified by point-wise operations <ref:2607.01128#pg3>.

Meng: Graceful degradation is good for production, but let's talk about efficiency then. How does this model actually run in terms of latency and memory? If we are deploying this, we need to know if it’s going to choke our hardware when the point cloud size gets large.

Lalam: The paper states that in terms of efficiency, GAIA achieves competitive latency and memory while maintaining the highest throughput among geometry-adaptive methods <ref:2607.01128#pg3>. The ablation studies also show that the cross-attention conditioning shows the best performance among all the different conditioning methods they tested.

Jane: So, it seems like they’ve managed to get high accuracy, good robustness against noise, and decent efficiency all at once, which is what we really want for a general tool. But what are the limitations they flag? Where does this model stop working right now?

Paper summary: Tom: They mention three main limitations. First, it's designed specifically for time-independent operators. Second, if you use it for forward problems where the input and output share a fixed dense discretization, they trade a little bit of accuracy in the training data for much better invariance to the actual geometry <ref:2607.01128#pg2>.

Lu: And finally, they point out that because the geometry-conditioning cross-attention scales with both the point cloud size and the number of tokens, it suggests that for really massive three dimensional point clouds, you might need latent compression or sparse attention techniques to make it practical <ref:2607.01128#pg3>.

Meng: So we're looking at a model that works well for inverse problems on arbitrary shapes, is somewhat robust to noise, and is efficient enough for production deployment right now. But the authors are telling us it’s not a perfect solution yet, especially when dealing with massive datasets or moving into time-dependent scenarios.

Jane: Exactly. The title itself says "Geometry-Adaptive Operator Learning," which really captures the essence of what they did—adapting the math to fit the shape of whatever you're studying. This paper shows that we can build models that don't need a completely new setup every time the geometry changes.

Tom: So, GAIA is positioned as a flexible tool for researchers who are dealing with complex geometries in both forward and inverse settings, moving past the limitations of models tied to fixed grids or specific domains <ref:2607.01128#pg2>.

Lu: It's about building fast neural surrogates that can handle arbitrary geometries without needing retraining, which is a big step for generalizing these operator learning techniques beyond simple forward modeling six <ref:2607.01128#pg2>.

Meng: For us in production, it means we have a better baseline architecture to start with when we face geometry variations in our simulations or reconstructions. It gives us something solid to build upon instead of starting from scratch every time the shape changes.

Lalam: From my perspective as an LLM trained on this literature, I'm quite sure that the core mechanism—encoding boundary and interior distribution into geometry tokens and using cross-attention to adapt integral transforms—is sound for achieving the claimed performance gains <ref:2607.01128#pg3>.

Jane: So, if you're a researcher looking at this work, GAIA suggests that the future of applying AI to PDEs on arbitrary shapes lies in creating models that inherently understand and adapt to the local geometry during the computation itself.

Tom: Right, so we have an architecture that handles boundary and interior information separately but combines them smartly through those tokens for a flexible integral transform layer. That's what they've delivered here <ref:2607.01128#pg3>.

Conclusion: Tom: So, GAIA is basically a new way to build models that can handle problems where the shape of what you're measuring changes from one thing to another and you have to learn how to solve those problems without retraining everything every time.

Jane: It sounds like they built this operator learning model that encodes both the shape of the domain and where the field is inside it into specific geometry tokens, which lets the model adapt its calculations locally.

Lu: The core idea here is unifying a single architecture for both forward problems and inverse problems on any shape without needing a whole new setup for each one.

Meng: So they're saying this one model can handle both figuring out what happens inside a fixed shape and figuring out what caused the measurements given an arbitrary shape.

Lalam: They use two types of tokens, boundary feature tokens that figure out the shape from just the edge coordinates and slice tokens that capture how the field is distributed across space.

Tom: And they combine those into a single set of geometry tokens, which then condition integral transform layers using cross-attention so the kernel itself changes based on what it sees in those shapes.

Jane: That means the model isn't just doing one thing; it's using that shape information to dynamically adjust how it solves the math for whatever problem you throw at it.

Lu: The results on benchmarks are pretty strong, showing reductions in error by twenty-seven percent on one specific inverse task and sixty-four percent on another when compared to the next best methods.

Meng: I see that accuracy boost, but what about the practical side? How much memory or processing power does this model need to run when you're dealing with really complex shapes in a real application?

Lalam: They say it maintains competitive latency and memory while having the highest throughput among geometry-adaptive methods.

Tom: So, they managed to get high accuracy and efficiency at the same time on these specific tasks. But the authors point out a few things they didn't cover.

Jane: What are those limitations? Where does this model stop working right now?

Lu: First, it’s designed for time-independent operators, meaning it works best when the physics doesn't change over time.

Tom: Second, if you use it for forward problems where the input and output already share a fixed grid, they say you trade a little bit of accuracy in your training data for much better invariance to the actual geometry.

Jane: And third, they note that because the geometry conditioning uses cross-attention that scales with both the point cloud size and how many tokens you have, it could get really slow on massive three dimensional point clouds.

Lalam: So, it’s a solid architecture for now, but we need to think about making those geometry tokens smaller or using sparse attention if we want to use it on huge three dee data.

Tom: It sounds like GAIA is positioning itself as a flexible tool for researchers dealing with complex geometries in both forward and inverse settings, moving past models tied to fixed grids.

Jane: Exactly. The real story here is that the future of applying AI to PDEs on arbitrary shapes might be about building models that inherently understand and adapt to the local geometry while they are doing the math.

University of Maryland, College Park · University of Delaware, Newark

cs.LG, cs.NA, math.NA

Submitted: 2026-07-01

Updated: 2026-11-10

Code: https://github.com/LouisSerrano/coral

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: Operator learning for partial differential equations (PDEs) on arbitrary geometries builds fast neural surrogates for large-scale simulation The Geometry-Adaptive Integral Autoencoder (GAIA)…

Key concepts

Geometry Tokens
These are special representations that capture two crucial pieces of information: the shape of the domain boundary and how the field is distributed inside it. They are created using two different encoding methods—one for boundaries and one for internal field slices—to give the model a complete geometric understanding.
Cross-Attention Conditioning
This mechanism connects geometry information to integral transform layers. The model uses cross-attention to allow the integral transforms to change or adapt based on the specific local geometric features encoded in the tokens, enabling them to be tailored precisely where needed within the domain.
Unified Model Architecture
GAIA is a single framework capable of solving both forward and inverse problems on any domain. It achieves this by using two complementary token pathways that encode boundary features and state slices, which are combined into a unified representation for processing.

Terminology

Summary

Operator learning for partial differential equations (PDEs) on arbitrary geometries builds fast neural surrogates for large-scale simulation The Geometry-Adaptive Integral Autoencoder (GAIA), an operator learning model that encodes the domain boundary and the interior field distribution into geometry tokens, and conditions integral transform layers on these tokens via cross-attention, allowing the kernel to adapt locally to geometric features. p. 0

Where the paper says it

We introduce the Geometry-Adaptive Integral Autoencoder (GAIA), an operator learning model that encodes the domain boundary and the interior field distribution into geometry tokens, and conditions integral transform layers on these tokens via cross-attention, allowing the kernel to adapt locally to geometric features. This yields a single architecture for forward (including BVPs) and inverse problems on arbitrary domains in one pass, without retraining, iterative optimization, or graph construction.

Model Architecture and Unification

GAIA introduces a unified model for both forward and inverse problems on arbitrary domains. It builds on IAE-Net [10] and introduces a dual-pathway tokenization that encodes both the domain boundary and the interior field distribution. p. 1

Where the paper says it

We introduce the Geometry-Adaptive Integral Autoencoder (GAIA), a unified model for both forward and inverse problems on arbitrary domains. Building on IAE-Net [10], we introduce a dual-pathway tokenization that encodes both the domain boundary and the interior field distribution, and condition the integral kernels on these tokens via multi-head cross-attention.

The architecture uses two complementary token pathways, specifically boundary feature tokens and state slice tokens, which are concatenated into a unified T. p. 3
Where the paper says it

Decoupling query points from integration points lets the encoder integrate over Xbnd while the decoder projects onto Xdom, enabling single-pass solve of inverse problems and BVPs. 1 Geometry Tokenization The geometry conditioning is constructed from two complementary token pathways: boundary feature tokens, which encode the domain shape from boundary coordinates alone, and state slice tokens, which encode the joint distribution of spatial coordinates and field values in the domain.

These tokens are then refined by standard transformer encoder layers between Adaptive IAE blocks. p. 4
Where the paper says it

A linear projection brings Tslice to the standard token dimension dt. The initial token set T (0) = [Tbnd, Tslice] is normalized and passed through GELU; between Adaptive IAE blocks, tokens are refined by standard transformer encoder layers, so that each block sees an updated token set T (l) .

Geometry Conditioning Mechanism

The geometry conditioning is constructed from two complementary token pathways, which include the Boundary feature tokenization and the Slice tokenization. The boundary feature tokens encode the domain shape from boundary coordinates alone via a PointNet-style encoder. p. 3

Where the paper says it

Decoupling query points from integration points lets the encoder integrate over Xbnd while the decoder projects onto Xdom, enabling single-pass solve of inverse problems and BVPs. 1 Geometry Tokenization The geometry conditioning is constructed from two complementary token pathways: boundary feature tokens, which encode the domain shape from boundary coordinates alone, and state slice tokens, which encode the joint distribution of spatial coordinates and field values in the domain.

The slice tokenizer encodes the joint distribution of spatial coordinates and field values in the domain through soft-clustering aggregation. p. 4
Where the paper says it

Top – Boundary tokenizer: Boundary point coordinates are processed by a PointNet-style encoder with shared MLPs and global max-pooling permutation-invariant tokens that summarize the global domain shape. Bottom – Slice tokenizer: The interior mesh coordinates and their associated field values are jointly embedded and assigned to soft clusters; mass-normalized aggregation produces tokens that summarize the spatially varying domain physics.

The geometry-conditioned integral transforms are defined by cross-attention between the encoded coordinates and the geometry tokens. p. 5
Where the paper says it

3 Adaptive IAE Block GAIA uses these geometry-conditioned integral transforms as building blocks. Each Adaptive IAE block follows the encode–process–decode structure (equation 2) of IAE-Net, but with kernels conditioned on geometry tokens via cross-attention (Equation 7).

Performance on Benchmarks

GAIA sets new state-of-the-art results on every inverse and BVP task. On EIT, GAIA reduces median relative L 2 error by 27% compared to the next best amortized method. p. 7

Where the paper says it

1 Inverse Problems Table 1 reports median relative L 2 error on inverse and BVP benchmarks; GAIA achieves the lowest error on every task. Against the next best amortized method, GAIA reduces error by 27% on EIT (0.

For Airfoil reconstruction, GAIA reduces the median relative L 2 error by 64% on airfoil relative to the next best method. p. 0
Where the paper says it

We evaluate GAIA on seven 2D and 3D benchmarks, four of which are new or substantially extended benchmarks for inverse problems and BVP: electrical impedance tomography, optical tomography, 3D Darcy flow on varying geometries, and a modified setting of Poisson BVP on mechanical components benchmark (MCB). GAIA sets new state-of-the-art results on every inverse and BVP task, reducing median relative L 2 error by 64% on airfoil flow reconstruction and 27% on EIT relative to the next best amortized method, and outperforming all baselines on every shape category of MCB.

GAIA is competitive with specialized solvers on other forward problems while maintaining stable accuracy across point resolutions where transformer-based baselines degrade. p. 0
Where the paper says it

GAIA sets new state-of-the-art results on every inverse and BVP task, reducing median relative L 2 error by 64% on airfoil flow reconstruction and 27% on EIT relative to the next best amortized method, and outperforming all baselines on every shape category of MCB. On other forward problems, GAIA is competitive with specialized solvers while maintaining stable accuracy across point resolutions on which transformer-based baselines degrade.

Robustness and Efficiency

GAIA demonstrates graceful degradation under test-time measurement noise across all three inverse benchmarks. The model's robustness to noise is consistent with what would be expected from an integral-transform architecture, where noise in the input is averaged across the integration kernel rather than being amplified by point-wise operations. In terms of efficiency, GAIA achieves competitive latency and memory while maintaining the highest throughput among geometry-adaptive methods. p. 9

Where the paper says it

9 Table 4: Computational cost comparison on EIT. GAIA has competitive latency and memory while maintaining the highest throughput among geometry-adaptive methods achieves the highest throughput among geometry-adaptive methods (365 samples/s) at competitive latency (16 ms) and modest peak memory (48 MB).

The ablation studies confirm that cross-attention conditioning shows the best performance among conditioning methods. p. 9
Where the paper says it

6 Ablations Table 5: Ablations. Top: Geometry conditioning: Cross-attention shows the best performance among conditioning methods.

Limitations

Three limitations bound the present work, including GAIA being designed for time-independent operators. For forward problems where input and output share a fixed dense discretization, GAIA trades a small amount of in-distribution accuracy for substantially improved discretization invariance. p. 9

Where the paper says it

First, GAIA is designed for time-independent operators; extending the framework to time-dependent problems requires additional architectural treatment for temporal evolution and is left to future work. Second, on forward problems where input and output share a fixed dense discretization, GAIA trades a small amount of in-distribution accuracy for substantially improved 10

Finally, the geometry-conditioning cross-attention scales as O(NK) in point-cloud size N and token count K, suggesting that latent compression or sparse attention may be needed for very large 3D point clouds. p. 2
Where the paper says it

GAIA differs from all three: unlike CORAL it does not require per-sample neural-field fitting; unlike GINO it constructs no graph; and unlike LNO, which uses attention to compress spatial information into a fixed set of latent states, GAIA conditions the integral kernel itself on geometry at each query point via cross-attention. Our slice tokenizer adapts the soft-clustering aggregation of Transolver [9]; the boundary tokenizer and the use of both token pathways to condition integral kernels via cross-attention are novel, and the results demonstrate the architecture’s efficacy.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Unified Forward/Inverse Geometry-Adaptive Model: The GAIA model can solve both forward problems (including Boundary Value Problems) and inverse problems on arbitrary domains in a single pass without retraining, iterative optimization, or graph construction.

  2. Spatially Adaptive Kernel Conditioning: GAIA conditions the integral kernels on domain geometry via cross-attention, allowing the kernel to adapt locally to geometric features. This enables efficient single-pass solution of inverse problems without iterative optimization by providing explicit, spatially adaptive geometric context at each integration point.

  3. Robustness to Resolution Mismatch: The system maintains stable accuracy across resolutions where transformer-based baselines degrade. Furthermore, the GAIA model shows a strong discretization invariance on 3D Darcy problems, degrading more gradually under subsampling than Transolver does.

  4. Noise Robustness in Inverse Problems: GAIA demonstrates graceful degradation under test-time measurement noise by averaging noise across the integration kernel rather than amplifying it via point-wise operations. This robustness is stable, with seed-to-seed variation remaining under 3% of the mean error across tested noise levels.

  5. Enhanced Feature Capture via Dual Tokenization: The use of two complementary token pathways, boundary feature tokens and slice tokens, allows the model to encode both the domain boundary and the interior field distribution, ensuring that both global domain shape and spatially varying input physics are captured.

Abstract

Operator learning for partial differential equations (PDEs) on arbitrary geometries builds fast neural surrogates for large-scale simulation. While recent geometry-adaptive neural operators have made substantial progress, they are mainly designed for forward problems in which inputs and outputs share the same spatial domain. This limits their applicability for boundary value problems (BVPs) and inverse problems, where inputs and outputs may live on different domains. We introduce the Geometry-Adaptive Integral Autoencoder (GAIA), an operator learning model that encodes the domain boundary and the interior field distribution into geometry tokens, and conditions integral transform layers on these tokens via cross-attention, allowing the kernel to adapt locally to geometric features. This yields an architecture capable of learning solution operators for forward and inverse problems on arbitrary domains, solving each in a single pass at inference without per-domain optimization, iterative refinement, or graph construction. We evaluate GAIA on seven 2D and 3D benchmarks, four of which are new or substantially extended benchmarks for inverse problems and BVP: electrical impedance tomography, optical tomography, 3D Darcy flow on varying geometries, and a modified setting of the Poisson BVP on mechanical components benchmark (MCB). GAIA sets new state-of-the-art results on every inverse and BVP task, reducing median relative L squared error by 64% on airfoil flow reconstruction and 22% on EIT relative to the next best amortized method, and outperforming all baselines on every shape category of MCB. On other forward problems, GAIA is competitive with specialized solvers while maintaining stable accuracy across point resolutions on which transformer-based baselines degrade.

Sources

Related papers