Cross-Domain Pretraining for Steady-State Neural CFD Surrogates

arXiv:2610.10398 · cs.LG · Submitted 2026-10-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Cross-Domain Pretraining for Steady-State Neural CFD Surrogates".

Jane: The gist: Cross-domain pretraining improves zero- and few-shot performance of CFD surrogates on unseen datasets, yielding lower errors and requiring fewer samples for deployment.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We started by looking at the title and authors of "Cross-Domain Pretraining for Steady-State Neural CFD Surrogates" and what that actually means for the field.

Jane: It’s a pretty straightforward title, but it signals that the focus is on using existing data from different CFD scenarios to make these neural surrogates more robust when they encounter something new.

Lu: The authors are Shirley Ho, Rudy Morel, Amir Barati Farimani, and Anthony Zhou, and they're looking at how pooling data across different geometries and boundary conditions helps the models generalize better #pg1.

Meng: So basically, it’s about moving away from training a model just on one specific test case to making it smart enough for any new test case in that CFD space.

Tom: It means we're not relying solely on generating a brand-new simulation every single time we need a result, which is always slow and expensive #pg2.

Jane: That’s right. The implication is that the main bottleneck isn't just building the neural network itself, but having access to diverse data for all those engineering scenarios.

Lu: They specifically point out that this study presents the first study of cross-domain pretraining for three dee, mesh-based CFD surrogates #pg2 <ref:2610.10398#pg1>.

Meng: So they’re tackling a problem where we have these complex industrial flows on unstructured meshes, and they're showing how to use existing data to tackle that complexity.

Tom: It means we can build tools that are much more versatile across different types of fluid dynamics problems without needing entirely new datasets for every single application #pg1.

The paper's summary: Jane: Now, let's break down the actual summary of "Cross-Domain Pretraining for Steady-State Neural CFD Surrogates" and what the mechanics of this approach are.

Lu: The authors aggregate a diverse set of steady-state, meshbased CFD datasets and pretrain a set of models on that pool before examining how they perform on held-out datasets #pg2.

Meng: So the process is: gather data from various sources, train one joint model on all that data, and then test its performance on completely new problems #pg2.

Tom: The key finding is that this joint model pretrained across CFD domains outperforms single-domain experts in zero- and few-shot t#pg1.

Jane: That's the punchline: the pooled model beats models trained only on one domain when you have very little data to guide it, which is exactly what we want for deployment #pg2.

Meng: That makes sense. Since steady-state problems are pretty well-behaved given just the geometry and parameters, you don't need to feed it extra simulation examples during the query phase #pg2.

Tom: So they’re showing that for steady-state work, the benefit comes from the pretraining itself rather than some kind of context injection at inference time.

Jane: It really shows that in this specific setting, a solid pretraining strategy is more important than trying to trick the model with extra examples during testing #pg2.

Lu: The implication is that for steady-state work right now, focusing on generating that diverse pretraining dataset is the most effective lever we have #pg1.

The paper's improvements: Tom: Moving on to the actual improvements the paper suggests for this approach, we're talking about what they suggest we should change in our modeling pipeline.

Jane: They look at normalization strategies, finding that global normalization works pretty well across different domains, which is good because it keeps the physical meaning consistent even when the underlying mesh or conditions change #pg1.

Lu: Global normalization sounds practical for deployment; it means we don't have to worry as much about perfectly matching every single grid detail in our pretraining data to a new simulation setup #pg2.

Meng: Global normalization sounds practical for deployment, because it means we don't have to worry as much about perfectly matching every single grid detail in our pretraining data to a new simulation setup #pg2.

Tom: And they also touched on how discretization, like the mesh itself, doesn't really matter for the cross-domain benefit when you’re dealing with steady-state flow problems #pg3.

Jane: That’s interesting because if we can rely on global normalization, it suggests that the core physics captured by the model is more important than exactly how fine our computational grid is #pg1.

Lu: They also tested querying finetuned models directly on the simulation mesh to prove that this cross-domain pretraining still helps even when you query in a specific context #pg2.

Tom: So they’ve covered how cross-domain pretraining helps CFD surrogates generalize better on new data, and we're wrapping up with the conclusion of this paper by Zhou et al., "Cross-Domain Pretraining for Steady-State Neural CFD Surrogates."

Jane: Essentially, they’re saying that using existing simulation data across different scenarios makes those neural surrogates much more accurate when you have a new problem to solve.

Lu: They show that this method cuts down the required training samples by up to eight times compared to training from scratch #pg2.

Meng: I agree on the sample efficiency part, because if we can generate fewer simulation runs just to get a good surrogate, that drastically lowers our operational cost for deployment #pg2.

Tom: The study also confirms that this benefit isn't tied to any specific neural network architecture, which means it’s a really flexible strategy for different engineering teams.

Jane: It sounds like the main message is that we can build these powerful CFD tools by focusing on creating broad, diverse pretraining datasets first.

Conclusion: Tom: So we've covered how this paper on "Cross-Domain Pretraining for Steady-State Neural CFD Surrogates" helps these models generalize better on new data.

Jane: Essentially, they’re showing that using existing simulation data across different scenarios makes those neural surrogates much more accurate when you have a new problem to solve.

Jane: It’s definitely a direction worth following as we try to make these AI surrogates more broadly applicable across different engineering fields.

Anthony Zhou, Amir Barati Farimani, Shirley Ho, Rudy Morel

Carnegie Mellon University · New York University · Princeton University · Polymathic AI Institute of Technology Center for Computational Astrophysics Institute for Computational Mathematics Flatiron Institute Scientific Computing Core

cs.LG

Submitted: 2026-10-07

Updated: 2026-10-07

Comments: 43 pages, 25 figures

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 85/100

The gist: The gist: Cross-domain pretraining improves zero- and few-shot performance of CFD surrogates on unseen datasets, yielding lower errors and requiring fewer samples for deployment.

Key concepts

Cross-Domain Pretraining: This involves training a neural network on one set of CFD problems (the source domain) and then using that knowledge to improve performance on entirely new, unseen CFD problems (the target domain). The goal is to transfer learning capabilities so the model understands the underlying physics better without needing extensive training for every new scenario.
Zero-Shot/Few-Shot Performance: Zero-shot performance means a model performs well on a new task with absolutely no specific examples during testing. Few-shot performance means it performs well when given only a very small number of examples (a few samples) from the new task. The study shows pretraining significantly boosts these capabilities.
Sample Efficiency: This measures how quickly and with how little data a model can achieve a desired level of accuracy. The paper demonstrates that cross-domain pretraining drastically improves sample efficiency, meaning fewer CFD simulations are needed to create an accurate surrogate model for a new application.

Summary

The gist: Cross-domain pretraining improves zero- and few-shot performance of CFD surrogates on unseen datasets, yielding lower errors and requiring fewer samples for deployment.

Cross-Domain Pretraining Benefits

The primary finding is that cross-domain pretraining improves zero- and few-shot performance on held-out datasets relative to both training from scratch and transferring from domain-specific experts The authors find that finetuning a pretrained, crossdomain model can achieve 2-3x lower errors at the same sample size and use 8x fewer samples to achieve the same error, compared to training from scratch. This benefit is architecture agnostic and improves with model size and pretraining dataset diversity. Furthermore, they find that simply pooling steady-state datasets is both sufficient and effective for cross-domain pretraining in CFD surrogates.

Model Architectures and Performance

The study evaluates several architectures to verify that the results are model-agnostic, including SMART (Hagnberger & Niepert, 2026), AB-UPT (Alkin et al., 2025a), and Transolver++ (Luo et al., 2025). The joint model consistently outperforms any expert model during zero-shot and few-shot evaluation across the held-out datasets. For instance, in Table 8, the Joint Model shows lower errors than all single-domain experts when evaluated on the pretraining datasets. The results suggest that cross-domain pretraining is likely a good strategy for future architectures as well and complements ongoing work in CFD surrogates.

Sample Efficiency and Scaling

The authors quantify sample efficiency by comparing the joint model to a fixed reference, showing that a joint model with only two samples outperforms a randomly initialized model that is provided the entire dataset. When increasing the finetuning budget to 4000 steps, the joint model achieves the same error at S ≈ 4-8 samples, which is 8-16 times fewer samples. The results suggest that generating a large and diverse pretraining dataset is the most effective lever for improving a cross-domain CFD surrogate. However, increasing model size or training compute beyond a certain point can raise the zero-shot error, suggesting that the model begins to overfit to the pretraining distribution.

Normalization and Discretization

The authors investigate normalization strategies, finding that global normalization can perform well, and they show that cross-domain pretraining is still helpful regardless of the discretization. They also test querying finetuned models on the simulation mesh, which validates that cross-domain pretraining is still helpful, regardless of the discretization. The study suggests that in steady-state CFD settings, context or additional information is likely redundant for cross-domain methods designed around PDE terms and coefficients.

In-Context Pretraining Limitations

The authors tested a joint in-context model (SMARTIC) and found that providing a context simulation yields no benefit, and in most cases the error is slightly higher. This is expected for steady-state problems because the forward problem is well-posed given only the query geometry Gq and parameters θq. The results suggest that cross-domain CFD surrogates may require different strategies from those developed for time-dependent PDEs.

Variable-Specific Errors

The work reports per-variable errors, showing that the joint model generally outperforms both the domain experts and the models trained from scratch across all variables. The main trends hold across all variables, with the joint model generally outperforming both the domain experts and the models trained from scratch.

Conclusion

Overall, this work shows that cross-domain pretraining improves the zero- and few-shot performance of CFD surrogates on unseen datasets. A pretrained model yields lower error and reduces the number of samples that must be generated to deploy a surrogate in a new application. These gains also grow with both the diversity of the pretraining data and the size of the model.

Improvements for AI systems

Bold header: Cross-Domain Pretraining for Generalization

Leverage existing CFD datasets to improve zero- and few-shot performance on held-out datasets; specifically, cross-domain pretraining improves zero- and few-shot performance on held-out datasets relative to both training from scratch and transferring from domain-specific experts. This allows the model to achieve 2–3x lower errors at the same sample size or use 8x fewer samples to achieve the same error, compared to training from scratch.

Bold header: Enhanced Sample Efficiency

The improved system will drastically reduce the data generation cost by requiring far fewer samples for adaptation; specifically, finetuning a pretrained model can achieve a target error with 8–16× fewer training samples compared to training from scratch. This makes deployment of surrogates to new applications significantly more feasible.

Bold header: Architecture Agnostic Transfer

The cross-domain pretraining benefit is demonstrated as being independent of the specific neural network structure; the paper states, This benefit is architecture agnostic and improves with model size and pretraining dataset diversity. This suggests that future architectures can leverage this pooling strategy without needing to redesign the core surrogate model.

Bold header: Robustness to Model Scaling

The system will maintain strong generalization even when scaling the model size or compute budget, as increasing the number of datasets seen during pretraining or the model size improves downstream performance. This provides a clear guideline for selecting optimal pretraining resources based on available computational budgets.

Bold header: Physical Meaning Preservation via Normalization

The system will utilize physical priors to ensure consistent performance across domains by employing global normalization, which preserves the physical meaning of each variable across datasets, such as ensuring that Cp = 1 consistently corresponds to a stagnation point, and Cf = 0 (vanishing wall-shear stress) consistently indicates flow separation.

Abstract

Neural surrogates for computational fluid dynamics (CFD) have the potential to greatly enhance engineering innovation through accelerating simulation. However, the primary limitation for neural surrogates is the lack of generalization to geometries and applications beyond the training set, which is significant given the diversity of engineering scenarios. Currently, this is addressed by generating a new dataset for a specific application; however, this requires running costly numerical solvers. In this work, we take a step toward addressing this by studying neural surrogates trained across different geometries, boundary conditions, and fidelities. We find that cross-domain pretraining improves zero- and few-shot performance on held-out datasets relative to both training from scratch and transferring from domain-specific experts. In particular, finetuning a pretrained, cross-domain model can achieve 2-3x lower errors at the same sample size and use 8x fewer samples to achieve the same error, compared to training from scratch. This benefit is architecture agnostic and improves with model size and pretraining dataset diversity. Furthermore, we study how and why cross-domain pretraining works in CFD surrogates, and find that simply pooling steady-state datasets is both sufficient and effective. Given the high cost of generating CFD data, leveraging existing datasets through cross-domain pretraining will likely be a valuable strategy as future surrogates expand to tackle new problems and use cases.