How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?

arXiv:2609.20814 · physics.comp-ph, cs.LG, physics.flu-dyn · Submitted 2026-09-17 · Read on arXiv

physics.comp-ph, cs.LG, physics.flu-dyn

Submitted: 2026-09-17

Updated: 2026-09-17

Comments: 15 pages, 5 figures. Representations for the Physical Sciences Workshop, NeurIPS 2026

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Fine-Tuning — 42× in this paper · explained in 5 episodes
• Accuracy — 15× in this paper · explained in 1 episode
• Distribution Shift — 12× in this paper · explained in 5 episodes
• Benchmark — 6× in this paper · explained in 2 episodes
• Foundation Model — 5× in this paper · explained in 3 episodes
• Foundation Models — 5× in this paper · explained in 7 episodes
• Generalization — 4× in this paper · explained in 13 episodes
• Transformer — 4× in this paper · explained in 2 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Neural Surrogates — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Uncertainty Quantification — 1× in this paper · explained in 8 episodes
• Zero-Shot Transfer — 1× in this paper · explained in 4 episodes

Sources

• Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• UniFoil: A Universal Dataset of Airfoils in Transitional and Turbulent Regimes for Subsonic and Transonic Flows
• SIMSHIFT: A Benchmark for Adapting Neural Surrogates to Distribution Shifts
• Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior
• Transolver: A Fast Transformer Solver for PDEs on General Geometries

Related papers

• Refining Machine Learning Potentials through Thermodynamic Theory of Phase Transitions
• Learning Spectrally Optimised Mesh-Free Discretisations
• Feature-preserving Latent-EnKF for Data Assimilation of Flows with Shocks
• chemtrain-deploy: A parallel and scalable framework for machine learning potentials in million-atom MD simulations
• Omnisolver: An extensible interface to Ising spin-glass and QUBO solvers: adding a distributed GPU brute-force plugin
• Breaking the Exascale Barrier for the Electronic Structure Problem in Ab-Initio Molecular Dynamics