Zero-shot generalization of transformer neural operators to larger domains

arXiv:2606.14597 · cs.LG · Submitted 2026-06-12 · Read on arXiv

cs.LG

Submitted: 2026-06-12

Updated: 2026-10-01

Code: https://github.com/cerea-daml/domain-extension

Terminology

• Transformer — 3× in this paper · explained in 2 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Zero-Shot Generalization — 2× in this paper · explained in 5 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes

Sources

• AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers
• Longformer: The Long-Document Transformer
• Extending Context Window of Large Language Models via Positional Interpolation
• Anchored-Branched Steady-state WInd Flow Transformer (AB-SWIFT): a metamodel for 3D atmospheric flow in urban environments
• Operator Learning with Domain Decomposition for Geometry Generalization in PDE Solving
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• Neural Operators with Localized Integral and Differential Kernels
• YaRN: Efficient Context Window Extension of Large Language Models
• Learning Mesh-Based Simulation with Graph Networks
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
• Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domains
• FlashBias: Fast Computation of Attention with Bias
• Transolver: A Fast Transformer Solver for PDEs on General Geometries
• Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks