Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models
cs.LG
Submitted: 2025-03-24
Updated: 2026-09-30
Terminology
Sources
- GPT-4 Technical Report
- Neural equilibria for long-term prediction of nonlinear conservation laws
- Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning
- Learning Physical Models that Can Respect Conservation Laws
- Poseidon: Efficient Foundation Models for PDEs
- Learning continuous models for continuous physics
- Fourier Neural Operator for Parametric Partial Differential Equations
- Learning differentiable solvers for systems with hard constraints
- Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs
- The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators
- Understanding the Implicit Biases of Design Choices for Time Series Foundation Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks