PhysicsBench: A Unified Leaderboard for Generative and Predictive Models in Engineering Design and Simulation
cs.LG, cs.CE
Submitted: 2026-08-25
Updated: 2026-08-25
Comments: 40 pages, 12 figures, 8 tables. Leaderboard: https://leaderboard.narnia.ai | Data: https://github.com/Narnialabs/leaderboard
Code: https://github.com/Narnialabs/leaderboard
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generative and predictive artificial intelligence models are increasingly used to generate geometry and to predict physical fields and scalar quantities in engineering design and simulation.
Terminology
Abstract
Generative and predictive artificial intelligence models are increasingly used to generate geometry and to predict physical fields and scalar quantities in engineering design and simulation. Yet these models are typically evaluated in isolation, on academic datasets at unconstrained scales, with inconsistent metrics and procedures. We present PhysicsBench, a unified benchmark and leaderboard that evaluates generative and predictive models under one standardized procedure. PhysicsBench spans seven generation and prediction tasks across 1D, 2D, and 3D domains and ranks 66 models on nine datasets, comprising industrial-scale CAD/CFD/FEA simulations and public references, expanded into 28 configurations. One procedure and ranking apply to both families, each ranked within its own tasks. Evaluation spans realistic, limited data scales from S to XL rather than the unlimited training sets common in academic benchmarks. A common metric suite captures geometric fidelity with distributional distances, physical-field and scalar accuracy, and engineering-specific field- and shape-validity. BenchRank debiases correlated metrics and ranks by PageRank over a head-to-head dominance graph, so every reported quality metric is also ranked, with computational cost in a separate efficiency view. Across tasks, an architecture's large-scale academic standing weakly predicts its small-data ranking. The top model changes with data scale in six of the seven tasks, and no model leads more than one task. PhysicsBench turns "state-of-the-art" from a self-reported claim into an openly published foundation for model selection.
Sources
- Fourier Neural Operator for Parametric Partial Differential Equations
- Neural Operator: Learning Maps Between Function Spaces
- Transolver: A Fast Transformer Solver for PDEs on General Geometries
- Generative Adversarial Networks
- Auto-Encoding Variational Bayes
- Denoising Diffusion Probabilistic Models
- Deep Generative Models in Engineering Design: A Review
- SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
- Measuring Massive Multitask Language Understanding
- CarBench: A Comprehensive Benchmark for Neural Surrogates on High-Fidelity 3D Car Aerodynamics
- Data efficient surrogate modeling for engineering design: Ensemble-free batch mode deep active learning for regression
- ShapeNet: An Information-Rich 3D Model Repository
- Learning Representations and Generative Models for 3D Point Clouds
- GNOT: A General Neural Operator Transformer for Operator Learning
- AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers
- Learning to Simulate Complex Physics with Graph Networks
- Learning Mesh-Based Simulation with Graph Networks
- Physics-Informed Neural Operator for Learning Partial Differential Equations
- SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
- Training data-efficient image transformers & distillation through attention
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks