On Neural Scaling Laws for Weather Emulation through Continual Training
cs.LG
Submitted: 2026-03-26
Updated: 2026-11-06
Code: https://github.com/facebookresearch/fvcore
Terminology
Sources
- Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models
- Skillful joint probabilistic weather forecasting from marginals
- The rise of data-driven weather forecasting
- Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast
- A Foundation Model for the Earth System
- Message Passing Neural PDE Solvers
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- AERIS: Argonne Earth Systems Model for Reliable and Skillful Predictions
- Query-Key Normalization for Transformers
- Training Compute-Optimal Large Language Models
- Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
- Comparing and Contrasting DLWP Backbones on Navier-Stokes and Atmospheric Dynamics
- Scaling Laws for Fine-Grained Mixture of Experts
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- Scaling transformer neural networks for skillful and reliable medium-range weather forecasting
- GenCast: Diffusion-based ensemble forecasting for medium-range weather
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Fixing the Double Penalty in Data-Driven Weather Forecasting Through a Modified Spherical Harmonic Loss Function
- Foundation Models for Discovery and Exploration in Chemical Space
- ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis