Thermodynamic cost of inference and learning in physical neural networks

arXiv:2503.09980 · cond-mat.stat-mech, cs.LG, physics.data-an, q-bio.NC · Submitted 2025-03-13 · Read on arXiv

cond-mat.stat-mech, cs.LG, physics.data-an, q-bio.NC

Submitted: 2025-03-13

Updated: 2026-08-25

Terminology

• Loss Function — 7× in this paper · explained in 2 episodes
• Artificial Intelligence — 3× in this paper
• Activation Function — 2× in this paper · explained in 1 episode
• Adaptation — 2× in this paper · explained in 1 episode
• Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Mixture of Experts — 1× in this paper · explained in 2 episodes
• Mixture of Experts (MoE) — 1× in this paper · explained in 5 episodes
• Scaling Laws — 1× in this paper · explained in 4 episodes
• Thermodynamic Uncertainty Relationship — 1× in this paper
• Transformer — 1× in this paper · explained in 2 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes

Sources

• Scaling Laws for Neural Language Models
• Training of Physical Neural Networks
• From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference
• DeepSeek-V3 Technical Report

Related papers

• Extreme First-Passage Time of Many Interacting Particles
• Decoherence-controlled collective criticality in a two-dimensional quantum Stag Hunt
• Measurement-induced phase transitions in disordered fermions
• Hierarchical Reduction of Invariant Subspaces: From Fragmentation to Many-Body Scars
• Quantum Thermalization beyond Non-Integrability and Quantum Scars in a Multispecies Bose-Josephson Junction
• Markov length can diverge in systems whose universal physics is spatially Markovian