The Novelty Horizon H N: Why Large Language Models Fail to Generate Scientific Data beyond Their Training Distribution
eess.SY, cs.SY
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 10 Figures, 61 pages
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large Language Models (LLMs) lack physical grounding and extrapolation.
Terminology
Abstract
Large Language Models (LLMs) lack physical grounding and extrapolation. This paper formalises the Novelty Horizon as tasks needing extrapolation beyond training support, causal reasoning absent from learned correlations, or post-frontier information. A forensic analysis tests LLMs on unguided, guided, and ground-truth data from an underrepresented diode-clamped short-circuit experiment using temporal, spectral, and statistical validation against a first-principles Modified Ohm's Law. Both script-based approaches fail catastrophically with physical and spectral inconsistencies, extreme statistical pathologies, zero Generative Fidelity Score, and linear-regime outputs. Direct prompting corroborates distinct failure modes. This paper introduces the Past Information Boundary and physics-based authentication. LLM data generation cannot extrapolate past the Novelty Horizon essential for novel discovery. Hybrid neuro-symbolic systems must preserve human causal reasoning and experimental design.
Sources
- Are Emergent Abilities in Large Language Models just In-Context Learning?
- Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data
- LLMs4All: A Review of Large Language Models Across Academic Disciplines
- The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities
- Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures
- Understanding the Role of Cross-Entropy Loss in Fairly Evaluating Large Language Model-based Recommendation
- Attention Is All You Need
- Attention as Conditioning: What Classical Learning Theory Predicts About Linear Transformers
- The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities
- Electricity Demand and Grid Impacts of AI Data Centers: Challenges and Prospects
- The Carbon Cost of Conversation, Sustainability in the Age of Language Models
- Hierarchical Reasoning Model
- Reinforcement Learning from Human Feedback
- A Survey on LLM Inference-Time Self-Improvement
- Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
- A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks
- Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training
- Model Collapse Demystified: The Case of Regression
- Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches
- On the Fundamental Limits of LLMs at Scale
Related papers
- Industrial Load Modeling and Optimization for Market-Based Interaction with Power Systems
- Exact and Approximate Convex Reformulation of Linear Stochastic Optimal Control with Chance Constraints
- Certified Set Convergence for Piecewise Affine Systems
- Symbolic Control for Stochastic Systems via Finite Parity Games
- Multiobjective optimization-based design and dispatch of islanded, hybrid microgrids for remote, off-grid communities in sub-Saharan Africa
- Physiology-informed layered sensing for intelligent human-exoskeleton interaction