Learning to Outgrow a Theory: Experimental Discovery Beyond the Initial Hypothesis Space
cs.AI
Submitted: 2026-10-06
Updated: 2026-10-06
License: http://creativecommons.org/licenses/by/4.0/
The gist: Scientific discovery systems typically optimize experiments within a fixed hypothesis space.
Abstract
Scientific discovery systems typically optimize experiments within a fixed hypothesis space. This creates a failure mode when all available candidates omit the same missing mechanism: candidate disagreement can collapse even while the model class is systematically wrong. We formulate experimental model-class revision, in which a discovery policy jointly proposes a structural edit and a diagnostic experiment that tests whether that edit is necessary. The method couples a class-level distinguishability objective, in which one shared parameterization must explain all selected experiments, with anytime-valid sequential evidence that triggers structural revision only after the current class is rejected. On 400 held-out controlled dynamical environments, the joint policy reaches 89.5% exact recovery with a budget of 32 real experiments, improving the strongest matched baseline by 10.0 percentage points while requiring fewer executed experiments and candidate fits. The learned revision-experiment pairing transfers across unseen mechanism combinations, held-out but expressible primitives, parameter extrapolation, and shifted experiment costs; when the true mechanism is outside the edit grammar, it detects library insufficiency in 88% of cases with a 5.5% false-support rate. Revision gains also transfer to ODEBench and ODEBase model-library tasks, as well as DiscoverPhysics worlds. These results support a view of scientific discovery in which deciding what mechanisms a theory should make expressible and where to collect evidence are treated as a single sequential decision problem.
Sources
- LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Search
- Evidence-Informed LLM Beliefs for Continual Scientific Discovery
- Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl
- ATLAS: Active Theory Learning for Automated Science
- Adversarial Causal Intervention Falsification
- Automated Hypothesis Validation with Agentic Sequential Falsifications
- LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMs
- Are all models wrong? Fundamental limits in distribution-free empirical model falsification
- LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models
- Representative, Informative, and De-Amplifying: Requirements for Robust Bayesian Active Learning under Model Misspecification
- Robust Simulation Based Inference
- A Probabilistic Framework for LLM-Based Model Discovery
- DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking
- The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
- EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems