Selecting LLM-Generated Hypotheses Raises Truth Beyond What an LLM Panel Expects: A Pre-registered Study on NAS-Bench-201 and HPOBench
cs.LG, cs.AI
Submitted: 2026-10-31
Updated: 2026-10-31
Terminology
Sources
- Evaluator Ensembles Under Reward Hacking: Covariance Geometry and Finite-Search Guarantees
- AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
- RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement
- BoxingGym: Benchmarking Progress in Automated Experimental Design and Model Discovery
- Accelerating scientific discovery with Co-Scientist
- Automated Hypothesis Validation with Agentic Sequential Falsifications
- Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation
- HindSight: Evaluating LLM-Generated Research Ideas via Future Impact
- Goodhart's Law in Reinforcement Learning
- Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment
- HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation
- IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation
- The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems
- Categorizing Variants of Goodhart's Law
- Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference
- The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
- Towards Execution-Grounded Automated AI Research
- On the Limits of LLM-as-Judge for Scientific Novelty Assessment
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis