Reinforcing Agentic Creativity in Scientific Ideation with Night Science
cs.AI, cs.CL
Submitted: 2026-09-28
Updated: 2026-09-28
Comments: Code: https://github.com/microsoft/ai_night_scientist Website: https://pkargupta.github.io/night_scientist.html
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) excel at structured, verifiable tasks, but their low-entropy bias can produce homogeneous and predictable outputs, limiting their utility for open-ended scientific
Terminology
Abstract
Large language models (LLMs) excel at structured, verifiable tasks, but their low-entropy bias can produce homogeneous and predictable outputs, limiting their utility for open-ended scientific ideation. Effective discovery, however, spans a broader creative spectrum: from structured day science to loosely structured, serendipitous night science that reaches ideas beyond those typically considered. We introduce AI Night-Scientist, an agentic framework that uses reinforcement learning to teach models when and how to depart from predictable reasoning. Grounded in cognitive science, we model creativity along three axes: action (what to do and how creatively), process (when to explore versus exploit), and outcome (the novelty and usefulness of the resulting idea). We use these axes to train models with GRPO, exposing them to varying degrees and forms of creativity throughout training. This produces substantially more diverse scientific proposals, expanding the range of research directions by 27.8% and contribution types by 14.9% over the base model. It also improves predicted citation impact by up to 32.0 percentage points and originality by 66.2 points. These gains cannot be reproduced by simply increasing decoding temperature; instead, we find that semantic guidance specifying what kind of creativity to pursue is critical. Overall, our results suggest that creativity is a learnable, multi-level ability that can be shaped to help researchers reach ideas beyond those typically explored by LLMs.
Sources
- Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback
- The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
- Measuring the Gap Between Human and LLM Research Ideas
- Accelerating scientific discovery with Co-Scientist
- LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Research
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- All That Glitters is Not Novel: Plagiarism in AI Generated Research
- GIANTS: Generative Insight Anticipation from Scientific Literature
- Cognitive Foundations for Reasoning and Their Manifestation in LLMs
- Sparking Scientific Creativity via LLM-Driven Interdisciplinary Inspiration
- Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code
- LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play
- Sparks of Science: Hypothesis Generation Using Structured Paper Data
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- HybridFlow: A Flexible and Efficient RLHF Framework
- AI Can Learn Scientific Taste
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example
- We're Different, We're the Same: Creative Homogeneity Across LLMs
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems