Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders

arXiv:2405.17044 · cs.AI, cs.CL, cs.DL, cs.LG · Submitted 2024-05-27 · Read on arXiv

cs.AI, cs.CL, cs.DL, cs.LG

Submitted: 2024-05-27

Updated: 2026-09-24

Comments: 15 pages; 7 figure, 2 tables; Appendix: 8 pages, 7 figures, 1 table

Code: https://github.com/artificial-scientist-lab/SciMuse

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Knowledge Graph — 37× in this paper · explained in 7 episodes
• AUC — 5× in this paper · explained in 2 episodes
• Accuracy — 3× in this paper · explained in 1 episode
• Knowledge graphs — 3× in this paper · explained in 2 episodes
• Artificial Intelligence (AI) — 2× in this paper · explained in 2 episodes
• Area Under the Curve — 1× in this paper
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Quantum Physics — 1× in this paper · explained in 10 episodes
• Rapid Automatic Key-word Extraction — 1× in this paper
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
• Forecasting high-impact research topics via machine learning on evolving knowledge graphs
• Large Language Models for Automated Open-domain Scientific Hypotheses Discovery
• ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
• GPT-4 Technical Report
• The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4
• OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
• Extracting human interpretable structure-property relationships in chemistry using XAI and large language models

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems