Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery
Taolin Han, Yuchen Zhang, Jinghang Wang, Yun Wu, Wai Yuet Chiu, Zhaohai Li, Yifei Zhang, Jinxin Wang, Yuhao Zhou, Chen Zhao, Jiajia Li, Jiaxin Li, Qile Jin, Kewei Sun, Shuang Wu, Weiqi Zhai, Renquan Lv, Junchao Li, Ruodan Chen, Qingteng Chen, Zhibo Yang, Hu Wei, Lin Qu, Shuai Bai, Bing Zhao
cs.AI
Submitted: 2026-08-07
Updated: 2026-08-10
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) are increasingly involved in scientific discovery, yet it remains unclear whether they can support complex real laboratory science.
Terminology
Abstract
Large language models (LLMs) are increasingly involved in scientific discovery, yet it remains unclear whether they can support complex real laboratory science. Here we introduce Science Edge Evaluation (SEE), a multimodal benchmark of expert-curated questions grounded in peer-reviewed literature and experimental practice in chemistry, biology, and materials science. Evaluation of 19 multimodal large language models (MLLMs) shows that even the best-performing model reaches only 48.7% accuracy. Moreover, general-purpose models outperform science-specialized models on average. In the visual-agent evaluation, the use of tools increases the best accuracy to 52.7%. Tool use can expand the information available to models, but more information does not necessarily lead to reliable scientific reasoning. The key challenge is whether models can manage tool-derived information within the boundaries of the original experimental evidence. Together, these findings reveal that current MLLMs still cannot reliably make justified and evidence-bounded inferences from experimental results, which is an essential capability in real scientific discovery. Bridging this gap requires MLLMs to transition from explaining established scientific concepts to deriving novel and evidence-based insights from experimental data.
Sources
- Measuring Massive Multitask Language Understanding
- MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- CMMLU: Measuring massive multitask language understanding in Chinese
- Humanity's Last Exam
- Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
- MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
- MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
- SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation
- MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models
- LAB-Bench: Measuring Capabilities of Language Models for Biology Research
- ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry
- Galactica: A Large Language Model for Science
- BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text
- MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
- ChemLLM: A Chemical Large Language Model
- Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models
- LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems