Benchmarking language models and agents for scientific reasoning in evolving materials laboratories
cs.AI, cond-mat.mtrl-sci, cs.CE, cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Code: https://github.com/dddxs/MatScope
Terminology
Sources
- MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science
- MaterialBENCH: Evaluating College-Level Materials Science Problem-Solving Abilities of Large Language Models
- MSQA: Benchmarking LLMs on Graduate-Level Materials Science Reasoning and Knowledge
- Towards Fully-Automated Materials Discovery via Large-Scale Synthesis Dataset and Expert-Level LLM-as-a-Judge
- MatTools: Benchmarking Large Language Models for Materials Science Tools
- LAB-Bench: Measuring Capabilities of Language Models for Biology Research
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems
- Holistic Evaluation of Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents