Iterated creation, grading, and revision of data science projects by language models
cs.AI, cs.CL
Submitted: 2026-11-03
Updated: 2026-11-03
Terminology
Sources
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- Do LLM Evaluators Prefer Themselves for a Reason?
- ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
- Same Prompt, Different Outcomes: Evaluating the Reproducibility of Data Analysis by LLMs
- Benchmarking AI Performance on End-to-End Data Science Projects
- DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?
- Into the Unknown: Benchmarking AI agents on open scientific research problems
- The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
- IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation
- Fisher-R1: Training LLM Agents for Reliable Hypothesis Testing
- Is Self-Repair a Silver Bullet for Code Generation?
- DSAgentBench: Can Agents Automate End-to-End Data-Science Workflows in Real Computer Environments?
- The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
- Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
- RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
- SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems
- Variation in Verification: Understanding Verification Dynamics in Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents