Can AI Scientists Coordinate at Runtime?
cs.MA, cs.AI
Submitted: 2026-10-01
Updated: 2026-10-01
Comments: 35 pages (9 pages main text), 4 figures, 10 tables. Code: https://github.com/systemind-team/Runtime-AI-Scientist
Code: https://github.com/systemind-team/Runtime-AI-Scientist
License: http://creativecommons.org/licenses/by/4.0/
The gist: Multi-agent AI scientists have shown improving performance across a diverse range of tasks.
Abstract
Multi-agent AI scientists have shown improving performance across a diverse range of tasks. Yet a common approach is design-time agentic orchestration, which typically relies on fixed workflows. In contrast, human scientists coordinate and adjust their division of labor at runtime. We therefore ask: can AI scientists also coordinate at runtime? To this end, we introduce Runtime Agent Coordination (RAC), which selects agents from existing AI-scientist hosts during execution, assigns scoped work contracts, and provides artifact-grounded verification. Verification informs subsequent agents without blocking transitions or discarding artifacts. We conduct a single-seed exploratory evaluation across Agent Laboratory, EvoScientist, and ARK on ResearchClawBench, preserving host models, tools, and permissions under host-calibrated budgets. Four cumulative conditions separate native execution, runtime communication, runtime selection, and the combined addition of contracts and verification. Runtime selection yields the highest observed mean score for each host; adding contracts and verification reduces these means, with host-dependent outcomes relative to native execution. These results motivate runtime coordination while exposing the limits of additional coordination mechanisms under constrained budgets. Code is available at https://github.com/systemind-team/Runtime-AI-Scientist.
Sources
- Why Do Multi-Agent LLM Systems Fail?
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks
- AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation
- Training AI Co-Scientists Using Rubric Rewards
- Accelerating scientific discovery with Co-Scientist
- MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
- AI Agents That Matter
- Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automation
- AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaboration
- A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery
- Can AI Evaluate AI Scientists? A Benchmarking Study of Autonomous Research Generation Systems Using Automated Multi-Model Review
- Agent Laboratory: Using LLM Agents as Research Assistants
- AI-Researcher: Autonomous Scientific Innovation
- AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
- ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research
- The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
- ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration
Related papers
- Highway Congestion Reduction through Reinforcement Learning Based Eulerian Headway Control
- ORACLE-SWE: Quantifying the Contribution of Critical Information Signals on SWE Agents
- You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agents
- Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems
- MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization
- PeroMAS: A Multi-agent System of Perovskite Material Discovery