WorkflowArena: Benchmarking Executable Workflow Orchestration in LLM-Based Multi-Agent Systems
cs.CL, cs.SE
Submitted: 2026-11-09
Updated: 2026-11-09
Project page: https://lab.einsia.ai/workflowarena
Terminology
Sources
- ReAct: Synergizing Reasoning and Acting in Language Models
- Reflexion: Language Agents with Verbal Reinforcement Learning
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
- Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence
- Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
- DynaFlow: Learning Free-Form Multi-Agent Orchestration
- An LLM Compiler for Parallel Function Calling
- AgentBench: Evaluating LLMs as Agents
- WebArena: A Realistic Web Environment for Building Autonomous Agents
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
- AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks
- MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
- Decentralized Multi-Agent Coordination and Communication in Shared Codebases
- FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents
- Benchmarking Agentic Workflow Generation
- OrchBench: Evaluating Multi-Agent Orchestration Plans in Isolation via Deterministic Simulation
- GAIA: a benchmark for General AI Assistants
- AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data