SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows
cs.SE, cs.AI
Submitted: 2026-06-29
Updated: 2026-11-07
Project page: https://spreadsheetbench.github.io
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making.
Terminology
Abstract
Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making. However, most existing spreadsheet benchmarks evaluate isolated operations such as single-formula generation or local cell edits, and therefore fail to capture end-to-end workflows in realistic business settings. We introduce SpreadsheetBench 2, a workflow-level benchmark for spreadsheet agents that covers three task categories: generation, debugging, and visualization. The benchmark is constructed from authentic business data, including financial reports and corporate filings, and is annotated and validated by domain experts. The benchmark contains 321 tasks; each instance averages 11.8 worksheets and requires 593.5 cell modifications, reflecting large multi-sheet workbooks with cross-sheet dependencies. We evaluate eight frontier large language models under a unified multi-turn agent scaffold, and additionally include several LLM-based spreadsheet products as complementary baselines. Results show that current systems remain far from reliable on real-world workflows: the best model achieves 34.89% overall task accuracy, and debugging accuracy is as low as 12.00%. Trajectory analysis and a failure taxonomy further indicate that insufficient spreadsheet inspection and incorrect target-cell selection are the dominant bottlenecks. Together, these findings position SpreadsheetBench 2 as a challenging testbed for advancing reliable spreadsheet automation. Project page: https://spreadsheetbench.github.io/
Sources
- SheetMind: Actions Set Accuracy, Agents Set the Failure Mode
- Large Language Models for Spreadsheets: Benchmarking Progress and Evaluating Performance with FLARE
- TAPEX: Table Pre-training via Learning a Neural SQL Executor
- MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark
- GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
- GLM-5: from Vibe Coding to Agentic Engineering
- Kimi K2.5: Visual Agentic Intelligence
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- TabFact: A Large-scale Dataset for Table-based Fact Verification
- SpreadsheetLLM: Encoding Spreadsheets for Large Language Models
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- WebArena: A Realistic Web Environment for Building Autonomous Agents
- AgentBench: Evaluating LLMs as Agents
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters