SynthCorp: A Platform for Creating Benchmarks for Data Agents
cs.AI, cs.DB
Submitted: 2026-11-10
Updated: 2026-11-10
Code: https://github.com/lerocha/chinook-database
Terminology
Sources
- REIN: A Comprehensive Benchmark Framework for Data Cleaning Methods in ML Pipelines
- PLG2: Multiperspective Processes Randomization and Simulation for Online and Offline Settings
- Automated Discovery of Business Process Simulation Models from Event Logs
- Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
- BEAVER: An Enterprise Benchmark for Text-to-SQL
- LAKEGEN: A LLM-based Tabular Corpus Generator for Evaluating Dataset Discovery in Data Lakes
- DABstep: Data Agent Benchmark for Multi-step Reasoning
- OrgForge: A Multi-Agent Simulation Framework for Verifiable Synthetic Corporate Corpora
- Autonomous Data Agents: A New Opportunity for Smart Data
- The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents
- SynSQL: Synthesizing Relational Databases for Robust Evaluation of Text-to-SQL Systems
- CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
- Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards
- AgentSimulator: An Agent-based Approach for Data-driven Business Process Simulation
- LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
- MiniCorp: The Last Mile to an AI Agent Corporation
- An Agentic Approach to Metadata Reasoning
- EvoSchema: Towards Text-to-SQL Robustness Against Schema Evolution
- CODA-BENCH: Can Code Agents Handle Data-Intensive Tasks?
- A Survey of Data Agents: Emerging Paradigm or Overstated Hype?
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents