Evaluating LLMs in Database Scenarios: A Lifecycle Benchmark for Assessing Their Potential in Core Database Tasks
Shunfan Zheng, Dongsheng Shi, Yue Li, Xin Yi, Linlin Wang, Gerard de Melo
cs.DB, cs.AI, cs.CL
Submitted: 2026-08-04
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Reinforcement Learning for Large Language Model Selective Evidence Adoption from Contaminated Retrieval Results
- Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queries
- Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL
- TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems
- Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
- Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs
- SURGENT: A Surgical Multi-Agent Assistance System Across the Perioperative Workflow
- SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning
- CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases
- SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Models
- A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
- Are Tools All We Need? Unveiling the Tool-Use Tax in LLM Agents
- LLM-Enhanced Data Management
Related papers
- SAGA: Synthetic Agentic Graph Architecture for Temporal Benchmark Generation
- Cost-Aware Mixture-of-Experts Coordination for Model Markets
- Amora: Communication-Efficient Private Approximate Nearest Neighbors Search
- Vibe Coding on Trial: Operating Characteristics of Unanimous LLM Juries
- Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering
- MetaJoin: Adaptive Join Selections for Heterogeneous Tables