SALUS: Automated Auditing of NL-to-SQL Benchmarks through Weak Supervision of Multi-Agent Output
cs.DB, cs.CL, cs.LG
Submitted: 2026-10-04
Updated: 2026-10-04
Code: https://github.com/ucisharadlab/SALUS
Terminology
Sources
- Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards
- Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
- RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics
- WRENCH: A Comprehensive Benchmark for Weak Supervision
- Stronger Than You Think: Benchmarking Weak Supervision on Realistic Tasks
- Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
Related papers
- Cost-Aware Mixture-of-Experts Coordination for Model Markets
- Vibe Coding on Trial: Operating Characteristics of Unanimous LLM Juries
- Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering
- MetaJoin: Adaptive Join Selections for Heterogeneous Tables
- Bridging Business Intent and Data: A Benchmark for Automatic Relational Data Product Generation
- DexterSQL: Deep Schema Exploration and Rule-based Correction for Text-to-SQL Generation