How far do on-premises open LLMs get on Text-to-SQL? A cross-family size and technique frontier on BIRD
The study benchmarked various open-weight models for Text-to-SQL on a BIRD database split, testing different model families and serving methods. It found that self-correction is the most effective technique across models, while schema linking provides no benefit. Self-consistency was deemed not worth the cost, suggesting practitioners should prioritize strong generation and add self-correction to achieve good performance.