On the Effectiveness of Large Language Models in Writing Alloy Formulas
The episode discusses a paper testing Large Language Models' effectiveness in writing Alloy formulas, a niche specification language. Researchers tested models like ChatGPT and DeepSeek on tasks including writing formulas from English descriptions, rewriting existing formulas, and completing partial sketches. The findings show the models can generate multiple correct answers and offer novel logical formulations, suggesting AI could become an integral part of formal methods.