DSL-Bench: Evaluating and Explaining LLM Code Generation Across Domain-Specific Languages
cs.SE
Submitted: 2026-11-03
Updated: 2026-11-03
Code: https://github.com/GDQuest/learn-gdscript
Project page: https://openvaf.github.io
Terminology
Sources
- Evaluating Large Language Models Trained on Code
- Program Synthesis with Large Language Models
- Measuring Coding Challenge Competence With APPS
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- LeanDojo: Theorem Proving with Retrieval-Augmented Language Models
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation
- MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics
- nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task
- Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
- RTLRepoCoder: Repository-Level RTL Code Completion through the Combination of Fine-Tuning and Retrieval Augmentation
- TerraFormer: Automated Infrastructure-as-Code with LLMs Fine-Tuned via Policy-Guided Verifier Feedback
- SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation
- QuanBench: Benchmarking Quantum Code Generation with Large Language Models
- A System Model Generation Benchmark from Natural Language Requirements
- Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback
- Teaching Large Language Models to Self-Debug
- ReAct: Synergizing Reasoning and Acting in Language Models
- LoRA: Low-Rank Adaptation of Large Language Models
- Enhancing Code Generation for Low-Resource Languages: No Silver Bullet
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters