Doc2Spec: Synthesizing Formal Programming Specifications from Natural Language via Grammar Induction
cs.PL, cs.AI, cs.SE
Submitted: 2026-01-30
Updated: 2026-10-07
Code: https://github.com/semgrep/semgrep
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Ensuring that API implementations and usage comply with natural language programming rules is critical for software correctness, security, and reliability.
Terminology
Abstract
Ensuring that API implementations and usage comply with natural language programming rules is critical for software correctness, security, and reliability. Formal verification can provide strong guarantees but requires precise specifications, which are difficult and costly to write manually. To address this challenge, we present Doc2Spec, a multi-agent framework that automatically induces a domain-specific grammar from natural-language API rules and uses it to guide specification generation. Doc2Spec fixes a domain-agnostic logical skeleton as a grammar template, prompts LLMs to infer domain-specific predicates and sorts, and formalizes each rule within the resulting grammar, turning an unreliable one-shot translation into a sequence of constrained, checkable steps. Across six benchmarks spanning Solidity and Rust, Doc2Spec improves precision by 0.28 and recall by 0.37 over baselines that lack grammar induction or perform it in one unstaged step, demonstrating the benefits of grammar-guided formalization and the staged pipeline. Moreover, its formalized rules enable symbolic-execution tools to uncover 142 previously unknown rule violations, confirming the rules' correctness and practical usefulness.
Sources
- Llemma: An Open Language Model For Mathematics
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Formal Specifications from Natural Language
- Finding Inductive Loop Invariants using Large Language Models
- Towards making formal methods normal: meeting developers where they are
- Evaluating Large Language Models in Semantic Parsing for Conversational Question Answering over Knowledge Graphs
- Leveraging Large Language Models for Automated Proof Synthesis in Rust
Related papers
- Language-Based Agent Control
- ANEForge: Python for direct computation on the Apple Neural Engine
- Volition Elicitation: Operational Semantics for People and Their Machines
- Griotte: Verified Compartmentalisation via Capabilities
- GraphMend: Code Transformations for Fixing Graph Breaks in PyTorch 2
- Two Birds One Stone: Effective Static Detection of Resource and Communication Deadlocks in Rust Programs