Evaluating Exact Output and Checkpoint-State Prediction in Real Programs

cs.SE, cs.AI

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: 15 pages. Accepted at the NeurIPS 2026 Workshop on AI for Verifiable Coding. Includes appendices

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Sources

• CWM: An Open-Weights LLM for Research on Code Generation with World Models
• Learning to Execute
• CodeMind: Evaluating Large Language Models for Code Reasoning
• Code Simulation Challenges for Large Language Models
• Show Your Work: Scratchpads for Intermediate Computation with Language Models

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties