Characterizing Overconfident Failure in LLM-Based Code Generation
cs.SE, cs.AI
Submitted: 2026-10-08
Updated: 2026-10-08
Project page: https://codeairesearch.github.io/CodeGenUncertainty
Sources
- Code Llama: Open Foundation Models for Code
- DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
- DOCE: Finding the Sweet Spot for Execution-Based Code Generation
- Training Language Models to Generate Quality Code with Program Analysis Feedback
- Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
- An Empirical Analysis of Uncertainty in Large Language Model Evaluations
- A Survey of Uncertainty Estimation Methods on Large Language Models
- Reconsidering LLM Uncertainty Estimation Methods in the Wild
- Why Language Models Hallucinate
- Calibration and Correctness of Language Models for Code
- Assessing Correctness in LLM-Based Code Generation via Uncertainty Estimation
- Code Is More Than Text: Uncertainty Estimation for Code Generation
- OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
- Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code
- TESTEVAL: Benchmarking Large Language Models for Test Case Generation
- Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data
- Mutual Information Alleviates Hallucinations in Abstractive Summarization
- Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
- Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models
- Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties