On the Robustness of LLMs' Internal Representation of Code Correctness
Francisco Ribeiro, Sohaila Abdulsattar, Renata Gonzalez, Mahmoud Kassem, Sarah Nadi
cs.SE, cs.AI, cs.LG
Submitted: 2026-08-08
Updated: 2026-08-11
Comments: 14 pages
Code: https://github.com/pylint-dev/astroid
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Evaluating Large Language Models Trained on Code
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- Representation Engineering: A Top-Down Approach to AI Transparency
- BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
- Program Synthesis with Large Language Models
- Language Models (Mostly) Know What They Know
- Qwen2.5-Coder Technical Report
- The Truthfulness Spectrum Hypothesis
- LLMs Encode Their Failures: Predicting Success from Pre-Generation Activations
- Localized Calibrated Uncertainty in Code Language Models
- Mechanistic Interpretability of Code Correctness in LLMs via Sparse Autoencoders
- CodeCircuit: Toward Inferring LLM-Generated Code Correctness via Attribution Graphs
- A Mixture of Linear Corrections Generates Secure Code
- Are Sparse Autoencoders Useful for Java Function Bug Detection?
- Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification
- No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation