Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs

arXiv:2606.32032 · cs.CL, cs.AI · Submitted 2026-06-30 · Read on arXiv

cs.CL, cs.AI

Submitted: 2026-06-30

Updated: 2026-10-12

Code: https://github.com/yale-nlp/RLMF

Terminology

• Metacognition — 3× in this paper · explained in 5 episodes
• Reinforcement Learning — 2× in this paper · explained in 4 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Active Learning — 1× in this paper · explained in 6 episodes
• Alignment — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes

Sources

• Linguistic Calibration of Long-Form Generations
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations
• Discovering Latent Knowledge in Language Models Without Supervision
• Finetuning Language Models to Emit Linguistic Expressions of Uncertainty
• SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Quantifying Faithful Confidence Expression in Large Reasoning Models
• The Llama 3 Herd of Models
• Improving Uncertainty Quantification in Large Language Models via Semantic Embeddings
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Measuring Massive Multitask Language Understanding
• Measuring Mathematical Problem Solving With the MATH Dataset
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling
• Can LLMs Estimate Cognitive Complexity of Reading Comprehension Items?
• Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations
• Reinforcement Learning from Human Feedback
• Taming Overconfidence in LLMs: Reward Calibration in RLHF
• HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
• Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence?
• C$^2$GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoning

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering