Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models

arXiv:2609.20846 · cs.CL, cs.LG · Submitted 2026-08-05 · Read on arXiv

cs.CL, cs.LG

Submitted: 2026-08-05

Updated: 2026-10-13

Comments: 19 pages, 9 figures

Code: https://github.com/polina-tsvilodub/reasoning-under-missing-info

Project page: https://polina-tsvilodub.github.io/reasoning-under-missing-info/experiments/task_identification_solution

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Abstention — 2× in this paper · explained in 1 episode
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• GRPO — 1× in this paper · explained in 6 episodes

Sources

• Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests
• STaR-GATE: Teaching Language Models to Ask Clarifying Questions
• Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Training Verifiers to Solve Math Word Problems
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information
• Measuring Massive Multitask Language Understanding
• Language Models (Mostly) Know What They Know
• AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
• Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
• QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?
• Fast on the Easy, Deep on the Hard: Efficient Reasoning via Powered Length Penalty
• Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
• Understanding R1-Zero-Like Training: A Critical Perspective
• Olmo 3
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
• Qwen3 Technical Report

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering