Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains

arXiv:2604.08454 · cs.LG · Submitted 2026-04-09 · Read on arXiv

cs.LG

Submitted: 2026-04-09

Updated: 2026-09-30

Terminology

• Trace — 2× in this paper · explained in 2 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Progressive Reasoning Gain — 1× in this paper
• Reasoning Distillation — 1× in this paper
• Reasoning Traces — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes

Sources

• On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
• Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?
• Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domains
• Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers
• LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
• Language Models (Mostly) Know What They Know
• Why Language Models Hallucinate
• Demystifying Domain-adaptive Post-training for Financial LLMs
• Let's Verify Step by Step
• Towards a Unified View of Large Language Model Post-Training
• AttackSeqBench: Benchmarking the Capabilities of LLMs for Attack Sequences Understanding
• Revisiting the Calibration of Modern Neural Networks
• Training language models to follow instructions with human feedback
• Understanding the Impact of Confidence in Retrieval Augmented Generation: A Case Study in the Medical Domain
• Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering?

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks