MedCalc-R1: Knowledge-Guided Reward Framework for Medical Mathematical Reasoning
Haotian Wang, Lian Yan, Xingzhi Yao, Fanshu Meng, Ye He, Jingchi Jiang, Yi Guan
cs.AI
Submitted: 2026-08-09
Updated: 2026-08-11
Comments: 17 pages, 11 figures, work in prograss
License: http://creativecommons.org/licenses/by/4.0/
The gist: In Reinforcement Learning with Verifiable Rewards (RLVR) frameworks for mathematical reasoning tasks, floating-point results are typically evaluated using a tolerance-based reward.
Terminology
Abstract
In Reinforcement Learning with Verifiable Rewards (RLVR) frameworks for mathematical reasoning tasks, floating-point results are typically evaluated using a tolerance-based reward. However, this strategy suffers from challenges such as difficulty in threshold calibration, unstable training dynamics, and limited accuracy, especially in clinical scenarios. To address these limitations, we propose a knowledge-guided hybrid reward framework (MedCalc-R1). Specifically, we introduce a knowledge verification reward mechanism that enforces explicit generation of computational formulas, which are further validated by an external verifier to enhance interpretability and reasoning reliability. Furthermore, we design a hybrid soft-hard reward scheme combining a hard constraint based on clinical safety thresholds with a soft, precision-sensitive reward that progressively guides learning within the acceptable range. Experimental results demonstrate that our method significantly outperforms existing baselines in both reasoning accuracy and generalization capability, validating the effectiveness and applicability in safety-critical domains.
Sources
- Program Synthesis with Large Language Models
- HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
- Training Verifiers to Solve Math Word Problems
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
- Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models
- Logical Reasoning in Large Language Models: A Survey
- REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization
- GPT-4o System Card
- Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?
- OpenAI o1 System Card
- Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
- Qwen2.5 Technical Report
- Proximal Policy Optimization Algorithms
- Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains
- A Survey on Large Language Models for Mathematical Reasoning
- Reinforcement Learning Enhanced LLMs: A Survey
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems