MediX-R1: Open Ended Medical Reinforcement Learning

arXiv:2602.23363 · cs.CV · Submitted 2026-02-26 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "MediX-R1: Open Ended Medical Reinforcement Learning".

Jane: Here's the gist of the paper. MediX-R1 introduces an open-ended Reinforcement Learning framework for medical multimodal large language models that enables clinically grounded, free-form answers beyond multiple-choice formats The paper presents MediX-R1, an open-ended RL framework that fine-tunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning This framework is significant because it provides a principled approach for training medical MLLMs with open-ended RL that integrates semantic correctness with structural and modality constraints Framework Overview MediX-R1 fine-tunes a baseline multimodal backbone for open-ended medical reasoning using Group Based RL Given an image I and question q, the vision encoder produces visual tokens that are fused with text tokens and fed to the LLM policy πθ The model generates structured outputs of the form:

modality tag: free-form clinical reasoningfinal concise answer.

Tom: This design encourages the model to generate accurate and interpretable reasoning paths.

Jane: How the training signal is built The composite reward is defined as r = wfmt Rformat + wllm Rllm + wemb Rembed + wmod Rmodality.

Tom: The specific coefficients were selected through a staged procedure, fixing the format reward weight to 0.10 and allocating the remaining 0.90 mass to task-facing signals.

Paper summary: Tom: So if we boil down what MediX-R1 is about: it’s taking those complex ideas—the composite reward structure and the unified evaluation framework—and making them practical for open ended medical reasoning.

Jane: Yeah, the paper by Sahal Shaji Mullappilly et al., titled "MediX-R1: Open Ended Medical Reinforcement Learning," shows how you can use reinforcement learning to train these multimodal models to give answers that aren't just simple picks, but actual free form clinical thoughts.

Lu: It’s about using a multi faceted reward signal, incorporating semantic checks from embeddings and LLM judges alongside structural requirements like the format reward. That gives the model discipline in generating those structured outputs we see in examples.

Meng: From a deployment standpoint, I'm looking at how they handle things at scale, but what this means is that if you want to deploy an AI for medical reports, you need these kinds of signals to ensure it’s not just guessing the right words.

Tom: Exactly. The implication is that we can move away from relying too much on brittle string matching metrics for evaluating complex medical reasoning tasks and use a more principled approach instead.

Jane: It moves us toward models that are better at handling real world uncertainty, like paraphrasing a question or changing the required output format, while still maintaining high levels of semantic correctness.

Lu: The authors also commit to releasing their training inference code, configurations, checkpoints, curated datasets and RL evaluation prompt templates under a CC BY NC SA four point zero license which makes it more accessible for others to build on this.

Meng: That’s good because transparency is key when you're working in sensitive areas like medicine. They also pointed out risks around demographic and dataset bias, suggesting future work should include fairness analyses.

Tom: So the title itself suggests a lot—open ended RL. It shows that open ended reinforcement learning can be a practical tool for making medical MLLMs more reliable and trustworthy in their output.

Conclusion: Tom: So we’ve looked at how MediX-R1 uses reinforcement learning to train these medical multimodal models for open ended answers, now we need to talk about what that title actually means.

Jane: It’s about taking those complex medical image and text questions and letting the AI reason through them freely, instead of just picking one pre written answer.

Lu: The authors are doing this by combining a few different signals for their training, like a format reward, an embedding based reward, and an llm based accuracy reward.

Meng: From where I sit at the startup level it looks like they’ve built a system that tries to make the AI not just get the right answer but get it in a way that looks clinically sound.

Tom: Yeah, so if you hear "Open Ended Medical Reinforcement Learning" you have to think about how this moves away from multiple choice tests and toward actual doctor-like explanations.

Jane: Right. It’s trying to make the AI generate these structured outputs, like it first says what kind of test it’s doing, then shows its reasoning step by step, and finally gives a concise answer.

Lu: The main point is that this multi signal design stabilizes the training process and helps reduce those weird reward hacking problems where you trick the model into giving you an easy answer instead of a good one.

Tom: It seems like they’ve found a way to use structure and multiple checks to get better performance than just using one simple accuracy check.

Jane: And they used this framework on real medical data, showing it performs well in those challenging image plus text tasks compared to other models we’ve seen.

Meng: I’m still checking how stable it is when you run it on high traffic because I don't trust benchmarks that aren't running on real user traffic.

Tom: Well that brings us to the evaluation part next, where they use a unified reference based llm as judge to check if the answers are actually good.

Jane: That’s the next big thing—moving away from just counting word overlap and using an instruction tuned model to actually judge if the reasoning makes sense clinically.

Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Omair Mohamed, Mohamed Zidan, Fahad Khan, Salman Khan, Rao Anwer, Hisham Cholakkal

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Jubilee Mission Medical College and Research Institute, JJM Medical College

cs.CV

Submitted: 2026-02-26

Updated: 2026-11-15

Comments: Accepted to EMNLP 2026 Findings

DOI: 10.18653/2026.findings-emnlp.2133

Code: https://github.com/abhinand5/MedEmbed

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 89/100

The gist: MediX-R1 introduces an open-ended Reinforcement Learning framework for medical multimodal large language models that enables clinically grounded…

Key concepts

MediX-R1 Framework
This is an open-ended RL system that fine-tunes a multimodal backbone for medical reasoning. It uses Group Based RL and a composite reward to train the model to produce detailed, free-form clinical answers instead of just selecting options. The goal is to achieve clinically relevant, interpretable outputs.
Composite Reward Function
The training signal is a combination of four rewards: format, LLM accuracy (checking semantic correctness), embedding similarity (checking meaning against a reference), and modality recognition. These weights are carefully tuned to ensure the model learns not just correct answers, but also the right structure and modality.
Reference-based LLM-as-Judge Evaluation
Instead of brittle string matching, this evaluation uses an instruction-tuned LLM (like vLLM) to judge generated answers against a reference using templates like BASE or MIMIC. This captures complex semantic correctness and reasoning adequacy across different medical tasks.
Reward Hacking Mitigation
The composite reward design, including penalties for format mismatches and modality errors, helps prevent 'reward hacking' where the model cheats to maximize the score without actually improving reasoning. Specific techniques like embedding gating further stabilize optimization.

Terminology

Summary

MediX-R1 introduces an open-ended Reinforcement Learning framework for medical multimodal large language models that enables clinically grounded, free-form answers beyond multiple-choice formats The paper presents MediX-R1, an open-ended RL framework that fine-tunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning This framework is significant because it provides a principled approach for training medical MLLMs with open-ended RL that integrates semantic correctness with structural and modality constraints

Framework Overview

MediX-R1 fine-tunes a baseline multimodal backbone for open-ended medical reasoning using Group Based RL Given an image I and question q, the vision encoder produces visual tokens that are fused with text tokens and fed to the LLM policy πθ The model generates structured outputs of the form: [modality tag]free-form clinical reasoningfinal concise answer. This design encourages the model to generate accurate and interpretable reasoning paths. p. 2

Where the paper says it

This reward is composed of: a) an LLM-based reward for evaluating the overall quality and correctness of the output; b) an embedding-based reward to ensure semantic alignment; c) a format reward to enforce the desired output structure (<think> and <answer> blocks); and d) a modality reward to ensure the response is grounded in the specified imaging modality. This reward-guided approach encourages the model to generate accurate and interpretable reasoning paths.

How the training signal is built

The composite reward is defined as r = wfmt Rformat + wllm Rllm + wemb Rembed + wmod Rmodality. The specific coefficients were selected through a staged procedure, fixing the format reward weight to 0. p. 3

Where the paper says it

Reward Design We define a composite reward: r = wfmt Rformat + wllm Rllm + wemb Rembed + wmod Rmodality with coefficients chosen via a simple, staged ablation procedure. In all experiments we keep a small, fixed weight on the format reward to enforce parseable outputs, and distribute the remaining weight across semantic correctness (LLM judge / embeddings) and modality grounding.

10 and allocating the remaining 0. p. 1
Where the paper says it

Moreover, using a single reward signal can induce instability or reward hacking, especially when the signal is noisy or overly permissive. Hence, it is desirable to have a principled approach for training medical MLLMs with open-ended RL that integrates semantic correctness with structural and modality constraints, while remaining data- and compute-efficient.

90 mass to task-facing signals. p. 7
Where the paper says it

10 in all experiments and allocated the remaining 0. 90 mass to the task-facing signals.

The LLM-Based Accuracy Reward (Rllm) enforces a strict YES/NO decision on semantic correctness using a local vLLM endpoint. p. 1
Where the paper says it

We present MediX-R1, an open-ended medical RL framework that fine-tunes a baseline multimodal backbone with Group Based RL (GRPO/GSPO/DAPO) using a composite reward tailored for clinical reasoning. Our design combines: (1) an LLM-based accuracy reward that enforces a strict YES/NO decision on semantic correctness, (2) a medical embedding-based semantic reward that captures paraphrases and terminology variants, (3) a lightweight format reward that elicits interpretable reasoning traces, and (4) a modality recognition reward that discourages crossmodality hallucinations by requiring explicit modality tags.

The Embedding-Based Reward (Rembed) computes cosine similarity between the predicted answer and the reference using a medical embedding model. p. 4
Where the paper says it

This captures correctness and robustness to paraphrasing while keeping the signal discrete. Embedding-Based Reward (Rembed) To further encourage semantic alignment, we compute cosine similarity between the predicted answer and the reference using a medical embedding model MedEmbed-large (Balachandran, 2024).

Where the gains come from

The gains stem from this multi-signal design which stabilizes optimization and supplies informative feedback where traditional verifiable or MCQ-only rewards fall short. The LLM judge is the strongest single signal for text correctness, and embeddings complement it by reducing false negatives from paraphrases. p. 6

Where the paper says it

5, which shows reduced volatility and fewer signs of reward hacking, these results suggest that the composite reward both improves aggregate performance and stabilizes optimization. Key takeaways: (i) The LLM judge is the strongest single signal for text correctness, and embeddings complement it by reducing false negatives from paraphrases.

Modality recognition is important for VLM tasks and drives substantial gains in image+text tasks. p. 6
Where the paper says it

(ii) Default string matching degrades substantially on open-ended multimodal evaluations. (iii) Modality recognition is important for VLM tasks and drives substantial gains in image+text tasks; the full composite delivers the best overall results.

The full composite design stabilizes optimization and reduces reward hacking, translating into the best average performance in Table 4. p. 1
Where the paper says it

The composite design (format + LLM judge + embeddings + modality recognition) stabilizes optimization and reduces reward hacking (Fig. 5), translating into the best average performance in Table 2.

Evaluation Methodology

To measure progress, a unified, three-stage Reference-based LLM-as-judge evaluation framework is proposed which supports both text-only and image+text tasks under a common protocol. This pipeline involves Generation via vLLM inference, Evaluation using Reference-based LLMas-judge with BASE and MIMIC templates, and Scoring through aggregation of judgment outputs. p. 3

Where the paper says it

0 Figure 3. Evaluation Framework Our three-stage evaluation pipeline: (1) Generation via vLLM inference on the model under test, (2) Evaluation using Reference-based LLMas-judge with BASE and MIMIC templates, and (3) Scoring through aggregation of judgment outputs.

This framework replaces brittle string-overlap metrics with instruction-tuned judges served via vLLM to capture semantic correctness, reasoning adequacy, and contextual alignment. p. 2
Where the paper says it

By replacing brittle string-overlap metrics with instruction-tuned judges served via vLLM (Kwon et al. , 2023), our evaluation captures semantic correctness, reasoning adequacy, and contextual alignment, and scales from short-form QA to long-form report generation.

Performance and Human Validation

MediX-R1 achieves the highest average score across all benchmarks, demonstrating state-of-the-art performance among open models. Human expert preference studies further corroborate its clinical answer quality, with MediX-R1 being selected as the best response in 72. p. 8

Where the paper says it

Empirically, MediX-R1 achieves strong results across diverse medical benchmarks and shows improved stability and resistance to reward hacking compared to single-signal RL variants. Human expert preference studies further corroborate its clinical answer quality, while qualitative examples illustrate faithful grounding and interpretable reasoning traces.

7% of cases. p. 8
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning The results demonstrate a strong preference for MediX-R1, which was selected as the best response in 72. 7% of the cases.

Qualitative examples illustrate faithful grounding and interpretable reasoning traces across modalities. p. 8
Where the paper says it

Empirically, MediX-R1 achieves strong results across diverse medical benchmarks and shows improved stability and resistance to reward hacking compared to single-signal RL variants. Human expert preference studies further corroborate its clinical answer quality, while qualitative examples illustrate faithful grounding and interpretable reasoning traces.

The results on real-world clinical data, such as MedPix 2. p. 8
Where the paper says it

Additional details on human expert evaluation is available in Sec. Evaluation on Real World Clinical Data To further assess the generalization ability of our model, we conducted additional evaluation on MedPix 2.

0, show that MediX-R1 consistently outperforms other medical visionlanguage models. p. 8
Where the paper says it

The evaluation on MedPix 2. 0 demonstrates that our model, MediX-R1, consistently outperforms other medical visionlanguage models.

Mitigation of Hacking and Bias

The composite objective mitigates reward hacking by penalizing mismatches in content or modality recognition. Specific mitigation strategies include embedding gating to set Rembed=0 for low-quality answers, and structural control enforced by Rformat. p. 7

Where the paper says it

” Mitigation in MediX-R1 To curb these failures, MediX-R1 employs a composite reward and input/output constraints: (i) Composite objective: Rllm + Rembed + Rmodality (with shared Rformat) reduces reliance on any single brittle signal and penalizes mismatches in content or modality recognition (Table 4). (ii) Embedding gating: set Rembed=0 for answers below a minimum character/word length, with high punctuation or non-alphanumeric ratio; strip punctuation before embedding; calibrate the similarity threshold.

The paper notes that the full composite reward yields the best overall results and shows reduced volatility and fewer signs of reward hacking compared to single-signal RL variants. p. 6
Where the paper says it

5, which shows reduced volatility and fewer signs of reward hacking, these results suggest that the composite reward both improves aggregate performance and stabilizes optimization. Key takeaways: (i) The LLM judge is the strongest single signal for text correctness, and embeddings complement it by reducing false negatives from paraphrases.

The work also highlights risks of demographic and dataset bias, suggesting future work should include fairness analyses. p. 9
Where the paper says it

While we used only publicly available, de-identified datasets under their respective licenses and did not conduct a prospective human-subjects study, residual privacy risks can remain; downstream users should perform auditing prior to redistribution or deployment. We also highlight risks of demographic and dataset bias and potential amplification of health disparities; future work should include fairness analyses where legally and ethically permissible, uncertainty calibration, bias-aware reward shaping, and clinician-in-the-loop evaluation.

Conclusion

Together, the framework demonstrates that carefully composed, structure-aware rewards plus standardized LLM-judge evaluation provide a practical path to scalable and interpretable medical multimodal RL. MediX-R1 achieves strong results across diverse medical benchmarks and shows improved stability and resistance to reward hacking compared to single-signal RL variants. p. 8

Where the paper says it

A unified vLLM-based evaluation pipeline enables consistent, paraphrase-robust scoring across both text-only and image+text tasks. Empirically, MediX-R1 achieves strong results across diverse medical benchmarks and shows improved stability and resistance to reward hacking compared to single-signal RL variants.

The work advances methods for open-ended RL of medical MLLMs, with the goal of improving semantic correctness under paraphrase, format compliance, and modality grounding in medical question answering and report-style tasks. p. 8
Where the paper says it

Altogether, the framework demonstrates that carefully composed, structure-aware rewards plus standardized LLM-judge evaluation provide a practical path to scalable and interpretable medical multimodal RL fine-tuning. Impact Statement This work advances methods for open-ended RL of medical MLLMs, with the goal of improving semantic correctness under paraphrase, format compliance, and modality grounding in medical question answering and report-style tasks.

It is recommended that downstream users perform auditing prior to redistribution or deployment. p. 9
Where the paper says it

Misuse risks include self-diagnosis, generation of misleading medical narratives, and adversarial prompting with malicious intent. While we used only publicly available, de-identified datasets under their respective licenses and did not conduct a prospective human-subjects study, residual privacy risks can remain; downstream users should perform auditing prior to redistribution or deployment.

The authors commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates under a CC-BY-NC-SA 4. p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

0 license.

Acknowledgments

This work was partially supported with NVIDIA Academic Grant 2025 and MBZUAI-IITD Research Collaboration Seed Grant. The authors disclose their use of generative AI tools, stating that assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually. p. 9

Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

The work is a research prototype and not intended for clinical or commercial deployment. p. 8
Where the paper says it

At the same time, the societal and ethical risks are nontrivial. MediX-R1 is a research prototype and is not intended for clinical or commercial deployment.

References

Balachandran, A. Medembed: Medical-focused embedding models, 2024. p. 9

Where the paper says it

References Balachandran, A. Medembed: Medical-focused embedding models, 2024.

p. 9
Where the paper says it

References Balachandran, A. Medembed: Medical-focused embedding models, 2024.

URL https://github. p. 9
Where the paper says it

Medembed: Medical-focused embedding models, 2024. URL https://github.

p. 9
Where the paper says it

Medembed: Medical-focused embedding models, 2024. URL https://github.

com/abhinand5/MedEmbed. p. 9
Where the paper says it

URL https://github. com/ abhinand5/MedEmbed.

p. 9
Where the paper says it

URL https://github. com/ abhinand5/MedEmbed.

Chen, J., Cai, Z., Ji, K., Wang, X., Liu, W., Wang, R., Hou, J., and Wang, B. p. 9
Where the paper says it

com/ abhinand5/MedEmbed. , and Wang, B.

Huatuogpt-o1. p. 0
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Sahal Shaji Mullappilly * 1 Mohammed Irfan Kurpath * 1 Omair Mohamed 2 Mohamed Zidan 3 Fahad Khan 1 Salman Khan 1 Rao Anwer 1 Hisham Cholakkal 1 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) 2 Jubilee Mission Medical College and Research Institute, 3 JJM Medical College 40% 45% 50% 55% 60% 65% 70% 75% Av era g e A c c ura c y (%) 10K 50K 200K 1M 5M 30M Training Dataset Size Params: ~2B ~8B ~30B = params size Data Available Data Not Available MediX-R1 MedGemma MedMO HuatuoGPT BiMediX2 MedVLM Less data, higher accuracy MedGemma 27B 68. 4% MedMO 8B 62.

Chen, J., Gui, C., Ouyang, R. p. 9
Where the paper says it

org/abs/2412. , Ouyang, R.

, Gao, A., Chen, S., Chen G. H.: towards injecting medical visual knowledge into multimodal llms at scale. p. 9
Where the paper says it

, and Wang, B. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale, 2024b.

Deria, A., Kumar, K. p. 9
Where the paper says it

org/abs/ 2406. , Kumar, K.

p. 9
Where the paper says it

org/abs/ 2406. , Kumar, K.

, Dukre, A. p. 9
Where the paper says it

, Kumar, K. , Dukre, A.

M.: Medmo: Grounding and understanding multimodal large language model for medical images. p. 9
Where the paper says it

, and Razzak, I. Medmo: Grounding and understanding multimodal large language model for medical images, 2026.

Guo, D., Yang, D., Zhang, H. p. 4
Where the paper says it

, 2018), PathVQA (He et al. , 2020), PMC-VQA (Zhang et al.

: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. p. 9
Where the paper says it

, Zhang, R. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.

Han, T., Adams, L. p. 9
Where the paper says it

12948, 2025. , Adams, L.

p. 9
Where the paper says it

12948, 2025. , Adams, L.

C.: Medalpaca–an open-source collection of medical conversational ai models and training data. p. 9
Where the paper says it

, and Bressem, ¨ K. Medalpaca–an open-source collection of medical conversational ai models and training data.

p. 9
Where the paper says it

, and Bressem, ¨ K. Medalpaca–an open-source collection of medical conversational ai models and training data.

He, X., Zhang, Y. p. 4
Where the paper says it

, 2018), PathVQA (He et al. , 2020), PMC-VQA (Zhang et al.

p. 4
Where the paper says it

, 2018), PathVQA (He et al. , 2020), PMC-VQA (Zhang et al.

, Mou, L.: Pathvqa: 30000+ questions for medical visual question answering. p. 9
Where the paper says it

, and Xie, P. Pathvqa: 30000+ questions for medical visual question answering, 2020.

Hendrycks, D. p. 4
Where the paper says it

The evaluation includes standard medical QA, multiple-choice, and open-ended report generation, as well as visual question answering and clinical image interpretation. The datasets used for evaluation are as follows: LLM (text-only) benchmarks: MMLU-Clinical, MMLUBio, MMLU-Med, MMLU-Genetics, MMLU-ProfMed, MMLU-Anat (Hendrycks et al.

, Burns, C. p. 9
Where the paper says it

Hendrycks, D. , Burns, C.

p. 9
Where the paper says it

Hendrycks, D. , Burns, C.

: Measuring massive multitask language understanding. p. 9
Where the paper says it

, and Steinhardt, J. Measuring massive multitask language understanding.

p. 9
Where the paper says it

, and Steinhardt, J. Measuring massive multitask language understanding.

Henigman, A. p. 8
Where the paper says it

0 (Siragusa et al. , 2025), a publicly available real-world clinical VQA dataset derived from the original MedPix (Henigman & Kennedy, 2025) database maintained by the U.

p. 8
Where the paper says it

0 (Siragusa et al. , 2025), a publicly available real-world clinical VQA dataset derived from the original MedPix (Henigman & Kennedy, 2025) database maintained by the U.

and Kennedy, B. p. 8
Where the paper says it

0 (Siragusa et al. , 2025), a publicly available real-world clinical VQA dataset derived from the original MedPix (Henigman & Kennedy, 2025) database maintained by the U.

p. 8
Where the paper says it

0 (Siragusa et al. , 2025), a publicly available real-world clinical VQA dataset derived from the original MedPix (Henigman & Kennedy, 2025) database maintained by the U.

: Medpix®: database of medical images, teaching cases, and clinical topics. p. 9
Where the paper says it

and Kennedy, B. Medpix®: database of medical images, teaching cases, and clinical topics.

p. 9
Where the paper says it

and Kennedy, B. Medpix®: database of medical images, teaching cases, and clinical topics.

Jin, D., Pan, E., Oufattole, N. p. 9
Where the paper says it

Medical Reference Services Quarterly, 44(3):328–333, 2025. , Oufattole, N.

: What disease does this patient have? p. 9
Where the paper says it

, and Szolovits, P. What disease does this patient have?

p. 9
Where the paper says it

, and Szolovits, P. What disease does this patient have?

a large-scale open domain question answering dataset from medical exams. p. 9
Where the paper says it

What disease does this patient have? a large-scale open domain question answering dataset from medical exams.

p. 9
Where the paper says it

What disease does this patient have? a large-scale open domain question answering dataset from medical exams.

Johnson, A. p. 4
Where the paper says it

, 2023), PubMedQA (Jin et al. , 2019), MIMIC-CXRSummarization (Johnson et al.

p. 4
Where the paper says it

, 2023), PubMedQA (Jin et al. , 2019), MIMIC-CXRSummarization (Johnson et al.

E., Pollard, T. p. 9
Where the paper says it

Johnson, A. , Pollard, T.

p. 9
Where the paper says it

Johnson, A. , Pollard, T.

J.: Mimic-iii: a freely accessible critical care database. p. 9
Where the paper says it

, and Mark, R. Mimic-iii, a freely accessible critical care database.

p. 9
Where the paper says it

, and Mark, R. Mimic-iii, a freely accessible critical care database.

Kwon, W., Li, Z.: Efficient memory management for large language model serving with pagedattention. p. 9
Where the paper says it

, and Stoica, I. Efficient memory management for large language model serving with pagedattention.

p. 9
Where the paper says it

, and Stoica, I. Efficient memory management for large language model serving with pagedattention.

Lau, J. J., Gayen, S. p. 9
Where the paper says it

In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. , Gayen, S.

p. 9
Where the paper says it

In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. , Gayen, S.

: A dataset of clinically generated visual questions and answers about radiology images. p. 9
Where the paper says it

, and DemnerFushman, D. A dataset of clinically generated visual questions and answers about radiology images.

p. 9
Where the paper says it

, and DemnerFushman, D. A dataset of clinically generated visual questions and answers about radiology images.

Marafioti, A. p. 2
Where the paper says it

Moreover, we have conducted experiments on different baseline VLMs, including Qwen2. 5-VL, Qwen3-VL (Team, 2025), and SmolVLM2 (Marafioti et al.

p. 2
Where the paper says it

Moreover, we have conducted experiments on different baseline VLMs, including Qwen2. 5-VL, Qwen3-VL (Team, 2025), and SmolVLM2 (Marafioti et al.

, Zohar, O.: Smolvlm: Redefining small and efficient multimodal models. p. 1
Where the paper says it

Hence, it is desirable to have a principled approach for training medical MLLMs with open-ended RL that integrates semantic correctness with structural and modality constraints, while remaining data- and compute-efficient. We present MediX-R1, an open-ended medical RL framework that fine-tunes a baseline multimodal backbone with Group Based RL (GRPO/GSPO/DAPO) using a composite reward tailored for clinical reasoning.

p. 1
Where the paper says it

Hence, it is desirable to have a principled approach for training medical MLLMs with open-ended RL that integrates semantic correctness with structural and modality constraints, while remaining data- and compute-efficient. We present MediX-R1, an open-ended medical RL framework that fine-tunes a baseline multimodal backbone with Group Based RL (GRPO/GSPO/DAPO) using a composite reward tailored for clinical reasoning.

Mullappilly, S. p. 0
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Sahal Shaji Mullappilly * 1 Mohammed Irfan Kurpath * 1 Omair Mohamed 2 Mohamed Zidan 3 Fahad Khan 1 Salman Khan 1 Rao Anwer 1 Hisham Cholakkal 1 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) 2 Jubilee Mission Medical College and Research Institute, 3 JJM Medical College 40% 45% 50% 55% 60% 65% 70% 75% Av era g e A c c ura c y (%) 10K 50K 200K 1M 5M 30M Training Dataset Size Params: ~2B ~8B ~30B = params size Data Available Data Not Available MediX-R1 MedGemma MedMO HuatuoGPT BiMediX2 MedVLM Less data, higher accuracy MedGemma 27B 68. 4% MedMO 8B 62.

p. 0
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Sahal Shaji Mullappilly * 1 Mohammed Irfan Kurpath * 1 Omair Mohamed 2 Mohamed Zidan 3 Fahad Khan 1 Salman Khan 1 Rao Anwer 1 Hisham Cholakkal 1 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) 2 Jubilee Mission Medical College and Research Institute, 3 JJM Medical College 40% 45% 50% 55% 60% 65% 70% 75% Av era g e A c c ura c y (%) 10K 50K 200K 1M 5M 30M Training Dataset Size Params: ~2B ~8B ~30B = params size Data Available Data Not Available MediX-R1 MedGemma MedMO HuatuoGPT BiMediX2 MedVLM Less data, higher accuracy MedGemma 27B 68. 4% MedMO 8B 62.

S., Kurpath, M. p. 0
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Sahal Shaji Mullappilly * 1 Mohammed Irfan Kurpath * 1 Omair Mohamed 2 Mohamed Zidan 3 Fahad Khan 1 Salman Khan 1 Rao Anwer 1 Hisham Cholakkal 1 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) 2 Jubilee Mission Medical College and Research Institute, 3 JJM Medical College 40% 45% 50% 55% 60% 65% 70% 75% Av era g e A c c ura c y (%) 10K 50K 200K 1M 5M 30M Training Dataset Size Params: ~2B ~8B ~30B = params size Data Available Data Not Available MediX-R1 MedGemma MedMO HuatuoGPT BiMediX2 MedVLM Less data, higher accuracy MedGemma 27B 68. 4% MedMO 8B 62.

p. 0
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Sahal Shaji Mullappilly * 1 Mohammed Irfan Kurpath * 1 Omair Mohamed 2 Mohamed Zidan 3 Fahad Khan 1 Salman Khan 1 Rao Anwer 1 Hisham Cholakkal 1 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) 2 Jubilee Mission Medical College and Research Institute, 3 JJM Medical College 40% 45% 50% 55% 60% 65% 70% 75% Av era g e A c c ura c y (%) 10K 50K 200K 1M 5M 30M Training Dataset Size Params: ~2B ~8B ~30B = params size Data Available Data Not Available MediX-R1 MedGemma MedMO HuatuoGPT BiMediX2 MedVLM Less data, higher accuracy MedGemma 27B 68. 4% MedMO 8B 62.

I.: Bimedix2: Biomedical expert lmm for diverse medical modalities. p. 1
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Model Diverse Medical Single-Stage Interpretable Open-Ended Annotation-Free Composite Modalities RL Reasoning Responses Reasoning RL Reward MedVLM-R1 (Pan et al. , 2025) ✗ ✓ ✓ ✗ ✓ ✗ BiMediX2 (Mullappilly et al.

p. 1
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Model Diverse Medical Single-Stage Interpretable Open-Ended Annotation-Free Composite Modalities RL Reasoning Responses Reasoning RL Reward MedVLM-R1 (Pan et al. , 2025) ✗ ✓ ✓ ✗ ✓ ✗ BiMediX2 (Mullappilly et al.

Pal, A., Umapathi, L. K.: Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. p. 9
Where the paper says it

What disease does this patient have? a large-scale open domain question answering dataset from medical exams.

p. 9
Where the paper says it

What disease does this patient have? a large-scale open domain question answering dataset from medical exams.

Pan, J., Liu, C.: Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. p. 9
Where the paper says it

, Zhang, R. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.

p. 9
Where the paper says it

, Zhang, R. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.

Pieri, S., Mullappilly, S. p. 0
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Sahal Shaji Mullappilly * 1 Mohammed Irfan Kurpath * 1 Omair Mohamed 2 Mohamed Zidan 3 Fahad Khan 1 Salman Khan 1 Rao Anwer 1 Hisham Cholakkal 1 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) 2 Jubilee Mission Medical College and Research Institute, 3 JJM Medical College 40% 45% 50% 55% 60% 65% 70% 75% Av era g e A c c ura c y (%) 10K 50K 200K 1M 5M 30M Training Dataset Size Params: ~2B ~8B ~30B = params size Data Available Data Not Available MediX-R1 MedGemma MedMO HuatuoGPT BiMediX2 MedVLM Less data, higher accuracy MedGemma 27B 68. 4% MedMO 8B 62.

p. 0
Where the paper says it

MediX-R1: Open Ended Medical Reinforcement Learning Sahal Shaji Mullappilly * 1 Mohammed Irfan Kurpath * 1 Omair Mohamed 2 Mohamed Zidan 3 Fahad Khan 1 Salman Khan 1 Rao Anwer 1 Hisham Cholakkal 1 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) 2 Jubilee Mission Medical College and Research Institute, 3 JJM Medical College 40% 45% 50% 55% 60% 65% 70% 75% Av era g e A c c ura c y (%) 10K 50K 200K 1M 5M 30M Training Dataset Size Params: ~2B ~8B ~30B = params size Data Available Data Not Available MediX-R1 MedGemma MedMO HuatuoGPT BiMediX2 MedVLM Less data, higher accuracy MedGemma 27B 68. 4% MedMO 8B 62.

S.: BiMediX: Bilingual medical mixture of experts LLM. p. 7
Where the paper says it

The outputs were anonymized and labeled as Model A, Model B, Model C and Model D with no identifiers provided to the reviewers. Medical experts were asked to evaluate the responses against the provided ground truth descriptions for each question.

p. 7
Where the paper says it

The outputs were anonymized and labeled as Model A, Model B, Model C and Model D with no identifiers provided to the reviewers. Medical experts were asked to evaluate the responses against the provided ground truth descriptions for each question.

Sellergren, A. p. 1
Where the paper says it

, 2024) ✓ ✗ ✗ ✓ ✗ ✗ HuatuoGPT-V (Chen et al. , 2024b) ✓ ✗ ✗ ✓ ✗ ✗ MedGemma (Sellergren et al.

p. 1
Where the paper says it

, 2024) ✓ ✗ ✗ ✓ ✗ ✗ HuatuoGPT-V (Chen et al. , 2024b) ✓ ✗ ✗ ✓ ✗ ✗ MedGemma (Sellergren et al.

, Kazemzadeh, S.: Medgemma technical report. p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

Shao, Z., Wang, P.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. p. 0
Where the paper says it

Abstract We introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical multimodal large language models (MLLMs) that enables clinically grounded, free-form answers beyond multiple-choice formats. MediX-R1 finetunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning: an LLM-based accuracy reward that judges semantic correctness with a strict YES/NO decision, a medical embeddingbased semantic reward to capture paraphrases and terminology variants, and lightweight format and modality rewards that enforce interpretable * Equal contribution.

p. 0
Where the paper says it

Abstract We introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical multimodal large language models (MLLMs) that enables clinically grounded, free-form answers beyond multiple-choice formats. MediX-R1 finetunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning: an LLM-based accuracy reward that judges semantic correctness with a strict YES/NO decision, a medical embeddingbased semantic reward to capture paraphrases and terminology variants, and lightweight format and modality rewards that enforce interpretable * Equal contribution.

Siragusa, I. p. 8
Where the paper says it

Evaluation on Real World Clinical Data To further assess the generalization ability of our model, we conducted additional evaluation on MedPix 2. 0 (Siragusa et al.

p. 8
Where the paper says it

Evaluation on Real World Clinical Data To further assess the generalization ability of our model, we conducted additional evaluation on MedPix 2. 0 (Siragusa et al.

, Contino, S.: Medpix 2. p. 8
Where the paper says it

Additional details on human expert evaluation is available in Sec. Evaluation on Real World Clinical Data To further assess the generalization ability of our model, we conducted additional evaluation on MedPix 2.

p. 8
Where the paper says it

Additional details on human expert evaluation is available in Sec. Evaluation on Real World Clinical Data To further assess the generalization ability of our model, we conducted additional evaluation on MedPix 2.

0: A comprehensive multimodal biomedical data set for advanced ai applications with retrieval augmented generation and knowledge graphs. Team, L., Xu, W.: Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning. p. 9
Where the paper says it

, and Razzak, I. Medmo: Grounding and understanding multimodal large language model for medical images, 2026.

p. 9
Where the paper says it

, and Razzak, I. Medmo: Grounding and understanding multimodal large language model for medical images, 2026.

Team, Q.: Qwen3 technical report. p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

p. 9
Where the paper says it

To support responsible use and scrutiny, we commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates, alongside a model card and clear usage restrictions, under a CC-BY-NC-SA 4. We also disclose our use of generative AI tools: assisted coding was limited to boilerplate scaffolding and minor refactors with all algorithmic logic authored and reviewed manually; writing-support models were used for grammar and style, while all technical claims and results were verified by the authors.

Yang, A., Li, A.: Qwen3 technical report. Yu, Q., Zhang, Z. p. 4
Where the paper says it

, 2018), PathVQA (He et al. , 2020), PMC-VQA (Zhang et al.

p. 4
Where the paper says it

, 2018), PathVQA (He et al. , 2020), PMC-VQA (Zhang et al.

p. 4
Where the paper says it

, 2018), PathVQA (He et al. , 2020), PMC-VQA (Zhang et al.

p. 4
Where the paper says it

, 2018), PathVQA (He et al. , 2020), PMC-VQA (Zhang et al.

: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. p. 5
Where the paper says it

These results highlight the effectiveness of our open-ended RL training and composite reward design, which enable MediX-R1 to generate accurate, semantically aligned, and clinically grounded responses beyond the capabilities of models trained only with supervised or MCQ objectives. We additionally report results on the Massive Multidiscipline Multimodal Understanding and Reasoning (MMMU) benchmark (Yue et al.

p. 5
Where the paper says it

These results highlight the effectiveness of our open-ended RL training and composite reward design, which enable MediX-R1 to generate accurate, semantically aligned, and clinically grounded responses beyond the capabilities of models trained only with supervised or MCQ objectives. We additionally report results on the Massive Multidiscipline Multimodal Understanding and Reasoning (MMMU) benchmark (Yue et al.

p. 5
Where the paper says it

These results highlight the effectiveness of our open-ended RL training and composite reward design, which enable MediX-R1 to generate accurate, semantically aligned, and clinically grounded responses beyond the capabilities of models trained only with supervised or MCQ objectives. We additionally report results on the Massive Multidiscipline Multimodal Understanding and Reasoning (MMMU) benchmark (Yue et al.

p. 5
Where the paper says it

These results highlight the effectiveness of our open-ended RL training and composite reward design, which enable MediX-R1 to generate accurate, semantically aligned, and clinically grounded responses beyond the capabilities of models trained only with supervised or MCQ objectives. We additionally report results on the Massive Multidiscipline Multimodal Understanding and Reasoning (MMMU) benchmark (Yue et al.

Yang, B., Hui, B.: Qwen3 technical report. Yu, Q., Zhang, Z.: Dapo: An open-source llm reinforcement learning system at scale. p. 2
Where the paper says it

Embedding Reward LLM Reward Modality Reward Format Reward think answer MRI XRAY CT MediX-R1 GRPO DAPO GSPO Figure 2. MediX-R1: Overall Architecture The MediX-R1 reinforcement learning framework for open-ended medical reasoning.

p. 2
Where the paper says it

Embedding Reward LLM Reward Modality Reward Format Reward think answer MRI XRAY CT MediX-R1 GRPO DAPO GSPO Figure 2. MediX-R1: Overall Architecture The MediX-R1 reinforcement learning framework for open-ended medical reasoning.

Yue, X., Ni, Y.: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. Zhang, X., Wu, C.: Pmc-vqa: Visual instruction tuning for medical visual question answering. p. 4
Where the paper says it

State-of-the-art Comparisons We evaluate MediX-R1 on a comprehensive suite of medical language and vision-language benchmarks, covering both text-only (LLM) and image+text (VLM) tasks. The evaluation includes standard medical QA, multiple-choice, and open-ended report generation, as well as visual question answering and clinical image interpretation.

p. 4
Where the paper says it

State-of-the-art Comparisons We evaluate MediX-R1 on a comprehensive suite of medical language and vision-language benchmarks, covering both text-only (LLM) and image+text (VLM) tasks. The evaluation includes standard medical QA, multiple-choice, and open-ended report generation, as well as visual question answering and clinical image interpretation.

Zheng, C., Liu, S.: Group sequence policy optimization. p. 2
Where the paper says it

An input of a medical image and a natural language question is processed by MediX-R1. The model’s policy is trained using Group Based RL, which leverages a multi-faceted reward signal.

p. 2
Where the paper says it

An input of a medical image and a natural language question is processed by MediX-R1. The model’s policy is trained using Group Based RL, which leverages a multi-faceted reward signal.

Zheng, Y., Lu, J.: Easyr1: An efficient, scalable multi-modality rl training framework. p. 1
Where the paper says it

Hence, it is desirable to have a principled approach for training medical MLLMs with open-ended RL that integrates semantic correctness with structural and modality constraints, while remaining data- and compute-efficient. We present MediX-R1, an open-ended medical RL framework that fine-tunes a baseline multimodal backbone with Group Based RL (GRPO/GSPO/DAPO) using a composite reward tailored for clinical reasoning.

p. 1
Where the paper says it

Hence, it is desirable to have a principled approach for training medical MLLMs with open-ended RL that integrates semantic correctness with structural and modality constraints, while remaining data- and compute-efficient. We present MediX-R1, an open-ended medical RL framework that fine-tunes a baseline multimodal backbone with Group Based RL (GRPO/GSPO/DAPO) using a composite reward tailored for clinical reasoning.

Appendix

A.1 Training Data and Modality Distribution

We trained MediX-R1 on 51335 multimodal medical instruction samples spanning 16 modality tags. Table 8 details the Modality Breakdown and Source Dataset composition of these samples. p. 4

Where the paper says it

For each dataset, we follow the evaluation protocol described in the previous section, using Reference-based LLMas-judge scoring for both short-form and long-form responses. Table 2 summarizes performance on our unified benchmark suite across several open-source medical models, including MedVLM-R1 (2B), BiMediX2 (8B), HuatuoGPTV (7B), MedGemma (4B/27B), MedMO (8B) (Deria et al.

A.4 Reward Function Source Code

The Format reward function checks for the presence of the required structural pattern. The LLM-based accuracy reward compares the predicted answer against the ground truth using an external helper. p. 4

Where the paper says it

” with ground truth “Low perfusion” a model response that includes hidden reasoning and the final answer “Low blood flow or less perfusion” is judged correct and assigned a score of 1. The judge compares predicted answers against references, accounting for paraphrase and clinically equivalent phrasing.

The Embedding-based semantic reward computes cosine similarity between embeddings and applies a threshold. p. 4
Where the paper says it

Embedding-Based Reward (Rembed) To further encourage semantic alignment, we compute cosine similarity between the predicted answer and the reference using a medical embedding model MedEmbed-large (Balachandran, 2024). We convert it to a binary reward via a threshold (default 0.

The Modality recognition reward requires the model to emit a correct modality tag before the thinking block. p. 4
Where the paper says it

This stabilizes training and improves interpretability of the medical reasoning. Modality recognition Reward (Rmodality) We encourage explicit grounding to the imaging modality by requiring the model to emit the predicted modality tag before the </think> block (case-insensitive).

A.7 Reinforcement Learning Training Prompt

The RL training prompt enforces an explicit modality tag, structured reasoning in tags, and a concise final answer in tags. This structure aligns with the format reward (Rformat) and modality reward (Rmodality) in our composite objective. p. 7

Where the paper says it

0) against the reference “Lung. ” Mitigation in MediX-R1 To curb these failures, MediX-R1 employs a composite reward and input/output constraints: (i) Composite objective: Rllm + Rembed + Rmodality (with shared Rformat) reduces reliance on any single brittle signal and penalizes mismatches in content or modality recognition (Table 4).

The final decision is evaluated solely from for Rllm and Rembed. p. 7
Where the paper says it

1Rformat + 0. For the combined semantic reward, we evaluated a small set of intuitive splits between Rllm and Rembed while keeping wfmt fixed; performance was similar across these choices on our validation benchmark, and we selected the configuration that slightly favored the LLM judge.

The RL training prompt enforces (i) an explicit modality tag, (ii) structured reasoning in tags, and (iii) a concise final answer in tags. p. 1
Where the paper says it

We present MediX-R1, an open-ended medical RL framework that fine-tunes a baseline multimodal backbone with Group Based RL (GRPO/GSPO/DAPO) using a composite reward tailored for clinical reasoning. Our design combines: (1) an LLM-based accuracy reward that enforces a strict YES/NO decision on semantic correctness, (2) a medical embedding-based semantic reward that captures paraphrases and terminology variants, (3) a lightweight format reward that elicits interpretable reasoning traces, and (4) a modality recognition reward that discourages crossmodality hallucinations by requiring explicit modality tags.

The structure aligns with the format reward (Rformat) and modality reward (Rmodality) in our composite objective. p. 7
Where the paper says it

0) against the reference “Lung. ” Mitigation in MediX-R1 To curb these failures, MediX-R1 employs a composite reward and input/output constraints: (i) Composite objective: Rllm + Rembed + Rmodality (with shared Rformat) reduces reliance on any single brittle signal and penalizes mismatches in content or modality recognition (Table 4).

Impact Statement

This work advances methods for open-ended RL of medical MLLMs, with the goal of improving semantic correctness under paraphrase, format compliance, and modality grounding in medical question answering and report-style tasks. If used appropriately, the proposed composite-reward RL and unified Referencebased LLM-as-judge evaluation framework may reduce reliance on brittle string-match metrics. p. 8

Where the paper says it

If used appropriately (e. , as a research and education aid), the proposed composite-reward RL and unified Referencebased LLM-as-judge evaluation framework may reduce reliance on brittle string-match metrics, enable more realistic benchmarking of free-form clinical responses, and improve transparency through structured outputs (modality tags and separated reasoning/answer blocks).

The work is a research prototype and is not intended for clinical or commercial deployment. p. 8
Where the paper says it

At the same time, the societal and ethical risks are nontrivial. MediX-R1 is a research prototype and is not intended for clinical or commercial deployment.

The authors commit to releasing training/inference code, configurations, checkpoints, curated datasets, and RL/evaluation prompt templates under a CC-BY-NC-SA 4.0 license.

Balachandran, A. Medembed: Medical-focused embedding models, 2024. URL https://github.com/abhinand5/MedEmbed. Chen, J., Cai, Z., Ji, K., Wang, X.: towards medical complex reasoning with llms Chen, J. p. 9

Where the paper says it

, and Wang, B. Huatuogpt-o1, towards medical complex reasoning with llms, 2024a.

, Gui, C.: towards injecting medical visual knowledge into multimodal llms at scale Deria, A. p. 9
Where the paper says it

, and Wang, B. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale, 2024b.

, Kumar, K.: Medmo: Grounding and understanding multimodal large language model for medical images Guo, D. p. 9
Where the paper says it

, and Razzak, I. Medmo: Grounding and understanding multimodal large language model for medical images, 2026.

, Yang, D.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning Han, T. p. 9
Where the paper says it

, Zhang, R. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.

, Adams, L. C.: Medalpaca–an open-source collection of medical conversational ai models and training data. He, X., Zhang, Y.: Pathvqa: 30000+ questions for medical visual question answering Hendrycks, D. p. 9
Where the paper says it

, and Xie, P. Pathvqa: 30000+ questions for medical visual question answering, 2020.

, Burns, C.: Measuring massive multitask language understanding. Henigman, A. and Kennedy, B.: Medpix®: database of medical images, teaching cases, and clinical topics. Jin, D., Pan, E.: What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Johnson, A. E., Pollard, T. J.: Mimic-iii: a freely accessible critical care database. Kwon, W., Li, Z.: Efficient memory management for large language model serving with pagedattention. Lau, J. J., Gayen, S.: A dataset of clinically generated visual questions and answers about radiology images. Marafioti, A., Zohar, O.: Smolvlm: Redefining small and efficient multimodal models. Mullappilly, S. S., Kurpath, M. I.: Bimedix2: Biomedical expert lmm for diverse medical modalities. Pal, A., Umapathi, L. K.: Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. Pan, J., Liu, C.: Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. Pieri, S., Mullappilly, S. S.: BiMediX: Bilingual medical mixture of experts LLM. Sellergren, A., Kazemzadeh, S.: Medgemma technical report. Shao, Z., Wang, P.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Siragusa, I., Contino, S.: Medpix 2.0: A comprehensive multimodal biomedical data set for advanced ai applications with retrieval augmented generation and knowledge graphs. Team, L., Xu, W.: Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning. Team, Q.: Qwen3 technical report. Yang, A., Li, A.: Qwen3 technical report. Yu, Q., Zhang, Z.: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. Yang, B., Hui, B.: Qwen3 technical report. Yu, Q., Zhang, Z.: Dapo: An open-source llm reinforcement learning system at scale. Yue, X., Ni, Y.: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. Zhang, X., Wu, C.: Pmc-vqa: Visual instruction tuning for medical visual question answering. Zheng, C., Liu, S.: Group sequence policy optimization. Zheng, Y., Lu, J.: Easyr1: An efficient, scalable multi-modality rl training framework.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Open-ended reasoning capability in medical MLLMs can be achieved by fine-tuning a baseline vision-language backbone with Group Based RL using a composite reward tailored for medical reasoning. This enables models to generate clinically grounded, free-form answers beyond multiple-choice formats.

  2. The system can produce structured outputs that enforce interpretability through the use of and blocks, which are stabilized by a lightweight format reward that elicits interpretable reasoning traces.

  3. The model can perform robust semantic alignment across complex medical phrasing by utilizing an embedding-based semantic reward, which captures paraphrases and terminology variants.

  4. Modality grounding can be enforced to prevent hallucinations by requiring explicit modality tags before the thinking block, which discourages cross-modality hallucinations by requiring explicit modality tags.

  5. A unified evaluation framework allows for measuring progress across both text-only and image+text tasks using a Reference-based LLM-as-judge, which captures semantic correctness, reasoning, and contextual alignment in a scalable manner.

  6. The system can generate detailed clinical reports by utilizing an instruction template that forces the model to first output a modality tag, then its thinking process in tags, and finally the answer in tags.

Abstract

In this work, we introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical vision language models (VLMs) that enables free-form answers beyond multiple-choice formats. MediX-R1 fine-tunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning: an LLM-based accuracy reward that judges semantic correctness with a strict YES/NO decision, a medical embedding-based semantic reward to capture paraphrases and terminology variants, and lightweight format and modality rewards that enforce interpretable reasoning and modality recognition. This multi-signal design provides stable, informative feedback for open-ended outputs where traditional verifiable or MCQ-only rewards fall short. To evaluate performance, we employ a unified evaluation framework for both text-only and multimodal (image+text) tasks that uses a Reference-based LLM-as-judge in place of brittle string-overlap metrics, capturing semantic correctness, clinical accuracy, and contextual alignment. Our results show that open-ended RL with comprehensive reward signals can effectively strengthen medical reasoning in multimodal models. Despite using only about51 K instruction examples, MediX-R1 achieves strong performance across standard medical LLM (text-only) and VLM (image + text) benchmarks, outperforming existing open-weight medical VLMs, with large gains particularly on open-ended clinical tasks. These improvements remain consistent across multiple RL algorithms and VLM backbones and are further supported by blinded evaluations with medical experts. Our trained models, curated datasets and source code are available at https://medix.cvmbzuai.com

Sources

Related papers