Gaokerena: A Small Persian Medical Language Model Family
cs.CL
Submitted: 2026-08-02
Updated: 2026-10-20
Comments: 29 pages, 9 figures
Code: https://github.com/Mehrdadghassabi/Gaokerena-V
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: The integration of artificial intelligence into medical question-answering systems has advanced rapidly; however, research remains predominantly focused on English, leaving low resource languages
Terminology
Abstract
The integration of artificial intelligence into medical question-answering systems has advanced rapidly; however, research remains predominantly focused on English, leaving low resource languages like Persian significantly underserved. To address this gap, this paper introduces Gaokerena, a novel family of compact Persian medical language models optimized for deployment on consumer grade hardware. As a foundational step toward localized digital healthcare, we first present Gaokerena-V, developed by training a baseline model on a newly curated 90-million-token Persian medical corpus and 20,000 expert-vetted physician Q&A pairs, which improved performance on a translated medical MMLU benchmark from 46.28% to 49.31%. Second, recognizing the critical demands of clinical reasoning, we developed Gaokerena-R by integrating a Chain-of-Thought approach with two novel Reinforcement Learning with AI Feedback (RLAIF) frameworks to optimize preference-based reasoning. Despite utilizing the same baseline architecture and a smaller dataset than Gaokerena-V, Gaokerena-R achieved a superior benchmark score of 52.98%. Furthermore, both models are equipped with custom-developed uncertainty heads that predict the model's confidence in its responses based solely on internal hidden states. While these results demonstrate significant progress in Persian medical language modeling and proactive safety estimation, current performance levels remain insufficient for direct clinical application, highlighting the necessity for further research into robust knowledge acquisition and rigorous safety verification prior to real world deployment.
Sources
- MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
- SINA-BERT: A pre-trained Language Model for Analysis of Medical Texts in Persian
- Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks
- MedMobile: A mobile-sized language model with clinical capabilities
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier
- PersianMind: A Cross-Lingual Persian-English Large Language Model
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The Medical Domain
- Datasets for Large Language Models: A Comprehensive Survey
- DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation
- HuatuoGPT, towards Taming Language Model to Be a Doctor
- MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data
- K-QA: A Real-World Medical Q&A Benchmark
- Measuring Massive Multitask Language Understanding
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- GPT-4 Technical Report
- RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Mistral 7B
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering