Calibration and Conformal Prediction for Medical Question Answering in Arabic and English: Evidence from Two Non-Parallel Benchmarks
cs.LG
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/Mariyaaleem/arabic-english-medqa-uncertainty
Terminology
Sources
- MedAraBench: Large-Scale Arabic Medical Question Answering Dataset and Benchmark
- A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
- Conformal Risk Control
- Selective Classification for Deep Neural Networks
- On Calibration of Modern Neural Networks
- What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams
- Language Models (Mostly) Know What They Know
- Correctness Coverage Evaluation for Medical Multiple-Choice Question Answering Based on the Enhanced Conformal Prediction Framework
- Conformal Prediction with Large Language Models for Multi-Choice Question Answering
- Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty
- Conformal Language Modeling
- Qwen2.5 Technical Report
- Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
- Mind the Gap: Benchmarking LLM Uncertainty and Calibration with Specialty-Aware Clinical QA and Reasoning-Based Behavioural Features
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis