From Compound Figures to Medical Multi-image Reasoning: Scaling Multimodal Large Language Models with Biomedical Literature
The research addresses the lack of multi-image reasoning in medical AI by using compound figures from biomedical literature as training data. The solution is M3LLM, a model trained through a five-stage instruction generation process that breaks down complex images into sub-tasks. This method enables the model to develop composite understanding, leading to superior performance in real clinical tasks like longitudinal X-ray analysis.