PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology

arXiv:2607.27378 · cs.CV, cs.MM · Submitted 2026-07-29 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology".

Jane: Here's the gist of the paper. Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation<ref:2607.27378#pg2> PanDent introduces a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations<ref:2607.27378#pg6> This dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced dentists and further validated by an oral and maxillofacial radiologist<ref:2607.27378#pg2> Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions<ref:2607.27378#pg2> This design enables evaluation of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings<ref:2607.27378#pg2> Dataset Construction and Curation PanDent is curated to support both rigorous benchmark evaluation and supervised training for dental radiology understanding<ref:2607.27378#pg2> The dataset is built from 9,854 raw annotated OPG records collected from multiple public-source batches and one in-house batch<ref:2607.27378#pg2> Each case is associated with structured annotations covering more than 20 clinically relevant findings observable from OPG images<ref:2607.27378#pg2> The annotations are labeled by four professional dentists with over ten years of clinical experience and further validated by an assistant professor in oral and maxillofacial radiology on a randomly sampled subset, providing clinically reliable supervision before conversion into clinician-guided radiology reports<ref:2607.27378#pg2> The final release contains 9,524 valid OPG cases, including 9,019 public-source cases and 505 in-house cases<ref:2607.27378#pg2> Annotation Structure and Normalization The raw annotations consist of heterogeneous clinical fields, which are consolidated into 21 reportable clinical findings for factual evaluation<ref:2607.27378#pg2> These findings capture dental findings across four aspects: regional abnormalities (e.g., maxillary or mandibular structural conditions), structural and periodontal conditions (e.g., angular and generalized crestal bone loss, furcation involvement), tooth-level pathologies and treatments (e.g., caries, periapical lesions and restorations), and surrounding anatomical structures (e.g., maxillary sinus and temporomandibular joint)<ref:2607.27378#pg2> The annotations are represented in two forms: tooth-index-aware findings and binary regional findings<ref:2607.27378#pg2> Tooth-index-aware findings refer to clinical attributes that require localization to specific teeth under the FDI numbering system<ref:2607.27378#pg2> Binary regional findings describe abnormalities assessed at the anatomical-region level<ref:2607.27378#pg2> These representations capture both tooth-specific and region-level clinical information, providing a structured basis for evaluating structure-language consistency and fine-grained clinical fidelity<ref:2607.27378#pg2> Evaluation Protocol The benchmark evaluates models on 500 held-out cases that are disjoint from the training split<ref:2607.27378#pg2> For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = 1, 2, 3, 4), METEOR, and ROUGE-L<ref:2607.27378#pg2> For clinical visual diagnostic accuracy, evaluation is conducted on attribute-targeted outputs instead of full free-text reports<ref:2607.27378#pg2> The normalized results in Tab.

Tom: 1 demonstrate that this procedure significantly reduces format-induced discrepancies<ref:2607.27378#pg2> Key Findings and Model Performance Experiments show that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, whereas fine-tuning on PanDent significantly improves dental-domain performance and brings model outputs closer to expert-validated annotations<ref:2607.27378#pg2> The results suggest that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis<ref:2607.27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, substantially enhancing visual localization accuracy and diagnostic correctness<ref:2607.27378#pg2> This paper introduces PanDent, a curated benchmark for evaluating multimodal models in dental panoramic radiography, with a focus on fine-grained tooth-level reasoning and structure-language consistency<ref:2607.27378#pg2> Experiments show that current MLLMs can produce fluent reports but still fail to achieve reliable tooth-level reasoning, revealing a substantial gap to expert dental interpretation<ref:2607.27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, visual grounding accuracy, and toothlevel diagnostic correctness<ref:2607.27378#pg2> Despite these advances, the benchmark remains limited by the long-tailed distribution of rare findings and the uneven coverage across certain clinical attributes<ref:2607.27378#pg2> Future work will focus on expanding pathology coverage, improving the representation and sampling of underrepresented attributes, and developing models that explicitly couple language generation with tooth-aware visual grounding<ref:2607.27378#pg2> References Abacha, A.B., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D., Müller, H., 2019.

Jane: VQA-Med: Overview of the medical visual question answering task at ImageCLEF 2019<ref:2607.27378#pg2> Alqaderi, H., Phillips, C., Jones, J.A., Ramchandani, K., Mitwalli, A.H., Youssef, N.A., Emami, S.E., Teixeira, T., Ferreira, P.S., Yousfi, H., Cheela, D., Bahaj, A.: Automated detection of dental caries and bone loss in bitewing radiographs using YOLOv8: A multicenter deep learning study<ref:2607.27378#pg2> Bai, S., et al., 2025.

Tom: Qwen2.5-VL technical report<ref:2607.27378#pg2> Bai, Y., et al., 2024.

Paper summary: Tom: So we're looking at this paper called "PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology." Basically, the authors are saying that right now, when we test multimodal models on panoramic X-rays, there just aren't enough benchmarks that actually reflect what an expert dentist would interpret.

Jane: That sounds like a big problem because if the benchmark isn't fine-grained enough, we can't really tell if an AI is actually getting the right details about individual teeth and not just guessing generally. What are they proposing with this PanDent dataset?

Tom: They’re introducing a large-scale dataset of nine thousand five hundred twenty-four high-quality panoramic X-rays that have been annotated with very specific, tooth-level findings <ref:2607.27378#pg1>. These aren't just simple "there is a cavity" labels; they come from experienced dentists and are further validated by an oral and maxillofacial radiologist.

Lu: What's important here is how they built the annotations. They focused on covering more than twenty clinically relevant findings for each case, broken down into four main categories: regional abnormalities, structural conditions, tooth-level pathologies and treatments, and surrounding anatomical structures.

Meng: Twenty findings per case sounds detailed, but what does that actually look like in practice? Is it just labeling every single tooth with a status for everything?

Jane: Well, they broke those raw clinical fields down into twenty-one reportable clinical findings for factual evaluation. These cover things like regional issues—structural conditions—tooth-level problems like caries or restorations, and even surrounding areas like the maxillary sinus and the temporomandibular joint.

Tom: They represented these findings in two ways: tooth-index aware findings, which need to be located on a specific tooth using an FDI numbering system, and binary regional findings that describe abnormalities at a broader anatomical region level.

Lalam: The tooth-index aware findings are crucial because they require localization to specific teeth under the FDI numbering system. This gives the model something concrete to ground its language in, rather than just describing a general area as abnormal.

Meng: So if we're talking about deployment, what's the structure of this data actually like when you look at it? Is it just one big table?

Lu: They have nine thousand five hundred twenty-four valid OPG cases in total <ref:2607.27378#pg1>. These are made up of nine thousand nineteen public-source cases and five hundred five in-house cases. The process also includes creating clinically consistent radiology reports using clinician-defined reporting logic to establish a direct link between the structured clinical evidence and the free text descriptions.

Jane: That connection is key because it lets them evaluate if an AI generates reports that aren't just fluent, but are actually consistent with those expert-validated tooth-level findings. It tests structure language consistency.

Tom: The evaluation protocol they used involves five hundred held-out cases disjoint from the training split. For linguistic coherence, they use standard metrics like BLEU-n, METEOR, and ROUGE L to see if the text flows well.

Paper summary: Meng: But what about accuracy? If we're moving toward production models, I need to know how this helps with actual diagnostic correctness instead of just checking fluency metrics.

Jane: They focused on attribute-targeted outputs for clinical visual diagnostic accuracy, which is different from just checking the whole report text. The normalized results showed that this method significantly reduces format-induced discrepancies between what's structured and what's written.

Lu: The main finding in their experiments was that current multimodal large language models can generate reports that sound fluent, but they still make substantial errors when it comes to fine-grained clinical findings.

Tom: Specifically, the paper suggests these models show substantial errors in fine-grained localization and tooth-level diagnosis. It's not just about the words sounding right; the AI is missing details on which specific tooth has which problem.

Meng: So if I were deploying a model right now, would I expect it to be accurate enough without this kind of fine-tuning?

Lu: Fine-tuning on PanDent significantly improves structure language consistency, substantially enhancing visual localization accuracy and diagnostic correctness. It moves the model closer to what an expert validates.

Jane: So what's the big picture implication for us in the AI community? What does PanDent actually change for how we think about these models?

Tom: The authors are introducing PanDent as a benchmark built on fine-grained, expert-validated tooth-level annotations. It highlights that while models can generate fluent reports, they still lack reliable tooth-level reasoning compared to actual expert dental interpretation.

Meng: It sounds like the gap between fluent text generation and clinically reliable diagnosis is pretty wide right now. What stops them from solving it completely?

Jane: The paper does admit there are limitations to this benchmark itself. They mention that the benchmark remains limited by the long-tailed distribution of rare findings, which means some findings might be underrepresented, and there's uneven coverage across certain clinical attributes.

Lu: Future work they point toward involves expanding pathology coverage and improving the representation and sampling of those underrepresented attributes. They also plan to develop models that explicitly couple language generation with tooth aware visual grounding.

Tom: So, in the end, PanDent provides a structured way to test if these next generation multimodal models can actually handle the necessary level of detail required for dental diagnosis.

Meng: It's a good step toward making these models more trustworthy for clinical settings, even if we still have to deal with those rare findings they mentioned.

Jane: Exactly. The implication is that moving forward, we need benchmarks that force models to go beyond surface-level fluency and demonstrate genuine understanding of the underlying clinical structure of the image.

Conclusion: Tom: So, PanDent is this new benchmark they put out for multimodal models in dental panoramic X-rays.

Jane: It’s about testing whether an AI can actually get the fine details of individual teeth, not just guess general problems on a picture.

Tom: Right, it’s focused on tooth-level structure and structure language consistency. The authors are saying that right now, models can sound fluent but miss the actual clinical specifics.

Lu: They built this dataset using nine thousand five hundred twenty-four high quality panoramic X-rays that have been checked by expert dentists and radiologists.

Meng: Ninety percent public source, so it’s a big enough collection to try out on something, right? But I'm always worried about the real-world performance when we deploy these things.

Jane: That's fair, Meng, but they say the key is linking structured clinical evidence with free text descriptions using expert logic. It checks if the AI's report matches what an actual doctor would find on a tooth level.

Tom: Exactly, so it’s not just about generating readable sentences anymore; it’s about checking if that sentence actually reflects correct tooth diagnosis across different regions of the image.

Lu: The experiment shows that when you fine tune the models on this PanDent data, their structure language consistency improves a lot, and their visual grounding accuracy goes up too.

Jane: That means we’re seeing a real improvement in how well these models ground their language in the actual anatomy of the radiograph. It moves them away from just saying "there is an issue" to pointing out exactly which tooth has which issue.

Tom: So, the authors are showing that this fine tuning actually brings those model outputs much closer to what an expert validates. That’s a concrete result there, not just a vague claim about being better.

Meng: It sounds like they’re proving that the gap between fluent text and reliable tooth-level reasoning is substantial right now. We still have a lot of work to do if we want these models to be truly dependable in dental practice.

Jane: Right, and while PanDent is a big step forward, they admit it still has limitations with rare findings being underrepresented in the dataset.

Tom: So, the authors are setting up this benchmark but also telling us exactly where we need to go next—focusing on covering those rare things and making sure the AI links language generation directly to tooth aware visual grounding.

Lu: That’s a clear roadmap for future work, showing that they see where the current method stops working.

Faculty of Dentistry, The University of Hong Kong Department of Computing, Imperial College London University of Science and Technology of China Department of Data and Systems Engineering The University of Hong Kong Department of Electronic Engineering The Chinese University of Hong Kong

cs.CV, cs.MM

Submitted: 2026-07-29

Updated: 2026-11-11

Code: https://github.com/HKUDentistry/PanDent

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 98/100

The gist: Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained…

Key concepts

PanDent
A large benchmark dataset for OPGs built upon fine-grained, expert-validated tooth-level annotations. It includes 9,524 high-quality OPGs annotated by experienced dentists and validated by oral and maxillofacial radiologists. This dataset is designed to test how well multimodal models can link visual evidence to detailed clinical findings.
Tooth-index-aware findings
Annotations that specify clinical attributes requiring localization to particular teeth using the FDI numbering system. These findings are crucial because they require models not just to describe a general area, but to pinpoint specific pathologies on individual teeth, providing fine-grained structural information.
Structure-language consistency
The measure of how well an MLLM's generated text matches the structured clinical evidence derived from the OPG. This concept tests whether the model produces reports that are not only grammatically correct but also clinically accurate and consistent with expert-validated tooth-level findings.
Fine-tuning on PanDent
The process of training a multimodal model further using the PanDent dataset. This fine-tuning significantly enhances the model's performance in dental domains, specifically improving structure-language consistency, visual localization accuracy, and overall diagnostic correctness compared to models trained without this specific data.

Terminology

Summary

Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation<ref:2607. p. 0

Where the paper says it

PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology Xiaohan Lia,1, Xinyu Liub,1, Chang Liuc,1, Sum Wing Au Yeunga , Jun Liud , Yixuan Yuane,∗∗, Hui Chena,∗ aFaculty of Dentistry, The University of Hong Kong, Hong Kong, China bDepartment of Computing, Imperial College London, London, United Kingdom cUniversity of Science and Technology of China, Hefei, China dDepartment of Data and Systems Engineering, The University of Hong Kong, Hong Kong, China eDepartment of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China Abstract Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations.

27378#pg2> PanDent introduces a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations<ref:2607. p. 0
Where the paper says it

PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology Xiaohan Lia,1, Xinyu Liub,1, Chang Liuc,1, Sum Wing Au Yeunga , Jun Liud , Yixuan Yuane,∗∗, Hui Chena,∗ aFaculty of Dentistry, The University of Hong Kong, Hong Kong, China bDepartment of Computing, Imperial College London, London, United Kingdom cUniversity of Science and Technology of China, Hefei, China dDepartment of Data and Systems Engineering, The University of Hong Kong, Hong Kong, China eDepartment of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China Abstract Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations.

27378#pg6> This dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced dentists and further validated by an oral and maxillofacial radiologist<ref:2607. p. 0
Where the paper says it

This work introduces PanDent, a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations. The dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced dentists and further validated by an oral and maxillofacial radiologist, providing clinically reliable supervision for tooth-level diagnosis and reasoning.

27378#pg2> Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions<ref:2607. p. 0
Where the paper says it

The dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced dentists and further validated by an oral and maxillofacial radiologist, providing clinically reliable supervision for tooth-level diagnosis and reasoning. Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions.

27378#pg2> This design enables evaluation of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings<ref:2607. p. 0
Where the paper says it

Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions. This design enables evaluation of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings.

27378#pg2>

Dataset Construction and Curation

PanDent is curated to support both rigorous benchmark evaluation and supervised training for dental radiology understanding<ref:2607.27378#pg2> The dataset is built from 9,854 raw annotated OPG records collected from multiple public-source batches and one in-house batch<ref:2607. p. 2

Where the paper says it

As illustrated in Fig. 1, the dataset is built from 9,854 raw annotated OPG records collected from multiple public-source batches and one in-house batch, 330 cases with invalid or unresolved image paths are removed.

27378#pg2> Each case is associated with structured annotations covering more than 20 clinically relevant findings observable from OPG images<ref:2607. p. 2
Where the paper says it

The final release contains 9,524 valid OPG cases, including 9,019 public-source cases and 505 in-house cases. Each case is associated with structured annotations covering more than 20 clinically relevant findings observable from OPG images.

27378#pg2> The annotations are labeled by four professional dentists with over ten years of clinical experience and further validated by an assistant professor in oral and maxillofacial radiology on a randomly sampled subset, providing clinically reliable supervision before conversion into clinician-guided radiology reports<ref:2607. p. 2
Where the paper says it

Each case is associated with structured annotations covering more than 20 clinically relevant findings observable from OPG images. These annotations are labeled by four professional dentists with over ten years of clinical experience and further validated by an assistant professor in oral and maxillofacial radiology on a randomly sampled subset, providing clinically reliable supervision before conversion into clinician-guided radiology reports.

27378#pg2> The final release contains 9,524 valid OPG cases, including 9,019 public-source cases and 505 in-house cases<ref:2607. p. 2
Where the paper says it

1, the dataset is built from 9,854 raw annotated OPG records collected from multiple public-source batches and one in-house batch, 330 cases with invalid or unresolved image paths are removed. The final release contains 9,524 valid OPG cases, including 9,019 public-source cases and 505 in-house cases.

27378#pg2>

Annotation Structure and Normalization

The raw annotations consist of heterogeneous clinical fields, which are consolidated into 21 reportable clinical findings for factual evaluation<ref:2607.27378#pg2> These findings capture dental findings across four aspects: regional abnormalities (e. p. 3

Where the paper says it

2 The raw annotations consist of heterogeneous clinical fields, which are consolidated into 21 reportable clinical findings for factual evaluation. From a clinical perspective, the annotation schema captures dental findings across four aspects: regional abnormalities (e.

g., maxillary or mandibular structural conditions), structural and periodontal conditions (e. p. 3
Where the paper says it

From a clinical perspective, the annotation schema captures dental findings across four aspects: regional abnormalities (e. , maxillary or mandibular structural conditions), structural and periodontal conditions (e.

g., angular and generalized crestal bone loss, furcation involvement), tooth-level pathologies and treatments (e. p. 3
Where the paper says it

, maxillary or mandibular structural conditions), structural and periodontal conditions (e. , angular and generalized crestal bone loss, furcation involvement), tooth-level pathologies and treatments (e.

g., caries, periapical lesions and restorations), and surrounding anatomical structures (e. p. 3
Where the paper says it

, angular and generalized crestal bone loss, furcation involvement), tooth-level pathologies and treatments (e. , caries, periapical lesions and restorations), and surrounding anatomical structures (e.

g., maxillary sinus and temporomandibular joint)<ref:2607. p. 3
Where the paper says it

, caries, periapical lesions and restorations), and surrounding anatomical structures (e. , maxillary sinus and temporomandibular joint).

27378#pg2> The annotations are represented in two forms: tooth-index-aware findings and binary regional findings<ref:2607. p. 3
Where the paper says it

Besides, an additional ans_comment field is included to record free-text clinical remarks for cases that cannot be fully captured by structured labels. For representation and downstream use, the annotations are represented in two forms: tooth-index-aware findings and binary regional findings, enabling consistent mapping to radiology reports and reliable downstream evaluation.

27378#pg2> Tooth-index-aware findings refer to clinical attributes that require localization to specific teeth under the FDI numbering system<ref:2607. p. 3
Where the paper says it

Tooth-Index-Aware Findings. Tooth-index-aware findings refer to clinical attributes that require localization to specific teeth under the FDI numbering system.

27378#pg2> Binary regional findings describe abnormalities assessed at the anatomical-region level<ref:2607. p. 3
Where the paper says it

Binary Regional Findings. Binary regional findings describe abnormalities assessed at the anatomical-region level.

27378#pg2> These representations capture both tooth-specific and region-level clinical information, providing a structured basis for evaluating structure-language consistency and fine-grained clinical fidelity<ref:2607. p. 3
Where the paper says it

These representations capture both tooth-specific and regionlevel clinical information, providing a structured basis for evaluating structure-language consistency and fine-grained clinical fidelity. Clinical Attribute Normalization Raw clinical annotations vary in format and level of detail.

27378#pg2>

Evaluation Protocol

The benchmark evaluates models on 500 held-out cases that are disjoint from the training split<ref:2607.27378#pg2> For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = 1, 2, 3, 4), METEOR, and ROUGE-L<ref:2607. p. 5

Where the paper says it

We measure model outputs from two dimensions: linguistic coherence and clinical accuracy. For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}), METEOR, and ROUGE-L (Papineni et al.

27378#pg2> For clinical visual diagnostic accuracy, evaluation is conducted on attribute-targeted outputs instead of full free-text reports<ref:2607. p. 5
Where the paper says it

These metrics measure lexical overlap and sequence-level similarity between the generated and reference reports, where higher scores indicate stronger surfaceform agreement5 . For clinical visual diagnostic accuracy, evaluation is conducted on attribute-targeted outputs instead of full free-text reports.

27378#pg2> The normalized results in Tab. p. 7
Where the paper says it

2 is applied, mapping all outputs into a unified concise reporting format. The normalized results in Tab.

1 demonstrate that this procedure significantly reduces format-induced discrepancies<ref:2607. p. 7
Where the paper says it

The normalized results in Tab. 1 demonstrate that this procedure significantly reduces format-induced discrepancies.

27378#pg2>

Key Findings and Model Performance

Experiments show that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, whereas fine-tuning on PanDent significantly improves dental-domain performance and brings model outputs closer to expert-validated annotations<ref:2607.27378#pg2> The results suggest that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis<ref:2607. p. 1

Where the paper says it

Accordingly, experiments are organized around three complementary aspects: report-level generation quality, fine-grained clinical factuality, and performance improvement through finetuning. Results suggest that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, whereas fine-tuning on PanDent significantly 2A detailed discussion of medical and dental radiology datasets is provided in Sec.

27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, substantially enhancing visual localization accuracy and diagnostic correctness<ref:2607. p. 0
Where the paper says it

Results show that current MLLMs can generate fluent reports, yet fail to produce clinically consistent descriptions, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness, and bringing model outputs closer to expert dental interpretation.

27378#pg2>

Conclusion

This paper introduces PanDent, a curated benchmark for evaluating multimodal models in dental panoramic radiography, with a focus on fine-grained tooth-level reasoning and structure-language consistency<ref:2607.27378#pg2> Experiments show that current MLLMs can produce fluent reports but still fail to achieve reliable tooth-level reasoning, revealing a substantial gap to expert dental interpretation<ref:2607. p. 0

Where the paper says it

Results show that current MLLMs can generate fluent reports, yet fail to produce clinically consistent descriptions, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness, and bringing model outputs closer to expert dental interpretation.

27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, visual grounding accuracy, and toothlevel diagnostic correctness<ref:2607. p. 0
Where the paper says it

Results show that current MLLMs can generate fluent reports, yet fail to produce clinically consistent descriptions, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness, and bringing model outputs closer to expert dental interpretation.

27378#pg2> Despite these advances, the benchmark remains limited by the long-tailed distribution of rare findings and the uneven coverage across certain clinical attributes<ref:2607. p. 5
Where the paper says it

This long-tailed distribution captures a wide spectrum of dental conditions beyond common pathologies. Rare attributes, such as temporomandibular joint abnormalities, maxillary sinus abnormalities, and broader maxillary or mandibular structural conditions, are retained for completeness.

27378#pg2> Future work will focus on expanding pathology coverage, improving the representation and sampling of underrepresented attributes, and developing models that explicitly couple language generation with tooth-aware visual grounding<ref:2607.27378#pg2>

References

Abacha, A.B., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D. p. 1

Where the paper says it

The first category directly supports the task of automatic radiology report generation. IU X-Ray (Demner-Fushman et al.

, Müller, H., 2019. VQA-Med: Overview of the medical visual question answering task at ImageCLEF 2019<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Alqaderi, H. p. 1
Where the paper says it

In OPG images, clinically meaningful reasoning depends on accurate tooth-level localization and interpretation of surrounding anatomical structures. However, on the dental side, existing datasets largely focus on isolated recognition tasks, such as tooth segmentation, numbering, or limited abnormality classification (Alqaderi et al.

, Phillips, C., Jones, J.A., Ramchandani, K., Mitwalli, A.H., Youssef, N.A., Emami, S.E., Teixeira, T., Ferreira, P.S., Yousfi, H., Cheela, D., Bahaj, A.: Automated detection of dental caries and bone loss in bitewing radiographs using YOLOv8: A multicenter deep learning study<ref:2607. p. 2
Where the paper says it

, 2023) organizes hierarchical supervision for dental enumeration, abnormal-tooth detection, and diagnosis on panoramic X-rays. More broadly, dental image-analysis studies also cover intra-oral radiographs and CBCT under tasks such as caries screening, periodontal analysis, landmark localization, and lesion segmentation (Singh and Raza, 2022).

27378#pg2> Bai, S. p. 6
Where the paper says it

058 Claude Opus 4. 7 (Bai et al.

, et al., 2025. Qwen2.5-VL technical report<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Bai, Y. p. 6
Where the paper says it

058 Claude Opus 4. 7 (Bai et al.

, et al., 2024. Constitutional AI: Harmlessness from AI feedback<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Banerjee, S. p. 5
Where the paper says it

For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}), METEOR, and ROUGE-L (Papineni et al. , 2002; Banerjee and Lavie, 2005; Lin, 2004).

, Lavie, A. p. 5
Where the paper says it

For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}), METEOR, and ROUGE-L (Papineni et al. , 2002; Banerjee and Lavie, 2005; Lin, 2004).

, 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments<ref:2607.27378#pg2> Bannur, S. p. 6
Where the paper says it

, 2026) 73 0. 235 MAIRA-2 (Bannur et al.

, et al., 2024. MAIRA-2: Grounded radiology report generation<ref:2607. p. 0
Where the paper says it

com/HKUDentistry/PanDent. Keywords: dental panoramic radiography, multimodal large language models, radiology report generation, tooth-level reasoning, clinical benchmark, structure-language consistency 1.

27378#pg2> Bustos, A. p. 0
Where the paper says it

, 2019), IU X-Ray (DemnerFushman et al. , 2016), and PadChest (Bustos et al.

, Pertusa, A., Salinas, J., de la Iglesia-Vayá, M., 2020. PadChest: A large chest x-ray image dataset with multi-label annotated reports<ref:2607. p. 1
Where the paper says it

, 2019) scales this setting to a large deidentified chest-radiograph corpus with associated reports, and PadChest (Bustos et al. , 2020) further enriches chest X-ray data with multi-label annotations derived from clinical reports.

27378#pg2> Chen, Z. p. 0
Where the paper says it

hk (Yixuan Yuan), amyhchen@hku. hk (Hui Chen) 1These authors contributed equally to this work.

, Varma, M., Delbrouck, J., Blankemeier, A., Blankemeier, L., Fu, C., Televic, A.: CheXagent: Towards a foundation model for chest x-ray interpretation<ref:2607. p. 0
Where the paper says it

However, progress is limited by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al. Existing public benchmarks for medical MMLMs are predominantly centered on chest X-ray imaging, including report-generation datasets such as MIMIC-CXR (Johnson et al.

27378#pg2> Demner-Fushman, D. p. 1
Where the paper says it

The first category directly supports the task of automatic radiology report generation. IU X-Ray (Demner-Fushman et al.

, Kohli, M.D., Rosenman, M.B., Shooshan, S.E., Rodriguez, L., Antani, S., Thoma, G.R., McDonald, C.J.: Preparing a collection of radiology examinations for distribution and retrieval<ref:2607. p. 2
Where the paper says it

Data Collection Multi-source OPG Acquisition p Open source datasets: ToothFairy, DENTEX, Mandible, Humans, NTU p In-house dataset: Collected from PPDH p Permanent dentition only p Exclude mixed dentition and non-dental images p Normalization Total: 9524 Cases Annotation by Dentists • Fine-grained tooth-level findings (FDI indices) • Treatment & restoration status • Regional / global observations • Other clinical attributes Validation by Senior Experts • Review and validate random sampled subsets • Ensure clinical correctness • Resolve inconsistencies Expert Annotation Clinician Report Construction Normalize Structured Annotations • Convert raw fields to unified format • Derive missing-tooth set • Normalize regional findings Clinician-Designed Reporting • Reporting flow designed by clinicians • Follow real clinical reasoning order: tooth-level → local findings → regional/global observations • Generated with LLM in clinically natural expressions Structure-Language Consistency Check • Verify alignment between structured findings and generated text • Ensure completeness and consistency Final Dataset 9,524 Radiograph-Report Pairs Training Set 9,024 cases (~ 94. 8%) Test Set 500 cases (~ 5.

27378#pg2> Dubey, A. p. 6
Where the paper says it

230 Llama-3. 2-90B-Vision-Instruct (Dubey et al.

, et al., 2024. The Llama 3 herd of models<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Fan, Y. p. 6
Where the paper says it

0 (HARI and MVL, 2026) 4 0. 365 OralGPT-Captioning-4B-Base (Fan et al.

, Hao, J., Chen, H., Bao, J., Shao, Y., Liang, Y., Hung, K.F.: OralGPT-Plus: Learning to use visual tools via reinforcement learning for panoramic x-ray analysis<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Gao, J. p. 0
Where the paper says it

, 2022; Liu et al. , 2024b,a; Gao et al.

, Liu, C., Li, Y.: S2D-Align: Shallow-to-Deep Auxiliary Learning for Anatomically-Grounded Radiology Report Generation<ref:2607. p. 4
Where the paper says it

The rule-based verifier operates on normalized structured annotations prior to report generation. It derives the missing-teeth set from the permanent dentition and removes absent teeth from all tooth-index-aware findings, preventing anatomically invalid assignments.

27378#pg2> Hamamci, I. p. 2
Where the paper says it

, 2023) releases a panoramic radiograph benchmark with baselines and an online platform, where the core annotations support tasks such as tooth segmentation and numbering. DENTEX (Hamamci et al.

E., Er, S., Simsar, E., Sekuboyina, A., Gundogar, M., Stadlinger, B.: DENTEX: An abnormal tooth detection with dental enumeration and diagnosis benchmark for panoramic x-rays<ref:2607. p. 2
Where the paper says it

DENTEX (Hamamci et al. , 2023) organizes hierarchical supervision for dental enumeration, abnormal-tooth detection, and diagnosis on panoramic X-rays.

27378#pg2> Hao, J. p. 2
Where the paper says it

As a result, they are valuable for dental visual recognition but cannot directly serve as benchmarks for radiology report generation or structure-language consistency evaluation. The most relevant resource to our curated dataset is MMOral (Hao et al.

, Fan, Y., Sun, Y., Guo, K., Lin, L., Yang, J., Ai, Q.Y.H.: Towards better dental AI: A multimodal benchmark and instruction dataset for panoramic x-ray analysis<ref:2607. p. 2
Where the paper says it

The most relevant resource to our curated dataset is MMOral (Hao et al. , 2025), which provides a large-scale multimodal instruction dataset for panoramic X-ray analysis around five diagnostic dimensions, including teeth condition, pathological findings, historical treatments, jawbone observations, and clinical summary and recommendation.

27378#pg2> He, X., Zhang, Y. p. 6
Where the paper says it

255 Libra-v1. 0-3B (Zhang et al.

p. 6
Where the paper says it

255 Libra-v1. 0-3B (Zhang et al.

, Mou, L., Xing, E., Xie, P.: PathVQA: 30000+ questions for medical visual question answering<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Jain, S. p. 1
Where the paper says it

The third category provides structured clinical supervision, either as disease labels or as report-derived entities and relations. For instances, PadChest includes imagelevel labels, while RadGraph (Jain et al.

, Agrawal, A., Saporta, A., Truong, S.Q.H., Duong, D.N., Bui, T., Chambon, P.: RadGraph: Extracting clinical entities and relations from radiology reports<ref:2607. p. 1
Where the paper says it

For instances, PadChest includes imagelevel labels, while RadGraph (Jain et al. , 2021) converts radiology reports into clinical entities and relations for factual evaluation.

27378#pg2> Johnson, A. p. 0
Where the paper says it

However, progress is limited by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al. Existing public benchmarks for medical MMLMs are predominantly centered on chest X-ray imaging, including report-generation datasets such as MIMIC-CXR (Johnson et al.

E.W., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.Y., Mark, R.G., Horng, S.: MIMIC-CXR: A large publicly available database of labeled chest radiographs<ref:2607. p. 0
Where the paper says it

However, progress is limited by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al. Existing public benchmarks for medical MMLMs are predominantly centered on chest X-ray imaging, including report-generation datasets such as MIMIC-CXR (Johnson et al.

27378#pg2> Liu, B. p. 0
Where the paper says it

It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza, 2022; Panetta et al. , 2022; Liu et al.

, Zhan, L.M., Xu, L., Ma, L., Wu, X., Lam, A.Y.S.: SLAKE: A semantically-labeled knowledge-enhanced dataset for medical visual question answering<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Liu, C. p. 0
Where the paper says it

It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza, 2022; Panetta et al. , 2022; Liu et al.

, Tian, Y., Chen, W., Song, Y., Zhang, Y.: Bootstrapping Large Language Models for Radiology Report Generation<ref:2607. p. 0
Where the paper says it

com/HKUDentistry/PanDent. Keywords: dental panoramic radiography, multimodal large language models, radiology report generation, tooth-level reasoning, clinical benchmark, structure-language consistency 1.

27378#pg2> Lutescu, E. p. 0
Where the paper says it

Recent work has explored data-driven models for extracting clinical findings and generating structured or free-text descriptions from OPG images. However, progress is limited by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al.

O., Chet-an, O., Draghici, E., Apostu, A., Ilie, M., Iliescu, M.: Can artificial intelligence in orthopantomography advance dental diagnostics through automated image analysis? p. 2
Where the paper says it

, 2023) organizes hierarchical supervision for dental enumeration, abnormal-tooth detection, and diagnosis on panoramic X-rays. More broadly, dental image-analysis studies also cover intra-oral radiographs and CBCT under tasks such as caries screening, periodontal analysis, landmark localization, and lesion segmentation (Singh and Raza, 2022).

<ref:2607.27378#pg2> Ma, X., et al., 2026. A safety report on GPT-5. p. 9
Where the paper says it

Case Study and Discussion Tab. 5 presents a representative case from the PanDent test benchmark, including the input radiograph, prompt instruction, ground-truth report with attribute annotations, and outputs from three representative models: GPT-5.

2, gemini 3 pro, Qwen3-VL, grok 4. p. 6
Where the paper says it

1 (OpenAI, 2023) N/A 0. 210 Gemini 3 Pro Preview (Gemini Team, 2023) N/A 0.

1 fast, nano banana pro, and seedream 4. p. 6
Where the paper says it

1 (OpenAI, 2023) N/A 0. 210 Gemini 3 Pro Preview (Gemini Team, 2023) N/A 0.

5<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> MedGemma Team, 2025. p. 6
Where the paper says it

, 2024) 90 0. 398 Medical-Domain MLLMs MedGemma-27B-IT (MedGemma Team, 2025) 27 0.

Medgemma technical report<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Mistral AI, 2024. p. 6
Where the paper says it

, 2024) 27 0. 222 Pixtral-Large-Instruct-2411 (Mistral AI, 2024) 124 0.

Pixtral 12b<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Moonshot AI, 2025. p. 6
Where the paper says it

216 MiniCPM-V-4_5 (OpenBMB Team, 2025) 8 0. 291 Kimi-VL-A3B-Thinking-2506 (Moonshot AI, 2025) 3 0.

Kimi-VL technical report<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> OpenAI, 2023. p. 6
Where the paper says it

Method Size (B) BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L Average General-Domain Closed-Source MLLMs GPT-5. 4 (OpenAI, 2023) N/A 0.

GPT-4 technical report<ref:2607. p. 9
Where the paper says it

Case Study and Discussion Tab. 5 presents a representative case from the PanDent test benchmark, including the input radiograph, prompt instruction, ground-truth report with attribute annotations, and outputs from three representative models: GPT-5.

27378#pg2> OpenBMB Team, 2025. p. 6
Where the paper says it

, 2025) 78 0. 216 MiniCPM-V-4_5 (OpenBMB Team, 2025) 8 0.

MiniCPM-V 4. p. 6
Where the paper says it

, 2025) 78 0. 216 MiniCPM-V-4_5 (OpenBMB Team, 2025) 8 0.

5: Cooking efficient MLLMs via architecture, data, and training recipe<ref:2607.27378#pg2> Panetta, K. p. 0
Where the paper says it

Introduction Dental panoramic radiography (orthopantomogram, OPG) is a cornerstone of routine oral healthcare. It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza, 2022; Panetta et al.

, Rajendran, R., Ramesh, A., Rao, S.P., Agaian, S.S.: Tufts dental database: A multimodal panoramic x-ray dataset for benchmarking diagnostic systems<ref:2607. p. 1
Where the paper says it

Particularly, the Tufts Dental Database (Panetta et al. , 2022) provides panoramic radiographs with tooth-level annotations and auxiliary multimodal signals, making it useful for benchmarking 2

27378#pg2> Papineni, K. p. 5
Where the paper says it

We measure model outputs from two dimensions: linguistic coherence and clinical accuracy. For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}), METEOR, and ROUGE-L (Papineni et al.

, Roukos, S., Ward, T., Zhu, W.J.: BLEU: A method for automatic evaluation of machine translation<ref:2607. p. 6
Where the paper says it

The best and second-best results are highlighted as in red and blue, respectively. Method Size (B) BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L Average General-Domain Closed-Source MLLMs GPT-5.

27378#pg2> Pham, T. p. 1
Where the paper says it

For example, dental caries are often annotated as a global label, whereas clinical practice requires identifying the exact tooth locations. As a result, such datasets do not provide sufficiently fine-grained, expert-validated annotations to support faithful evaluation against clinical standards (Pham and Al-Hebshi, 2026), making it difficult to determine whether models truly capture clinically relevant findings or merely produce plausible reports.

B.T., Al-Hebshi, N. p. 1
Where the paper says it

For example, dental caries are often annotated as a global label, whereas clinical practice requires identifying the exact tooth locations. As a result, such datasets do not provide sufficiently fine-grained, expert-validated annotations to support faithful evaluation against clinical standards (Pham and Al-Hebshi, 2026), making it difficult to determine whether models truly capture clinically relevant findings or merely produce plausible reports.

M.: Classification and prevalence of dental pathology in pediatric panoramic radiographs: A retrospective study<ref:2607. p. 1
Where the paper says it

Yet, their dominant modality remains on chest radiography or pathology microscopy. Therefore, they do not directly address the tooth-index-aware reasoning and dental terminology required by panoramic dental radiology analysis.

27378#pg2> Qwen Team, 2025. p. 6
Where the paper says it

, 2026) N/A 0. 211 General-Domain Open-Source MLLMs Qwen3-VL-235B-A22B-Instruct (Qwen Team, 2025) 235 0.

Qwen3-VL technical report<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Qwen Team, 2026. p. 6
Where the paper says it

, 2026) N/A 0. 211 General-Domain Open-Source MLLMs Qwen3-VL-235B-A22B-Instruct (Qwen Team, 2025) 235 0.

Qwen3.5 Hugging Face model card<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Silva, B. p. 2
Where the paper says it

diagnostic systems and studying expert interpretation behavior. OdontoAI (Silva et al.

, Pinheiro, L., Sobrinho, B., Lima, F., Sobrinho, B., Lima, K.A.B.: Boosting research on dental panoramic radiographs: A challenging data set<ref:2607. p. 1
Where the paper says it

Particularly, the Tufts Dental Database (Panetta et al. , 2022) provides panoramic radiographs with tooth-level annotations and auxiliary multimodal signals, making it useful for benchmarking 2

27378#pg2> Singh, N. p. 0
Where the paper says it

Introduction Dental panoramic radiography (orthopantomogram, OPG) is a cornerstone of routine oral healthcare. It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza, 2022; Panetta et al.

K., Raza, K.: Progress in deep learning-based dental and maxillofacial image analysis: A systematic review<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Wu, Z., et al., 2024. DeepSeek-VL2: Mixture-of-experts vision-language models for advanced multimodal understanding<ref:2607. p. 1
Where the paper says it

, 2021), and PathVQA (He et al. While these datasets have advanced medical vision-language modeling, they primarily focus on global image-level understanding and do not reflect the structure of dental panoramic interpretation.

27378#pg2> Xu, J., et al., 2026. Pulsemind: A multi-modal medical model for real-world clinical diagnosis<ref:2607. p. 7
Where the paper says it

Experimental Setups Benchmarked Multi-modal Large Language Models (MLLMs). To comprehensively evaluate mainstream multimodal large language models (MLLMs), three categories are considered: closed-source models, open-source models, and medical-domain models.

27378#pg2> Zhang, X. p. 6
Where the paper says it

255 Libra-v1. 0-3B (Zhang et al.

, Meng, Z., Lever, J., Ho, E.S.L.: Libra: Leveraging temporal images for biomedical radiology analysis<ref:2607. p. 0
Where the paper says it

, 2020), as well as medical visual question answering benchmarks such as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu Preprint submitted to Medical Image Analysis July 31, 2026 arXiv:2607.

27378#pg2> Zhipu AI, 2025. p. 6
Where the paper says it

291 Kimi-VL-A3B-Thinking-2506 (Moonshot AI, 2025) 3 0. 1V-9B-Thinking (Zhipu AI, 2025) 9 0.

GLM-4.1V-Thinking: Towards versatile multimodal reasoning with scalable reinforcement learning<ref:2607. p. 0
Where the paper says it

com/HKUDentistry/PanDent. Keywords: dental panoramic radiography, multimodal large language models, radiology report generation, tooth-level reasoning, clinical benchmark, structure-language consistency 1.

27378#pg2> Zhu, J. p. 6
Where the paper says it

, 2025) 72 0. 220 InternVL3-78B (Zhu et al.

, et al., 2025. InternVL3: Exploring advanced training and testtime recipes for open-source multimodal models<ref:2607. p. 7
Where the paper says it

Experimental Setups Benchmarked Multi-modal Large Language Models (MLLMs). To comprehensively evaluate mainstream multimodal large language models (MLLMs), three categories are considered: closed-source models, open-source models, and medical-domain models.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness. This enables models to produce reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings.

  2. The system can perform evaluation across three complementary aspects: report-level generation quality, fine-grained clinical factuality, and performance improvement through finetuning. This allows for a rigorous assessment of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings.

  3. The improved system can achieve higher performance on attribute-targeted outputs by utilizing the exact match accuracy metric (Accj). This measures how well the model generates a statement matching the ground truth for specific attributes, such as Generalized crestal bone loss in the mandible, directly improving clinical diagnostic accuracy beyond conventional report-level metrics.

Abstract

Accurate evaluation of multimodal large language models in dental panoramic radiography is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale benchmark built on expert-validated tooth-level annotations. It comprises 9,554 panoramic radiographs with comprehensive structured findings produced by experienced dentists and further validated by an oral and maxillofacial radiologist. Clinician-guided reports are constructed from these findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions. The dataset supports complementary evaluation of report-level textual agreement and tooth-level clinical consistency, with 9,049 public-source cases for supervised training and an independent 505-case in-house benchmark. Experiments cover proprietary, general-domain open-source, and medical-specific models. Current models generate fluent reports but exhibit substantial errors in fine-grained localization and tooth-level diagnosis. Supervised fine-tuning on PanDent markedly improves report alignment and diagnostic accuracy, bringing predictions closer to expert dental interpretation. These results establish PanDent as a rigorous benchmark and training resource for clinically grounded dental multimodal reasoning. Dataset sample: https://huggingface.co/datasets/Desperado1103/Pandent Sample; code: https://github.com/HKUDentistry/PanDent. Full data will be released upon publication under source redistribution terms.

Sources

Related papers