PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology".
Jane: Here's the gist of the paper. Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation<ref:2607.27378#pg2> PanDent introduces a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations<ref:2607.27378#pg6> This dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced dentists and further validated by an oral and maxillofacial radiologist<ref:2607.27378#pg2> Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions<ref:2607.27378#pg2> This design enables evaluation of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings<ref:2607.27378#pg2> Dataset Construction and Curation PanDent is curated to support both rigorous benchmark evaluation and supervised training for dental radiology understanding<ref:2607.27378#pg2> The dataset is built from 9,854 raw annotated OPG records collected from multiple public-source batches and one in-house batch<ref:2607.27378#pg2> Each case is associated with structured annotations covering more than 20 clinically relevant findings observable from OPG images<ref:2607.27378#pg2> The annotations are labeled by four professional dentists with over ten years of clinical experience and further validated by an assistant professor in oral and maxillofacial radiology on a randomly sampled subset, providing clinically reliable supervision before conversion into clinician-guided radiology reports<ref:2607.27378#pg2> The final release contains 9,524 valid OPG cases, including 9,019 public-source cases and 505 in-house cases<ref:2607.27378#pg2> Annotation Structure and Normalization The raw annotations consist of heterogeneous clinical fields, which are consolidated into 21 reportable clinical findings for factual evaluation<ref:2607.27378#pg2> These findings capture dental findings across four aspects: regional abnormalities (e.g., maxillary or mandibular structural conditions), structural and periodontal conditions (e.g., angular and generalized crestal bone loss, furcation involvement), tooth-level pathologies and treatments (e.g., caries, periapical lesions and restorations), and surrounding anatomical structures (e.g., maxillary sinus and temporomandibular joint)<ref:2607.27378#pg2> The annotations are represented in two forms: tooth-index-aware findings and binary regional findings<ref:2607.27378#pg2> Tooth-index-aware findings refer to clinical attributes that require localization to specific teeth under the FDI numbering system<ref:2607.27378#pg2> Binary regional findings describe abnormalities assessed at the anatomical-region level<ref:2607.27378#pg2> These representations capture both tooth-specific and region-level clinical information, providing a structured basis for evaluating structure-language consistency and fine-grained clinical fidelity<ref:2607.27378#pg2> Evaluation Protocol The benchmark evaluates models on 500 held-out cases that are disjoint from the training split<ref:2607.27378#pg2> For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = 1, 2, 3, 4), METEOR, and ROUGE-L<ref:2607.27378#pg2> For clinical visual diagnostic accuracy, evaluation is conducted on attribute-targeted outputs instead of full free-text reports<ref:2607.27378#pg2> The normalized results in Tab.
Tom: 1 demonstrate that this procedure significantly reduces format-induced discrepancies<ref:2607.27378#pg2> Key Findings and Model Performance Experiments show that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, whereas fine-tuning on PanDent significantly improves dental-domain performance and brings model outputs closer to expert-validated annotations<ref:2607.27378#pg2> The results suggest that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis<ref:2607.27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, substantially enhancing visual localization accuracy and diagnostic correctness<ref:2607.27378#pg2> This paper introduces PanDent, a curated benchmark for evaluating multimodal models in dental panoramic radiography, with a focus on fine-grained tooth-level reasoning and structure-language consistency<ref:2607.27378#pg2> Experiments show that current MLLMs can produce fluent reports but still fail to achieve reliable tooth-level reasoning, revealing a substantial gap to expert dental interpretation<ref:2607.27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, visual grounding accuracy, and toothlevel diagnostic correctness<ref:2607.27378#pg2> Despite these advances, the benchmark remains limited by the long-tailed distribution of rare findings and the uneven coverage across certain clinical attributes<ref:2607.27378#pg2> Future work will focus on expanding pathology coverage, improving the representation and sampling of underrepresented attributes, and developing models that explicitly couple language generation with tooth-aware visual grounding<ref:2607.27378#pg2> References Abacha, A.B., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D., Müller, H., 2019.
Jane: VQA-Med: Overview of the medical visual question answering task at ImageCLEF 2019<ref:2607.27378#pg2> Alqaderi, H., Phillips, C., Jones, J.A., Ramchandani, K., Mitwalli, A.H., Youssef, N.A., Emami, S.E., Teixeira, T., Ferreira, P.S., Yousfi, H., Cheela, D., Bahaj, A.: Automated detection of dental caries and bone loss in bitewing radiographs using YOLOv8: A multicenter deep learning study<ref:2607.27378#pg2> Bai, S., et al., 2025.
Tom: Qwen2.5-VL technical report<ref:2607.27378#pg2> Bai, Y., et al., 2024.
Paper summary: Tom: So we're looking at this paper called "PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology." Basically, the authors are saying that right now, when we test multimodal models on panoramic X-rays, there just aren't enough benchmarks that actually reflect what an expert dentist would interpret.
Jane: That sounds like a big problem because if the benchmark isn't fine-grained enough, we can't really tell if an AI is actually getting the right details about individual teeth and not just guessing generally. What are they proposing with this PanDent dataset?
Tom: They’re introducing a large-scale dataset of nine thousand five hundred twenty-four high-quality panoramic X-rays that have been annotated with very specific, tooth-level findings <ref:2607.27378#pg1>. These aren't just simple "there is a cavity" labels; they come from experienced dentists and are further validated by an oral and maxillofacial radiologist.
Lu: What's important here is how they built the annotations. They focused on covering more than twenty clinically relevant findings for each case, broken down into four main categories: regional abnormalities, structural conditions, tooth-level pathologies and treatments, and surrounding anatomical structures.
Meng: Twenty findings per case sounds detailed, but what does that actually look like in practice? Is it just labeling every single tooth with a status for everything?
Jane: Well, they broke those raw clinical fields down into twenty-one reportable clinical findings for factual evaluation. These cover things like regional issues—structural conditions—tooth-level problems like caries or restorations, and even surrounding areas like the maxillary sinus and the temporomandibular joint.
Tom: They represented these findings in two ways: tooth-index aware findings, which need to be located on a specific tooth using an FDI numbering system, and binary regional findings that describe abnormalities at a broader anatomical region level.
Lalam: The tooth-index aware findings are crucial because they require localization to specific teeth under the FDI numbering system. This gives the model something concrete to ground its language in, rather than just describing a general area as abnormal.
Meng: So if we're talking about deployment, what's the structure of this data actually like when you look at it? Is it just one big table?
Lu: They have nine thousand five hundred twenty-four valid OPG cases in total <ref:2607.27378#pg1>. These are made up of nine thousand nineteen public-source cases and five hundred five in-house cases. The process also includes creating clinically consistent radiology reports using clinician-defined reporting logic to establish a direct link between the structured clinical evidence and the free text descriptions.
Jane: That connection is key because it lets them evaluate if an AI generates reports that aren't just fluent, but are actually consistent with those expert-validated tooth-level findings. It tests structure language consistency.
Tom: The evaluation protocol they used involves five hundred held-out cases disjoint from the training split. For linguistic coherence, they use standard metrics like BLEU-n, METEOR, and ROUGE L to see if the text flows well.
Paper summary: Meng: But what about accuracy? If we're moving toward production models, I need to know how this helps with actual diagnostic correctness instead of just checking fluency metrics.
Jane: They focused on attribute-targeted outputs for clinical visual diagnostic accuracy, which is different from just checking the whole report text. The normalized results showed that this method significantly reduces format-induced discrepancies between what's structured and what's written.
Lu: The main finding in their experiments was that current multimodal large language models can generate reports that sound fluent, but they still make substantial errors when it comes to fine-grained clinical findings.
Tom: Specifically, the paper suggests these models show substantial errors in fine-grained localization and tooth-level diagnosis. It's not just about the words sounding right; the AI is missing details on which specific tooth has which problem.
Meng: So if I were deploying a model right now, would I expect it to be accurate enough without this kind of fine-tuning?
Lu: Fine-tuning on PanDent significantly improves structure language consistency, substantially enhancing visual localization accuracy and diagnostic correctness. It moves the model closer to what an expert validates.
Jane: So what's the big picture implication for us in the AI community? What does PanDent actually change for how we think about these models?
Tom: The authors are introducing PanDent as a benchmark built on fine-grained, expert-validated tooth-level annotations. It highlights that while models can generate fluent reports, they still lack reliable tooth-level reasoning compared to actual expert dental interpretation.
Meng: It sounds like the gap between fluent text generation and clinically reliable diagnosis is pretty wide right now. What stops them from solving it completely?
Jane: The paper does admit there are limitations to this benchmark itself. They mention that the benchmark remains limited by the long-tailed distribution of rare findings, which means some findings might be underrepresented, and there's uneven coverage across certain clinical attributes.
Lu: Future work they point toward involves expanding pathology coverage and improving the representation and sampling of those underrepresented attributes. They also plan to develop models that explicitly couple language generation with tooth aware visual grounding.
Tom: So, in the end, PanDent provides a structured way to test if these next generation multimodal models can actually handle the necessary level of detail required for dental diagnosis.
Meng: It's a good step toward making these models more trustworthy for clinical settings, even if we still have to deal with those rare findings they mentioned.
Jane: Exactly. The implication is that moving forward, we need benchmarks that force models to go beyond surface-level fluency and demonstrate genuine understanding of the underlying clinical structure of the image.
Conclusion: Tom: So, PanDent is this new benchmark they put out for multimodal models in dental panoramic X-rays.
Jane: It’s about testing whether an AI can actually get the fine details of individual teeth, not just guess general problems on a picture.
Tom: Right, it’s focused on tooth-level structure and structure language consistency. The authors are saying that right now, models can sound fluent but miss the actual clinical specifics.
Lu: They built this dataset using nine thousand five hundred twenty-four high quality panoramic X-rays that have been checked by expert dentists and radiologists.
Meng: Ninety percent public source, so it’s a big enough collection to try out on something, right? But I'm always worried about the real-world performance when we deploy these things.
Jane: That's fair, Meng, but they say the key is linking structured clinical evidence with free text descriptions using expert logic. It checks if the AI's report matches what an actual doctor would find on a tooth level.
Tom: Exactly, so it’s not just about generating readable sentences anymore; it’s about checking if that sentence actually reflects correct tooth diagnosis across different regions of the image.
Lu: The experiment shows that when you fine tune the models on this PanDent data, their structure language consistency improves a lot, and their visual grounding accuracy goes up too.
Jane: That means we’re seeing a real improvement in how well these models ground their language in the actual anatomy of the radiograph. It moves them away from just saying "there is an issue" to pointing out exactly which tooth has which issue.
Tom: So, the authors are showing that this fine tuning actually brings those model outputs much closer to what an expert validates. That’s a concrete result there, not just a vague claim about being better.
Meng: It sounds like they’re proving that the gap between fluent text and reliable tooth-level reasoning is substantial right now. We still have a lot of work to do if we want these models to be truly dependable in dental practice.
Jane: Right, and while PanDent is a big step forward, they admit it still has limitations with rare findings being underrepresented in the dataset.
Tom: So, the authors are setting up this benchmark but also telling us exactly where we need to go next—focusing on covering those rare things and making sure the AI links language generation directly to tooth aware visual grounding.
Lu: That’s a clear roadmap for future work, showing that they see where the current method stops working.
Faculty of Dentistry, The University of Hong Kong Department of Computing, Imperial College London University of Science and Technology of China Department of Data and Systems Engineering The University of Hong Kong Department of Electronic Engineering The Chinese University of Hong Kong
cs.CV, cs.MM
Submitted: 2026-07-29
Updated: 2026-11-11
Code: https://github.com/HKUDentistry/PanDent
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 98/100
The gist: Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained…
Key concepts
- PanDent
- A large benchmark dataset for OPGs built upon fine-grained, expert-validated tooth-level annotations. It includes 9,524 high-quality OPGs annotated by experienced dentists and validated by oral and maxillofacial radiologists. This dataset is designed to test how well multimodal models can link visual evidence to detailed clinical findings.
- Tooth-index-aware findings
- Annotations that specify clinical attributes requiring localization to particular teeth using the FDI numbering system. These findings are crucial because they require models not just to describe a general area, but to pinpoint specific pathologies on individual teeth, providing fine-grained structural information.
- Structure-language consistency
- The measure of how well an MLLM's generated text matches the structured clinical evidence derived from the OPG. This concept tests whether the model produces reports that are not only grammatically correct but also clinically accurate and consistent with expert-validated tooth-level findings.
- Fine-tuning on PanDent
- The process of training a multimodal model further using the PanDent dataset. This fine-tuning significantly enhances the model's performance in dental domains, specifically improving structure-language consistency, visual localization accuracy, and overall diagnostic correctness compared to models trained without this specific data.
Terminology
Summary
Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation<ref:2607. p. 0 PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental
Radiology
Xiaohan Lia,1, Xinyu Liub,1, Chang Liuc,1, Sum Wing Au Yeunga
, Jun Liud
, Yixuan Yuane,∗∗, Hui Chena,∗
aFaculty of Dentistry, The University of Hong Kong, Hong Kong, China
bDepartment of Computing, Imperial College London, London, United Kingdom
cUniversity of Science and Technology of China, Hefei, China
dDepartment of Data and Systems Engineering, The University of Hong Kong, Hong Kong, China
eDepartment of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China
Abstract
Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG)
is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations. PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental
Radiology
Xiaohan Lia,1, Xinyu Liub,1, Chang Liuc,1, Sum Wing Au Yeunga
, Jun Liud
, Yixuan Yuane,∗∗, Hui Chena,∗
aFaculty of Dentistry, The University of Hong Kong, Hong Kong, China
bDepartment of Computing, Imperial College London, London, United Kingdom
cUniversity of Science and Technology of China, Hefei, China
dDepartment of Data and Systems Engineering, The University of Hong Kong, Hong Kong, China
eDepartment of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China
Abstract
Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG)
is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations. This work introduces PanDent, a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations. The
dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced
dentists and further validated by an oral and maxillofacial radiologist, providing clinically reliable supervision for tooth-level diagnosis and reasoning. The
dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced
dentists and further validated by an oral and maxillofacial radiologist, providing clinically reliable supervision for tooth-level diagnosis and reasoning. Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined
reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions. Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined
reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions. This design
enables evaluation of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with
expert-validated tooth-level findings.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Dataset Construction and Curation
PanDent is curated to support both rigorous benchmark evaluation and supervised training for dental radiology understanding<ref:2607.27378#pg2> The dataset is built from 9,854 raw annotated OPG records collected from multiple public-source batches and one in-house batch<ref:2607. p. 2 As illustrated in Fig. 1, the dataset is built from 9,854 raw
annotated OPG records collected from multiple public-source
batches and one in-house batch, 330 cases with invalid or unresolved image paths are removed. The final release contains
9,524 valid OPG cases, including 9,019 public-source cases
and 505 in-house cases. Each case is associated with structured
annotations covering more than 20 clinically relevant findings
observable from OPG images. Each case is associated with structured
annotations covering more than 20 clinically relevant findings
observable from OPG images. These annotations are labeled
by four professional dentists with over ten years of clinical
experience and further validated by an assistant professor in
oral and maxillofacial radiology on a randomly sampled subset,
providing clinically reliable supervision before conversion into
clinician-guided radiology reports. 1, the dataset is built from 9,854 raw
annotated OPG records collected from multiple public-source
batches and one in-house batch, 330 cases with invalid or unresolved image paths are removed. The final release contains
9,524 valid OPG cases, including 9,019 public-source cases
and 505 in-house cases.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Annotation Structure and Normalization
The raw annotations consist of heterogeneous clinical fields, which are consolidated into 21 reportable clinical findings for factual evaluation<ref:2607.27378#pg2> These findings capture dental findings across four aspects: regional abnormalities (e. p. 3 2 The raw annotations consist of heterogeneous clinical fields, which are consolidated into 21 reportable clinical findings for factual evaluation. From a clinical perspective, the annotation schema captures dental findings across four aspects: regional abnormalities (e. From a clinical perspective, the annotation schema captures dental findings across four aspects: regional abnormalities (e. , maxillary
or mandibular structural conditions), structural and periodontal conditions (e. , maxillary
or mandibular structural conditions), structural and periodontal conditions (e. , angular and generalized crestal bone loss,
furcation involvement), tooth-level pathologies and treatments
(e. , angular and generalized crestal bone loss,
furcation involvement), tooth-level pathologies and treatments
(e. , caries, periapical lesions and restorations), and surrounding anatomical structures (e. , caries, periapical lesions and restorations), and surrounding anatomical structures (e. , maxillary sinus and temporomandibular joint). Besides, an additional ans_comment field
is included to record free-text clinical remarks for cases that
cannot be fully captured by structured labels. For representation and downstream use, the annotations are represented in two
forms: tooth-index-aware findings and binary regional findings,
enabling consistent mapping to radiology reports and reliable
downstream evaluation. Tooth-Index-Aware Findings. Tooth-index-aware findings
refer to clinical attributes that require localization to specific
teeth under the FDI numbering system. Binary Regional Findings. Binary regional findings describe abnormalities assessed at the anatomical-region level. These representations capture both tooth-specific and regionlevel clinical information, providing a structured basis for evaluating structure-language consistency and fine-grained clinical
fidelity. Clinical Attribute Normalization
Raw clinical annotations vary in format and level of detail.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Evaluation Protocol
The benchmark evaluates models on 500 held-out cases that are disjoint from the training split<ref:2607.27378#pg2> For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = 1, 2, 3, 4), METEOR, and ROUGE-L<ref:2607. p. 5 We measure model outputs
from two dimensions: linguistic coherence and clinical accuracy. For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}),
METEOR, and ROUGE-L (Papineni et al. These metrics measure lexical overlap
and sequence-level similarity between the generated and reference reports, where higher scores indicate stronger surfaceform agreement5
. For clinical visual diagnostic accuracy, evaluation is conducted on attribute-targeted outputs instead of full free-text reports. 2 is applied, mapping all
outputs into a unified concise reporting format. The normalized
results in Tab. The normalized
results in Tab. 1 demonstrate that this procedure significantly
reduces format-induced discrepancies.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Key Findings and Model Performance
Experiments show that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, whereas fine-tuning on PanDent significantly improves dental-domain performance and brings model outputs closer to expert-validated annotations<ref:2607.27378#pg2> The results suggest that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis<ref:2607. p. 1 Accordingly, experiments are organized around three complementary aspects: report-level generation quality, fine-grained
clinical factuality, and performance improvement through finetuning. Results suggest that current MLLMs can generate fluent reports while still making substantial errors in fine-grained
clinical findings, whereas fine-tuning on PanDent significantly
2A detailed discussion of medical and dental radiology datasets is provided
in Sec. Results show that current MLLMs can generate fluent
reports, yet fail to produce clinically consistent descriptions, exhibiting substantial errors in fine-grained localization and tooth-level
diagnosis. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness, and bringing model outputs closer to expert dental interpretation.Where the paper says it
Where the paper says it
Conclusion
This paper introduces PanDent, a curated benchmark for evaluating multimodal models in dental panoramic radiography, with a focus on fine-grained tooth-level reasoning and structure-language consistency<ref:2607.27378#pg2> Experiments show that current MLLMs can produce fluent reports but still fail to achieve reliable tooth-level reasoning, revealing a substantial gap to expert dental interpretation<ref:2607. p. 0 Results show that current MLLMs can generate fluent
reports, yet fail to produce clinically consistent descriptions, exhibiting substantial errors in fine-grained localization and tooth-level
diagnosis. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness, and bringing model outputs closer to expert dental interpretation. Results show that current MLLMs can generate fluent
reports, yet fail to produce clinically consistent descriptions, exhibiting substantial errors in fine-grained localization and tooth-level
diagnosis. Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness, and bringing model outputs closer to expert dental interpretation. This long-tailed distribution captures a wide spectrum of dental conditions beyond common pathologies. Rare attributes, such as temporomandibular joint abnormalities, maxillary sinus abnormalities, and broader maxillary or mandibular
structural conditions, are retained for completeness.Where the paper says it
Where the paper says it
Where the paper says it
References
Abacha, A.B., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D. p. 1 The first category directly supports the task of automatic radiology report
generation. IU X-Ray (Demner-Fushman et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. In OPG images, clinically meaningful reasoning depends on
accurate tooth-level localization and interpretation of surrounding anatomical structures. However, on the dental side, existing datasets largely focus on isolated recognition tasks, such as
tooth segmentation, numbering, or limited abnormality classification (Alqaderi et al. , 2023) organizes hierarchical supervision for dental enumeration, abnormal-tooth
detection, and diagnosis on panoramic X-rays. More broadly,
dental image-analysis studies also cover intra-oral radiographs
and CBCT under tasks such as caries screening, periodontal
analysis, landmark localization, and lesion segmentation (Singh
and Raza, 2022). 058
Claude Opus 4. 7 (Bai et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. 058
Claude Opus 4. 7 (Bai et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}),
METEOR, and ROUGE-L (Papineni et al. , 2002; Banerjee and
Lavie, 2005; Lin, 2004). For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}),
METEOR, and ROUGE-L (Papineni et al. , 2002; Banerjee and
Lavie, 2005; Lin, 2004). , 2026) 73 0. 235
MAIRA-2 (Bannur et al. com/HKUDentistry/PanDent. Keywords: dental panoramic radiography, multimodal large language models, radiology report generation, tooth-level reasoning,
clinical benchmark, structure-language consistency
1. , 2019), IU X-Ray (DemnerFushman et al. , 2016), and PadChest (Bustos et al. , 2019) scales this setting to a large deidentified chest-radiograph corpus with associated reports, and
PadChest (Bustos et al. , 2020) further enriches chest X-ray
data with multi-label annotations derived from clinical reports. hk (Yixuan Yuan),
amyhchen@hku. hk (Hui Chen)
1These authors contributed equally to this work. However, progress is limited
by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al. Existing public benchmarks for medical MMLMs are predominantly centered on
chest X-ray imaging, including report-generation datasets such
as MIMIC-CXR (Johnson et al. The first category directly supports the task of automatic radiology report
generation. IU X-Ray (Demner-Fushman et al. Data Collection
Multi-source OPG Acquisition
p Open source datasets:
ToothFairy, DENTEX,
Mandible, Humans, NTU
p In-house dataset:
Collected from PPDH
p Permanent dentition only
p Exclude mixed dentition
and non-dental images
p Normalization
Total: 9524 Cases
Annotation by Dentists
• Fine-grained tooth-level
findings (FDI indices)
• Treatment & restoration status
• Regional / global observations
• Other clinical attributes
Validation by Senior Experts
• Review and validate random
sampled subsets
• Ensure clinical correctness
• Resolve inconsistencies
Expert Annotation Clinician Report Construction
Normalize Structured Annotations
• Convert raw fields to unified
format
• Derive missing-tooth set
• Normalize regional findings
Clinician-Designed Reporting
• Reporting flow designed by clinicians
• Follow real clinical reasoning order:
tooth-level → local findings →
regional/global observations
• Generated with LLM in clinically
natural expressions
Structure-Language Consistency
Check
• Verify alignment between structured
findings and generated text
• Ensure completeness and
consistency
Final Dataset
9,524 Radiograph-Report Pairs
Training Set
9,024 cases
(~ 94. 8%)
Test Set
500 cases
(~ 5. 230
Llama-3. 2-90B-Vision-Instruct (Dubey et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. 0 (HARI and MVL, 2026) 4 0. 365
OralGPT-Captioning-4B-Base (Fan et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. , 2022; Liu et al. , 2024b,a; Gao et al. The rule-based verifier operates
on normalized structured annotations prior to report generation. It derives the missing-teeth set from the permanent dentition and removes absent teeth from all tooth-index-aware
findings, preventing anatomically invalid assignments. , 2023) releases a panoramic radiograph
benchmark with baselines and an online platform, where the
core annotations support tasks such as tooth segmentation and
numbering. DENTEX (Hamamci et al. DENTEX (Hamamci et al. , 2023) organizes hierarchical supervision for dental enumeration, abnormal-tooth
detection, and diagnosis on panoramic X-rays. As a result, they are valuable for dental visual recognition
but cannot directly serve as benchmarks for radiology report
generation or structure-language consistency evaluation. The
most relevant resource to our curated dataset is MMOral (Hao
et al. The
most relevant resource to our curated dataset is MMOral (Hao
et al. , 2025), which provides a large-scale multimodal instruction dataset for panoramic X-ray analysis around five diagnostic dimensions, including teeth condition, pathological findings,
historical treatments, jawbone observations, and clinical summary and recommendation. 255
Libra-v1. 0-3B (Zhang et al. 255
Libra-v1. 0-3B (Zhang et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. The third category provides structured
clinical supervision, either as disease labels or as report-derived
entities and relations. For instances, PadChest includes imagelevel labels, while RadGraph (Jain et al. For instances, PadChest includes imagelevel labels, while RadGraph (Jain et al. , 2021) converts radiology reports into clinical entities and relations for factual evaluation. However, progress is limited
by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al. Existing public benchmarks for medical MMLMs are predominantly centered on
chest X-ray imaging, including report-generation datasets such
as MIMIC-CXR (Johnson et al. However, progress is limited
by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al. Existing public benchmarks for medical MMLMs are predominantly centered on
chest X-ray imaging, including report-generation datasets such
as MIMIC-CXR (Johnson et al. It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza,
2022; Panetta et al. , 2022; Liu et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza,
2022; Panetta et al. , 2022; Liu et al. com/HKUDentistry/PanDent. Keywords: dental panoramic radiography, multimodal large language models, radiology report generation, tooth-level reasoning,
clinical benchmark, structure-language consistency
1. Recent work has explored data-driven models for extracting clinical findings and generating structured or free-text descriptions from OPG images. However, progress is limited
by the lack of large, clinically reliable benchmarks for expertlevel evaluation (Lutescu et al. , 2023) organizes hierarchical supervision for dental enumeration, abnormal-tooth
detection, and diagnosis on panoramic X-rays. More broadly,
dental image-analysis studies also cover intra-oral radiographs
and CBCT under tasks such as caries screening, periodontal
analysis, landmark localization, and lesion segmentation (Singh
and Raza, 2022). Case Study and Discussion
Tab. 5 presents a representative case from the PanDent test
benchmark, including the input radiograph, prompt instruction,
ground-truth report with attribute annotations, and outputs from
three representative models: GPT-5. 1 (OpenAI, 2023) N/A 0. 210
Gemini 3 Pro Preview (Gemini Team, 2023) N/A 0. 1 (OpenAI, 2023) N/A 0. 210
Gemini 3 Pro Preview (Gemini Team, 2023) N/A 0. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. , 2024) 90 0. 398
Medical-Domain MLLMs
MedGemma-27B-IT (MedGemma Team, 2025) 27 0. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. , 2024) 27 0. 222
Pixtral-Large-Instruct-2411 (Mistral AI, 2024) 124 0. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. 216
MiniCPM-V-4_5 (OpenBMB Team, 2025) 8 0. 291
Kimi-VL-A3B-Thinking-2506 (Moonshot AI, 2025) 3 0. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. Method Size (B) BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L Average
General-Domain Closed-Source MLLMs
GPT-5. 4 (OpenAI, 2023) N/A 0. Case Study and Discussion
Tab. 5 presents a representative case from the PanDent test
benchmark, including the input radiograph, prompt instruction,
ground-truth report with attribute annotations, and outputs from
three representative models: GPT-5. , 2025) 78 0. 216
MiniCPM-V-4_5 (OpenBMB Team, 2025) 8 0. , 2025) 78 0. 216
MiniCPM-V-4_5 (OpenBMB Team, 2025) 8 0. Introduction
Dental panoramic radiography (orthopantomogram, OPG) is
a cornerstone of routine oral healthcare. It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza,
2022; Panetta et al. Particularly, the Tufts Dental Database (Panetta et al. , 2022) provides
panoramic radiographs with tooth-level annotations and auxiliary multimodal signals, making it useful for benchmarking
2 We measure model outputs
from two dimensions: linguistic coherence and clinical accuracy. For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = {1, 2, 3, 4}),
METEOR, and ROUGE-L (Papineni et al. The best and second-best results are
highlighted as in red and blue, respectively. Method Size (B) BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L Average
General-Domain Closed-Source MLLMs
GPT-5. For example, dental caries are often annotated as a global label, whereas clinical practice requires identifying the exact tooth locations. As a result, such datasets do not
provide sufficiently fine-grained, expert-validated annotations
to support faithful evaluation against clinical standards (Pham
and Al-Hebshi, 2026), making it difficult to determine whether
models truly capture clinically relevant findings or merely produce plausible reports. For example, dental caries are often annotated as a global label, whereas clinical practice requires identifying the exact tooth locations. As a result, such datasets do not
provide sufficiently fine-grained, expert-validated annotations
to support faithful evaluation against clinical standards (Pham
and Al-Hebshi, 2026), making it difficult to determine whether
models truly capture clinically relevant findings or merely produce plausible reports. Yet, their dominant modality remains on chest radiography or pathology microscopy. Therefore, they do not directly
address the tooth-index-aware reasoning and dental terminology required by panoramic dental radiology analysis. , 2026) N/A 0. 211
General-Domain Open-Source MLLMs
Qwen3-VL-235B-A22B-Instruct (Qwen Team, 2025) 235 0. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. , 2026) N/A 0. 211
General-Domain Open-Source MLLMs
Qwen3-VL-235B-A22B-Instruct (Qwen Team, 2025) 235 0. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. diagnostic systems and studying expert interpretation behavior. OdontoAI (Silva et al. Particularly, the Tufts Dental Database (Panetta et al. , 2022) provides
panoramic radiographs with tooth-level annotations and auxiliary multimodal signals, making it useful for benchmarking
2 Introduction
Dental panoramic radiography (orthopantomogram, OPG) is
a cornerstone of routine oral healthcare. It provides a comprehensive view of the dentition and surrounding anatomical structures, forming the primary basis for clinical evaluation, treatment planning, and radiological diagnosis (Singh and Raza,
2022; Panetta et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. , 2021), and PathVQA (He et al. While these
datasets have advanced medical vision-language modeling,
they primarily focus on global image-level understanding and
do not reflect the structure of dental panoramic interpretation. Experimental Setups
Benchmarked Multi-modal Large Language Models
(MLLMs). To comprehensively evaluate mainstream multimodal large language models (MLLMs), three categories
are considered: closed-source models, open-source models,
and medical-domain models. 255
Libra-v1. 0-3B (Zhang et al. , 2020), as
well as medical visual question answering benchmarks such
as ImageCLEF-VQA-Med (Abacha et al. , 2019), SLAKE (Liu
Preprint submitted to Medical Image Analysis July 31, 2026
arXiv:2607. 291
Kimi-VL-A3B-Thinking-2506 (Moonshot AI, 2025) 3 0. 1V-9B-Thinking (Zhipu AI, 2025) 9 0. com/HKUDentistry/PanDent. Keywords: dental panoramic radiography, multimodal large language models, radiology report generation, tooth-level reasoning,
clinical benchmark, structure-language consistency
1. , 2025) 72 0. 220
InternVL3-78B (Zhu et al. Experimental Setups
Benchmarked Multi-modal Large Language Models
(MLLMs). To comprehensively evaluate mainstream multimodal large language models (MLLMs), three categories
are considered: closed-source models, open-source models,
and medical-domain models.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Fine-tuning on PanDent significantly improves structure-language consistency, substantially enhancing visual localization accuracy and diagnostic correctness. This enables models to produce reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings.
-
The system can perform evaluation across three complementary aspects: report-level generation quality, fine-grained clinical factuality, and performance improvement through finetuning. This allows for a rigorous assessment of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings.
-
The improved system can achieve higher performance on attribute-targeted outputs by utilizing the
exact match accuracy
metric (Accj). This measures how well the model generates a statement matching the ground truth for specific attributes, such asGeneralized crestal bone loss in the mandible,
directly improving clinical diagnostic accuracy beyond conventional report-level metrics.
Abstract
Accurate evaluation of multimodal large language models in dental panoramic radiography is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale benchmark built on expert-validated tooth-level annotations. It comprises 9,554 panoramic radiographs with comprehensive structured findings produced by experienced dentists and further validated by an oral and maxillofacial radiologist. Clinician-guided reports are constructed from these findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions. The dataset supports complementary evaluation of report-level textual agreement and tooth-level clinical consistency, with 9,049 public-source cases for supervised training and an independent 505-case in-house benchmark. Experiments cover proprietary, general-domain open-source, and medical-specific models. Current models generate fluent reports but exhibit substantial errors in fine-grained localization and tooth-level diagnosis. Supervised fine-tuning on PanDent markedly improves report alignment and diagnostic accuracy, bringing predictions closer to expert dental interpretation. These results establish PanDent as a rigorous benchmark and training resource for clinically grounded dental multimodal reasoning. Dataset sample: https://huggingface.co/datasets/Desperado1103/Pandent Sample; code: https://github.com/HKUDentistry/PanDent. Full data will be released upon publication under source redistribution terms.
Sources
- Qwen2.5-VL Technical Report
- Constitutional AI: Harmlessness from AI Feedback
- MAIRA-2: Grounded Radiology Report Generation
- A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation
- The Llama 3 Herd of Models
- OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis
- Gemini: A Family of Highly Capable Multimodal Models
- Gemma 3 Technical Report
- DENTEX: Dental Enumeration and Tooth Pathosis Detection Benchmark for Panoramic X-ray
- Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis
- PathVQA: 30000+ Questions for Medical Visual Question Answering
- RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
- MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
- Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
- A Systematic Review of Deep Learning-based Research on Radiology Report Generation
- A Safety Report on GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5
- MedGemma Technical Report
- Pixtral 12B
- Kimi-VL Technical Report
- GPT-4 Technical Report
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs