PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology
summary
The gist
Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained…
In short
PanDent is a large-scale benchmark for evaluating multimodal models in dental panoramic radiography (OPG). It addresses the lack of fine-grained, clinically reliable benchmarks by providing 9,524 OPGs with expert-validated tooth-level annotations. The study shows that while current models generate fluent reports, they struggle with accurate tooth-level reasoning and structure-language consistency. Fine-tuning on PanDent significantly improves these critical dental diagnostic capabilities.
Key concepts
- PanDent
- A large benchmark dataset for OPGs built upon fine-grained, expert-validated tooth-level annotations. It includes 9,524 high-quality OPGs annotated by experienced dentists and validated by oral and maxillofacial radiologists. This dataset is designed to test how well multimodal models can link visual evidence to detailed clinical findings.
- Tooth-index-aware findings
- Annotations that specify clinical attributes requiring localization to particular teeth using the FDI numbering system. These findings are crucial because they require models not just to describe a general area, but to pinpoint specific pathologies on individual teeth, providing fine-grained structural information.
- Structure-language consistency
- The measure of how well an MLLM's generated text matches the structured clinical evidence derived from the OPG. This concept tests whether the model produces reports that are not only grammatically correct but also clinically accurate and consistent with expert-validated tooth-level findings.
- Fine-tuning on PanDent
- The process of training a multimodal model further using the PanDent dataset. This fine-tuning significantly enhances the model's performance in dental domains, specifically improving structure-language consistency, visual localization accuracy, and overall diagnostic correctness compared to models trained without this specific data.
Terminology used across episodes
This episode discusses
- PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology · Paper Radio
- Qwen2.5-VL Technical Report
- Constitutional AI: Harmlessness from AI Feedback
- MAIRA-2: Grounded Radiology Report Generation
- A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation
- The Llama 3 Herd of Models · Paper Radio
- OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis
- Gemini: A Family of Highly Capable Multimodal Models
- Gemma 3 Technical Report
- DENTEX: Dental Enumeration and Tooth Pathosis Detection Benchmark for Panoramic X-ray
- Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis
- PathVQA: 30000+ Questions for Medical Visual Question Answering
- RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
- MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
- Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning · Paper Radio
- A Systematic Review of Deep Learning-based Research on Radiology Report Generation
- A Safety Report on GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5
- MedGemma Technical Report · Paper Radio
- Pixtral 12B
- Kimi-VL Technical Report
- GPT-4 Technical Report
The paper
PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology · Read on arXiv
Faculty of Dentistry, The University of Hong Kong Department of Computing, Imperial College London University of Science and Technology of China Department of Data and Systems Engineering The University of Hong Kong Department of Electronic Engineering The Chinese University of Hong Kong
Accurate evaluation of multimodal large language models in dental panoramic radiography is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation. This work introduces PanDent, a large-scale benchmark built on expert-validated tooth-level annotations. It comprises 9,554 panoramic radiographs with comprehensive structured findings produced by experienced dentists and further validated by an oral and maxillofacial radiologist. Clinician-guided reports are constructed from these findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions. The dataset supports complementary evaluation of report-level textual agreement and tooth-level clinical consistency, with 9,049 public-source cases for supervised training and an independent 505-case in-house benchmark. Experiments cover proprietary, general-domain open-source, and medical-specific models. Current models generate fluent reports but exhibit substantial errors in fine-grained localization and tooth-level diagnosis. Supervised fine-tuning on PanDent markedly improves report alignment and diagnostic accuracy, bringing predictions closer to expert dental interpretation. These results establish PanDent as a rigorous benchmark and training resource for clinically grounded dental multimodal reasoning. Dataset sample: https://huggingface.co/datasets/Desperado1103/Pandent Sample; code: https://github.com/HKUDentistry/PanDent. Full data will be released upon publication under source redistribution terms.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology".
Jane: Here's the gist of the paper. Accurate evaluation of multimodal large language models (MLLMs) in dental panoramic radiography (orthopantomogram, OPG) is limited by the lack of fine-grained, clinically reliable benchmarks that reflect expert interpretation<ref:2607.27378#pg2> PanDent introduces a large-scale, clinically grounded OPG benchmark built upon fine-grained, expert-validated tooth-level annotations<ref:2607.27378#pg6> This dataset comprises 9,524 high-quality OPGs, each associated with comprehensive structured annotations produced by experienced dentists and further validated by an oral and maxillofacial radiologist<ref:2607.27378#pg2> Clinically consistent radiology reports are constructed from expert-validated findings using clinician-defined reporting logic, establishing explicit correspondence between structured clinical evidence and free-text descriptions<ref:2607.27378#pg2> This design enables evaluation of whether MLLMs generate reports that are not only linguistically coherent but also clinically consistent with expert-validated tooth-level findings<ref:2607.27378#pg2> Dataset Construction and Curation PanDent is curated to support both rigorous benchmark evaluation and supervised training for dental radiology understanding<ref:2607.27378#pg2> The dataset is built from 9,854 raw annotated OPG records collected from multiple public-source batches and one in-house batch<ref:2607.27378#pg2> Each case is associated with structured annotations covering more than 20 clinically relevant findings observable from OPG images<ref:2607.27378#pg2> The annotations are labeled by four professional dentists with over ten years of clinical experience and further validated by an assistant professor in oral and maxillofacial radiology on a randomly sampled subset, providing clinically reliable supervision before conversion into clinician-guided radiology reports<ref:2607.27378#pg2> The final release contains 9,524 valid OPG cases, including 9,019 public-source cases and 505 in-house cases<ref:2607.27378#pg2> Annotation Structure and Normalization The raw annotations consist of heterogeneous clinical fields, which are consolidated into 21 reportable clinical findings for factual evaluation<ref:2607.27378#pg2> These findings capture dental findings across four aspects: regional abnormalities (e.g., maxillary or mandibular structural conditions), structural and periodontal conditions (e.g., angular and generalized crestal bone loss, furcation involvement), tooth-level pathologies and treatments (e.g., caries, periapical lesions and restorations), and surrounding anatomical structures (e.g., maxillary sinus and temporomandibular joint)<ref:2607.27378#pg2> The annotations are represented in two forms: tooth-index-aware findings and binary regional findings<ref:2607.27378#pg2> Tooth-index-aware findings refer to clinical attributes that require localization to specific teeth under the FDI numbering system<ref:2607.27378#pg2> Binary regional findings describe abnormalities assessed at the anatomical-region level<ref:2607.27378#pg2> These representations capture both tooth-specific and region-level clinical information, providing a structured basis for evaluating structure-language consistency and fine-grained clinical fidelity<ref:2607.27378#pg2> Evaluation Protocol The benchmark evaluates models on 500 held-out cases that are disjoint from the training split<ref:2607.27378#pg2> For linguistic coherence, standard natural language generation metrics are used, including BLEU-n (n = 1, 2, 3, 4), METEOR, and ROUGE-L<ref:2607.27378#pg2> For clinical visual diagnostic accuracy, evaluation is conducted on attribute-targeted outputs instead of full free-text reports<ref:2607.27378#pg2> The normalized results in Tab.
Tom: 1 demonstrate that this procedure significantly reduces format-induced discrepancies<ref:2607.27378#pg2> Key Findings and Model Performance Experiments show that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, whereas fine-tuning on PanDent significantly improves dental-domain performance and brings model outputs closer to expert-validated annotations<ref:2607.27378#pg2> The results suggest that current MLLMs can generate fluent reports while still making substantial errors in fine-grained clinical findings, exhibiting substantial errors in fine-grained localization and tooth-level diagnosis<ref:2607.27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, substantially enhancing visual localization accuracy and diagnostic correctness<ref:2607.27378#pg2> This paper introduces PanDent, a curated benchmark for evaluating multimodal models in dental panoramic radiography, with a focus on fine-grained tooth-level reasoning and structure-language consistency<ref:2607.27378#pg2> Experiments show that current MLLMs can produce fluent reports but still fail to achieve reliable tooth-level reasoning, revealing a substantial gap to expert dental interpretation<ref:2607.27378#pg2> Fine-tuning on PanDent significantly improves structurelanguage consistency, visual grounding accuracy, and toothlevel diagnostic correctness<ref:2607.27378#pg2> Despite these advances, the benchmark remains limited by the long-tailed distribution of rare findings and the uneven coverage across certain clinical attributes<ref:2607.27378#pg2> Future work will focus on expanding pathology coverage, improving the representation and sampling of underrepresented attributes, and developing models that explicitly couple language generation with tooth-aware visual grounding<ref:2607.27378#pg2> References Abacha, A.B., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D., Müller, H., 2019.
Jane: VQA-Med: Overview of the medical visual question answering task at ImageCLEF 2019<ref:2607.27378#pg2> Alqaderi, H., Phillips, C., Jones, J.A., Ramchandani, K., Mitwalli, A.H., Youssef, N.A., Emami, S.E., Teixeira, T., Ferreira, P.S., Yousfi, H., Cheela, D., Bahaj, A.: Automated detection of dental caries and bone loss in bitewing radiographs using YOLOv8: A multicenter deep learning study<ref:2607.27378#pg2> Bai, S., et al., 2025.
Tom: Qwen2.5-VL technical report<ref:2607.27378#pg2> Bai, Y., et al., 2024.
Paper summary: Tom: So we're looking at this paper called "PanDent: Toward Comprehensive Tooth-Level Structure-Language Consistency in Dental Radiology." Basically, the authors are saying that right now, when we test multimodal models on panoramic X-rays, there just aren't enough benchmarks that actually reflect what an expert dentist would interpret.
Jane: That sounds like a big problem because if the benchmark isn't fine-grained enough, we can't really tell if an AI is actually getting the right details about individual teeth and not just guessing generally. What are they proposing with this PanDent dataset?
Tom: They’re introducing a large-scale dataset of nine thousand five hundred twenty-four high-quality panoramic X-rays that have been annotated with very specific, tooth-level findings <ref:2607.27378#pg1>. These aren't just simple "there is a cavity" labels; they come from experienced dentists and are further validated by an oral and maxillofacial radiologist.
Lu: What's important here is how they built the annotations. They focused on covering more than twenty clinically relevant findings for each case, broken down into four main categories: regional abnormalities, structural conditions, tooth-level pathologies and treatments, and surrounding anatomical structures.
Meng: Twenty findings per case sounds detailed, but what does that actually look like in practice? Is it just labeling every single tooth with a status for everything?
Jane: Well, they broke those raw clinical fields down into twenty-one reportable clinical findings for factual evaluation. These cover things like regional issues—structural conditions—tooth-level problems like caries or restorations, and even surrounding areas like the maxillary sinus and the temporomandibular joint.
Tom: They represented these findings in two ways: tooth-index aware findings, which need to be located on a specific tooth using an FDI numbering system, and binary regional findings that describe abnormalities at a broader anatomical region level.
Lalam: The tooth-index aware findings are crucial because they require localization to specific teeth under the FDI numbering system. This gives the model something concrete to ground its language in, rather than just describing a general area as abnormal.
Meng: So if we're talking about deployment, what's the structure of this data actually like when you look at it? Is it just one big table?
Lu: They have nine thousand five hundred twenty-four valid OPG cases in total <ref:2607.27378#pg1>. These are made up of nine thousand nineteen public-source cases and five hundred five in-house cases. The process also includes creating clinically consistent radiology reports using clinician-defined reporting logic to establish a direct link between the structured clinical evidence and the free text descriptions.
Jane: That connection is key because it lets them evaluate if an AI generates reports that aren't just fluent, but are actually consistent with those expert-validated tooth-level findings. It tests structure language consistency.
Tom: The evaluation protocol they used involves five hundred held-out cases disjoint from the training split. For linguistic coherence, they use standard metrics like BLEU-n, METEOR, and ROUGE L to see if the text flows well.
Paper summary: Meng: But what about accuracy? If we're moving toward production models, I need to know how this helps with actual diagnostic correctness instead of just checking fluency metrics.
Jane: They focused on attribute-targeted outputs for clinical visual diagnostic accuracy, which is different from just checking the whole report text. The normalized results showed that this method significantly reduces format-induced discrepancies between what's structured and what's written.
Lu: The main finding in their experiments was that current multimodal large language models can generate reports that sound fluent, but they still make substantial errors when it comes to fine-grained clinical findings.
Tom: Specifically, the paper suggests these models show substantial errors in fine-grained localization and tooth-level diagnosis. It's not just about the words sounding right; the AI is missing details on which specific tooth has which problem.
Meng: So if I were deploying a model right now, would I expect it to be accurate enough without this kind of fine-tuning?
Lu: Fine-tuning on PanDent significantly improves structure language consistency, substantially enhancing visual localization accuracy and diagnostic correctness. It moves the model closer to what an expert validates.
Jane: So what's the big picture implication for us in the AI community? What does PanDent actually change for how we think about these models?
Tom: The authors are introducing PanDent as a benchmark built on fine-grained, expert-validated tooth-level annotations. It highlights that while models can generate fluent reports, they still lack reliable tooth-level reasoning compared to actual expert dental interpretation.
Meng: It sounds like the gap between fluent text generation and clinically reliable diagnosis is pretty wide right now. What stops them from solving it completely?
Jane: The paper does admit there are limitations to this benchmark itself. They mention that the benchmark remains limited by the long-tailed distribution of rare findings, which means some findings might be underrepresented, and there's uneven coverage across certain clinical attributes.
Lu: Future work they point toward involves expanding pathology coverage and improving the representation and sampling of those underrepresented attributes. They also plan to develop models that explicitly couple language generation with tooth aware visual grounding.
Tom: So, in the end, PanDent provides a structured way to test if these next generation multimodal models can actually handle the necessary level of detail required for dental diagnosis.
Meng: It's a good step toward making these models more trustworthy for clinical settings, even if we still have to deal with those rare findings they mentioned.
Jane: Exactly. The implication is that moving forward, we need benchmarks that force models to go beyond surface-level fluency and demonstrate genuine understanding of the underlying clinical structure of the image.
Conclusion: Tom: So, PanDent is this new benchmark they put out for multimodal models in dental panoramic X-rays.
Jane: It’s about testing whether an AI can actually get the fine details of individual teeth, not just guess general problems on a picture.
Tom: Right, it’s focused on tooth-level structure and structure language consistency. The authors are saying that right now, models can sound fluent but miss the actual clinical specifics.
Lu: They built this dataset using nine thousand five hundred twenty-four high quality panoramic X-rays that have been checked by expert dentists and radiologists.
Meng: Ninety percent public source, so it’s a big enough collection to try out on something, right? But I'm always worried about the real-world performance when we deploy these things.
Jane: That's fair, Meng, but they say the key is linking structured clinical evidence with free text descriptions using expert logic. It checks if the AI's report matches what an actual doctor would find on a tooth level.
Tom: Exactly, so it’s not just about generating readable sentences anymore; it’s about checking if that sentence actually reflects correct tooth diagnosis across different regions of the image.
Lu: The experiment shows that when you fine tune the models on this PanDent data, their structure language consistency improves a lot, and their visual grounding accuracy goes up too.
Jane: That means we’re seeing a real improvement in how well these models ground their language in the actual anatomy of the radiograph. It moves them away from just saying "there is an issue" to pointing out exactly which tooth has which issue.
Tom: So, the authors are showing that this fine tuning actually brings those model outputs much closer to what an expert validates. That’s a concrete result there, not just a vague claim about being better.
Meng: It sounds like they’re proving that the gap between fluent text and reliable tooth-level reasoning is substantial right now. We still have a lot of work to do if we want these models to be truly dependable in dental practice.
Jane: Right, and while PanDent is a big step forward, they admit it still has limitations with rare findings being underrepresented in the dataset.
Tom: So, the authors are setting up this benchmark but also telling us exactly where we need to go next—focusing on covering those rare things and making sure the AI links language generation directly to tooth aware visual grounding.
Lu: That’s a clear roadmap for future work, showing that they see where the current method stops working.
More episodes
- 2603.28113-The multiplicative Lipschitz bound as an inductive bias
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound