From Documents to Spans: Span-Centric Learning for LLM-based Explainable Medical Coding

The gist The authors propose Span-Centric Learning (SCL), a training framework that strengthens Large Language Models’ coding ability at the span level and transfers this capability to full clinical documents, achieving an 8.2-point improvement in macro-F1 at only 20% of standard SFT training cost while providing explicit supporting evidence for each predicted code.

Problem Formulation and Motivation

ICD coding is essential for healthcare billing and clinical analysis, but existing public datasets typically provide only code labels without evidence annotations, limiting models’ ability to learn evidence-grounded predictions <ref:2603.15270#pg5>. Manual coding is time-consuming and errorprone, requiring each assigned code to be traceable to explicit supporting evidence in the medical record <ref:2603.15270#pg9>. Early discriminative models struggle to approximate supporting evidence, whereas LLMs offer a natural alternative by generating supporting evidence before predicting codes, enabling a human-in-the-loop workflow <ref:2603.15270#pg10>. The paper argues that dense document-level evidence annotation is not always necessary for learning evidence-based coding, suggesting that document-level ICD coding can be decomposed into three sub-tasks: evidence localization, evidence aggregation and code assignment <ref:2603.15270#pg2>.

Span-Centric Learning (SCL) Framework

SCL is a training framework that explicitly separates span-level code knowledge learning from document-level information aggregation learning <ref:2603.15270#pg5>. It employs a mixed training strategy optimizing the objective min θ E(x,E,C)∼D∗ doc LSFTfθ(x),(E, C) + E(e,c)∼Dspan LSFTfθ(e), c <ref:2603.15270#pg5>. This involves:

1. Using a small set of documents with annotated evidence spans to supervise evidence aggregation and code assignment over full documents <ref:2603.15270#pg5>.

2. Leveraging a large collection of lightweight evidence spans to reinforce span-level reasoning <ref:2603.15270#pg5>.

This approach enables small-scale LLMs to approach the performance of much larger proprietary models while retaining interpretability and supporting human intervention <ref:2603.15270#pg5>.

Code-Centric Data Expansion

To construct large-scale span-level data, the authors propose a code-centric data expansion strategy that improves coverage of code-specific evidence <ref:2603.15270#pg5>. This strategy builds a multi-tier span-level knowledge base:

** Gold pairs from human knowledge bases:**

The gold pairs are obtained by treating Alphabetic Index terms paired with their default ICD codes as gold evidence-code pairs <ref:2603.15270#pg5> and utilizing these resources for retrieval-augmented generation <ref:2603.15270#pg5>.

** Silver pairs from public datasets:**

Silver pairs are constructed via a two-stage LLM pipeline using Llama-3.1-70B, where the first stage extracts supporting evidence spans and the second stage summarizes them into representative expressions <ref:2603.15270#pg5>.

** Synthetic pairs for uncovered codes:**

To achieve full code coverage, synthetic pairs are synthesized using GPT-5.1 guided by ICD knowledge to infer evidence that plausibly supports target codes based on their nearest neighbors in the ICD-10-CM hierarchy <ref:2603.15270#pg5>.

Training and Inference

The LLM is fine-tuned on mixed document-level and span-level data under a standard autoregressive objective min θ LSFT(θ) = X N i=1 X Ti t=1 − log pθ y(i) t x(i), y(i). The document-level data teaches the model to aggregate evidence across the full clinical context and predict ICD codes, while code-centric data injects code-specific knowledge beyond document-level coverage <ref:2603.15270#pg5>. At inference time, the model is prompted to identify relevant evidence spans before assigning ICD codes while leveraging the code knowledge learned during code-centric learning <ref:2603.15270#pg5>.

Empirical Validation and Findings

Experiments validate that SCL improves coding accuracy, unseen code learning, and evidence extraction <ref:2603.15270#pg5>.

** Finding 1 (Unseen codes):**

SCL brings substantial accuracy improvements on unseen codes using spans only, indicating the model can learn previously unseen codes from spans alone and transfer this knowledge to document-level ICD coding <ref:2603.15270#pg5>.

** Finding 2 (Evidence extraction):**

Adding span-level data further improves evidence extraction, especially Recall, confirming that the benefit of span-level supervision is not limited to the final code assignment but also helps the model identify more complete and human-consistent supporting evidence <ref:2603.15270#pg5>.

Conclusion

SCL outperforms code-only fine-tuning in both accuracy and training efficiency, while producing explicit supporting evidence that enables human-in-the-loop auditing and revision <ref:2603.15270#pg5>. The span-centric perspective offers a general strategy for annotation-efficient training in clinical NLP tasks where document-level supervision is costly but span-level knowledge is more accessible <ref:2603.15270#pg5>. Future work could investigate the relationship between span quality and coding performance <ref:2603.15270#pg5>.

--- Page 1 ---

From Documents to Spans: Scalable Supervision for Evidence-Based ICD Coding with LLMs Xu Zhang, Wenxin Ma, Chenxu Wu, Rongsheng Wang, Zhiyang He, Xiaodong Tao, Kun Zhang and S. Kevin Zhou. School of Biomedical Engineering, Division of Life Sciences and Medicine, USTC 2 MIRACLE Center, Suzhou Institute for Advance Research USTC 3 Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology 4 State Key Laboratory of Precision and Intelligent Chemistry USTC 5 iFlyHealth xu zhang@mail.ustc.edu.cn kkzhang@ustc.edu.cn skevinzhou@ustc.edu.cn Abstract International Classification of Diseases (ICD) coding assigns diagnosis codes to clinical documents and is essential for healthcare billing and clinical analysis. Reliable coding requires that each predicted code be supported by explicit textual evidence. However, existing public datasets provide only code labels, without evidence annotations, limiting models’ ability to learn evidence-grounded predictions. In this work, we argue that dense, document-level evidence annotation is not always necessary for learning evidence-based coding. Instead, models can learn code-specific evidence patterns from local spans and use these patterns to support document-level evidence-based coding. Based on this insight, we propose Span-Centric Learning (SCL), a training framework that strengthens LLMs’ coding ability at the span level and transfers this capability to full clinical documents. Specifically, we use a small set of annotated documents to supervise evidence recognition, aggregation, and code assignment, while leveraging a large collection of lightweight evidence spans to reinforce span-level reasoning. Due to their compactness, span annotations are scalable and can be further augmented through synthesis. Under the same Llama3.1-8B backbone, our approach achieves an 8.2-point improvement in macro-F1 at only 20% of the training cost of standard SFT, and provides explicit supporting evidence for each predicted code, enabling human auditing and revision.

--- Page 2 ---

1 Introduction ICD coding is the task of assigning standardized diagnosis codes to long, clinical documents and serves as a foundational component of modern healthcare, directly affecting insurance reimbursement, epidemiological surveillance, and health data analysis. Manual coding is time-consuming and errorprone; even experienced coders make frequent mistakes [3, 11, 8], motivating decades of research on automated solutions. Crucially, each assigned code should be traceable to explicit supporting evidence in the medical record, consistent with official coding guidelines [5]. Yet such evidence annotations are extremely scarce: existing datasets typically provide only code labels. To the best of our knowledge, the only publicly available dataset with expert-annotated evidence is MDACE [4], a small subset of MIMIC-III [14], whose limited scale restricts its use to evaluation rather than training.

1. Bold Header: Span-Centric Learning (SCL) Framework Implementation

This framework decouples span-level code knowledge learning from document-level information aggregation learning by using a mixed training strategy: "a small number of documents with annotated evidence spans are used to supervise evidence aggregation and code assignment over full documents, while a large collection of standalone evidence spans is leveraged to learn robust evidence-to-code associations. This allows small LLMs to approach the performance of much larger proprietary models while retaining interpretability and supporting human intervention."

2. Bold Header: Code Knowledge Expansion via Multi-Tier Data Strategy

To achieve full code coverage, the system utilizes a multi-tier span-level code knowledge base composed of gold, silver, and synthetic evidence–code pairs, which includes methods like mining silver pairs from MIMIC-IV documents and synthesizing pairs for uncovered codes using GPT-5.1 guided by ICD knowledge.

3. Bold Header: Enhanced Evidence Extraction Fidelity

The system can improve evidence extraction beyond the final code prediction by leveraging span-level supervision; specifically, adding 150k evidence spans further improves evidence extraction, showing that SCL enhances evidence identification. This results in models that produce more complete and human-consistent supporting evidence.

4. Bold Header: Unseen Code Generalization

The improved system can learn previously unseen codes from spans alone by demonstrating that span-level supervision introduces knowledge of codes absent from document-level training data, leading to an improvement in accuracy on unseen codes using only spans only, without additional documents.

5. Bold Header: Human-in-the-Loop Auditing

The system enables a practical workflow where clinicians can review and revise LLM-generated evidence because the framework produces explicit supporting evidence for each predicted code, enabling human auditing and revision.

Medical coding, which assigns standardized ICD codes to clinical documents, is a fundamental yet challenging task in modern healthcare. It requires extensive knowledge of a large code space and the ability to identify and interpret sparse supporting evidence in long, noisy clinical documents. Most existing methods rely on document-level supervision, which provides limited guidance on supporting evidence and insufficient training signals for rare codes. To address these limitations, we propose Span-Centric Learning (SCL), which separates coding knowledge acquisition from its application in full documents. SCL first learns coding knowledge from short spans, then learns to use this knowledge in full documents from a small set of annotated data. Experiments show that, our approach enables an 8B LLM to surpass state-of-the-art methods using only 248 complete documents for training. It also provides explicit supporting evidence for each predicted code, with strong alignment to human annotations. Further analysis shows that the model can learn knowledge of codes absent from documents through spans and apply that knowledge to full documents.