ALTER: Modeling Longitudinal Changes via Regional Differencing for 3D CT Report Generation

arXiv:2608.05615 · cs.CV · Submitted 2026-08-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "ALTER: Modeling Longitudinal Changes via Regional Differencing for 3D CT Report Generation".

Jane: Here's the gist of the paper. Computed tomography (CT) is widely used for clinical diagnosis and longitudinal follow-up, yet automatically generating accurate and complete radiology reports from threedimensional (3D) CT remains challenging Existing methods improve fine-grained correspondence between images and text by modeling anatomical regions, but remain centered on the current examination Consequently, patient-specific longitudinal changes within individual regions remain insufficiently modeled.

Tom: We propose Anatomically Localized Temporal Evidence Representation (ALTER) to address these limitations.

Jane: Proposed Framework Components ALTER coordinates three complementary mechanisms: prior integration, regional differencing, and interval change fusion.

Tom: Global Prior Integration (GPI) establishes study-level historical context by interacting current and prior volume representations through gated attention and incorporating the encoded prior report.

Paper summary: Lu: So, to wrap up on "ALTER: Modeling Longitudinal Changes via Regional Differencing for three dee CT Report Generation", this framework coordinates multimodal history with anatomy-specific temporal comparison and report-level change aggregation <ref:2608.05615#pg1,ALTER: Modeling Longitudinal Changes via Regional Differencing for 3D CT Report Generation>. Meng It’s essentially using regional proxy differencing to get localized interval representations from a single shared encoding of the prior volume, which then feeds into interval change fusion to create prompts that guide the language model.

Tom: The implication here is that we can achieve reporting that is not just temporally consistent across scans, but also anatomically specific in its description of change. Jane It moves us past just saying "something changed" to pinpointing exactly what changed, and where it happened relative to other structures.

Lu: The paper demonstrates that combining prior integration with these two regional mechanisms leads to state-of-the-art results on the RadGenome-ChestCT and CTRG-Chest548K test sets. Meng It’s a solid piece of work if you’re looking for a strong performance boost on these benchmarks, but the authors also flag that they haven't fully explored how this works when patient history isn't available.

Jane: That is a key caveat, because clinical workflows don't always have perfect prior data readily available. Tom So the future work seems focused on making this robust enough to handle those real-world constraints, like using it with or without accessible patient history in a more flexible way.

Conclusion: Tom: So, we're wrapping up on ALTER which models longitudinal changes in three dimensional CT reports using anatomical regions for localized evidence.

Jane: The authors are Dongchen Li, Jitao Liang, Wei Li at Northeastern University. It’s about getting better at telling a story across multiple scans instead of just looking at one snapshot.

Lu: Basically, they coordinate three things: integrating the prior study history with the current scan, using regional proxy differencing to find what changed locally in each area, and then fusing those changes into prompts for the language model.

Meng: If it works, it means a report isn't just saying "there's a nodule" on scan one; it can say "that nodule got bigger by twenty percent in the right lung lobe between scans one and two." That’s what matters for doctors.

Tom: Exactly. So, the main implication is that we move from global comparisons to very precise, anatomically grounded descriptions of disease progression over time.

Jane: It supports the idea that language models can become tools not just for summarizing images, but for tracking subtle patient journeys through medical imaging history.

Lu: The authors showed state-of-the-art results on two major chest CT benchmarks, RadGenome and CTRG Chest 548K. They even showed that adding the regional differencing part gives them the best scores on eight out of nine metrics.

Meng: Those numbers are interesting, but we gotta remember they only tested it on those specific datasets with prior history available in some cases. It’s a strong signal, but deployment needs to be careful about how much historical data you need upfront.

Tom: Right, so the title itself highlights that this isn't just about general language models anymore; it's about using spatial information—the anatomy—to make those temporal changes meaningful.

Jane: It opens up a new way for AI to assist clinicians in making sense of complex, time-dependent medical histories.

Lu: The next thing we look at is how they handle the situation when you don't have that prior history at all; that’s a big question for real-world use.

College of Computer Science and Engineering, Northeastern University · National Frontiers Science Center for Industrial Intelligence and Systems Optimization · Key Laboratory of Intelligent Computing in Medical Image (MIIC), Northeastern University

cs.CV

Submitted: 2026-08-06

Updated: 2026-11-16

Code: https://github.com/peytonkarlie/ALTER

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: Computed tomography (CT) is widely used for clinical diagnosis and longitudinal follow-up, yet automatically generating accurate and complete radiology reports from threedimensional (3D) CT remains…

Key concepts

Global Prior Integration (GPI)
This mechanism establishes a study-level historical context by interacting the current CT volume with its prior version and incorporating the encoded prior report. It uses gated attention to fuse this information, allowing the model to understand the overall history of the examination.
Regional Proxy Differencing (RPD)
RPD enables each current anatomical region to retrieve a historical proxy from a single shared encoding of the prior volume. This allows for localized interval evidence by comparing current regional tokens against these retrieved historical proxies, capturing changes specific to that area.
Interval Change Fusion (ICF)
ICF combines two signals: the current abnormality states and the differences distributed across anatomical regions. It converts this joint representation into change-aware soft prompts, which guide the language model in generating a report that explicitly describes how findings have changed over time.

Terminology

Summary

Computed tomography (CT) is widely used for clinical diagnosis and longitudinal follow-up, yet automatically generating accurate and complete radiology reports from threedimensional (3D) CT remains challenging Existing methods improve fine-grained correspondence between images and text by modeling anatomical regions, but remain centered on the current examination Consequently, patient-specific longitudinal changes within individual regions remain insufficiently modeled. p. 0 p. 0 We propose Anatomically Localized Temporal Evidence Representation (ALTER) to address these limitations. p. 0 p. 0

Proposed Framework Components

ALTER coordinates three complementary mechanisms: prior integration, regional differencing, and interval change fusion. Global Prior Integration (GPI) establishes study-level historical context by interacting current and prior volume representations through gated attention and incorporating the encoded prior report. p. 1 Regional Proxy Differencing (RPD) enables each current anatomical region to retrieve a historical proxy from a single shared encoding of the prior volume to derive localized interval evidence. p. 0 Interval Change Fusion (ICF) combines current abnormality states with region-distributed differences, converting their joint representation into change-aware soft prompts that guide report generation. p. 0

Training Signal Construction

The report generation is formulated as pθ(Yt It) = YYtj=1 pθ(yt,j yt,<j, It). GPI constructs a fixed-length study-level representation through two sequential interactions with historical information, where the current representation serves as the query and the historical source provides complementary context. p. 2 Visual history is distinguished by learned temporal embeddings et and et−1, yielding Ct = C+et and Pt−1 = P+et−1. p. 2 p. 2 Report history is incorporated asynchronously, where the visually contextualized sequence E is projected into the decoder space as E = Φc(G) ∈ RM×dl. p. 2 ICF constructs n report-level soft prompt tokens from two complementary signals: the status pathway identifies which abnormalities should be described, while the change pathway summarizes how their regional manifestations differ from the prior examination. p. 3

Performance Gains and Ablations

ALTER achieves state-of-the-art results on most evaluation metrics across the RadGenome-ChestCT validation and CTRG-Chest-548K test sets. p. 0 The complete configuration of GPI, RPD, and ICF achieves the best or jointly best results on eight of the nine metrics. p. 5 Ablation studies show that GPI yields the largest initial gains, increasing CIDEr from 9. p. 50 to 25.5 on RadGenome-ChestCT. p. 0 Adding RPD further improves B-4, R-L, CE-R, and CE-F1 while yielding the highest RG score of 27. p. 54.

Clinical Efficacy and Regional Analysis

ALTER improves weighted CE precision from 42.4 to 47.5, recall from 18. p. 50 to 19.1, and F1 from 23.8 to 26.2. The class-wise results also exhibit substantial nonuniformity, with largest gains observed for consolidation, lung opacity, cardiomegaly, and pleural effusion. p. 9 Regional results show that ALTER improves text overlap and clinical semantic quality across multiple anatomical regions. p. 6

Conclusion

We introduced ALTER for longitudinal 3D CT report generation, combining multimodal history, region-specific interval evidence, and change-aware prompting. Results on RadGenome-ChestCT and CTRG-Chest-548K demonstrate improved anatomically grounded reporting and cross-dataset transfer. p. 6 ALTER is intended to assist clinicians in longitudinal CT interpretation and report drafting, with future studies needed to further evaluate its performance in real-world clinical settings. p. 6

--- Page 1 ---

ALTER Modeling Longitudinal Changes via Regional Differencing for 3D CT Report Generation Dongchen Li1 ∗, Jitao Liang1∗, Wei Li2,3 † 1 College of Computer Science and Engineering, Northeastern University, Shenyang, China 2 National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Shenyang, China 3 Key Laboratory of Intelligent Computing in Medical Image (MIIC), Northeastern University, Shenyang, China Abstract Computed tomography (CT) is widely used for clinical diagnosis and longitudinal follow-up yet automatically generating accurate and complete radiology reports from threedimensional (3D) CT remains challenging. Existing methods improve fine-grained correspondence between images and text by modeling anatomical regions, but remain centered on the current examination. p. 0 Consequently, patient-specific longitudinal changes within individual regions remain insufficiently modeled. Meanwhile, interval changes are often distributed across multiple anatomical regions, complicating a coherent assessment of the overall longitudinal state. p. 0 We propose Anatomically Localized Temporal Evidence Representation (ALTER) to address these limitations. Global Prior Integration (GPI) incorporates the prior CT and report to establish historical context for the current examination. p. 0 Regional Proxy Differencing (RPD) enables each current anatomical region to retrieve a historical proxy from a single shared encoding of the prior volume and to derive localized interval evidence. p. 0 Interval Change Fusion (ICF) further combines current abnormality states with region-distributed differences, converting their joint representation into change-aware soft prompts that guide report generation. p. 0 ALTER achieves state-of-the-art results on most evaluation metrics across the RadGenome-ChestCT validation and CTRG-Chest-548K test sets. Code and data preprocessing details are available at https://github. p. 0com/peytonkarlie/ALTER/tree/main. p. 0 Introduction Computed tomography (CT) is widely used in disease diagnosis, treatment planning, and longitudinal follow-up. p. 0 Radiology reports serve as the primary medium for communicating CT findings, requiring radiologists to translate complex observations from volumetric images into coherent descriptions of anatomical structures, abnormal findings, and diagnostic impressions. p. 0 In clinical practice, radiologists must not only identify abnormalities in the current examination but also compare them with patient-specific prior scans and reports to characterize interval changes and assess disease progression. p. 0 This process is particularly demanding because a single CT examination typically comprises hundreds of slices and may contain multiple clinically relevant findings distributed across different anatomical regions. p. 0 Radiologists must therefore review extensive volumes of images, retrieve relevant historical evidence, and synthesize current and prior findings into a temporally consistent report, resulting in a substantial clinical workload. p. 0 Early radiology report generation focused primarily on 2D chest radiographs, with subsequent memory-augmented Transformers and large language models improving semantic modeling and report fluency. p. 0 As the task has extended to 3D CT volumes, recent methods have moved from whole-volume encoding toward anatomical regions and localized abnormalities to preserve fine-grained spatial information. p. 0 However, these methods generally remain centered on the current examination and describe where findings occur without explicitly modeling how they have changed over time. p. 0 Longitudinal report generation has incorporated prior images, reports, or both, but has been developed predominantly for 2D chest radiographs. p. 0 A small number of recent studies have begun to extend longitudinal report generation to 3D CT. p. 0 Even in these studies, prior examinations are incorporated through arXiv:2608. p. 005615v1 [cs. p. 0CV] 6 Aug 2026. p. 0

--- Page 2 ---

global interaction or generic discrepancy modeling, rather than by explicitly retrieving historical evidence corresponding to each current anatomical region. Consequently, subtle region-specific changes may be diluted, or distinct anatomical trajectories may be conflated. p. 1 This limitation becomes more pronounced when multiple findings evolve differently across a CT volume. p. 1 Their changes may vary in direction, magnitude, and clinical relevance, making it difficult to derive a coherent longitudinal description from coarse historical context alone. p. 1 Moreover, findings in the current examination do not by themselves reveal whether an abnormality is new, persistent, resolving, or worsening, while a single global comparison may blur distinct trajectories across anatomical locations. p. 1 Figure 1 illustrates that findings across anatomical regions can follow distinct trajectories between consecutive CT examinations, whereas coarse whole-examination comparison may fail to distinguish such region-specific changes. p. 1 Consequently, existing models may overlook subtle interval changes or reduce heterogeneous disease evolution to an overly coarse longitudinal progression. p. 1 To address these limitations, we propose Anatomically Localized Temporal Evidence Representation (ALTER), a framework for longitudinal 3D CT report generation. p. 1 Given a current CT volume, its anatomical regions, and, when available, a prior CT–report pair, ALTER coordinates three complementary mechanisms: prior integration, regional differencing, and interval change fusion. p. 1

--- Page 3 ---

For study-wide historical conditioning, Global Prior Integration (GPI) first interacts current and prior volume representations through gated attention and then incorporates the encoded prior report. For localized temporal comparison, Regional Proxy Differencing (RPD) uses each current anatomical region to query a historical proxy from a single shared encoding of the prior volume. p. 1 Finally, Interval Change Fusion (ICF) combines the current abnormality profile with difference evidence distributed across anatomical regions and maps the resulting representation into change-aware soft prompts for report generation. p. 1

--- Page 4 ---

We introduce ALTER, a framework that extends anatomyaware 3D CT reporting to patient-specific longitudinal modeling by coordinating multimodal historical context, anatomy-specific temporal comparison, and report-level change aggregation, while supporting studies both with and without accessible patient history. We develop Regional Proxy Differencing to derive localized interval representations from a single shared encoding of the prior volume, together with Interval Change Fusion to consolidate distributed regional differences and current abnormality states into change-aware prompts for report generation. p. 1 We conduct comprehensive experiments on two public 3D CT benchmarks, including component ablations, prior availability analyses, and conditioning diagnostics, to evaluate the effectiveness and complementarity of the proposed design. p. 1

--- Page 4 ---

Related Work Vision–Language Alignment Vision–language alignment establishes semantic correspondence between visual representations and natural-language descriptions, requiring radiology models to map subtle imaging findings to clinically precise expressions while preserving anatomical context. Early encoder–decoder approaches use co-attention or joint image–text representations to connect radiographic patterns with report semantics. p. 1 Cross-modal memory networks and curriculum learning subsequently improve the retention and acquisition of aligned clinical information. p. 1 Knowledge-guided methods incorporate retrieved templates, structured graphs, and prior clinical knowledge to organize report content. p. 1 More recent studies establish fine-grained correspondence among localized visual features, disease concepts, and anatomical structures through hierarchical or region-aware modeling. p. 1 Detected regions and region-referenced decoding further provide explicit spatial constraints for grounding individual findings in their corresponding anatomical structures. p. 1 For volumetric CT, pathological graphs model spatially distributed abnormalities and their clinical relationships. p. 1 Despite these advances, existing methods mainly address spatial image–text alignment within individual examinations, while longitudinal correspondence between the same anatomical regions across successive examinations remains underexplored. p. 1

--- Page 4 ---

Longitudinal Report Generation Longitudinal report generation incorporates patient-specific history, including prior images, reports, or both, to describe interval changes and generate the current report. Comparison-prior and pre-filling methods leverage historical reports or draft content to provide contextual findings and guide subsequent generation. p. 1 Recent work further strengthens the utilization of patient-specific history through contrastive pretraining and coarse-to-fine decoding. p. 1 Multimodal analogical reasoning explicitly characterizes disease evolution across examinations. p. 1 Temporal representation learning methods model relationships between paired chest radiographs through contrastive objectives, longitudinal encoders, or cross-examination feature interactions. p. 1

--- Page 4 ---

CT2Rep extends longitudinal modeling to 3D CT report generation by incorporating priorimage cross-attention and memory mechanisms. DiffVP similarly encodes global and local scan-to-reference discrepancies as visual prefixes for language-model-based CT report generation. p. 2 Nevertheless, most longitudinal systems are designed for two-dimensional radiographs, whereas existing studies on 3D CT remain limited and generally represent temporal context only at the examination level. p. 2 Such global comparison may dilute subtle changes confined to individual anatomical structures or conflate the distinct trajectories of different regions. p. 2 Consequently, fine-grained longitudinal comparison of the same anatomical regions across volumetric CT examinations remains insufficiently explored. p. 2

--- Page 4 ---

ALTER ALTER generates an anatomically grounded report for the current examination by jointly interpreting the current CT, its anatomical regions, and the preceding examination. Let Xt and Yt = (yt,1,. p. 2.., yt,Yt) denote the current volume and report, respectively, and let It = (Xt, Rr t) R r=1 denote the R anatomical crops extracted from the current CT. p. 2 A shared Vision Transformer (ViT) and Perceiver resampler encode the current and prior volumes into study-level tokens C, P ∈ RM×dv, respectively, and each current crop into regional tokens Rr ∈ R N×dv. p. 2 As illustrated in Figure 2, ALTER organizes longitudinal evidence at three complementary levels. p. 2 GPI establishes studylevel historical context from the prior CT and report, RPD resolves prior visual information into anatomy-specific proxies and localized longitudinal differences, and ICF consolidates the current abnormality profile with changes distributed across anatomical regions. p. 2

--- Page 4 ---

Global Prior Integration GPI constructs a fixed-length study-level representation through two sequential interactions with historical information, where the current representation serves as the query that determines the requested information, while the corresponding historical source provides complementary context. Visual history is distinguished by learned temporal embeddings et and et−1, yielding Ct = C+et and Pt−1 = P+et−1. The resulting gated fusion is G = Hv(Ct, P t−1). p. 2 In parallel, a frozen text encoder and a learned projection map the prior report to T ∈R L×dl. p. 2 The aligned textual response is then computed as At = A(E, T). p. 2

--- Page 4 ---

Regional Proxy Differencing RPD recovers anatomy-specific historical information from the shared prior-volume sequence P. For each region r, the current regional tokens Rr query the prior study tokens, and an output projection preserves the regional token layout: P˜r = Φo(A(Rr, P)). p. 3 The resulting proxy P˜r ∈ R N×dv has the same token layout as Rr, while its content is retrieved from P through an anatomy-specific query. p. 3

--- Page 4 ---

Implicit regional differencing The signed token difference Dr = Rr − P˜r characterizes deviations of the current region from its retrieved historical proxy. RPD therefore computes a gate as zr = σ(Fz([Rr∥P˜r∥Dr])) and decomposes the representation update into a gated change residual and a moderated proxy residual. p. 3

--- Page 4 ---

Explicit regional difference The implicit change information remains distributed across N regional tokens and may therefore be difficult for the decoder to access as a concise comparison cue. RPD further refines the pre-fusion difference as Hr = A(Dr, Dr) an… p. 3

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Global Prior Integration (GPI) establishes study-level historical context by interacting current and prior volume representations through gated attention, allowing complementary visual and textual evidence to contextualize the current examination.

  2. Regional Proxy Differencing (RPD) enables each current anatomical region to retrieve a historical proxy from a single shared encoding of the prior volume to derive localized interval evidence without separately encoding or registering prior regional crops.

  3. Interval Change Fusion (ICF) combines current abnormality states with region-distributed differences into change-aware soft prompts that guide report generation, mapping the resulting representation into change-aware soft prompts for the report decoder.

  4. The improved AI system can generate anatomically grounded reports for 3D CT by coordinating multimodal historical context, anatomy-specific temporal comparison, and report-level change aggregation.

Sources

Related papers