AdaPCLA: Adaptive Prior-Calibrated Logit Adjustment for Long-Tailed Longitudinal EHR Generation

arXiv:2607.12645 · cs.LG · Submitted 2026-07-14 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "AdaPCLA: Adaptive Prior-Calibrated Logit Adjustment for Long-Tailed Longitudinal EHR Generation".

Jane: The gist The AdaPCLA framework enables generative models to adaptively fit and generate EHR data through a data distribution-aware training strategy,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: We’ve just walked through the AdaPCLA framework. To recap, the core thesis of this paper is that standard autoregressive models fail to capture the co-occurrence structure of tail events in Electronic Health Records, which hurts their ability to generate data for rare subpopulations Jane They propose AdaPCLA as a way to fix this by using a data distribution-aware training strategy that internalizes knowledge parameters via simulated annealing training Lu

Jane: So, the framework claims two main things. First, it enables adaptive fitting and generation of EHR data through this distribution-aware training strategy Tom And second, it supports training-free adaptation to a diverse clinical population for generation through zero-shot distribution control Meng

Lu: The abstract states that this is achieved by internalizing data knowledge parameters by simulated annealing training Tom It also mentions that the theoretical analysis characterizes rare-code logit updates through the label-wise empirical NTK and derives a prior-internalization bound for how annealing speed and NTK conditioning affect retained prior signals Jane

Meng: So, to put it plainly, they are using simulated annealing training to bake the data knowledge into the model parameters Tom And they’ve also got this zero-shot control mechanism that lets you adjust the target prior at inference time without retraining Lu

Jane: It means that instead of just relying on a fixed training bias, you get a dynamic way to protect those rare updates during training and then learn the structure itself Tom This is designed to support clean inference without an external bias after training Meng

Lu: The framework consists of three core modules. First, an EHR Generator Backbone, which is an autoregressive Transformer that captures longitudinal clinical dependencies Tom Second, Statistical Prior Modeling, which converts long-tailed empirical prevalences into a label-wise logit scaffold bstat Jane And third, Curriculum Prior Injection, which anneals this scaffold so that early rare-event updates are protected while the final generator can operate without an external bias Meng

Tom: The backbone is just there to model the sequence of visits over time using those clinical dependencies Lu And then they have this statistical prior modeling step to convert frequencies into that logit scaffold bstat Jane

Jane: And then the injection module controls how much of that scaffold gets used at different training steps via alpha tau Tom That’s a lot of moving parts controlling the balance between learning rare events and learning common ones Meng

Lu: It’s about making sure that the early updates to rare events are protected while allowing the final generator to operate without an external bias Tom This structure is designed for long-tailed EHR generation, addressing issues where optimization is dominated by frequent patterns rather than rare but important dependencies Jane

Meng: So, it's tackling the problem of learning those rare but clinically important dependencies that existing methods often struggle with because optimization gets stuck on easy patterns Tom And this paper proposes AdaPCLA as a way to change that dynamic in training Jane

Lu: This approach addresses the difficulty where synthetic EHR generators might fit common clinical patterns but fail to preserve the temporal and co-occurrence structure involving rare clinical events Tom

Conclusion: Tom: So, wrapping up the AdaPCLA paper. The title is Adaptive Prior-Calibrated Logit Adjustment for Long-Tailed Longitudinal EHR Generation Jane It’s essentially a framework that uses dynamic training to bake in data knowledge and then provides zero-shot control for cross-population adaptation Lu

Jane: In simple terms, the authors are showing that you can make generative models better at producing synthetic EHR data for rare events without needing to retrain them for every new population Tom The big implication is that we can generate more realistic and faithful synthetic clinical records, which is crucial for privacy-preserving research Meng

Lu: It suggests that dynamic annealing helps the backbone learn useful prior information within its own logits instead of depending on a fixed external correction during sampling Jane This means the model learns the structure it needs to learn Tom

Meng: So, for us shipping models, it’s about getting better performance metrics like tail plausibility and downstream utility over existing baselines Jane And we can use that zero-shot control feature to quickly adapt models to new hospital data without a full retraining cycle Lu

Tom: It seems the framework successfully strengthens rare-code learning during optimization without needing an external bias during inference Jane This is the main point I’m taking away from this paper Lu

Jane: It moves us toward having more robust synthetic EHR generation tools for the future where we can handle those long-tail complexities more effectively Tom

Shuai Cui, Chen Wenxuan, Wenjie Du, Jian Lou, Dan Li, Wenjie Feng

University of Science and Technology of China · Sun Yat-sen University

cs.LG

Submitted: 2026-07-14

Updated: 2026-10-14

Comments: The authors have identified a critical error/issue in the paper that affects the validity of the results. We are withdrawing to address these issues

Code: https://github.com/worldcs27/worldcs27.github.io

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 88/100

The gist: The gist The AdaPCLA framework enables generative models to adaptively fit and generate EHR data through a data distribution-aware training strategy, which internalizes data knowledge parameters by

Key concepts

AdaPCLA Framework: This is a curriculum prior-calibrated method for EHR generation. It starts by injecting a statistical prior bias into the model's logits early in training, then gradually removes this bias (anneals it) so the model learns the rare event structure intrinsically, leading to clean inference.
Statistical Prior Modeling: This module converts long-tailed empirical data frequencies into a 'logit scaffold' called bstat. This scaffold acts as a prior correction term added to the model's initial logits, specifically designed to make early optimization more sensitive and effective for learning rare clinical codes.
Zero-Shot Distribution Control: This capability allows the model to adapt its generation style to a new clinical population without any retraining. It achieves this by applying a target-prior correction difference during inference time, steering the generated probabilities toward the desired target distribution.
Internalization Bound: A theoretical analysis that characterizes how quickly and smoothly the framework anneals the prior bias. It provides a mathematical guarantee on how annealing speed and conditioning affect the retained prior signals, showing that smoother adjustments lead to lower error.

Terminology

Adaptation — 2× in this paper · explained in 1 episode
Fidelity — 1× in this paper · explained in 2 episodes
Generative Model — 1× in this paper · explained in 2 episodes
Generative Modeling — 1× in this paper · explained in 3 episodes
Generative Models — 1× in this paper · explained in 3 episodes

Summary

The gist The AdaPCLA framework enables generative models to adaptively fit and generate EHR data through a data distribution-aware training strategy, which internalizes data knowledge parameters by simulated annealing training and supports zero-shot distribution control

AdaPCLA Framework Overview

AdaPCLA is proposed as a curriculum prior-calibrated framework for long-tailed EHR generation that applies prior-aware logit adjustment early in training to maintain effective rare-code updates, then gradually anneals this correction so that rare-code temporal and co-occurrence dependencies are internalized into the model parameters This design supports clean inference without an external bias after training, and it also enables zero-shot distribution control by applying target-prior corrections without retraining The framework consists of three core modules: (i) an EHR Generator Backbone, which is an autoregressive Transformer that captures longitudinal clinical dependencies (ii) Statistical Prior Modeling, which converts long-tailed empirical prevalences into a label-wise logit scaffold bstat and (iii) Curriculum Prior Injection, which anneals this scaffold so that early rare-event updates are protected while the final generator can operate without an external bias

EHR Generator Backbone

The EHR Generator Backbone uses an autoregressive Transformer to model longitudinal EHR trajectories, where the hidden state used for predicting the t-th step visit depends only on previous records and respects the temporal order The next visit is modeled with Bernoulli output heads, and the training objective is the Binary Cross-Entropy loss over clinical events

Optimization Scaffold: Statistical Prior

The statistical prior modeling converts long-tailed empirical prevalences into a label-wise logit scaffold bstat, defined as bstat(c) = -logitπˆϵ(c) = log 1 − πˆ(c) + ϵ / πˆ(c) + ϵ At the start of training, the model logit for c is close to its empirical log-odds, z0,c(x) ≈ logitπˆϵ(c), and after adding bstat(c), the effective logit becomes z˜0,c(x) = z0,c(x) + bstat(c) ≈ 0 This bias is used to make early optimization more sensitive, especially for rare codes

Curriculum Prior Injection and Internalization

The Curriculum Prior Injection uses the modified logits z˜τ (x) = zτ (x)+α(τ)bstat, where α(τ) ∈ [0, 1] controls the proportion of the prior bias used at step τ The training loss is the BCE loss computed with the effective logits: LAdaPCLA = LBCEy, σ(˜zτ (x)) At the end of training, α(T) = 0, so the generator uses only its own logits, z˜T (x) = zT (x)

Zero-Shot Cross-Population Adaptation

AdaPCLA adaptively supports cross-population adaptation by introducing a target-prior correction difference at inference time, without retraining Let ∆b(c) = logitπtarget(c) − logitπˆ(c), and modify the logits as z′t = zt(θ) + λ∆b for inference, where λ ∈ [0, 1] controls the strength of the adjustment Under the label-wise prior-shift assumption, this correction steers labelwise occurrence probabilities toward the target population while reusing longitudinal and co-occurrence structure learned from source EHR data

Theoretical Analysis and Internalization Bound

The theoretical analysis characterizes rare-code logit updates through the label-wise empirical NTK and derives a prior-internalization bound for how annealing speed and NTK conditioning affect retained prior signals The internalization error eτ (x) satisfies∥eτ (x)∥K−1 ≤ e−µλmin(K)τ∥e0(x)∥K−1 + B⋆(x)µλmin(K)sup s∈[0,T]α˙(s) This bound suggests that smoother annealing reduces sups α˙(s), which lowers the internalization error caused by a moving scaffold

Experimental Results

Experiments on real-world data show that AdaPCLA achieves consistent gains in tail plausibility, downstream utility, and zero-shot control For instance, it improves TailPairSeen over HALO by 114.2% on MIMIC-III and 65.1% on MIMIC-IV In the zero-shot generation setting based on MIMIC-IV training to MIMIC-III inference, AdaPCLA reaches AUPRC of 0.922 without retraining, outperforming GPT-style generation by 3.5% F1

Model Analysis and Clinical Validation

The ablation study shows that Static-PCLA (Train-only) fails after the bias is removed, indicating that a fixed training bias alone does not make the model learn the rare-event structure AdaPCLA achieves the best metrics in this ablation, suggesting that dynamic annealing helps the backbone learn useful prior information in its own logits instead of depending on a fixed external correction Case studies qualitatively illustrate that AdaPCLA correctly recovers sophisticated treatment logic, such as predicting Long-term use of anticoagulants (V5861), which is tied to the history of Atrial Fibrillation

Prediction Reliability

AdaPCLA consistently outperforms GPT-style and HALO in both overall and tail-only settings regarding prediction reliability metrics like ECE, Brier Score, and NLL In the tail-only scope, AdaPCLA achieves the best overall and tail-only scores

Bias-Free Inference Algorithm

Algorithm 1 Bias-free inference process of AdaPCLA outlines the procedure where intrinsic logits are computed without an external bias during sampling The final step involves mapping the logits to probabilities and sampling the next visit independently based on these probabilities

Conclusion

In this paper, AdaPCLA supports zero-shot generation adaptation for crosspopulation setting in a training-free manner The framework successfully strengthens rare-code learning during optimization without requiring an external bias during inference Experiments on real-world dataset show that AdaPCLA improves tail plausibility, downstream utility, and zero-shot distribution control over strong EHR generation baselines The paper concludes by demonstrating that dynamic annealing helps the backbone learn useful prior information in its own logits instead of depending on a fixed external correction during sampling

Limitations

The current evaluation limited to diagnosis-code sequences from two public EHR benchmarks, and more validation for AdaPCLA over diverse hospitals and richer EHR modalities beyond diagnosis codes is expected as future work The paper also reports that the analysis of λmin(K) and related-rare AUC/gain is non-conclusive evidence

References

[1] Kim-Anh-Nhi Nguyen, Dhavalkumar Patel, Masoud Edalati, Maria Sevillano, Prem Timsina, Robert Freeman, Matthew A. Levin, David L.

Improvements for AI systems

Bold header: Prior-Calibrated Logit Adjustment for Rare Events

This enables generative models to adaptively fit and generate EHR data through a data distribution-aware training strategy; this is achieved by internalizing data knowledge parameters by simulated annealing training. This allows the system to capture rare clinical events, which provide limited positive evidence and weak stable learning signals that standard autoregressive models underrepresent.

Bold header: Bias-Free Inference via Dynamic Annealing

The system supports clean inference without an external bias after training by treating the prior bias as a temporary optimization scaffold that is gradually annealed away. This ensures that the generation process relies on model-encoded rare-event dependencies rather than an external correction, which is crucial for clinical reliability.

Bold header: Zero-Shot Cross-Population Adaptation

AdaPCLA enables adaptation to new clinical populations without retraining by applying a target-prior correction difference at inference time. Specifically, it modifies logits as z′t = zt(θ) + λ∆b, steering labelwise occurrence probabilities toward the target population while preserving learned structure.

Abstract

Generative modeling of longitudinal Electronic Health Records is increasingly important for privacy-preserving research, yet standard autoregressive models tend to underrepresent the co-occurrence structure of tail events (i.e., diseases, symptoms), reducing the fidelity and faithfulness of generated data for rare subpopulations. To this end, we propose AdaPCLA framework, which enables generative models to adaptively fit and generate EHR data through a data distribution-aware training strategy; this is achieved by internalizing data knowledge parameters by simulated annealing training. It also supports training-free adaptation to a diverse clinical population for generation through zero-shot distribution control. Moreover, our theoretical analysis characterizes rare-code logit updates through the label-wise empirical NTK and derives a prior-internalization bound for how annealing speed and NTK conditioning affect retained prior signals. Experiments on real-world data show that AdaPCLA achieves consistent gains in tail plausibility, downstream utility, and zero-shot control; in particular, it improves TailPairSeen over HALO by 114.2% on MIMIC-III and 65.1% on MIMIC-IV, outperforms GPT-style generation by 3.5% F1 for zero-shot cross-population adaptation.