Multi-state Models For Disease Histories Based On Longitudinal Data
summary
The gist
Multi-state models are suitable to investigate transitions between different disease stages in presence of competing risks The authors investigate the extension of piecewise exponential additive…
In short
The research extends piecewise exponential additive models (PAMs) to analyze disease transitions across multiple stages, tackling challenges like dependent left-truncation, multiple time scales, and index event bias. Simulations show PAMs handle these complexities well. Application to a UK Biobank dataset reveals that early Chronic Kidney Disease onset and increasing age are the highest risks for progression.
Key concepts
- Multi-state models
- These statistical models are used to study how patients move between different stages of a disease over time, especially when other diseases could also happen simultaneously (competing risks). They help track the progression through various disease states.
- Piecewise exponential additive models (PAMs)
- This is an advanced modeling technique used here to fit the data. It allows for different rates of change (piecewise) and incorporates additive effects, making it flexible enough to handle complex patterns in longitudinal disease histories.
- Dependent left-truncation
- This occurs when patients enter the risk group for a specific disease stage at different times, depending on their individual history. This makes standard modeling difficult because the starting points for tracking progression are not synchronized across all subjects.
- Index event bias
- This is a statistical issue where certain risk factors appear correlated within specific groups of patients, even if they aren't related in the general population. The study shows this bias depends on how thoroughly the researchers adjust for these covariates.
Terminology used across episodes
This episode discusses
- Multi-state Models For Disease Histories Based On Longitudinal Data · Paper Radio
- pammtools: Piece-wise exponential Additive Mixed Modeling tools
- Reduction Techniques for Survival Analysis
- Capturing heterogeneous time-variation in covariate effects in non-proportional hazard regression models
The paper
Multi-state Models For Disease Histories Based On Longitudinal Data · Read on arXiv
Simon Wiegrebe, Johannes Piller, Mathias Gorski, Merle Behr, Helmut K¨uchenhoff, Iris M. Heid, Andreas Bender
Statistical Consulting Unit StaBLab, Department of Statistics, LMU Munich, Munich, Germany. · Department of Genetic Epidemiology, University of Regensburg, Regensburg, Germany. · Munich Center for Machine Learning, LMU Munich, Munich, Germany. · Faculty of Informatics and Data Science at University of Regensburg · Department of Statistics at LMU Munich
Multi-stage disease histories derived from longitudinal data are becoming increasingly available as registry data and biobanks expand. Multi-state models are suitable to investigate transitions between different disease stages in presence of competing risks. In this context, however, their estimation is complicated by dependent left-truncation, multiple time scales, index event bias, and interval-censoring. In this work, we investigate the extension of piecewise exponential additive models (PAMs) to this setting and their applicability given the above challenges. In simulation studies we show that PAMs can handle dependent left-truncation and accommodate multiple time scales. Compared to a stratified single time scale model, a multiple time scales model is found to be less robust to the data generating process. We also quantify the extent of index event bias in multiple settings, demonstrating its dependence on the completeness of covariate adjustment. In general, PAMs recover baseline and fixed effects well in most settings, except for baseline hazards in interval-censored data. Finally, we apply our framework to estimate multi-state transition hazards and probabilities of chronic kidney disease (CKD) onset and progression in a UK Biobank dataset (n=142,667). We observe CKD progression risk to be highest for individuals with early CKD onset and to further increase over age. In addition, the well-known genetic variant rs77924615 in the UMOD locus is found to be associated with CKD onset hazards, but not with risk of further CKD progression.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Multi-state Models For Disease Histories Based On Longitudinal Data".
Jane: Here's the gist of the paper. Multi-state models are suitable to investigate transitions between different disease stages in presence of competing risks The authors investigate the extension of piecewise exponential additive models (PAMs) to this setting and their applicability given the above challenges Modeling Challenges Addressed The paper identifies four statistical challenges when applying multi-state modeling to multi-stage disease histories: dependent left-truncation, choice of time scales, index event bias, and interval-censoring 1.
Tom: Dependent left-truncation occurs because subjects enter the risk set for transition k at different time points due to subject-specific state-entry times 2.
Jane: Choice of time scales is problematic because hazards are commonly modeled on a single time scale, but multiple relevant scales exist in disease histories, such as chronological age and time since CKD onset 3.
Tom: Index event bias arises from negative correlation between otherwise uncorrelated risk factors induced within subsets of diseased individuals 4.
Paper summary: Tom: So Jane you have the paper "Multi-state Models For Disease Histories Based On Longitudinal Data" in front of us. Basically, it's tackling the problem of using the lots of longitudinal data we get from registries and biobanks to track how diseases progress through different stages.
Jane: Right, so that means when a patient has chronic kidney disease for example, they aren't just one state; they move through mild, moderate, severe stages. And the paper claims that multi-state models are the right tool for figuring out those transitions between those stages while also accounting for other diseases that could happen at the same time.
Lu: The core idea here is extending piecewise exponential additive models called PAMs to handle these tricky real-world complexities, which they say are dependent left-truncation, multiple time scales, index event bias, and interval-censoring.
Meng: Dependent left-truncation sounds like when a person enters the risk set for a certain stage at different times depending on their personal history, which makes things messy to model. What's the big win here for us in terms of actual data pipelines?
Lalam: The paper shows that PAMs can actually handle dependent left-truncation and they can deal with multiple time scales, which is a huge improvement over simpler models we might use.
Tom: And they found that when comparing a model that only looks at one time scale versus one that uses multiple time scales, the latter actually performs worse when the way the data is generated is complex.
Jane: That's interesting because usually people think more time scales mean more detail, but this study suggests it can actually make things less robust to how the data was created.
Lu: They also quantified index event bias in multiple settings and showed that this bias really depends on how complete the covariate adjustments are in the model.
Meng: So if we're trying to ship a model for production, knowing exactly how much bias we need to correct based on our data quality is crucial, so that measurement about covariate adjustment completeness is important.
Tom: And generally speaking, PAMs do a pretty good job of recovering the baseline hazards and the fixed effects in most situations we test.
Jane: But they hit a snag when dealing with interval-censored data, which means we only know for sure that an event happened sometime within a certain time window, not exactly when it happened.
Lu: And specifically, they found that PAMs actually recover the baseline hazards well in most settings except for those involving interval-censored data.
Meng: So if our clinical data has a lot of censored observations, we might have to be extra careful about how much we trust the baseline hazard prediction from this model.
Paper summary: Tom: Finally, they applied this framework to estimate transition hazards and probabilities for chronic kidney disease onset and progression using a UK Biobank dataset with one hundred forty-two thousand six hundred sixty-seven individuals.
Jane: The findings on the CKD progression risk are quite telling; they observed that the risk of progressing in CKD is highest for people who started with early CKD and that this risk increases as they get older.
Lu: And they also found a specific genetic variant, rs77924615 in the UMOD locus, which is associated with the hazard of CKD onset but not with the risk of further CKD progression.
Meng: That's concrete because it gives us a specific target for genetic risk assessment versus progression risk in this context.
Tom: It’s important to remember that while PAMs are good at handling these complexities, they still have those limitations when dealing with interval-censored data, as noted in the paper.
Jane: So what does this mean for the broader research community looking at multi-stage disease histories? It suggests that we need to be very specific about the data structure we're feeding into these models to get reliable results.
Lu: It pushes us toward developing methods that are specifically tailored not just to fitting the model, but to correctly addressing those four challenges—truncation, time scales, bias, and censoring—simultaneously.
Meng: From an engineering standpoint, it means we can't just plug any longitudinal dataset into a standard multi-state tool; we have to preprocess or adapt the data structure first for something like PAMs.
Tom: Overall, the paper shows that these models are applicable, but you need to be careful about what assumptions you make regarding time scales and how well your covariates are adjusted for bias.
Jane: So when we think about the title "Multi-state Models For Disease Histories Based On Longitudinal Data," it tells us that tracking disease progression over time using complex data sources is definitely a valid, though challenging, research area.
Lu: The authors are essentially providing a framework that tries to bridge the gap between the theoretical need for multi-state modeling and the messy reality of real-world longitudinal health data.
Meng: If we were deploying this for something like personalized risk stratification, we'd have to spend significant time calibrating those multiple scales and checking that index event bias correction works properly on our specific patient cohorts.
Tom: It seems like a solid piece of work because they provide evidence that PAMs are a viable extension, even if we still have to manage those known caveats with interval-censoring data.
Jane: So for the listener, it means when you look at complex health records that track conditions like kidney disease over time, researchers need to be aware of these specific statistical hurdles before they start building their models.
Conclusion: Tom: So we've been looking at how these complex disease histories are being modeled using multi-state models, and this paper is called "Multi-state Models For Disease Histories Based On Longitudinal Data."
Jane: That title sounds pretty technical, so what's the actual big picture here for someone just driving or cooking?
Tom: It's about taking all that messy data from registries and biobanks—where people move between different disease stages—and figuring out the best way to model those transitions, even when other diseases are happening at the same time.
Lu: The authors focus on making these models work even when things get complicated, like having subjects enter the risk group at different times, or dealing with multiple time scales and interval-censoring data.
Meng: So they are trying to build a model that can handle all that mess instead of just simplifying it down to something too basic for real-world deployment.
Lalam: Yes, the main point is they extend these models using piecewise exponential additive models, PAMs, and show that this framework actually handles dependent left-truncation and multiple time scales in their simulation studies.
Jane: So what does this mean practically? If we're looking at a patient's health record, how does knowing the right model help us understand their risk?
Tom: It helps us see that models that try to use multiple time scales, rather than just one single time scale for everything, can actually be less robust if the way the data was collected is complex.
Lu: They also quantified index event bias and showed it really depends on how complete you make your covariate adjustments in the model. It’s a measurement of how much correction you need to apply based on your data quality.
Meng: That's important because if we deploy these models, we need to know exactly where that bias correction comes from so we don't ship something that breaks at scale when the patient population shifts.
Lalam: Generally speaking, PAMs do a pretty good job of recovering the baseline hazards and fixed effects in most settings, but they still struggle with interval-censored data specifically when predicting the baseline hazard.
Jane: So it’s not a perfect solution yet, especially when we can't pinpoint the exact moment an event happened.
Tom: And then they applied this to a UK Biobank set of over one hundred forty-two thousand people, looking at chronic kidney disease progression.
Lu: They found that the progression risk in kidney disease is highest for people who started with early CKD and that this risk tends to increase as age goes up.
Jane: And they also found a specific genetic variant, rs77924615 in the UMOD locus, which seems linked to getting CKD on, but not linked to it getting worse later.
Tom: It’s a concrete finding for risk stratification—you can separate the initial trigger from the long-term progression risk based on that specific genetic marker.
Jane: This paper shows that multi-state modeling is definitely applicable here, even with these tricky longitudinal data formats, but you still have to be careful about those time scales and the censoring issues.
Lu: The implication is that for complex disease histories, we need methods specifically built to manage all four of those challenges—truncation, time scales, bias, and censoring—all at once.
Meng: From a deployment standpoint, it means we can’t just use off-the-shelf tools; you have to adapt the data structure first for something like these PAMs before you start training.
Tom: It’s a solid piece of work because they prove this extension is viable, but those caveats with interval-censored data still mean we can't treat it as a completely risk-free solution yet.
Jane: So when we look at these papers, the message is that tracking disease progression over time is possible, but you have to be meticulous about what you assume about the time scales and how clean your covariate adjustments are.
Lu: It pushes us toward developing methods that are specifically tailored to handle those four challenges simultaneously, instead of just fitting a model and hoping it works on the data.
Meng: If we're doing personalized risk stratification, we need to spend significant time calibrating those multiple scales and checking that index event bias correction actually works properly on our specific patient cohorts before we trust the output.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates