TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins
summary
The gist
Precision oncology requires forecasting clinical events and trajectories, yet modeling sparse, multimodal clinical time series remains a critical challenge TwinWeaver introduces an open-source…
In short
TwinWeaver creates an open-source framework that serializes complex patient histories into text to build a foundation model called Genie Digital Twin (GDT). GDT predicts continuous biomarkers and clinical events across 20 cancer types, significantly outperforming existing models in forecasting error and risk stratification. This demonstrates LLMs can function as powerful, general-purpose patient digital twins.
Key concepts
- TwinWeaver Framework
- A general framework designed to build LLM-based patient digital twins. It works by taking multi-modal clinical data (like lab results and history) and converting it into a standardized text format that fine-tunes large language models for patient understanding.
- Genie Digital Twin (GDT)
- A pan-cancer foundation model built using TwinWeaver. GDT learns from serialized patient journeys to jointly forecast continuous biomarkers and predict specific clinical events, such as disease progression or therapy switching, across many different cancer types.
- Data Serialization
- The process of converting complex, structured longitudinal patient data into a single, human-readable text prompt. This textual representation is the input used to train the LLM so it can learn from the entire patient history at once.
- Forecasting and Event Prediction Tasks
- Two main prediction goals: 1) Time-series forecasting for continuous values like blood biomarkers, predicting future measurements. 2) Landmark event prediction, classifying whether a specific clinical event (like survival or progression) will occur in the future.
Terminology used across episodes
This episode discusses
- TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins · Paper Radio
- AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
- OncoReason: Structuring Clinical Reasoning in LLMs for Robust and Interpretable Survival Prediction
- EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Qwen3 Technical Report
The paper
TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins · Read on arXiv
Computational Sciences Center of Excellence, Roche Penzberg Germany Computational Health Center Helmholtz Munich Department of Biology Ludwig Maximilian University of Munich Early Development Oncology Roche Innovation Center Zurich Computational Sciences Center of Excellence Genentech South San Francisco Department of Computer Science Stanford University Center for Data Science New York University Computational Sciences Center of Excellence Roche Basel Department of Biochemistry and Pharmacology Bio21 Molecular Science and Biotechnology
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins".
Jane: Here's the gist of the paper. Precision oncology requires forecasting clinical events and trajectories, yet modeling sparse, multimodal clinical time series remains a critical challenge TwinWeaver introduces an open-source framework that serializes longitudinal patient histories into text, enabling unified event prediction as well as forecasting with large language models This framework is used to build Genie Digital Twin (GDT) on 93,054 patients across 20 cancer types GDT significantly reduces forecasting error, achieving a median Mean Absolute Scaled Error (MASE) of 0.87 compared to 0.97 for the strongest time-series baseline Furthermore, GDT improves risk stratification, achieving an average concordance index (C-index) of 0.703 across survival, progression, and therapy switching tasks, surpassing the best baseline of 0.662 Framework Architecture TwinWeaver is a general framework for constructing LLM-based patient digital twins from longitudinal, multi-modal data by fine-tuning LLMs on procedurally generated text representations of longitudinal patient journeys The framework serializes multi-modal EHR data into text to train the Genie Digital Twin (GDT) pan-cancer foundation model GDT jointly forecasts continuous biomarkers and predicts landmark clinical events Data Serialization and Training The training pipeline preprocesses longitudinal data, generates input-target pairs by splitting patient trajectories at critical time points, converts the pairs into text prompts, and fine-tunes a pretrained LLM with them The input to the model is defined as Xp(t) = (sp, Hp(t)), where Hp(t) =
hpi hpi ∈ Hp, hpi,timestamp ≤ t: The target Yp(t) is defined as a set of future predictions of various tasks The framework transforms the structured patient history Xp(t) and multi-task targets Yp(t) into a single, human-readable text prompt Forecasting and Event Prediction Tasks The first task type is time-series forecasting, used to predict frequently measured values such as blood biomarkers or vital signs The forecasting target Yforecast p (t forecast, V ′) comprises future values for the sampled variables V′ up to the forecasting horizon t forecast = t + ∆t forecast with ∆t forecast ∈ N + The second task is predicting a patient’s event status at a future time point using the landmarking framework The set of event labels is Y =
occurred, not occurred, censored: Performance and Generalization GDT consistently outperforms forecasting baselines across all indications, achieving a median MASE of 0.867 (IQR 0.186) aggregated across all variables GDT shows statistically significant improvements across the majority of modeled clinical outcomes for survival, progression, and therapy switching GDT demonstrates robust ranking capabilities in survival, progression, and switching therapy tasks The model also generalizes to out-of-distribution clinical trials, matching trained baselines at zero-shot and surpassing them with fine-tuning Reasoning Extension The TwinWeaver framework enables an interpretable clinical reasoning extension by fine-tuning LLMs on a synthetic corpus, followed by alignment via Group Relative Policy Optimization (GRPO) This extension provides transparent, biologically grounded context for hypothesis generation by extracting structured chains-of-thought conditioned on patient history and ground-truth outcomes Keyword analysis confirms that GDT generated reasoning chains correspond to clinically relevant features and genetic contexts Clinical Trial Performance In out-of-distribution clinical trial tasks, fine-tuned GDT shows strong performance, achieving the lowest MASE in blood biomarker forecasting (median MASE 0.883) and highest C-Indexes for survival and progression events The model demonstrates strong generalization in cold start scenarios where historical data is sparse Overall, our findings suggest that the TwinWeaver framework transforms pretrained LLMs using specific fine-tuning into clinical prediction engines, exemplified with GDT The ability of GDT to generalize to new clinical trials as well as its pan-cancer capabilities offers a promising solution for low-data scenarios Future work should focus on systematic hyperparameter optimization and integrating modalities such as histopathology images TwinWeaver indicates that LLMs can be adapted successfully as a platform for patient digital twins Software and Data Sources All framework and model code can be found at http://github.com/MendenLab/TwinWeaver The FH-FMI CGDB data that support the findings of this study were originated by and are the property of Flatiron Health, Inc.
Tom: and Foundation Medicine, Inc.
Jane: Anonymized records for individual patients across more than one data source external to Roche cannot, and should not, be linked due to a potential increase in risk of patient re-identification The study was funded by F.
Tom: Hoffmann-La Roche and the European Union’s Horizon 2020 Research and Innovation Programme Author Contributions N.M.
Paper summary: Tom: So we're looking at this paper called TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins, and basically, the main idea is that modeling those messy, sparse patient histories—the multimodal time series data—is really hard. This paper introduces TwinWeaver as this open-source framework that serializes all that longitudinal patient history into text.
Jane: That sounds like a big move because if you can turn patient journeys into text, it opens up a whole new way to use large language models for making predictions and forecasting. What's the core claim they are making about this framework?
Tom: They claim that by using TwinWeaver to build something called Genie Digital Twin or GDT on ninety-three thousand patients across twenty cancer types, you can actually get better at both predicting continuous biomarkers and forecasting those discrete clinical events.
Jane: So, the implication is that this isn't just a clever data processing trick; it's a way to use these models to do unified event prediction and forecasting using large language models. How does the paper back up that claim with some actual numbers?
Lu: The framework trains a pre-trained large language model by fine-tuning it on procedurally generated text representations of patient journeys, which they call the Genie Digital Twin foundation model. This process involves transforming structured longitudinal data into text prompts and then fine-tuning the LLM with those pairs.
Meng: Wait, so they are generating their own training data through this serialization process? What does that mean for the cost and complexity of building these digital twins at scale?
Lalam: The framework transforms the structured patient history Xp(t) and multi-task targets Yp(t) into a single, human-readable text prompt, which is how the model learns to connect everything together across different cancer types.
Tom: And the results are pretty strong. They say GDT significantly reduces forecasting error, achieving a median Mean Absolute Scaled Error or MASE of zero point eight seven compared to zero point nine seven for the strongest time-series baseline they tested, which is statistically significant at a p value less than zero point zero zero one <ref:2601.20906#pg1,MASE of 0.87 compared to 0.97 for the strongest time>.
Paper summary: Jane: That reduction from zero point nine seven down to zero point eight seven sounds substantial when you're dealing with sparse data and trying to predict future clinical events, right <ref:2601.20906#pg1>? What else are they showing us regarding how this digital twin performs for risk assessment?
Lu: Beyond forecasting, they also show that GDT improves risk stratification, achieving an average concordance index or C-index of zero point seven zero three across survival, progression, and therapy switching tasks <ref:2601.20906#pg1,C-index of 0.703 across survival, progression, and therapy switching tasks>. That's compared to the best baseline which scored a C-index of zero point six six two <ref:2601.20906#pg1>.
Meng: A C-index of zero point seven is decent for risk stratification in this context, but I wonder how robust these results are when you try to deploy this at scale on real patient data that isn't perfectly preprocessed like in their study <ref:2601.20906#pg1>.
Tom: They test the generalization a bit there too, and they find that GDT shows robust ranking capabilities across survival, progression, and switching therapy tasks. Plus, it generalizes to out-of-distribution clinical trials by matching trained baselines at zero shot and surpassing them with fine tuning.
Jane: So it suggests that this approach isn't just good for the specific data they used in their study; it has some real promise for applying these digital twins to new clinical situations or different patient populations. What about how this framework helps clinicians actually think?
Lu: They extended the reasoning capability by fine-tuning LLMs on a synthetic corpus, and then aligning that with group relative policy optimization or GRPO. This creates an interpretable clinical reasoning extension that gives transparent, biologically grounded context for generating hypotheses by extracting structured chains of thought conditioned on patient history and ground truth outcomes.
Lalam: Keyword analysis confirms that the reasoning chains generated by GDT correspond to clinically relevant features and genetic contexts, which is important because it means the model isn't just spitting out random text; it's linking the prediction back to known biology.
Paper summary: Meng: From an engineering standpoint, I’m interested in how they handle the inference part when you actually want a prediction for a new patient. They generate independent completions and average them for forecasting, which sounds like it adds a layer of stability to those predictions.
Tom: And for event prediction, they use a two-step post-processing strategy involving conditioning followed by isotonic calibration using the pool adjacent violators algorithm or PAVA to recover a consistent cumulative incidence function for individual patient analysis without needing retraining. That’s a solid engineering detail there.
Jane: So, while the performance metrics look strong across all those tasks—forecasting and event prediction—the paper also points out some limitations regarding its application in new clinical trials, especially concerning cold start scenarios where historical data is sparse.
Lu: Exactly. They note that while it performs well when fine-tuned, it still shows strong performance in out-of-distribution clinical trial tasks, specifically achieving the lowest MASE of zero point eight eight three in blood biomarker forecasting and highest C-indexes for survival and progression events there <ref:2601.20906#pg1>.
Tom: It sounds like the key thing here is that TwinWeaver proves that these large language models can be adapted successfully as a platform for patient digital twins, especially because they show this ability to generalize across different cancer types within one pan-cancer model.
Jane: So, looking at the whole picture of TwinWeaver: An LLM-Based Foundation Model Framework for Pan-Cancer Digital Twins, it seems like the authors are demonstrating a unified way to tackle the challenge of sparse, multimodal clinical time series by turning history into text for better prediction and risk stratification.
Tom: Yeah, it’s about taking this complex longitudinal data and giving the large language model a structured way to understand it so it can forecast what's coming next while also helping clinicians make better risk decisions based on that combined understanding.
Jane: And the implication is that we might see models like GDT being used not just for prediction, but for generating reasoned explanations about patient trajectories in a way that feels more grounded in clinical context.
Conclusion: Tom: So, TwinWeaver is this open source framework for building digital twins using large language models across all kinds of cancer data.
Jane: It basically takes all that messy patient history and turns it into text so the model can actually learn from it.
Tom: The authors, N.M., E.V.P., J.W., D.M., M.B., L.V, V, S and H are the team behind this thing; they built the whole system from scratch with a focus on using these LLMs for patient trajectories across twenty different cancer types at once—pan-cancer modeling is key here for them.
Lu: The core idea they present is serializing those longitudinal patient histories into text to train a foundation model, the Genie Digital Twin or GDT.
Meng: What I see as important from an engineering standpoint is that they're fine-tuning a large language model on these generated text representations of the patient journeys.
Lalam: Right, and they use this process to jointly forecast continuous measurements like blood markers and also predict specific clinical events.
Jane: The conclusion of the paper suggests that this framework shows how large language models can be adapted successfully to become actual engines for clinical prediction.
Tom: They show GDT does a really good job generalizing to new patient data, even in those out-of-distribution clinical trials, which is pretty cool for real-world use.
Lu: It also includes this reasoning extension using group relative policy optimization or GRPO, which gives the model a way to explain its predictions with structured chains of thought.
Jane: So it suggests that we're moving toward these LLMs being used not just as pattern matchers, but as tools that can actually generate grounded clinical reasoning.
Tom: Exactly; the implications here are pretty huge because if this works reliably across so many cancer types, it opens up a new way to handle the complexity of patient data in oncology.
Meng: The caveat is that they still point out cold start scenarios where there's very little historical data, and we need more work on optimizing the hyper-parameters for those sparse cases.
Jane: So while they’ve shown strong results, the next step for this technology has to be making it even better at handling those initial, sparsely populated patient profiles.
More episodes
- 2603.28113-The multiplicative Lipschitz bound as an inductive bias
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound