3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology".
Jane: Here's the gist of the paper. A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well.
Tom: The authors present HoloTea, a 3D-aware flowmatching framework that imputes spot-level gene expression from H&E while explicitly using information from adjacent sections.
Jane: Problem Motivation Existing spatial transcriptomics platforms operate on 2D sections with a limited field of view, capturing only a thin slice of a much larger 3D tissue volume.
Tom: This restricts the ability to probe vertical cell–cell communication, 3D tissue niches, and morphomolecular architectures that span multiple planes.
Paper summary: Jane: So, thinking about the whole title and everything from this paper, what does this mean for how we actually use spatial transcriptomics in labs today?
Lu: It means we can move toward a way to get a holistic understanding of tissue organization because this model allows us to look at vertical cell cell communication that spans multiple planes.
Tom: The authors are essentially showing that you can take cheap and scalable H andE staining and use it to learn the three dee structure of gene expression without having to do the incredibly expensive labor of dense sectioning for every single spot <ref:2511.14613#pg1>.
Meng: If this works as claimed, it changes the cost-effectiveness of analyzing complex three dee tissue niches in health studies significantly <ref:2511.14613#pg1>.
Jane: It also means we can finally get better insights into disease mechanisms because we are no longer limited to just looking at a thin slice of tissue.
Lu: The paper demonstrates that flow matching, when guided by explicit spatial context from neighbors and controlled by biology compatible priors, can produce much stronger spot and gene wise correlations than prior H andE to ST methods.
Tom: And the architecture scales favorably with stack depth, meaning we can actually use these deep tissue stacks without hitting the memory walls that used to stop some three dimensional models.
Jane: The main limitation they point out is that it’s still a generative model, so it’s not just a perfect predictor of every single detail.
Lu: They are being careful about that because the method does what it does well: generating three dee consistent spatial transcriptomics from serial H andE images <ref:2511.14613#pg1>.
Conclusion: Jane: So we've been talking about this paper and how they used flow matching to turn regular H andE slides into three-dimensional spatial transcriptomics, and now we're getting to the end of this talk.
Tom: Yeah, so let's look at the title 'three dee-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology'. It’s long, but it tells you exactly what they did: they used flow matching to make three dee data out of serial H andE slides in a scalable way.
Lu: From my view, the core idea here is bridging that gap between 2D sections and the actual three-dimensional structure of tissue <ref:2511.14613#pg1>. They aren't just predicting spots one by one; they are using flow matching to move from a simple image like H andE to a complex, structured gene expression map across multiple planes.
Meng: Scalable is a big word. I’m thinking about deployment—if this actually runs on real tissue stacks, does it handle the memory load? Because when we think three dee models, they often blow up the computer when you try to process huge volumes.
Tom: That's exactly where the paper gets interesting because they introduced a scalable transformer backbone with low-rank global context. It means as the stack gets deeper, the cost doesn't just grow exponentially; it stays close to linear with the number of spots and the depth of those sections.
Jane: So what does that mean for us in a lab? It suggests we can finally look at things like cell-cell communication that happen vertically across different tissue layers, not just what’s happening on one flat slice.
Lu: They also did this by making it three dee-aware through conditioning. They pull information from adjacent sections—sections z plus and z minus—using feature space similarity to guide the process, and then they inject that spatial context into the generation process using something called ControlNet.
Meng: ControlNet sounds like another layer of complexity we have to manage during training. It adds a lot of tuning work, but if it makes the final output much more coherent across planes than just looking at one section in isolation, I can see why they put it in there.
Tom: They also introduced some priors over gene relations, and they even used a learned prior called ZINB for their flow matching distributions. It’s like giving the model a better sense of what kind of gene expression patterns are biologically sensible before it even starts generating the final image.
Jane: So, to sum it up, this work shows that by combining flow matching with explicit three dee context from neighboring slides and using smarter biological priors, you can generate much more accurate and globally consistent three dee spatial transcriptomics from cheap H andE stains.
Lu: It’s about getting a holistic view of tissue architecture by leveraging the structure already present in dense histology.
Tom: It means we could potentially get much better insights into complex diseases by looking at the whole three dee environment, not just isolated spots. What happens next in this research?
Cambridge Stem Cell Institute, University of Cambridge · Cambridge Center for AI in Medicine, University of Cambridge · Department of Medicine, University of Cambridge · Computer Science Department, University of Oxford · Computer Science Department, University of Helsinki · Institute for Molecular Medicine Finland, University of Helsinki · Wellcome Sanger Institute
cs.CV
Submitted: 2025-11-18
Updated: 2026-11-10
Comments: Accepted at MLCB 2026 (oral). 30 pages (10 main, 3 references, 17 supplementary)
Project page: https://graph-and-geometriclearning.github.io/projects/stflow
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 91/100
The gist: A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease Most predictive…
Key concepts
- Flow Matching (FM)
- This is a modeling technique where the process of transforming data (like from H&E to ST) is modeled as a continuous path. The model learns this path by predicting intermediate steps between a starting point and the final desired data distribution, allowing it to generate realistic spatial expression patterns.
- Adjacent-Section Conditioning
- This involves using information from neighboring slides (sections z±1) to guide the generation process for a specific spot. The model retrieves morphologically similar spots from these adjacent sections based on feature similarity and injects this contextual information into the generation network, helping it understand 3D structure.
- Global Set Attention (GSA)
- GSA is a scalable attention mechanism designed to efficiently capture global context across an entire slide. Unlike standard attention which can become too slow with many spots, GSA uses a two-stage block that scales linearly with the number of spots and stack depth, ensuring comprehensive spatial understanding without prohibitive computational costs.
Terminology
Summary
A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. p. 0 3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial
Transcriptomics from Serial Histology
Mohammad Vali Sanian5,6,7,* Arshia Hemmat4,7,* Amirhossein Vahidi2,7,*
Jonas Maaskola7
Jimmy Tsz Hang Lee7 Stanislaw Makarchuk7
Yeliz Demirci7 Nana-Jane Chipampe7 Muzlifah Haniffa2,3,7
Omer Bayraktar7,† Lassi Paavolainen6,† Mohammad Lotfollahi1,2,7,†
*Equal contribution
† Correspondence Authors
1Cambridge Stem Cell Institute, University of Cambridge
2Cambridge Center for AI in Medicine, University of Cambridge
3Department of Medicine, University of Cambridge
4Computer Science Department, University of Oxford
5Computer Science Department, University of Helsinki
6
Institute for Molecular Medicine Finland, University of Helsinki
7Wellcome Sanger Institute
Abstract
A scalable and robust 3D tissue transcriptomics profile can
enable a holistic understanding of tissue organization and
provide deeper insights into human biology and disease. Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure,
while existing 3D-aware approaches are not generative and
do not scale well. Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure,
while existing 3D-aware approaches are not generative and
do not scale well. We present Holographic Tissue Expression Inpainting and Analysis (HoloTea), a 3D-aware flowmatching framework that imputes spot-level gene expression
from H&E while explicitly using information from adjacent
sections.Where the paper says it
Where the paper says it
Problem Motivation
Existing spatial transcriptomics platforms operate on 2D sections with a limited field of view, capturing only a thin slice of a much larger 3D tissue volume. This restricts the ability to probe vertical cell–cell communication, 3D tissue niches, and morphomolecular architectures that span multiple planes. p. 0 , Visium, Xenium, CosMx) operate
on 2D sections with a limited field of view, capturing only a
thin slice of a much larger 3D tissue volume. This restricts
our ability to probe vertical cell–cell communication, 3D
tissue niches, and morphomolecular architectures that span
multiple planes. Another
approach is to densely section the tissue and perform ST on
many serial sections [32, 33], but this remains prohibitively
expensive and labor-intensive to reconstruct full 3D volumes
in routine studies. In contrast, acquiring histology is cheap and scalable:
tissues can be densely sectioned, H&E-stained, and imaged
as serial whole-slide stacks, with ST performed only on a
subset of sections. In contrast, acquiring histology is cheap and scalable:
tissues can be densely sectioned, H&E-stained, and imaged
as serial whole-slide stacks, with ST performed only on a
subset of sections. This naturally motivates computational
models that leverage these dense histology stacks and sparse
ST measurements to learn the 3D spatial structure of gene
expression.Where the paper says it
Where the paper says it
Where the paper says it
Flow Matching Foundation
The authors model imputation as transport from a start distribution p0 over gene vectors to the data distribution p1 via a time-dependent field learned with flow matching (FM). For each spot i, they define a linear (displacement) path to the target x1,i≡yi,xt,i = (1 − t) x0,i + t x1,i. p. 3 We model imputation as transport from a
start distribution p0 over gene vectors to the data distribution
p1 via a time-dependent field learned with flow matching
(FM). For each spot i, we draw a start sample x0,i ∼p0 and
define a linear (displacement) path to the target x1,i ≡ yi
,
xt,i = (1 − t) x0,i + t x1,i, t ∈ [0, 1], (1)
where t is the scalar time along the path. For each spot i, we draw a start sample x0,i ∼p0 and
define a linear (displacement) path to the target x1,i ≡ yi
,
xt,i = (1 − t) x0,i + t x1,i, t ∈ [0, 1], (1)
where t is the scalar time along the path. A time-conditioned
network fθ (parameters θ) predicts the clean endpoint from
partial information,
yˆi = fθ
xt,i, ci
, t
, ci = [vi
, si
, kNN(i)], (2)
where ci concatenates the image features vi
, the coordinates
si
, and a neighborhood summary kNN(i) (the indices or
pooled features of the k nearest spots to i within its section). A time-conditioned
network fθ (parameters θ) predicts the clean endpoint from
partial information,
yˆi = fθ
xt,i, ci
, t
, ci = [vi
, si
, kNN(i)], (2)
where ci concatenates the image features vi
, the coordinates
si
, and a neighborhood summary kNN(i) (the indices or
pooled features of the k nearest spots to i within its section). We minimize the supervised endpoint-regression loss
Lfm = Ei,t
∥yˆi − yi∥
2
2
, following standard FM practice
for displacement or straight paths [6, 22, 23].Where the paper says it
Where the paper says it
Where the paper says it
Novel Conditioning and Priors
HoloTea introduces three main components: (i) conditioning from adjacent spots on the closest slides together with a prior over gene–gene relations in training dataset, (ii) applying conditioning and z-variation through ControlNet, and (iii) building hierarchical feature selection efficiently via Global set attention (GSA). Adjacent-section conditioning retrieves morphologically corresponding candidates on sections z±1 using feature-space cosine similarity and injects it with a spot-wise ControlNet. p. 1 To address these gaps, we introduce HoloTea, a 3D H&E
to ST leveraging flow matching [22, 24]. For each query
spot on section z, we retrieve morphologically corresponding candidates on sections z±1 using (x, y) proximity and
feature-space cosine similarity, then inject this adjacent context into selected denoiser blocks via a lightweight ControlNet [45]. This design promotes anatomical continuity
across sections without heavy preprocessing. Because flow
matching decouples the probability path from the start distribution, we can instantiate biology-compatible priors: a
learned ZINB prior using the scVI parameterization [9, 25]
and a 3D-consistent spatial–empirical prior constructed from
neighboring sections.Where the paper says it
Where the paper says it
Scalability and Performance
The method achieves scalability by introducing a scalable transformer backbone with low-rank global context, where the computational cost scales approximately linearly with the number of spots per slide and with the depth of the stack. The Global set attention (GSA) uses a two-stage attention block that scales as O(Nm d) rather than O(N2d), ensuring global information is propagated across an entire slide without attention cost growing with the number of spots. p. 4 We apply prenorm residual blocks around both stages (LayerNorm →
MHA → Dropout → Residual) followed by a positionwise MLP with residual. The computational cost scales
as O(Nm d) rather than O(N2d), so global information
is propagated across an entire slide without attention cost
growing with the number of spots. A global attention block introduces 3D H&E scaling linearly with the number of spots in the slide, enabling
training and inference on large 3D ST datasets. Across three
spatial transcriptomics datasets spanning different tissue
types and resolutions, HoloTea consistently improves 3D
expression accuracy and generalization compared to 2D
and 3D baselines. Across
public breast datasets and a large in-house embryo stack,
HoloTea delivers stronger spot and gene-wise correlations
and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from
the learned prior, ControlNet conditioning, and the crossslide branch. The architecture also scales favorably with
stack depth: training and inference remain near-linear in
the number of sections, avoiding the memory blow-ups that
affect some 3D baselines.Where the paper says it
Where the paper says it
Where the paper says it
Ablation Study Findings
The ablation study confirms complementary gains from the learned prior, ControlNet conditioning, and the cross-slide branch. Introducing the learned ZINB prior has a mixed but generally positive effect relative to Vanilla STFlow. p. 7 Effect of the learned prior. Introducing the learned ZINB
prior has a mixed but generally positive effect relative to
Vanilla STFlow. Effect of ControlNet conditioning. Adding ControlNet
on top of the learned prior yields consistent gains over both
Vanilla STFlow and the prior only model. Our full model,
which augments ControlNet with the cross-slide cosine
branch, delivers the strongest overall performance. At 100
and 200 genes it achieves the best scores across all three
metrics (e. Across
public breast datasets and a large in-house embryo stack,
HoloTea delivers stronger spot and gene-wise correlations
and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from
the learned prior, ControlNet conditioning, and the crossslide branch. The architecture also scales favorably with
stack depth: training and inference remain near-linear in
the number of sections, avoiding the memory blow-ups that
affect some 3D baselines. Across
public breast datasets and a large in-house embryo stack,
HoloTea delivers stronger spot and gene-wise correlations
and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from
the learned prior, ControlNet conditioning, and the crossslide branch. The architecture also scales favorably with
stack depth: training and inference remain near-linear in
the number of sections, avoiding the memory blow-ups that
affect some 3D baselines.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
HoloTea is a 3D-aware H&E to ST generative model that couples flow-matching with (i) 3D context by retrieving morphologically corresponding spots from adjacent sections and ControlNet-based injection of those cues, (ii) Generalization with biology-compatible priors over gene counts, and (iii) Scalability: we introduce a scalable transformer backbone with low-rank global context. Across public breast datasets and a large in-house embryo stack, HoloTea delivers stronger spot and gene-wise correlations and reduced reconstruction error than prior H&E to ST methods. p. 8 Conclusion
We presented HoloTea, a 3D-aware H&E to ST generative
model that couples flow-matching with (i) 3D context by retrieving morphologically corresponding spots from adjacent
sections and ControlNet-based injection of those cues, (ii)
Generalization with biology-compatible priors over gene
counts, and (iii) Scalability: we introduce a scalable transformer backbone with low-rank global context. Across
public breast datasets and a large in-house embryo stack,
HoloTea delivers stronger spot and gene-wise correlations
and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from
the learned prior, ControlNet conditioning, and the crossslide branch.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Adjacent-slide conditioning enables 3D context by retrieving
morphologically corresponding spots on neighboring slides in a shared feature space and fuse this cross section context into a lightweight ControlNet, allowing conditioning to follow anatomical continuity.
This allows the system to generate expression profiles that are explicitly informed by the spatial arrangement of adjacent sections, addressing the gap where "most H&E to ST predictors are trained per section and do not explicitly enforce 3D coherence across serial stacks." -
Z-aware priors provide biology-compatible constraints on gene counts by introducing a
learned ZINB prior using the scVI parameterization
and a3D-consistent spatial–empirical prior constructed from neighboring sections.
This allows the flow matching process to be tailored to the nature of count data, which is described as having anoverdispersed nature,
thereby improving the accuracy of spot-level gene expression imputation. -
Global set attention scales the model efficiently by using a two-stage attention block that achieves a complexity of O(Nm d) instead of O(N2d), ensuring that
global information is propagated across an entire slide without attention cost growing with the number of spots.
This allows for training and inference on large 3D datasets, such as the in-house embryo stack, while avoiding GPU out-of-memory errors encountered by baselines like ASIGN 3D. -
ControlNet injection refines the generative trajectory by using a
lightweight CNN (ControlNet) processes a coarse gene map
to sample per-spot control tokens which areinjected into selected transformer blocks as a residual modulation.
This allows for explicit conditioning of the flow matching process based on anatomical continuity, leading to stronger performance across ablation studies. -
The full model, combining all components including the
cross-slide cosine branch,
achieves the strongest overall performance, demonstrating that these cues providecomplementary long-range context beyond what ControlNet and the learned prior capture.
This results in superior spot and gene-wise correlations when tested on both public breast datasets and the in-house embryo dataset.
Abstract
A 3D transcriptomic profile of a tissue can give a holistic view of its organization in health and disease, but spatial transcriptomics (ST) is costly, so usually only some serial sections are profiled, while H&E is available for all. Most methods that infer ST from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. We present Holographic Tissue Expression Inpainting and Analysis (HoloTea), a 3D-aware flow-matching framework that imputes spot-level gene expression on an unprofiled section from its H&E and the measured expression of adjacent profiled sections. Our key idea is to take the measured expression at the same registered position on neighboring sections and feed this cross-section context to the flow model through a lightweight ControlNet, so that predictions follow anatomical continuity along z. To capture the discrete, zero-inflated nature of counts, the flow starts from a learned zero-inflated negative binomial (ZINB) prior that, on an unprofiled section, is predicted from the measured expression of the nearest profiled section. A global attention block with a small set of inducing tokens adds whole-slide context at a cost linear in the number of spots, so training and inference scale to large 3D ST datasets. Across three ST datasets spanning different tissue types and resolutions, HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines. We envision HoloTea advancing the creation of accurate 3D virtual tissues, ultimately accelerating biomarker discovery and deepening our understanding of disease. Code is available at https://github.com/Lotfollahi-lab/HoloTea.
Sources
- Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
- Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching
- Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
- SpaRED benchmark: Enhancing Gene Expression Prediction from Histology Images with Spatial Transcriptomics Completion
- Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs