3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology

arXiv:2511.14613 · cs.CV · Submitted 2025-11-18 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology".

Jane: Here's the gist of the paper. A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well.

Tom: The authors present HoloTea, a 3D-aware flowmatching framework that imputes spot-level gene expression from H&E while explicitly using information from adjacent sections.

Jane: Problem Motivation Existing spatial transcriptomics platforms operate on 2D sections with a limited field of view, capturing only a thin slice of a much larger 3D tissue volume.

Tom: This restricts the ability to probe vertical cell–cell communication, 3D tissue niches, and morphomolecular architectures that span multiple planes.

Paper summary: Jane: So, thinking about the whole title and everything from this paper, what does this mean for how we actually use spatial transcriptomics in labs today?

Lu: It means we can move toward a way to get a holistic understanding of tissue organization because this model allows us to look at vertical cell cell communication that spans multiple planes.

Tom: The authors are essentially showing that you can take cheap and scalable H andE staining and use it to learn the three dee structure of gene expression without having to do the incredibly expensive labor of dense sectioning for every single spot <ref:2511.14613#pg1>.

Meng: If this works as claimed, it changes the cost-effectiveness of analyzing complex three dee tissue niches in health studies significantly <ref:2511.14613#pg1>.

Jane: It also means we can finally get better insights into disease mechanisms because we are no longer limited to just looking at a thin slice of tissue.

Lu: The paper demonstrates that flow matching, when guided by explicit spatial context from neighbors and controlled by biology compatible priors, can produce much stronger spot and gene wise correlations than prior H andE to ST methods.

Tom: And the architecture scales favorably with stack depth, meaning we can actually use these deep tissue stacks without hitting the memory walls that used to stop some three dimensional models.

Jane: The main limitation they point out is that it’s still a generative model, so it’s not just a perfect predictor of every single detail.

Lu: They are being careful about that because the method does what it does well: generating three dee consistent spatial transcriptomics from serial H andE images <ref:2511.14613#pg1>.

Conclusion: Jane: So we've been talking about this paper and how they used flow matching to turn regular H andE slides into three-dimensional spatial transcriptomics, and now we're getting to the end of this talk.

Tom: Yeah, so let's look at the title 'three dee-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology'. It’s long, but it tells you exactly what they did: they used flow matching to make three dee data out of serial H andE slides in a scalable way.

Lu: From my view, the core idea here is bridging that gap between 2D sections and the actual three-dimensional structure of tissue <ref:2511.14613#pg1>. They aren't just predicting spots one by one; they are using flow matching to move from a simple image like H andE to a complex, structured gene expression map across multiple planes.

Meng: Scalable is a big word. I’m thinking about deployment—if this actually runs on real tissue stacks, does it handle the memory load? Because when we think three dee models, they often blow up the computer when you try to process huge volumes.

Tom: That's exactly where the paper gets interesting because they introduced a scalable transformer backbone with low-rank global context. It means as the stack gets deeper, the cost doesn't just grow exponentially; it stays close to linear with the number of spots and the depth of those sections.

Jane: So what does that mean for us in a lab? It suggests we can finally look at things like cell-cell communication that happen vertically across different tissue layers, not just what’s happening on one flat slice.

Lu: They also did this by making it three dee-aware through conditioning. They pull information from adjacent sections—sections z plus and z minus—using feature space similarity to guide the process, and then they inject that spatial context into the generation process using something called ControlNet.

Meng: ControlNet sounds like another layer of complexity we have to manage during training. It adds a lot of tuning work, but if it makes the final output much more coherent across planes than just looking at one section in isolation, I can see why they put it in there.

Tom: They also introduced some priors over gene relations, and they even used a learned prior called ZINB for their flow matching distributions. It’s like giving the model a better sense of what kind of gene expression patterns are biologically sensible before it even starts generating the final image.

Jane: So, to sum it up, this work shows that by combining flow matching with explicit three dee context from neighboring slides and using smarter biological priors, you can generate much more accurate and globally consistent three dee spatial transcriptomics from cheap H andE stains.

Lu: It’s about getting a holistic view of tissue architecture by leveraging the structure already present in dense histology.

Tom: It means we could potentially get much better insights into complex diseases by looking at the whole three dee environment, not just isolated spots. What happens next in this research?

Cambridge Stem Cell Institute, University of Cambridge · Cambridge Center for AI in Medicine, University of Cambridge · Department of Medicine, University of Cambridge · Computer Science Department, University of Oxford · Computer Science Department, University of Helsinki · Institute for Molecular Medicine Finland, University of Helsinki · Wellcome Sanger Institute

cs.CV

Submitted: 2025-11-18

Updated: 2026-11-10

Comments: Accepted at MLCB 2026 (oral). 30 pages (10 main, 3 references, 17 supplementary)

Project page: https://graph-and-geometriclearning.github.io/projects/stflow

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease Most predictive…

Key concepts

Flow Matching (FM)
This is a modeling technique where the process of transforming data (like from H&E to ST) is modeled as a continuous path. The model learns this path by predicting intermediate steps between a starting point and the final desired data distribution, allowing it to generate realistic spatial expression patterns.
Adjacent-Section Conditioning
This involves using information from neighboring slides (sections z±1) to guide the generation process for a specific spot. The model retrieves morphologically similar spots from these adjacent sections based on feature similarity and injects this contextual information into the generation network, helping it understand 3D structure.
Global Set Attention (GSA)
GSA is a scalable attention mechanism designed to efficiently capture global context across an entire slide. Unlike standard attention which can become too slow with many spots, GSA uses a two-stage block that scales linearly with the number of spots and stack depth, ensuring comprehensive spatial understanding without prohibitive computational costs.

Terminology

Summary

A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. p. 0

Where the paper says it

3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology Mohammad Vali Sanian5,6,7,* Arshia Hemmat4,7,* Amirhossein Vahidi2,7,* Jonas Maaskola7 Jimmy Tsz Hang Lee7 Stanislaw Makarchuk7 Yeliz Demirci7 Nana-Jane Chipampe7 Muzlifah Haniffa2,3,7 Omer Bayraktar7,† Lassi Paavolainen6,† Mohammad Lotfollahi1,2,7,† *Equal contribution † Correspondence Authors 1Cambridge Stem Cell Institute, University of Cambridge 2Cambridge Center for AI in Medicine, University of Cambridge 3Department of Medicine, University of Cambridge 4Computer Science Department, University of Oxford 5Computer Science Department, University of Helsinki 6 Institute for Molecular Medicine Finland, University of Helsinki 7Wellcome Sanger Institute Abstract A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease. Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well.

The authors present HoloTea, a 3D-aware flowmatching framework that imputes spot-level gene expression from H&E while explicitly using information from adjacent sections. p. 0
Where the paper says it

Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. We present Holographic Tissue Expression Inpainting and Analysis (HoloTea), a 3D-aware flowmatching framework that imputes spot-level gene expression from H&E while explicitly using information from adjacent sections.

Problem Motivation

Existing spatial transcriptomics platforms operate on 2D sections with a limited field of view, capturing only a thin slice of a much larger 3D tissue volume. This restricts the ability to probe vertical cell–cell communication, 3D tissue niches, and morphomolecular architectures that span multiple planes. p. 0

Where the paper says it

, Visium, Xenium, CosMx) operate on 2D sections with a limited field of view, capturing only a thin slice of a much larger 3D tissue volume. This restricts our ability to probe vertical cell–cell communication, 3D tissue niches, and morphomolecular architectures that span multiple planes.

While densely sectioning the tissue is prohibitively expensive and labor-intensive to reconstruct full 3D volumes in routine studies, acquiring histology is cheap and scalable for dense serial sections. p. 1
Where the paper says it

Another approach is to densely section the tissue and perform ST on many serial sections [32, 33], but this remains prohibitively expensive and labor-intensive to reconstruct full 3D volumes in routine studies. In contrast, acquiring histology is cheap and scalable: tissues can be densely sectioned, H&E-stained, and imaged as serial whole-slide stacks, with ST performed only on a subset of sections.

This motivates computational models that leverage these dense histology stacks and sparse ST measurements to learn the 3D spatial structure of gene expression. p. 1
Where the paper says it

In contrast, acquiring histology is cheap and scalable: tissues can be densely sectioned, H&E-stained, and imaged as serial whole-slide stacks, with ST performed only on a subset of sections. This naturally motivates computational models that leverage these dense histology stacks and sparse ST measurements to learn the 3D spatial structure of gene expression.

Flow Matching Foundation

The authors model imputation as transport from a start distribution p0 over gene vectors to the data distribution p1 via a time-dependent field learned with flow matching (FM). For each spot i, they define a linear (displacement) path to the target x1,i≡yi,xt,i = (1 − t) x0,i + t x1,i. p. 3

Where the paper says it

We model imputation as transport from a start distribution p0 over gene vectors to the data distribution p1 via a time-dependent field learned with flow matching (FM). For each spot i, we draw a start sample x0,i ∼p0 and define a linear (displacement) path to the target x1,i ≡ yi , xt,i = (1 − t) x0,i + t x1,i, t ∈ [0, 1], (1) where t is the scalar time along the path.

A time-conditioned network fθ (parameters θ) predicts the clean endpoint from partial information yˆi = fθ xt,i, ci, t,. p. 3
Where the paper says it

For each spot i, we draw a start sample x0,i ∼p0 and define a linear (displacement) path to the target x1,i ≡ yi , xt,i = (1 − t) x0,i + t x1,i, t ∈ [0, 1], (1) where t is the scalar time along the path. A time-conditioned network fθ (parameters θ) predicts the clean endpoint from partial information, yˆi = fθ xt,i, ci , t , ci = [vi , si , kNN(i)], (2) where ci concatenates the image features vi , the coordinates si , and a neighborhood summary kNN(i) (the indices or pooled features of the k nearest spots to i within its section).

The loss minimized is the supervised endpoint-regression loss Lfm = Ei,t∥yˆi − yi∥2. p. 3
Where the paper says it

A time-conditioned network fθ (parameters θ) predicts the clean endpoint from partial information, yˆi = fθ xt,i, ci , t , ci = [vi , si , kNN(i)], (2) where ci concatenates the image features vi , the coordinates si , and a neighborhood summary kNN(i) (the indices or pooled features of the k nearest spots to i within its section). We minimize the supervised endpoint-regression loss Lfm = Ei,t ∥yˆi − yi∥ 2 2 , following standard FM practice for displacement or straight paths [6, 22, 23].

Novel Conditioning and Priors

HoloTea introduces three main components: (i) conditioning from adjacent spots on the closest slides together with a prior over gene–gene relations in training dataset, (ii) applying conditioning and z-variation through ControlNet, and (iii) building hierarchical feature selection efficiently via Global set attention (GSA). Adjacent-section conditioning retrieves morphologically corresponding candidates on sections z±1 using feature-space cosine similarity and injects it with a spot-wise ControlNet. p. 1

Where the paper says it

To address these gaps, we introduce HoloTea, a 3D H&E to ST leveraging flow matching [22, 24]. For each query spot on section z, we retrieve morphologically corresponding candidates on sections z±1 using (x, y) proximity and feature-space cosine similarity, then inject this adjacent context into selected denoiser blocks via a lightweight ControlNet [45].

Furthermore, they introduce z-aware, biology-compatible prior distributions for flow matching, including a learned ZINB prior and a spatial–empirical prior constructed from neighboring sections. p. 1
Where the paper says it

This design promotes anatomical continuity across sections without heavy preprocessing. Because flow matching decouples the probability path from the start distribution, we can instantiate biology-compatible priors: a learned ZINB prior using the scVI parameterization [9, 25] and a 3D-consistent spatial–empirical prior constructed from neighboring sections.

Scalability and Performance

The method achieves scalability by introducing a scalable transformer backbone with low-rank global context, where the computational cost scales approximately linearly with the number of spots per slide and with the depth of the stack. The Global set attention (GSA) uses a two-stage attention block that scales as O(Nm d) rather than O(N2d), ensuring global information is propagated across an entire slide without attention cost growing with the number of spots. p. 4

Where the paper says it

We apply prenorm residual blocks around both stages (LayerNorm → MHA → Dropout → Residual) followed by a positionwise MLP with residual. The computational cost scales as O(Nm d) rather than O(N2d), so global information is propagated across an entire slide without attention cost growing with the number of spots.

Across three spatial transcriptomics datasets spanning different tissue types and resolutions, HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines. p. 0
Where the paper says it

A global attention block introduces 3D H&E scaling linearly with the number of spots in the slide, enabling training and inference on large 3D ST datasets. Across three spatial transcriptomics datasets spanning different tissue types and resolutions, HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines.

The in-house embryo dataset demonstrates that the method scales favorably with stack depth, avoiding the memory blow-ups that affect some 3D baselines. p. 8
Where the paper says it

Across public breast datasets and a large in-house embryo stack, HoloTea delivers stronger spot and gene-wise correlations and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from the learned prior, ControlNet conditioning, and the crossslide branch. The architecture also scales favorably with stack depth: training and inference remain near-linear in the number of sections, avoiding the memory blow-ups that affect some 3D baselines.

Ablation Study Findings

The ablation study confirms complementary gains from the learned prior, ControlNet conditioning, and the cross-slide branch. Introducing the learned ZINB prior has a mixed but generally positive effect relative to Vanilla STFlow. p. 7

Where the paper says it

Effect of the learned prior. Introducing the learned ZINB prior has a mixed but generally positive effect relative to Vanilla STFlow.

Adding ControlNet on top of the learned prior yields consistent gains over both Vanilla STFlow and the prior only model. p. 7
Where the paper says it

Effect of ControlNet conditioning. Adding ControlNet on top of the learned prior yields consistent gains over both Vanilla STFlow and the prior only model.

The full model, which augments ControlNet with the cross-slide cosine branch, delivers the strongest overall performance across all three metrics. p. 7
Where the paper says it

Our full model, which augments ControlNet with the cross-slide cosine branch, delivers the strongest overall performance. At 100 and 200 genes it achieves the best scores across all three metrics (e.

The architecture also scales favorably with stack depth: training and inference remain near-linear in the number of sections, avoiding the memory blow-ups that affect some 3D baselines. p. 8
Where the paper says it

Across public breast datasets and a large in-house embryo stack, HoloTea delivers stronger spot and gene-wise correlations and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from the learned prior, ControlNet conditioning, and the crossslide branch. The architecture also scales favorably with stack depth: training and inference remain near-linear in the number of sections, avoiding the memory blow-ups that affect some 3D baselines.

p. 8
Where the paper says it

Across public breast datasets and a large in-house embryo stack, HoloTea delivers stronger spot and gene-wise correlations and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from the learned prior, ControlNet conditioning, and the crossslide branch. The architecture also scales favorably with stack depth: training and inference remain near-linear in the number of sections, avoiding the memory blow-ups that affect some 3D baselines.

Conclusion

HoloTea is a 3D-aware H&E to ST generative model that couples flow-matching with (i) 3D context by retrieving morphologically corresponding spots from adjacent sections and ControlNet-based injection of those cues, (ii) Generalization with biology-compatible priors over gene counts, and (iii) Scalability: we introduce a scalable transformer backbone with low-rank global context. Across public breast datasets and a large in-house embryo stack, HoloTea delivers stronger spot and gene-wise correlations and reduced reconstruction error than prior H&E to ST methods. p. 8

Where the paper says it

Conclusion We presented HoloTea, a 3D-aware H&E to ST generative model that couples flow-matching with (i) 3D context by retrieving morphologically corresponding spots from adjacent sections and ControlNet-based injection of those cues, (ii) Generalization with biology-compatible priors over gene counts, and (iii) Scalability: we introduce a scalable transformer backbone with low-rank global context. Across public breast datasets and a large in-house embryo stack, HoloTea delivers stronger spot and gene-wise correlations and reduced reconstruction error than prior H&E to ST methods, with ablations confirming complementary gains from the learned prior, ControlNet conditioning, and the crossslide branch.

The architecture also scales favorably with stack depth: training and inference remain near-linear in the number of sections, avoiding the memory blow-ups that affect some 3D baselines.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adjacent-slide conditioning enables 3D context by retrieving morphologically corresponding spots on neighboring slides in a shared feature space and fuse this cross section context into a lightweight ControlNet, allowing conditioning to follow anatomical continuity. This allows the system to generate expression profiles that are explicitly informed by the spatial arrangement of adjacent sections, addressing the gap where "most H&E to ST predictors are trained per section and do not explicitly enforce 3D coherence across serial stacks."

  2. Z-aware priors provide biology-compatible constraints on gene counts by introducing a learned ZINB prior using the scVI parameterization and a 3D-consistent spatial–empirical prior constructed from neighboring sections. This allows the flow matching process to be tailored to the nature of count data, which is described as having an overdispersed nature, thereby improving the accuracy of spot-level gene expression imputation.

  3. Global set attention scales the model efficiently by using a two-stage attention block that achieves a complexity of O(Nm d) instead of O(N2d), ensuring that global information is propagated across an entire slide without attention cost growing with the number of spots. This allows for training and inference on large 3D datasets, such as the in-house embryo stack, while avoiding GPU out-of-memory errors encountered by baselines like ASIGN 3D.

  4. ControlNet injection refines the generative trajectory by using a lightweight CNN (ControlNet) processes a coarse gene map to sample per-spot control tokens which are injected into selected transformer blocks as a residual modulation. This allows for explicit conditioning of the flow matching process based on anatomical continuity, leading to stronger performance across ablation studies.

  5. The full model, combining all components including the cross-slide cosine branch, achieves the strongest overall performance, demonstrating that these cues provide complementary long-range context beyond what ControlNet and the learned prior capture. This results in superior spot and gene-wise correlations when tested on both public breast datasets and the in-house embryo dataset.

Abstract

A 3D transcriptomic profile of a tissue can give a holistic view of its organization in health and disease, but spatial transcriptomics (ST) is costly, so usually only some serial sections are profiled, while H&E is available for all. Most methods that infer ST from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. We present Holographic Tissue Expression Inpainting and Analysis (HoloTea), a 3D-aware flow-matching framework that imputes spot-level gene expression on an unprofiled section from its H&E and the measured expression of adjacent profiled sections. Our key idea is to take the measured expression at the same registered position on neighboring sections and feed this cross-section context to the flow model through a lightweight ControlNet, so that predictions follow anatomical continuity along z. To capture the discrete, zero-inflated nature of counts, the flow starts from a learned zero-inflated negative binomial (ZINB) prior that, on an unprofiled section, is predicted from the measured expression of the nearest profiled section. A global attention block with a small set of inducing tokens adds whole-slide context at a cost linear in the number of spots, so training and inference scale to large 3D ST datasets. Across three ST datasets spanning different tissue types and resolutions, HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines. We envision HoloTea advancing the creation of accurate 3D virtual tissues, ultimately accelerating biomarker discovery and deepening our understanding of disease. Code is available at https://github.com/Lotfollahi-lab/HoloTea.

Sources

Related papers