CHRep: Cross-modal Histology Representation and Post-hoc Calibration for Spatial Gene Expression Prediction

arXiv:2604.21573 · cs.CV, q-bio.QM · Submitted 2026-04-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "CHRep: Cross-modal Histology Representation and Post-hoc Calibration for Spatial Gene Expression Prediction".

Jane: Here's the gist of the paper. Spatial transcriptomics (ST) enables spatially resolved gene profiling but remains expensive and lowthroughput, limiting large-cohort studies and routine clinical use <ref:2604.21573#pg2> Predicting spatial gene expression from routine hematoxylin and eosin (H&E) slides is a promising alternative, yet under realistic leave-one-slide-out evaluation, existing models often suffer from slide-level appearance shifts and regression-driven over-smoothing that suppress biologically meaningful variation <ref:2604.21573#pg2>.

Tom: Framework Overview CHRep introduces a two-phase framework for robust histology-to-expression prediction <ref:2604.21573#pg3>.

Jane: In the training phase, CHRep learns a structureaware representation by jointly optimizing correlationaware regression, symmetric image–expression alignment, and coordinate-induced spatial topology regularization <ref:2604.21573#pg4>.

Tom: The inference phase then applies a lightweight calibration module trained on the training slides to improve cross-slide robustness without backbone fine-tuning <ref:2604.21573#pg5>.

Paper summary: Tom: So, we're looking at this paper now titled CHRep: Cross-modal Histology Representation and Post-hoc Calibration for Spatial Gene Expression Prediction. Essentially, they're trying to take something that's super expensive and low throughput—spatial transcriptomics—and try to predict it just from a routine hematoxylin and eosin slide.

Jane: That’s the core idea, right? The paper argues that there are real promises here for getting spatial gene expression data without needing those massive sequencing runs, but they acknowledge that existing models struggle when you test them on new slides in a leave-one-slide-out evaluation because things shift visually and the models get too smooth.

Lu: What CHRep introduces is this two phase framework. In the training phase, it learns a structureaware representation by optimizing three different things at once: correlation aware regression, symmetric image expression alignment, and coordinate induced spatial topology regularization.

Meng: Three objectives in one goes deep into the mechanics of what they're trying to achieve. Correlation aware regression is supervised by an objective that includes a term for Pearson correlation, which means it’s looking at how well the prediction correlates with the actual gene expression across spots.

Lalam: And then they have this cross modal alignment loss, which aligns the coordinate guided morphology representation with the gene representation using a symmetric contrastive objective. That sounds like they're forcing those two different data types—the image structure and the gene data—to talk to each other in a shared space.

Jane: Before we get into how it works, what does this mean for us on the ground? Why is predicting spatial gene expression from H andE slides a big deal right now?

Tom: Because spatial transcriptomics is fantastic for seeing where genes are located within a tissue, but it takes too much time and money to do that for huge cohorts or even routine clinical use. This paper suggests that if we can get good predictions from just H andE slides, we unlock a totally different way to study gene expression spatially.

Lu: The key insight here seems to be how they handle the structure. They have this coordinate branch that maps spot coordinates to coordinate features, and they fuse those with the histology features to form this coordinate guided morphology representation.

Meng: So they’re building a map of the tissue structure itself, not just looking at individual spots in isolation. That's ambitious because those spatial relationships are often what matter most for biology.

Paper summary: Jane: And then they add this topology regularization term, which is designed to preserve that coordinate induced spatial structure by regularizing the similarity pattern of the gene representation. This keeps the biological layout intact during training even when things get messy.

Lalam: It's like they’re telling the model, "Hey, don't lose track of where spot one relates spatially to spot two, because that spatial relationship is biologically important."

Tom: Okay, moving into how they actually predict on a new slide. The inference phase is where things get clever because it separates the learning from the final deployment calibration. They have an Estimate Module and a Correction Module working together.

Jane: Can you unpack that for someone who doesn't know machine learning internals? What does this estimate and correction module do when they are making a prediction on a slide they haven't seen before?

Tom: The Estimate Module uses the training gallery to compute a similarity weighted neighbor based estimate. So, it looks at what it knows from all the previous slides to make an initial guess.

Meng: That sounds like using past examples to guide the prediction, which is smart for cross slide robustness, but what about when that initial guess is systematically biased? That's where the Correction Module steps in.

Lu: The Correction Module outputs a magnitude regularized residual adjustment from the frozen embedding. This adjustment compensates for systematic bias without having to fine tune any of the original backbone learning components.

Jane: So, it’s not retraining the whole thing every time you get a new slide; you just apply this lightweight correction layer based on similarity-weighted estimates and a residual adjustment. That sounds like a big win for deployment speed and robustness.

Tom: It is, because the authors found that they can do this without fine tuning the backbone, which is huge for making these models practical instead of just theoretical. They optimized the correction module while keeping the image branch frozen, using an objective that mixes mean squared error plus mean absolute error plus a PCC form together with residual regularization.

Lalam: That combination of loss terms sounds like they’re trying to keep the regression accurate on average, but also control how much it drifts away from what’s expected based on the training data.

Jane: So, when we look at the experimental results, what did they find when they tested CHRep against other methods? Are these gains real or just on a specific dataset?

Tom: They ran extensive experiments on three datasets: cSCC twenty-one, HER2 plus twenty-two, and Alex plus ten times twenty-three. They consistently showed improvements under leave-one-slide-out evaluation, which is the tough test.

Paper summary: Meng: The Alex plus ten times dataset seems to have been a real stress test for them, and they found that CHRep consistently improved gene wise correlation there, with the largest gains actually coming from that specific set.

Lu: Relative to a baseline called mclSTExp eighteen, they reported that the Pearson correlation between ACG improves by thirty-nine point five percent on Alex plus ten times. Plus, they saw reductions in mean squared error and mean absolute error of nine point seven percent and nine point zero percent respectively.

Jane: That’s a lot of numerical evidence showing tangible improvement over existing methods, but I want to know what the authors say is the most important design choice in that success.

Tom: The analysis of post-hoc calibration on Alex plus ten times showed that the estimate plus correction design provided the strongest overall gains. They also pointed out that a correction magnitude constraint was critical because without it, their estimate plus correction design saw a marked degradation in squared error.

Meng: A constraint on the magnitude of the correction adjustment is important because it stops the model from just wildly overcompensating for noise, which is exactly what we worry about at scale.

Lu: They also found that combining regression, cross modal alignment, and topology preservation into one full objective yielded the most balanced performance and strongest cross slide generalization.

Jane: So to wrap up this paper on CHRep: it’s a two phase framework that learns structure aware representations during training while using a lightweight calibration module for inference to improve robustness without retraining the main model. It tackles the problem of appearance shifts in H andE slides head-on by preserving spatial topology and aligning image and expression features.

Tom: The authors suggest this separation between representation learning and deployment time calibration is a practical direction for building models that work reliably in large cohorts or even real world settings.

Lalam: I'm confident in the findings here because they show consistent gains across different datasets when tested under that rigorous leave-one-slide-out evaluation.

Jane: So, to summarize what we’ve heard about CHRep: it’s a method for predicting spatial gene expression from routine H andE slides by learning structure aware representations during training and using a post hoc calibration module to improve cross slide robustness without backbone fine tuning.

Tom: The implication is that we can get much more reliable histology to expression models for real world deployment if we focus on these structural preservation techniques combined with lightweight inference adjustments.

Conclusion: Tom: So, CHRep is about taking that expensive spatial transcriptomics data and trying to predict gene expression just from a routine H andE slide using this two-phase approach.

Jane: It sounds complicated, but it’s basically building a smart system that learns how tissue structure relates to gene activity during training, and then adds a quick fix for when you actually use the model in the real world.

Lu: The authors are focusing on making sure this representation learning actually respects the physical layout of the slide—the coordinates—so it doesn't just see random pixels.

Meng: From an engineering standpoint, they’re trying to make this practical so we don't need massive computing power every time we want to check a tissue sample.

Lalam: What they are doing is separating the learning of the underlying structure from the final calibration step, which should make deployment much faster and more stable.

Tom: The title itself, CHRep—Cross-modal Histology Representation—just tells you they’re bridging that gap between looking at a picture and understanding what’s happening inside it.

Jane: And it's interesting because they aren't just doing one thing; they are jointly optimizing several things during training, which means the learned representation is more balanced.

Lu: They optimize correlation aware regression, symmetric image expression alignment, and coordinate induced spatial topology regularization all at once for every spot. That’s a lot of constraints on the model architecture.

Meng: I wonder if that complexity translates to a model that actually runs fast enough on our production servers without needing constant retraining.

Lalam: The paper suggests this separation—learning the structure first, then calibrating—is a really practical direction for making these histology models usable in large cohorts.

Tom: It means we might be able to get reliable spatial gene expression predictions without needing that huge sequencing cost every single time we analyze a slide.

Beijing University of Posts and Telecommunications

cs.CV, q-bio.QM

Submitted: 2026-04-23

Updated: 2026-11-11

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 89/100

The gist: Spatial transcriptomics (ST) enables spatially resolved gene profiling but remains expensive and lowthroughput…

Key concepts

Structure-aware Representation Learning
This involves training a model to understand both the visual structure of the histology slide and the gene expression data simultaneously. The model learns how spatial coordinates relate to gene patterns, ensuring that predictions respect the underlying tissue organization rather than just local pixel features.
Correlation-aware Regression
During training, one objective forces the model to accurately predict gene expression based on its correlation with existing labels. This ensures that the predicted gene patterns are biologically meaningful and maintain a strong statistical relationship with known expression levels in the training data.
Post-hoc Calibration Mechanism
This is a lightweight module used during prediction, separate from the main learning process. It uses embeddings from a training set to estimate biases and then corrects them using residual adjustments. This allows the model to become more reliable on unseen slides without needing extensive retraining.

Terminology

Summary

Spatial transcriptomics (ST) enables spatially resolved gene profiling but remains expensive and lowthroughput, limiting large-cohort studies and routine clinical use <ref:2604. p. 021573#pg2> Predicting spatial gene expression from routine hematoxylin and eosin (H&E) slides is a promising alternative, yet under realistic leave-one-slide-out evaluation, existing models often suffer from slide-level appearance shifts and regression-driven over-smoothing that suppress biologically meaningful variation <ref:2604. p. 021573#pg2>.

Framework Overview

CHRep introduces a two-phase framework for robust histology-to-expression prediction <ref:2604.21573#pg3>. In the training phase, CHRep learns a structureaware representation by jointly optimizing correlationaware regression, symmetric image–expression alignment, and coordinate-induced spatial topology regularization <ref:2604. p. 021573#pg4>. The inference phase then applies a lightweight calibration module trained on the training slides to improve cross-slide robustness without backbone fine-tuning <ref:2604. p. 021573#pg5>.

Representation Learning Objectives

The training phase involves jointly optimizing three objectives for each spot, including correlation-aware regression, cross-modal alignment, and spatial topology regularization <ref:2604.21573#pg6>. Specifically, the regression branch is supervised by a correlation-aware objective that includes a term for Pearson correlation <ref:2604. p. 321573#pg7>. Furthermore, CHRep explicitly aligns the coordinate-guided morphology representation with the gene representation using a symmetric contrastive objective <ref:2604. p. 321573#pg8>. Finally, it regularizes the gene-side representation with a topology-consistency term derived from spot coordinates to preserve higher-order spatial structure <ref:2604. p. 321573#pg9>.

Post-hoc Calibration Mechanism

The inference phase improves robustness by decoupling representation learning from deploymenttime calibration <ref:2604.21573#pg10>. This involves an Estimate Module that produces a similarity-weighted neighbor-based estimate from a training gallery embedding bank. p. 1 A Correction Module then predicts a regularized residual adjustment from the frozen embedding to compensate for systematic bias without backbone fine-tuning. p. 1 This correction module is optimized while keeping the image branch frozen, and its objective applies MSE+MAE+PCC form together with residual regularization. p. 4

Experimental Findings

Extensive experiments on the cSCC [21], HER2+ [22], and Alex+10x [23] datasets demonstrate consistent improvements under leave-one-slide-out evaluation. CHRep consistently improves gene-wise correlation under leave-one-slide-out evaluation, with the largest gains observed on Alex+10x <ref:2604. p. 021573#pg9>. Relative to mclSTExp [18], PCC(ACG) improves by 39. p. 05% on Alex+10x, together with 9. p. 07% and 9.0% reductions in mean squared error and mean absolute error, respectively <ref:2604. p. 021573#pg9>.

Design Component Analysis

The analysis of post-hoc calibration on Alex+10x shows that the Estimate+Correction design provides the strongest overall gains. The correction-magnitude constraint is critical, as without it, the Estimate+Correction design exhibits a marked degradation in squared error. p. 9 The full objective combining regression, cross-modal alignment, and topology preservation yields the most balanced performance and the strongest cross-slide generalization. p. 1

Conclusion

CHRep addresses spatial gene expression prediction from H&E histology under strict leave-one-slide-out evaluation through a two-phase design that separates structure-aware representation learning from inference-time bias correction. The framework improves robustness to cross-slide variation by preserving image–expression correspondence and coordinateinduced tissue organization during training, while refining inference through similarity-guided estimation and regularized residual correction. p. 1 These findings suggest that decoupling representation learning from lightweight inference-phase calibration offers a practical direction for building more reliable and transferable histology-to-expression models in large-cohort and real-world deployment settings. p. 1

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Structural representation learning integrates three coordinated inputs to learn from H&E patches, spot coordinates, and gene expression vectors jointly. This creates a representation that is predictive for gene expression, aligned across modalities, and consistent with tissue spatial organization.

  2. Inference phase robustness is improved by decoupling representation learning from deployment-time calibration. The system uses a lightweight calibration module combining a non-parametric estimate from a training gallery with a magnitude-regularized correction module to improve robustness without backbone fine-tuning.

  3. Cross-slide generalization is strengthened by using the post-hoc calibration stage which applies an Estimate Module that produces a similarity-weighted non-parametric prediction and a Correction Module that predicts an additive residual, ensuring the test slide is excluded from both gallery construction and correction module training to match strict leave-one-slide-out evaluation.

  4. The unified representation learning objective combines regression, cross-modal alignment, and topology preservation: The overall training objective combines regression, cross-modal alignment, and topology preservation. This ensures the learned representation preserves gene-wise trend fidelity while maintaining tissue structural consistency by jointly optimizing correlation-aware regression (Lreg), symmetric contrastive alignment (Lcon), and coordinate-induced spatial topology regularization (Lspa).

  5. The system can perform accurate gene expression prediction from routine H&E slides in large cohorts without additional wet-lab assays. This is achieved by consistently improving gene-wise correlation under leave-one-slide-out evaluation across datasets like cSCC, HER2+, and Alex+10x, reaching the largest gains observed on Alex+10x.

Related papers