Representation-driven Endoscopic Visual Embedding Alignment for Latent Generation

The paper introduces REVEAL (Representation-driven Endoscopic Visual Embedding Alignment), described as the largest generative foundation model for endoscopy to date, trained on GastroNet-5M (GN-5M), a multicenter dataset of 5 million endoscopic frames. The authors argue that developing foundation generative models for endoscopy is limited by the gap between natural and clinical images and the computational cost of training large Diffusion Transformers.

The central motivation stems from the limitations of existing representation alignment strategies: "Prevailing alignment strategies that rely on general-purpose vision encoders assume that semantic richness is the primary driver of synthesis quality. We argue that for medical foundation models, this reliance on out-of-domain semantic proxies may lead to suboptimal morphological fidelity and a failure to capture the clinical nuances of the endoscopic manifold. The paper emphasizes that endoscopic scenes are characterized by subtle mucosal textures, specular reflections, and intricate anatomical morphologies, in which global semantic labels provide insufficient signal."

The paper lists three specific contributions:

• (a) an extensive benchmark of representation alignment design elements, evaluating the influence of target representations and alignment strategies on the morphological fidelity and convergence of DiTs

• (b) "an evaluation of model features across multiple independent endoscopic datasets, including a corrupted robustness benchmark with realistic imaging artifacts, demonstrating generalization beyond the training distribution while maintaining anatomical integrity"

• (c) "the introduction of the largest foundation generative model for endoscopy to date, a high-capacity backbone trained on GN-5M that simultaneously serves as a competitive feature extractor, exceeding established endoscopic foundation models such as EndoViT and Endo-FM, and enables immediate application to downstream tasks such as inpainting, outpainting, and specialized conditional generation"

REVEAL is described as a latent generative model that synthesizes high-fidelity medical images by aligning the feature space of a Scalable Interpolant Transformer (SiT) with a specialized, pretrained vision encoder. The architecture operates within a compressed latent space where an input image x is mapped to z = E(x) via a pretrained VAE encoder. The generative backbone reverses a stochastic interpolation process while a Representation Alignment (REPA) strategy injects domain-specific knowledge.

The alignment objective maximizes cosine similarity between the pretrained teacher representation y* and the student's hidden states ht, formalized as:

L REPA(θ, φ):= −E[(1/N) Σ sim(y*[n], hφ(ht[n]))]

The paper adopts the iREPA strategy with two architectural refinements to the standard alignment recipe: (1) replacing the conventional Multilayer Perceptron (MLP) projection with a lightweight convolutional layer to introduce a local inductive bias that preserves the spatial relationships between adjacent patch tokens; and (2) applying a spatial normalization layer to the original target representations to increase the signal-to-noise ratio of local anatomical structures by sacrificing global information to improve spatial contrast.

The full training objective combines denoising and alignment terms: L = L Denoising + λ · L REPA, with λ set to 1.

The framework supports two operational inference-time modes: high-fidelity sampling by traversing the learned reverse SiT trajectory, and feature extraction where the frozen SiT blocks serve as a robust feature extractor, yielding high-dimensional patch descriptors extracted from intermediate transformer layers at t=0.

• Pretraining: GN-5M, "a large, multicenter dataset of 4,820,653 unlabeled endoscopic images collected from eight Dutch hospitals, across multiple endoscopic procedures (colonoscopy, gastroscopy, capsule endoscopy) and imaging modalities (white light, narrow-band, blue light, and linked color imaging)." Component analysis used 250,000 images; final models used the full set.

• Classification: The POLAR benchmark (polyp optical diagnosis, neoplastic vs. non-neoplastic) and a private Barrett's Esophagus neoplasia (BE) dataset enriched with subtle cases of early BE neoplasia.

• Robustness: BE-C, a corrupted variant of the BE test set with eight corruption types commonly encountered in clinical endoscopy practice: motion blur, defocus blur, overexposure, hue shift, saturation, contrast, sharpness, and brightness distortion, yielding 819 corrupted images.

Component Analysis: The ablations establish that SD2 provides the optimal balance for endoscopic synthesis, with its compact 4-channel bottleneck being more easily modulated by the SiT backbone. The domain-specific DINOv3-B (GN-5M) teacher outperforms all alternatives in FID and POLAR. Scaling from SiT-S to SiT-L yields a substantial improvement in image fidelity, and transitioning from the 250k subset to the full GN-5M dataset yields immediate and drastic improvements across all metrics, with the best results at 250k iterations on the full dataset achieving FID 5.32, BE accuracy 0.904, and POLAR accuracy 0.773.

Classification Performance: Despite being a generative model not explicitly optimized for discriminative tasks, REVEAL achieves these results using only the first 8 layers of the transformer backbone. It achieves BE AUC 0.786/AUPRC 0.728 and POLAR AUC 0.758/AUPRC 0.935, convincingly outperform[ing] specialized models like EndoViT and Endo-FM, and surpass[ing] all general-purpose encoders on both benchmarks. The authors note EndoViT underperforms general-purpose encoders DINOv2 and DINOv3 on both benchmarks, suggesting that domain specificity alone is insufficient without adequate pretraining scale and objective expressiveness.

Robustness Performance: Under corrupted imaging conditions (BE-C), REVEAL achieves AUC 0.754/AUPRC 0.679, surpass[ing] the endoscopy-specific models EndoViT and Endo-FM by a substantial margin. Notably, EndoViT degrades catastrophically under corruption, suggesting that its representations are highly sensitive to low-level imaging artifacts despite being domain-specific. The authors note REVEAL is evaluated at the standard noise level t = 0; intermediate diffusion timesteps, which naturally denoise corrupted inputs, could be expected to yield further gains.

Qualitative Results: Unconditional generation reveals that the model can synthesize plausible endoscopic scenes, capturing the intricate light reflections and mucosal textures typical of the GN-5M distribution. Inpainting and outpainting experiments demonstrate that "REVEAL accurately reconstructs missing anatomical regions and enlarges the visible area while preserving structural continuity, indicating that aligning latent representations with a specialized teacher effectively models gastrointestinal morphology beyond the pixel level."

The authors conclude that large-scale generative pretraining [is] a viable and complementary path to masked image modeling for building clinical visual representations. The aligned latent space is described as "a natural substrate for a broad range of downstream applications, such as conditional synthesis for targeted pathology generation, rare lesion augmentation, segmentation via generative features, and out-of-distribution detection through likelihood estimation." Future work includes scaling to higher resolution, larger transformer backbones, conditional finetuning on labeled subsets, generative classifiers, and robustness validation across devices and centers. The code and weights are publicly released at caetas.github.io/reveal.html.

Improvements you can make:

• Train the alignment teacher encoder on large-scale, domain-specific clinical data (e.g., 5M endoscopy frames) instead of using general-purpose vision encoders, ensuring the latent space captures mucosal textures, specular reflections, and subtle anatomical morphology rather than global semantic proxies.

• Replace the standard MLP representation-alignment projector with a lightweight convolutional projection and add spatial normalization to the target teacher features — this preserves local spatial relationships between patch tokens and increases the signal-to-noise ratio of fine anatomical structures, improving morphological fidelity.

• Use a Scalable Interpolant Transformer (SiT) backbone trained with a combined denoising + representation-alignment objective (λ=1) so the same network serves both as a high-fidelity generative model and as a robust feature extractor.

• Exploit the frozen intermediate transformer layers of the generative backbone (e.g., first 8 layers at t=0) as patch-level descriptors for discriminative tasks, rather than training a separate encoder — this yields competitive or superior classification performance compared to dedicated medical foundation models.

• Leverage the diffusion process for robustness: sample or extract features at intermediate denoising timesteps to inherently counteract clinical imaging artifacts such as motion blur, defocus, overexposure, hue shifts, and contrast distortion, instead of relying on pre-processing or data augmentation.

• Scale pretraining data and backbone capacity together (e.g., from 250k to 4.82M images, from SiT-S to SiT-L) to unlock drastic improvements in FID and downstream classification accuracy.

The improved AI system can:

• Generate realistic, high-fidelity endoscopic images with accurate mucosal texture, light reflections, and anatomical continuity, enabling unconditional synthesis, inpainting of missing regions, outpainting of the visible area, and targeted conditional generation (e.g., rare lesion augmentation).

• Perform polyp optical diagnosis and Barrett’s esophagus neoplasia classification with higher accuracy than existing endoscopic foundation models like EndoViT and Endo-FM, while also surpassing general-purpose encoders such as DINOv2 and DINOv3.

• Maintain high classification performance (e.g., AUC 0.75, AUPRC 0.68) on heavily corrupted clinical images with realistic artifacts, avoiding the catastrophic degradation seen in domain-specific encoders like EndoViT.

• Provide a single, reusable feature extractor for downstream tasks — including segmentation, out-of-distribution detection, and generative-classifier pipelines — without separate pretraining, since frozen SiT layers yield rich patch-level descriptors.

• Generalize across multiple endoscopic procedures (colonoscopy, gastroscopy, capsule endoscopy) and imaging modalities (white light, narrow-band, blue light, linked color imaging) from multiple hospitals, reducing the need for dataset-specific fine-tuning.

• Be extended to higher resolutions, larger backbones, and conditional fine-tuning on labeled subsets, enabling robust deployment across new medical centers and devices.

Developing foundation generative models for endoscopy is limited by the gap between natural and clinical images and the computational cost of training large Diffusion Transformers. Although representation alignment has improved efficiency in general computer vision, its role within the highly specialized endoscopic image space remains unclear. We introduce REVEAL (Representation-driven Endoscopic Visual Embedding Alignment), the largest generative foundation model for endoscopy to date, trained on GastroNet-5M (GN-5M), a multicenter dataset of 5 million endoscopic frames. Instead of depending on out-of-domain priors, REVEAL employs encoders pretrained directly on the endoscopic distribution to align diffusion latents with domain-specific visual features, preserving fine textures and intricate anatomical structures. Beyond image generation, REVEAL also serves as a powerful feature extractor; in multiple benchmarks, it delivers performance that is competitive with, and in several cases exceeds, endoscopic foundation models such as EndoViT and Endo-FM, specifically tuned for classification tasks, while demonstrating strong representation robustness under realistic imaging corruptions. REVEAL produces high-fidelity images and maintains robust structural coherence in latent-space edits such as inpainting and outpainting. This high-capacity backbone lowers the computational threshold for building specialized clinical tools, offering an open, versatile foundation for conditional synthesis, segmentation, and out-of-distribution detection in future intelligent gastroenterology systems.