eXact-Prior Variational Autoencoder (X-VAE): Learning Data-Adaptive Gaussian Mixture Priors for Latent Distributions
Qijun Chen, Shaofan Li
stat.ML, cs.LG
Submitted: 2026-06-30
License: http://creativecommons.org/licenses/by/4.0/
The gist: Variational Autoencoders (VAEs) commonly assume a standard isotropic Gaussian prior over the latent space, an assumption that often fails to capture the true distribution of latent representations
Terminology
Abstract
Variational Autoencoders (VAEs) commonly assume a standard isotropic Gaussian prior over the latent space, an assumption that often fails to capture the true distribution of latent representations for complex datasets. This mismatch can limit reconstruction accuracy, reduce sample quality, and constrain the expressive power of the learned latent space. We propose the eXact-Prior Variational Autoencoder (X-VAE), a framework that replaces the conventional standard normal prior with a Gaussian prior derived from the latent representations of a pretrained autoencoder (AE). Specifically, the empirical mean and standard deviation of the AE latent codes are used to parameterize a data-adaptive prior that more closely reflects the underlying structure of the training data. During generation, X-VAE introduces a latent scaling factor that enables explicit control over the variance of the sampled latent vectors, providing a simple mechanism for balancing sample diversity and fidelity. This flexibility makes the proposed approach particularly well suited for applications such as industrial and engineering design, where generated solutions must satisfy strict structural or functional constraints while still permitting meaningful design exploration. We present the mathematical formulation of well-suited X-VAE, derive the corresponding KL divergence objective for the proposed prior, and evaluate the method on standard benchmark datasets. Experimental results demonstrate that X-VAE preserves reconstruction quality while producing latent representations that better align with the empirical data distribution, leading to improved controllability and more realistic generated samples.
Sources
- Fixing a Broken ELBO
- Latent Space Oddity: on the Curvature of Deep Generative Models
- dpVAEs: Fixing Sample Generation for Regularized VAEs
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Learning Disentangled Joint Continuous and Discrete Representations
- Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Categorical Reparameterization with Gumbel-Softmax
- Auto-Encoding Variational Bayes
- Adam: A Method for Stochastic Optimization
- Improving Variational Inference with Inverse Autoregressive Flow
- Autoencoding beyond pixels using a learned similarity metric
- Deep Learning Face Attributes in the Wild
- Adversarial Autoencoders
- Generating Diverse High-Fidelity Images with VQ-VAE-2
- Variational Inference with Normalizing Flows
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Distribution Matching in Variational Inference
- Improved Techniques for Training GANs
- Ladder Variational Autoencoders
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches