Marformer: A Transformer for Predicting Missing Data Distributions
The Marformer is a Transformer model designed to predict missing data distributions directly from observed values. It uses an attention mechanism to refine representations of each variable, allowing it to compute conditional marginals needed for Bayes risk and Value of Information. This approach is fast, requires no prior knowledge of the data generation process, and outperforms classical methods on synthetic and real datasets.