Every Measurement, Every Direction, All at Once: Multimodal Flow Matching for Molecules and Spectra

arXiv:2611.22098 · cs.LG, physics.chem-ph · Submitted 2026-11-17 · Read on arXiv

cs.LG, physics.chem-ph

Submitted: 2026-11-17

Updated: 2026-11-19

Comments: 30 pages, 12 figures. Accepted at NeurIPS 2026

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

The gist: Generative modeling in science is often data-limited, where experiments are expensive.

Terminology

Abstract

Generative modeling in science is often data-limited, where experiments are expensive. However, each experiment may yield several coupled measurements of the same physical entity. Standard generative models often train only on a single direction, discarding supervision from the rest. We argue that each experiment contains more training signal than this asymmetry allows the model to exploit: random masking enables measurements to serve as either conditioning inputs or prediction targets, yielding multiple supervised denoising tasks from a single experiment. We investigate this premise with MOSAIC, a multimodal flow matching transformer over molecular 3D structure and UV, IR, and Raman spectra. MOSAIC is a single model that is capable of any-to-any generation: generating any subset of modalities from any disjoint subset of the others. On QM9S, MOSAIC achieves 88.8% molecular graph recovery, outperforming the strongest spectra-to-structure baseline by 23.2% in structure generation, while simultaneously outperforming a structure-to-spectra specialist model in spectral generation. Ablations show that the any-to-any objective improves structure generation over structure-only training across model sizes. Furthermore, MOSAIC supports arbitrary subsets of available measurements and enables molecular identification by matching generated spectra to experimental references.

Related papers