COSMIO: A Benchmark for Cross-Survey Modality Imputation
summary
The gist
Large-scale astronomical surveys define a heterogeneous, partially observed multisurvey learning setting: each object is observed by only a subset of surveys…
In short
The paper introduces COSMIO, a benchmark for cross-survey astronomical modality imputation. This involves inferring an object's appearance in a target survey using observations from other surveys with different characteristics like resolution and wavelength. The study shows that methods from medical imaging fail to transfer well, and rare events like strong lensing present significant challenges.
Key concepts
- Cross-survey Modality Imputation
- This is the task of guessing how a celestial object looks in one survey when you only have data from other, different surveys. The surveys differ in things like image quality or wavelength, and the goal is to fill in the missing information to create a complete picture.
- COSMIO Benchmark
- COSMIO is a test set containing realistic observations of galaxies rendered under different survey conditions (LSST, Euclid, Roman). It includes regular galaxies and strong-lensing systems, allowing researchers to systematically evaluate how well imputation methods work across various missing data patterns.
- Rare Event Imputation
- This refers to imputing data for scientifically valuable but infrequent phenomena, such as strong gravitational lensing. The paper finds that models struggle when the rare event distribution shifts, meaning they perform poorly on these important but underrepresented astronomical features.
Terminology used across episodes
This episode discusses
- COSMIO: A Benchmark for Cross-Survey Modality Imputation · Paper Radio
- Cosmology with the SPHEREX All-Sky Spectral Survey
- Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
- SLSim: a strong lensing population simulation package
- The Saas Fee Lectures on Strong Gravitational Lensing
The paper
COSMIO: A Benchmark for Cross-Survey Modality Imputation · Read on arXiv
Dichang Zhang, Yixuan Shao, Jiaqi Deng, Jiali Cui, Haotian Yin, Yuanpeng Liu, Xiang Gao, Zhiqiang Lao
Department of Computer Science, Stony Brook University, USA · Department of Physics and Astronomy, Stony Brook University, USA · School of Computer Science, University of Technology Sydney Australia · Futurewei Technologies USA · Department of Computer Science, New Jersey Institute of Technology USA
Large-scale astronomical surveys define a heterogeneous, partially observed multi-survey learning setting: each object is observed by only a subset of surveys, while survey observations differ in resolutions, wavelength coverage, noise characteristics, and footprints. This setting gives rise to a natural machine learning problem: given observations of an object in some surveys, how to infer its appearance in other surveys. We formalize this as the problem of cross-survey modality imputation and introduce COSMIO, a multi-survey benchmark of aligned observations obtained by forward-rendered shared astrophysical scenes across LSST, Euclid, and Roman under survey-specific instrument characteristics. The benchmark enables systematic evaluation of imputation across source-target survey combinations and missing-survey patterns. We evaluate a broad range of machine learning methods for imputation, which reveal a clear domain gap: methods developed for existing imputation settings transfer poorly to the survey setting. We identify rare-event imputation as a central challenge. This regime arises in the early stages of new surveys, when rare but scientifically valuable phenomena may be available only from prior surveys. We use strong gravitational lensing, where light from a background galaxy is gravitationally deflected by a foreground galaxy, as a representative rare-event setting. We evaluate model performance on zero-shot imputation of gravitational lenses and on downstream scientific tasks using the imputed lens observations. On our constructed evaluation set of galaxy-galaxy strong lenses, performance degrades under rare-event distribution shift, and this degradation further propagates to the downstream lens-detection task. Code: https://github.com/fw-ic/COSMIO.
Transcript
Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.
Vera: We're looking at the paper: "COSMIO: A Benchmark for Cross-Survey Modality Imputation".
Jocelyn: Here's the gist of the paper. Large-scale astronomical surveys define a heterogeneous, partially observed multisurvey learning setting: each object is observed by only a subset of surveys, while survey observations differ in resolutions, wavelength coverage, noise characteristics, and footprints Given observations of an object in some surveys, how to infer its appearance in other surveys The benchmark enables systematic evaluation of imputation across source-target survey combinations and missing-survey patterns.
Vera: The Problem Formalization The setting is formalized as the problem of cross-survey modality imputation, where the task is to infer the appearance of the same astrophysical source in a target survey from its available observations in one or more source surveys.
Jocelyn: This involves modeling the conditional distribution pθ x(T) x(O), where x(O) = y(k): k ∈ O and x(T) = y(k): k ∈ T.
Vera: The problem is particularly consequential during the initial observation phase of a new survey, when only a limited region of the sky has been covered.
Paper summary: Vera: So we're looking at this paper titled "COSMIO: A Benchmark for Cross-Survey Modality Imputation." Basically, they've set up this setup where you have multiple large astronomical surveys, and each object is only seen by a piece of those surveys. The core idea is figuring out how to guess what that object looks like in the surveys it hasn't been observed in yet.
Jocelyn: It’s about modeling this conditional distribution, which means taking what you see in some surveys and trying to infer the appearance of the same object in a target survey. They frame this as cross-survey modality imputation, and they want to see if models can do it well across different survey types and missing data patterns.
Subrahmanyan: It’s important because this is happening right now with big projects like LSST, Euclid, and Roman that are all observing the same sky but with different characteristics—different resolutions, wavelength coverage, noise levels. This whole problem is consequential when we're starting out with a new survey where only a small patch of the sky has been covered.
Vera: Exactly. The paper introduces COSMIO as this benchmark dataset. They’ve rendered one hundred fifteen thousand regular galaxy samples and fourteen thousand, eight hundred twenty five strong-lensing systems under realistic LSST, Euclid, and Roman survey configurations. This gives them a real test bed for the problem they are studying.
Jocelyn: They use this benchmark to ask three main questions: can models actually infer missing observations from what we have? Can models trained on common objects generalize when we look at scientifically important rare-event regimes? And finally, can these imputed images actually help us do downstream scientific tasks?
Subrahmanyan: That third question is critical. The authors found three main things. First, methods that work well for the most established imputation setting—multi-contrast brain MRI imputation—they transfer poorly to astronomical survey data and get beaten by simpler diffusion baselines, which shows a clear gap between the domains.
Vera: That’s a big finding because it suggests that just taking a model from one area and applying it to astronomy isn't going to work well. And second, they found that models trained on common galaxies actually degrade when you test them on rare galaxy-galaxy strong-lensing systems.
Jocelyn: And third, while the imputed observations can help with downstream tasks like detecting strong lenses, their performance still doesn't match what you'd see with real target survey observations, and it really depends on how good the reconstruction quality is.
Subrahmanyan: So what this means for the bigger picture is that they’ve established cross-survey imputation as a separate setting entirely. They highlight that we need dedicated methods instead of just reusing tools from other fields to solve these multi-survey problems.
Vera: It really does, and they emphasize that missing-modality rare-event imputation is a major challenge when we look at the early stages of new surveys. Strong gravitational lensing is used as a representative example of this rare event setting.
Paper summary: Jocelyn: And they point out that performance degrades under shifts in the rare-event distribution, which means if you miss a scientifically valuable phenomenon, your model struggles even more with it. This degradation then carries over to the downstream lens detection task.
Subrahmanyan: That connects back to structure formation and cosmology because if we can't reliably detect these lenses from imputed data, we lose information about the large-scale structure of the universe that those lenses help us map out.
Vera: The authors also mentioned their limitation, which is that the dataset itself is constructed through forward physical rendering. They admit this introduces a sim-toreal gap, and because scientifically important phenomena are rare, they feel simulation is necessary to test these concepts.
Jocelyn: That’s a fair point on the limitation. But they've made this benchmark interesting precisely because it forces us to confront that gap and test how well different imputation approaches handle it before we apply them to real data.
Subrahmanyan: The results also show that asynchronous diffusion has the lowest reported error measure, CRPS, in each target-survey group. This suggests that async diffusion and conditional diffusion consistently beat the medical imputation baselines they compared against, indicating architecture choices tailored for medical imaging don't translate well here.
Vera: It confirms what Jocelyn said about the domain gap. The methods developed for MRI don't generalize to this astronomical setting; you need something built specifically for this cross-survey modality problem.
Jocelyn: And they point out that the choice of statistical statistic matters a lot in their ranking, with a dedicated estimator having better statistical properties than just looking at image-level generative imputation through morphology and rare-event detection.
Subrahmanyan: That’s a good caution for anyone trying to use these results. You need to think about what you are actually measuring when you compare the performance of different methods.
Vera: And finally, for downstream utility, they measure things like AUROC and AUPRC for lens classification, which directly tests how useful the imputed images are for actual scientific tasks. They found that asynchronous diffusion is still the best performer in those metrics.
Jocelyn: So to wrap up this paper "COSMIO: A Benchmark for Cross-Survey Modality Imputation," it shows that cross-survey imputation is a distinct learning problem, different from traditional imputation domains like brain MRI. It highlights rare-event imputation as a key bottleneck, especially when we are studying early observations in new surveys.
Subrahmanyan: The implication is that we need to develop methods specifically for multi-survey settings, because off-the-shelf models don't work well here, and current models degrade when faced with scientifically important rare events like strong lensing.
Vera: The authors are looking ahead by planning to crossmatch Rubin LSST and Euclid observations with real data, which is a smart next step to see if their benchmark conclusions hold up when applied to actual telescope data.
Jocelyn: So the main point is that we need new tools for this, and the paper gives us a good idea of what those tools should look like by showing where current approaches fail and where they succeed.
Conclusion: Vera: So we're wrapping up this discussion on COSMIO, which is this benchmark for cross-survey modality imputation, and who wrote it—the authors are working on modeling how to guess what an object looks like in a survey it hasn't been observed in yet.
Jocelyn: Yeah, so essentially they built this test case using data from LSST and Euclid and Roman surveys to see if we can actually fill in the blanks when we only have partial data for an object.
Subrahmanyan: It’s about modeling that conditional distribution, ptheta x(T) given x(O), which is like taking what you see in survey one and trying to figure out what it looks like in survey two.
Vera: And the authors are showing that their setup is distinct from the typical imputation problems we usually see, so we need new tools for this kind of cross-survey scenario.
Jocelyn: They’re pointing out that methods developed for other things, like brain MRI imaging, just don't transfer well to astronomical survey data.
Subrahmanyan: That’s a big thing because it tells us that the architecture choices we use need to be tailored specifically for these multi-survey settings.
Vera: And they flag missing-modality rare-event imputation as a major hurdle, especially when we're looking at new surveys where rare but important things might only show up in older data.
Jocelyn: They used strong gravitational lensing as an example of a rare event setting that makes this problem particularly tricky.
Subrahmanyan: It means if we miss those scientifically valuable phenomena from prior surveys, our models will struggle even more with them later on, and that affects how well we can do the science.
Vera: And while they've built this benchmark through physical rendering, they admit it has a sim-toreal gap, so it’s important to remember that the results are based on a simulation.
Jocelyn: So the main point is that off-the-shelf methods don't work here and we need new approaches for these multi-survey problems.
Subrahmanyan: It really highlights that when we look at early observations of new surveys, we need to focus on figuring out how to handle those rare events correctly.
Vera: Next up, Jocelyn and I are going to look at what this means for real follow-up observations and planning future telescope time.
More episodes
- 2607.17929-gevolution 2.0: GPU-accelerated relativistic N-body simulations for cosmology
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2509.02657-On the synergetic use of Ariel and JWST for exoplanet atmospheric science
- 2609.10042-Modelling Fourier filtering wavefront sensors for PSD-based methods: the AOPERA tool
- 2511.05639-Performance of the AstroPix Prototype Module for the Barrel Imaging Calorimeter at the ePIC Detector and in Space-Based Payloads
- 2610.19909-RPCA: a Resolved PSF reconstruction method for undersampled images
- 2512.03440-An Analysis of LIGO Glitches Using t-SNE During the First Part of the Fourth LIGO-Virgo-KAGRA Observing Run
- 2608.10284-Lunar Reflective Interferometry
- 2610.16633-Institutional broadening of the Japanese astronomy and astrophysics research community during 1996-2025