Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.
Vera: We're looking at the paper: "Learning What's Real".
Jocelyn: Here's the gist of the paper. Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument.
Vera: This secondary signal acts as a confounding factor, limiting our ability to extract information about the underlying physics.
Jocelyn: To disentangle these factors of variation, we propose a dual-encoder architecture with a counterfactual generation objective that leverages overlapping observations.
Vera: The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference, and instrumentindependent similarity search—all unconfounded by measurement artifacts.
Title and authors: Vera: So, to summarize what they’ve done in "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," they propose a dual-encoder architecture with a counterfactual generation objective. This framework is designed specifically to separate the intrinsic physical signal from measurement artifacts caused by different sensors or instruments.
Jocelyn: They achieve this by using overlapping observations across multiple instruments—like combining data from the Legacy Survey and Hyper Suprime-Cam—to create training triplets that force the encoders to learn distinct representations for physics and instrument effects.
Subrahmanyan: The paper’s summary focuses heavily on how they construct these training triplets: an anchor observation, an instrument-augmented observation, and a physics-augmented observation. They then use the physics encoder to capture invariant features while the instrument encoder isolates the measurement distortions.
Vera: They show that by using this setup, you get separate latent variables where one space focuses on physical properties and the other on instrumental characteristics. This is what allows for downstream tasks like outlier detection or parameter inference without being confused by instrument-specific noise patterns.
Jocelyn: What’s really interesting about the summary is that they treat sensor- or modality-specific effects as augmentations during training, which helps build those robust, invariant representations through the counterfactual generation objective. It’s a recipe for self-supervised pretraining in science.
Subrahmanyan: From a theory standpoint, it moves away from relying on hand-engineered loss functions and instead enforces structural disentanglement through this specific architecture and the flow matching mechanism, which connects it to ideas about recovering independent causal mechanisms three <ref:2604.09787#pg1>.
The paper's summary: Vera: Looking at the improvements they suggest in "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," they focus on the fact that this approach is architecture-driven rather than relying on pre-defined loss functions. They’re enforcing structural disentanglement through their specific design.
Jocelyn: The improvements highlight how they use multi-view observations—seeing the same underlying content under different conditions or at different times—as a natural source of structure for disentanglement, which is something we see everywhere in real observation data.
Subrahmanyan: I think the most concrete improvement is the asymmetry in property prediction they found. The physics latent space substantially outperforms instrument latents when predicting physical properties, while the pattern reverses when it comes to instrumental properties. This asymmetry proves that their architecture actually succeeds in separating those factors efficiently.
Vera: That’s what they mean by achieving a more efficient separation of physics and instrumental properties compared to monolithic foundation models. The physics latent space recovers physical properties at a level comparable to AION-one which is a strong benchmark for this kind of feature recovery <ref:2604.09787#pg1>.
Jocelyn: And they also show that the instrument latent space does a better job at predicting those instrument-related properties than baseline models, which is useful for understanding the systematics themselves.
Subrahmanyan: If we think about the implications, this means we can trust inferred galaxy properties more across different surveys because our method is explicitly designed to be robust to those instrumental variations, even if those variations are complex.
The paper's improvements: Vera: So, wrapping up "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," the main implication is that we can now build a framework for scientific self-supervised pretraining. It’s about constructing training pairs from overlapping observations of the same physical system.
Jocelyn: This means we treat sensor or modality effects as augmentations when training and learn invariant representations through counterfactual generation, which lets us do things like instrument-independent similarity search without being confounded by artifacts.
Subrahmanyan: For cosmology, this suggests that we can more reliably use these latent spaces to streamline the discovery of rare astrophysical phenomena because the physics latent space effectively isolates and groups complex physical features.
Vera: They also confirm that their generative decoder acts like a learned survey simulator, performing instrument-aware super-resolution and denoising. This is useful for reducing search space for follow-up observations by generating informed predictions rather than just noisy pixels.
Jocelyn: It’s a general recipe here: construct training pairs from overlapping data, treat system effects as augmentations, and learn invariant representations this way. It’s applicable beyond just galaxies, to other areas of astrophysics and even tens of millions of light curves from missions like TESS or Kepler.
Subrahmanyan: I just want to say that the physics latent space recovering properties at a level comparable to AION-one is a solid piece of evidence for how effective this architecture is at isolating intrinsic features <ref:2604.09787#pg1>. It shows the method has real power for uncovering those complex structures we’re looking for in the sky.
Vera: Exactly, and it confirms that by explicitly disentangling the factors, we get a more efficient separation than monolithic foundation models. We have to keep an eye on how they extend this to settings where you don't have such perfect overlap yet.
Conclusion: Vera: So we've been looking at "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," and essentially what they did is build a dual-encoder system that separates the actual physical signal from all the noise caused by different instruments.
Jocelyn: Right, so it uses overlapping observations—like combining data from different surveys—to create these training sets where you have the same physical thing seen through different lenses.
Subrahmanyan: The core idea is to force two encoders to learn separate things: one encoder learns the physics, and the other learns how each specific instrument messes up the data.
Vera: Exactly. The results show that if you look at those latent spaces, you get a physics space that recovers real physical properties pretty well, even matching models like AION-one.
Jocelyn: And the instrument space is surprisingly good at capturing those measurement distortions themselves, which is useful for understanding the noise conditions of different instruments.
Subrahmanyan: From a cosmic view, this means we can now trust inferred galaxy properties more across different surveys because they aren't just affected by the specific sensor they were taken with.
Vera: It also gives us tools like counterfactual generation, which lets you predict what an observation would look like under different conditions or even generate a high-resolution image from a low-resolution one.
Jocelyn: That means we can use it to simulate observations or even do some kind of instrument-independent similarity search, which is pretty neat for finding similar sources across different datasets.
Subrahmanyan: The limitation they mention is that this method currently needs those overlapping observations across instruments, so you're mostly limited to regions where the sky coverage from those surveys overlaps.
Vera: It’s a powerful tool for self-supervised pretraining because it gives us a general recipe: take overlapping data, treat the sensor effects as augmentations during training, and learn these invariant representations.
Jocelyn: It’s a solid framework for moving forward, especially since they show how this architecture can be applied to tens of millions of light curves from missions like TESS or Kepler.
Subrahmanyan: That's where the real potential lies, applying it to those massive datasets to find patterns in structure formation that we just can't see otherwise.
Vera: So, for anyone listening, this paper is about building a way to clean up messy observational data by learning what's physically real versus what the equipment is adding on top of it.
Jocelyn: It shows how combining different instruments can actually help us understand the underlying physics better than using just one telescope alone.
Subrahmanyan: We'll keep an eye on how they extend this to unpaired settings, because that's where the real challenge in observational science often lies.
Massachusetts Institute of Technology · Flatiron Institute, Simons Foundation · AstroAI, CfA | Harvard & Smithsonian
astro-ph.IM, astro-ph.GA, cs.LG
Submitted: 2026-04-10
Updated: 2026-11-07
Comments: Accepted to the 40th Conference on Neural Information Processing Systems (NeurIPS 2026)
Code: https://github.com/pablomerc/galaxy-counter
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 91/100
The gist: Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from…
Key concepts
- Dual-Encoder Architecture
- This architecture uses two specialized neural networks: a physics encoder and an instrument encoder. The physics encoder learns the intrinsic properties of the source while ignoring instrumental noise, and the instrument encoder learns the specific distortions introduced by each sensor.
- Counterfactual Generation Objective
- This training objective leverages overlapping observations to create 'training triplets' (anchor, different instrument observation, different source observation). This forces the model to learn representations that are invariant to measurement artifacts by generating synthetic views that differ only in the instrumental or source context.
- Physics Latent Space
- This latent space is designed specifically to capture underlying physical properties of astronomical objects, such as galaxy morphology. It is trained to be robust against sensor-specific distortions, meaning it recovers accurate physical characteristics even when data comes from different instruments.
Terminology
Summary
Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. p. 0 Freeman
Massachusetts Institute of Technology
billf@mit. edu
Abstract
Data collected from the physical world is always a combination of multiple sources:
an underlying signal from the physical process of interest and a signal from
measurement-dependent artifacts from the sensor or instrument. edu
Abstract
Data collected from the physical world is always a combination of multiple sources:
an underlying signal from the physical process of interest and a signal from
measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the
underlying physics. Moreover, it poses significant challenges for combining data
in heterogeneous or multi-instrument frameworks. To disentangle these factors of
variation, we propose a dual-encoder architecture with a counterfactual generation
objective that leverages overlapping observations. To disentangle these factors of
variation, we propose a dual-encoder architecture with a counterfactual generation
objective that leverages overlapping observations. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can
be used for counterfactual view generation, parameter inference, and instrumentindependent similarity search—all unconfounded by measurement artifacts. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can
be used for counterfactual view generation, parameter inference, and instrumentindependent similarity search—all unconfounded by measurement artifacts. We
demonstrate the effectiveness of our approach in a multi-instrument setting on
astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and
the Hyper Suprime-Cam (HSC) Survey. We
demonstrate the effectiveness of our approach in a multi-instrument setting on
astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and
the Hyper Suprime-Cam (HSC) Survey. This framework provides a general recipe
for scientific self-supervised pretraining: construct training pairs from overlapping
observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual
generation.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The Core Methodology
The framework introduces a dual-encoder architecture with a counterfactual generation objective designed to learn representations that disentangle physical properties from measurement systematics. This objective leverages large-scale scientific datasets with overlapping observations from multiple instruments to construct training triplets: an anchor, an observation of the same source (but different instrument), and an observation on the same instrument (but different source). p. 2 Our framework leverages large-scale scientific datasets with
overlapping observations from multiple instruments. In this setting, natural augmentations can be
used to construct triplets: an anchor, an observation of the same source (but different instrument),
and an observation on the same instrument (but different source), as shown in Fig. In this setting, natural augmentations can be
used to construct triplets: an anchor, an observation of the same source (but different instrument),
and an observation on the same instrument (but different source), as shown in Fig. A physics encoder receives a sample from the target source but from a different instrument, so it
is encouraged to reduce the loss by capturing the underlying physical properties of the source and
becoming invariant to the instrumental distortions. A physics encoder receives a sample from the target source but from a different instrument, so it
is encouraged to reduce the loss by capturing the underlying physical properties of the source and
becoming invariant to the instrumental distortions. Similarly, an instrument encoder receives a sample
from the target instrument but a different source, so it is encouraged to capture instrument-specific
distortions while ignoring the content specific to the source. Similarly, an instrument encoder receives a sample
from the target instrument but a different source, so it is encouraged to capture instrument-specific
distortions while ignoring the content specific to the source. The decoder combines these two latent
embeddings to reconstruct an unseen target or anchor observation.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Disentanglement and Representation Learning
The model is trained using a conditional flow matching objective, learning a velocity field that transports samples from a Gaussian base distribution to the target distribution. The training loss is defined as LFM(θ) = E t∼U[0,1] j∼[N],k∼[M] xj,k∼pdata x0∼N(0,I) h uθ xt, t, zphy(xj,k′)k′̸=k zins(xj′,k)j′̸=j − (xj,k − x0) 2 i (1). p. 3 We model this conditional distribution using flow matching [14], learning a velocity field
uθ(x, t, zphy, zins) that transports samples from Gaussian base distribution to the target distribution. The training loss is given by the error on the predicted velocity field,
LFM(θ) = E t∼U[0,1]
j∼[N],k∼[M]
xj,k∼pdata
x0∼N(0,I)
h
uθ
xt, t, {zphy(xj,k′ )}k′̸=k, {zins(xj
′
,k)}j
′̸=j
− (xj,k − x0)
2
i
(1)
where xt = (1−t)x0 +txj,k. That is, we sample a source j, a target instrument k, a noise sample x0,
and a timestep t, and train the velocity field to reconstruct the observation xj,k from a noisy state xt
conditioned on physics latents from other instruments and instrument latents from different sources. Once trained, the physics embedding supports downstream tasks such as outlier detection, parameter
inference, classification, or embedding-based retrieval, while the instrument space allows to find
similar systematics and observing conditions. That is, we sample a source j, a target instrument k, a noise sample x0,
and a timestep t, and train the velocity field to reconstruct the observation xj,k from a noisy state xt
conditioned on physics latents from other instruments and instrument latents from different sources. Once trained, the physics embedding supports downstream tasks such as outlier detection, parameter
inference, classification, or embedding-based retrieval, while the instrument space allows to find
similar systematics and observing conditions.Where the paper says it
Where the paper says it
Where the paper says it
Application in Astrophysics
The model is demonstrated on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey. The preprocessing pipeline aligns overlapping filters across both surveys into a unified four-channel representation and treats both sources as the same modality. p. 3 Despite the shared designations, they differ significantly in their central wavelengths, pixel scales,
zero-points, and noise characteristics. Our preprocessing pipeline follows the strategy established
by [8], with one key difference: rather than treating all nine bands as distinct channels, we align the
overlapping (g, r, i, z) filters across both surveys into a unified four-channel representation and treat
both sources as the same modality. Here, we show that counterfactual reconstruction improves
robustness to instrument systematics while enabling new foundational capabilities, including crossinstrument generation, instrument-independent retrieval, and data-driven denoising. We take a cross-matched dataset from the Multimodal Universe that includes ∼100,000 cross-matched
galaxy images from the DESI Legacy Imaging Surveys [Legacy, 15] and the Hyper Suprime-Cam
[HSC, 16] survey as a representative multi-instrument setting.Where the paper says it
Where the paper says it
Performance Metrics and Capabilities
The effectiveness is evaluated by probing the latent spaces with a three-layer MLP to predict physical and instrumental properties from embeddings extracted by each encoder. The physics latent space recovers physical properties at a level comparable to AION-1, while the instrument latent space dominates on instrument-related properties. p. 5 Details on the
properties and source catalogs are provided in Appendix A. The physics latent space recovers physical properties at a level comparable to AION-1. The generative decoder enables generating counterfactual views. In astrophysics, a particularly promising use case is to inform follow-up observations by
predicting high-resolution observations from expensive facilities using low resolution observations
from a different instrument. Data-driven noise model. The instrument encoder and generative decoder together yield a datadriven noise model: by manipulating the instrument conditioning at inference time, we can simulate
observations under different conditions while holding the underlying physics fixed.Where the paper says it
Where the paper says it
Where the paper says it
Limitations and Future Work
Our approach currently requires overlapping observations across instruments, which limits applicability to regions of shared sky coverage. Future work plans include extending the framework to unpaired settings where such overlap is unavailable or limited. p. 9 Our approach currently requires overlapping observations across
instruments, which limits applicability to regions of shared sky coverage. In future work, we aim to
extend the framework to unpaired settings where such overlap is unavailable or limited. Moreover,
while we have focused on galaxy imaging, the framework is general and applicable to other domains
in astrophysics and beyond. In particular, we plan to apply it to the tens of millions of light curves
from NASA’s Transiting Exoplanet Survey Satellite [TESS, 22] and Kepler [23, 24] missions.Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The key insight is that cross-matched observations form natural pairs of same physics, different instrument, providing a training signal for separating the two sets of factors without requiring an explicit forward model of either instrument. This formulation allows the model to leverage all available multi-instrument data: the physics of the target source is informed by its appearances across different instruments, while the instrument response is informed by how other sources appear under similar observing conditions. p. 3 The set {zins(xj
′
,k)}j
′̸=j are the instrument embeddings extracted from observations of other
sources j
′ by the same instrument k. This formulation allows the model to leverage all available
multi-instrument data: the physics of the target source is informed by its appearances across different
instruments, while the instrument response is informed by how other sources appear under similar
observing conditions.Where the paper says it
How the training signal is built
Training triplets are constructed from an anchor observation (signal s from instrument i), an instrument-augmented observation (same source s, a different instrument i′), and a physics-augmented observation (a different source s′, same instrument i). The physics encoder extracts latents from the instrument-augmented pair to capture invariant physical features, while the instrument encoder processes the physics-augmented observation to isolate measurement artifacts. p. 1 Training uses data triplets:
an anchor observation (signal s from instrument i), an instrument-augmented observation (same
source s, a different instrument i
′
), and a physics-augmented observation (a different source s
′
, same
instrument i). The physics encoder extracts latents from the instrument-augmented pair to capture
invariant physical features, while the instrument encoder processes the physics-augmented observation to isolate measurement artifacts.Where the paper says it
Where the gains come from
The asymmetry in property prediction shows that our architecture successfully separates physics properties from instrumentspecific information. The physics latents substantially outperform instrument latents at predicting (mostly) physical properties, whereas the pattern reverses for instrument properties. p. 6 The full results are shown in Appendix A. The key diagnostic is the asymmetry in the split of properties: the physics latents substantially
outperform instrument latents at predicting (mostly) physical properties, whereas the pattern reverses
for instrument properties. On the other hand, we note that AION-1 embeddings retain sensitivity
to instrumental properties, but our instrument latent space outperforms both baselines in predicting
instrument-related properties. This highlights the primary advantage of our architecture: by explicitly
disentangling the latent factors, we achieve a more efficient separation of physics and instrumental
properties than monolithic foundation models. predicting physics-related properties from the instrument latents),
indicating that the model has actively learned to erase instrument-specific information from the
physics space and vice versa. On the other hand, we note that AION-1 embeddings retain sensitivity
to instrumental properties, but our instrument latent space outperforms both baselines in predicting
instrument-related properties.Where the paper says it
Where the paper says it
Where the paper says it
The training objective is designed to learn representations that disentangle physical properties from measurement systematics. The generative component can also be viewed as a data-driven instrument model, learned implicitly from crossmatched observations. p. 9 Compared to previous contrastive approaches, our method enables counterfactual generation by synthesizing how a source would appear under different observing
conditions, and it is designed to scale to settings with more than two instruments. The generative
component can also be viewed as a data-driven instrument model, learned implicitly from crossmatched observations. The model successfully generates progressively cleaner images; the
average SNR shown is that of the images used to condition the reconstruction. Bottom (Systematic
Artifact Removal): By conditioning the physics encoder on an artifact-corrupted HSC image and
the instrument encoder on a random set of HSC images, the model successfully removes severe
instrumental distortions while preserving the underlying galaxy morphology.Where the paper says it
Where the paper says it
The framework is general and applicable to other domains in astrophysics and beyond. p. 9 In future work, we aim to
extend the framework to unpaired settings where such overlap is unavailable or limited. Moreover,
while we have focused on galaxy imaging, the framework is general and applicable to other domains
in astrophysics and beyond. multiple HSC images that exhibit strong morphological evidence to be considered candidate lenses. This confirms that the physics latent space effectively isolates and groups complex physical features,
streamlining the discovery of rare astrophysical phenomena. multiple HSC images that exhibit strong morphological evidence to be considered candidate lenses. This confirms that the physics latent space effectively isolates and groups complex physical features,
streamlining the discovery of rare astrophysical phenomena. multiple HSC images that exhibit strong morphological evidence to be considered candidate lenses. This confirms that the physics latent space effectively isolates and groups complex physical features,
streamlining the discovery of rare astrophysical phenomena. HSC’s higher-resolution allows to resolve features, such as faint arcs, that are blurred or
undetected in Legacy imaging. Our model can generate the expected HSC-like appearance for
any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs
instrument-aware super-resolution and denoising. HSC’s higher-resolution allows to resolve features, such as faint arcs, that are blurred or
undetected in Legacy imaging. Our model can generate the expected HSC-like appearance for
any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs
instrument-aware super-resolution and denoising. HSC’s higher-resolution allows to resolve features, such as faint arcs, that are blurred or
undetected in Legacy imaging. Our model can generate the expected HSC-like appearance for
any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs
instrument-aware super-resolution and denoising.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The model successfully captures realistic morphological structure and noise properties rather than pixel-level artifacts or overly smoothed outputs. The posterior uncertainty bands appropriately expand at higher spatial frequencies, where noise dominates. p. 7 Column 3 (Ground
Truth): The original target image. Columns 4–5 (Posterior Samples): Independent samples
generated by the decoder, demonstrating the model’s ability to capture noise and uncertainty. First, in a controlled SNR traversal, we reconstruct
an HSC target galaxy from its Legacy counterpart five times, fixing the initial noise sample x0 but
varying the instrument conditioning across the 5th, 25th, 50th, 75th, and 95th percentiles of the HSC
SNR distribution (defined as the mean pixel-wise flux divided by the flux error, averaged across
channels). Traversing this axis produces progressively cleaner reconstructions, even surpassing the
apparent SNR of the ground truth—indicating an exploitable, manipulable noise model rather than
memorization.Where the paper says it
Where the paper says it
The framework is general and applicable to other domains in astrophysics and beyond. This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena. The model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising. The generated images are best understood as informed predictions that reduce the search space for follow-up, rather than replacements for real, deeper observations. p. 8 Features well below the Legacy detection threshold will not appear in
the counterfactual. The generated images are best understood as informed predictions that reduce the
search space for follow-up, rather than replacements for real, deeper observations. Features well below the Legacy detection threshold will not appear in
the counterfactual. The generated images are best understood as informed predictions that reduce the
search space for follow-up, rather than replacements for real, deeper observations.Where the paper says it
Where the paper says it
The physics latent space recovers physical properties at a level comparable to AION-1. The instrument latent space dominates on instrument-related properties while the physics latents collapse near the random-baseline level on those same properties. p. 6 Notably, our latent spaces often contain less information than the baselines
when used for cross-prediction (e. predicting physics-related properties from the instrument latents),
indicating that the model has actively learned to erase instrument-specific information from the
physics space and vice versa. This cross-prediction baseline measures the shared spatial component:
any predictive power it achieves reflects information that is inherently common to both surveys
and therefore accessible to either encoder by construction. We find that predictions from physics
latents are always worse than the cross-predict baseline, verifying that the physics latent space is not
capturing private instrument information.Where the paper says it
Where the paper says it
The model successfully removes severe instrumental distortions while preserving the underlying galaxy morphology. The framework is general and applicable to other domains in astrophysics and beyond. This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena. The model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising. The generated images are best understood as informed predictions that reduce the search space for follow-up, rather than replacements for real, deeper observations.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Disentangled Representation Learning enables robust inference by separating intrinsic physical properties from sensor-specific distortions through a dual-encoder architecture. This allows for
outlier detection unconfounded by instrument systematics,
meaning models can identify physically unusual sources without being biased by instrumental artifacts like saturated pixels or noise patterns. -
Counterfactual Generation allows for instrument independent similarity search and novel data generation. The framework enables the model to
generate counterfactual views,
which can act as alearned survey simulator that performs instrument aware super-resolution and denoising,
allowing researchers to predict high resolution observations from low resolution ones for follow up decisions. -
Parameter Inference Robustness allows for reliable property estimation despite instrumental variations. The physics latent space recovers physical properties at a level comparable to existing foundation models while being robust to instrumental systematics, meaning the resulting inferred galaxy properties are more trustworthy across different instruments than those from monolithic models.
Abstract
Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the physics underlying the phenomena we observe. Furthermore, it complicates the combination of observations in heterogeneous or multi-instrument settings. We propose a deep learning framework that leverages overlapping observations, a dual-encoder architecture, and a counterfactual generation objective to disentangle these factors of variation. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference unconfounded by measurement distortions, and instrument-independent similarity search. We demonstrate the effectiveness of our approach on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey as a representative multi-instrument setting. This framework provides a general recipe for scientific and multi-modal self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation.
Sources
- SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery
- Towards Galaxy Foundation Models with Hybrid Contrastive Learning
- Euclid Quick Data Release (Q1) Exploring galaxy properties with a multi-modal foundation model
- Causal Foundation Models: Disentangling Physics from Instrument Properties
- Flow Matching for Generative Modeling
- Deep Residual Learning for Image Recognition
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- AstroCLIP: A Cross-Modal Foundation Model for Galaxies
Related papers
- On-sky demonstration of a vector Zernike wavefront sensor in a cascaded adaptive optics system
- Euclid Data Release 1 (DR1): Calibration strategy and observations
- The General Antiparticle Spectrometer (GAPS) Antarctic Balloon Payload
- Detection and excision of radio-frequency interference in radio arrays
- Unified matched-filter and excess-power pipeline for gravitational-wave detector characterization
- Ushering in sub-arcsecond angular resolution observatories in the far-infrared: The case for a double-Fourier interferometer