Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics

arXiv:2604.09787 · astro-ph.IM, astro-ph.GA, cs.LG · Submitted 2026-04-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.

Vera: We're looking at the paper: "Learning What's Real".

Jocelyn: Here's the gist of the paper. Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument.

Vera: This secondary signal acts as a confounding factor, limiting our ability to extract information about the underlying physics.

Jocelyn: To disentangle these factors of variation, we propose a dual-encoder architecture with a counterfactual generation objective that leverages overlapping observations.

Vera: The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference, and instrumentindependent similarity search—all unconfounded by measurement artifacts.

Title and authors: Vera: So, to summarize what they’ve done in "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," they propose a dual-encoder architecture with a counterfactual generation objective. This framework is designed specifically to separate the intrinsic physical signal from measurement artifacts caused by different sensors or instruments.

Jocelyn: They achieve this by using overlapping observations across multiple instruments—like combining data from the Legacy Survey and Hyper Suprime-Cam—to create training triplets that force the encoders to learn distinct representations for physics and instrument effects.

Subrahmanyan: The paper’s summary focuses heavily on how they construct these training triplets: an anchor observation, an instrument-augmented observation, and a physics-augmented observation. They then use the physics encoder to capture invariant features while the instrument encoder isolates the measurement distortions.

Vera: They show that by using this setup, you get separate latent variables where one space focuses on physical properties and the other on instrumental characteristics. This is what allows for downstream tasks like outlier detection or parameter inference without being confused by instrument-specific noise patterns.

Jocelyn: What’s really interesting about the summary is that they treat sensor- or modality-specific effects as augmentations during training, which helps build those robust, invariant representations through the counterfactual generation objective. It’s a recipe for self-supervised pretraining in science.

Subrahmanyan: From a theory standpoint, it moves away from relying on hand-engineered loss functions and instead enforces structural disentanglement through this specific architecture and the flow matching mechanism, which connects it to ideas about recovering independent causal mechanisms three <ref:2604.09787#pg1>.

The paper's summary: Vera: Looking at the improvements they suggest in "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," they focus on the fact that this approach is architecture-driven rather than relying on pre-defined loss functions. They’re enforcing structural disentanglement through their specific design.

Jocelyn: The improvements highlight how they use multi-view observations—seeing the same underlying content under different conditions or at different times—as a natural source of structure for disentanglement, which is something we see everywhere in real observation data.

Subrahmanyan: I think the most concrete improvement is the asymmetry in property prediction they found. The physics latent space substantially outperforms instrument latents when predicting physical properties, while the pattern reverses when it comes to instrumental properties. This asymmetry proves that their architecture actually succeeds in separating those factors efficiently.

Vera: That’s what they mean by achieving a more efficient separation of physics and instrumental properties compared to monolithic foundation models. The physics latent space recovers physical properties at a level comparable to AION-one which is a strong benchmark for this kind of feature recovery <ref:2604.09787#pg1>.

Jocelyn: And they also show that the instrument latent space does a better job at predicting those instrument-related properties than baseline models, which is useful for understanding the systematics themselves.

Subrahmanyan: If we think about the implications, this means we can trust inferred galaxy properties more across different surveys because our method is explicitly designed to be robust to those instrumental variations, even if those variations are complex.

The paper's improvements: Vera: So, wrapping up "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," the main implication is that we can now build a framework for scientific self-supervised pretraining. It’s about constructing training pairs from overlapping observations of the same physical system.

Jocelyn: This means we treat sensor or modality effects as augmentations when training and learn invariant representations through counterfactual generation, which lets us do things like instrument-independent similarity search without being confounded by artifacts.

Subrahmanyan: For cosmology, this suggests that we can more reliably use these latent spaces to streamline the discovery of rare astrophysical phenomena because the physics latent space effectively isolates and groups complex physical features.

Vera: They also confirm that their generative decoder acts like a learned survey simulator, performing instrument-aware super-resolution and denoising. This is useful for reducing search space for follow-up observations by generating informed predictions rather than just noisy pixels.

Jocelyn: It’s a general recipe here: construct training pairs from overlapping data, treat system effects as augmentations, and learn invariant representations this way. It’s applicable beyond just galaxies, to other areas of astrophysics and even tens of millions of light curves from missions like TESS or Kepler.

Subrahmanyan: I just want to say that the physics latent space recovering properties at a level comparable to AION-one is a solid piece of evidence for how effective this architecture is at isolating intrinsic features <ref:2604.09787#pg1>. It shows the method has real power for uncovering those complex structures we’re looking for in the sky.

Vera: Exactly, and it confirms that by explicitly disentangling the factors, we get a more efficient separation than monolithic foundation models. We have to keep an eye on how they extend this to settings where you don't have such perfect overlap yet.

Conclusion: Vera: So we've been looking at "Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics," and essentially what they did is build a dual-encoder system that separates the actual physical signal from all the noise caused by different instruments.

Jocelyn: Right, so it uses overlapping observations—like combining data from different surveys—to create these training sets where you have the same physical thing seen through different lenses.

Subrahmanyan: The core idea is to force two encoders to learn separate things: one encoder learns the physics, and the other learns how each specific instrument messes up the data.

Vera: Exactly. The results show that if you look at those latent spaces, you get a physics space that recovers real physical properties pretty well, even matching models like AION-one.

Jocelyn: And the instrument space is surprisingly good at capturing those measurement distortions themselves, which is useful for understanding the noise conditions of different instruments.

Subrahmanyan: From a cosmic view, this means we can now trust inferred galaxy properties more across different surveys because they aren't just affected by the specific sensor they were taken with.

Vera: It also gives us tools like counterfactual generation, which lets you predict what an observation would look like under different conditions or even generate a high-resolution image from a low-resolution one.

Jocelyn: That means we can use it to simulate observations or even do some kind of instrument-independent similarity search, which is pretty neat for finding similar sources across different datasets.

Subrahmanyan: The limitation they mention is that this method currently needs those overlapping observations across instruments, so you're mostly limited to regions where the sky coverage from those surveys overlaps.

Vera: It’s a powerful tool for self-supervised pretraining because it gives us a general recipe: take overlapping data, treat the sensor effects as augmentations during training, and learn these invariant representations.

Jocelyn: It’s a solid framework for moving forward, especially since they show how this architecture can be applied to tens of millions of light curves from missions like TESS or Kepler.

Subrahmanyan: That's where the real potential lies, applying it to those massive datasets to find patterns in structure formation that we just can't see otherwise.

Vera: So, for anyone listening, this paper is about building a way to clean up messy observational data by learning what's physically real versus what the equipment is adding on top of it.

Jocelyn: It shows how combining different instruments can actually help us understand the underlying physics better than using just one telescope alone.

Subrahmanyan: We'll keep an eye on how they extend this to unpaired settings, because that's where the real challenge in observational science often lies.

Massachusetts Institute of Technology · Flatiron Institute, Simons Foundation · AstroAI, CfA | Harvard & Smithsonian

astro-ph.IM, astro-ph.GA, cs.LG

Submitted: 2026-04-10

Updated: 2026-11-07

Comments: Accepted to the 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

Code: https://github.com/pablomerc/galaxy-counter

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from…

Key concepts

Dual-Encoder Architecture
This architecture uses two specialized neural networks: a physics encoder and an instrument encoder. The physics encoder learns the intrinsic properties of the source while ignoring instrumental noise, and the instrument encoder learns the specific distortions introduced by each sensor.
Counterfactual Generation Objective
This training objective leverages overlapping observations to create 'training triplets' (anchor, different instrument observation, different source observation). This forces the model to learn representations that are invariant to measurement artifacts by generating synthetic views that differ only in the instrumental or source context.
Physics Latent Space
This latent space is designed specifically to capture underlying physical properties of astronomical objects, such as galaxy morphology. It is trained to be robust against sensor-specific distortions, meaning it recovers accurate physical characteristics even when data comes from different instruments.

Terminology

Summary

Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. p. 0

Where the paper says it

Freeman Massachusetts Institute of Technology billf@mit. edu Abstract Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument.

This secondary signal acts as a confounding factor, limiting our ability to extract information about the underlying physics. p. 0
Where the paper says it

edu Abstract Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the underlying physics.

To disentangle these factors of variation, we propose a dual-encoder architecture with a counterfactual generation objective that leverages overlapping observations. p. 0
Where the paper says it

Moreover, it poses significant challenges for combining data in heterogeneous or multi-instrument frameworks. To disentangle these factors of variation, we propose a dual-encoder architecture with a counterfactual generation objective that leverages overlapping observations.

The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference, and instrumentindependent similarity search—all unconfounded by measurement artifacts. p. 0
Where the paper says it

To disentangle these factors of variation, we propose a dual-encoder architecture with a counterfactual generation objective that leverages overlapping observations. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference, and instrumentindependent similarity search—all unconfounded by measurement artifacts.

We demonstrate the effectiveness of our approach in a multi-instrument setting on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey. p. 0
Where the paper says it

The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference, and instrumentindependent similarity search—all unconfounded by measurement artifacts. We demonstrate the effectiveness of our approach in a multi-instrument setting on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey.

This framework provides a general recipe for scientific self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation. p. 0
Where the paper says it

We demonstrate the effectiveness of our approach in a multi-instrument setting on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey. This framework provides a general recipe for scientific self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation.

The Core Methodology

The framework introduces a dual-encoder architecture with a counterfactual generation objective designed to learn representations that disentangle physical properties from measurement systematics. This objective leverages large-scale scientific datasets with overlapping observations from multiple instruments to construct training triplets: an anchor, an observation of the same source (but different instrument), and an observation on the same instrument (but different source). p. 2

Where the paper says it

Our framework leverages large-scale scientific datasets with overlapping observations from multiple instruments. In this setting, natural augmentations can be used to construct triplets: an anchor, an observation of the same source (but different instrument), and an observation on the same instrument (but different source), as shown in Fig.

A physics encoder receives a sample from the target source but from a different instrument, so it is encouraged to reduce the loss by capturing the underlying physical properties of the source and becoming invariant to instrumental distortions. p. 2
Where the paper says it

In this setting, natural augmentations can be used to construct triplets: an anchor, an observation of the same source (but different instrument), and an observation on the same instrument (but different source), as shown in Fig. A physics encoder receives a sample from the target source but from a different instrument, so it is encouraged to reduce the loss by capturing the underlying physical properties of the source and becoming invariant to the instrumental distortions.

Similarly, an instrument encoder receives a sample from the target instrument but a different source, so it is encouraged to capture instrument-specific distortions while ignoring the content specific to the source. p. 2
Where the paper says it

A physics encoder receives a sample from the target source but from a different instrument, so it is encouraged to reduce the loss by capturing the underlying physical properties of the source and becoming invariant to the instrumental distortions. Similarly, an instrument encoder receives a sample from the target instrument but a different source, so it is encouraged to capture instrument-specific distortions while ignoring the content specific to the source.

The decoder then combines these two latent embeddings to reconstruct an unseen target or anchor observation. p. 2
Where the paper says it

Similarly, an instrument encoder receives a sample from the target instrument but a different source, so it is encouraged to capture instrument-specific distortions while ignoring the content specific to the source. The decoder combines these two latent embeddings to reconstruct an unseen target or anchor observation.

Disentanglement and Representation Learning

The model is trained using a conditional flow matching objective, learning a velocity field that transports samples from a Gaussian base distribution to the target distribution. The training loss is defined as LFM(θ) = E t∼U[0,1] j∼[N],k∼[M] xj,k∼pdata x0∼N(0,I) h uθ xt, t, zphy(xj,k′)k′̸=k zins(xj′,k)j′̸=j − (xj,k − x0) 2 i (1). p. 3

Where the paper says it

We model this conditional distribution using flow matching [14], learning a velocity field uθ(x, t, zphy, zins) that transports samples from Gaussian base distribution to the target distribution. The training loss is given by the error on the predicted velocity field, LFM(θ) = E t∼U[0,1] j∼[N],k∼[M] xj,k∼pdata x0∼N(0,I) h uθ xt, t, {zphy(xj,k′ )}k′̸=k, {zins(xj ′ ,k)}j ′̸=j − (xj,k − x0) 2 i (1) where xt = (1−t)x0 +txj,k.

The physics embedding supports downstream tasks such as outlier detection, parameter inference, classification, or embedding-based retrieval. p. 3
Where the paper says it

That is, we sample a source j, a target instrument k, a noise sample x0, and a timestep t, and train the velocity field to reconstruct the observation xj,k from a noisy state xt conditioned on physics latents from other instruments and instrument latents from different sources. Once trained, the physics embedding supports downstream tasks such as outlier detection, parameter inference, classification, or embedding-based retrieval, while the instrument space allows to find similar systematics and observing conditions.

Furthermore, the instrument space allows to find similar systematics and observing conditions. p. 3
Where the paper says it

That is, we sample a source j, a target instrument k, a noise sample x0, and a timestep t, and train the velocity field to reconstruct the observation xj,k from a noisy state xt conditioned on physics latents from other instruments and instrument latents from different sources. Once trained, the physics embedding supports downstream tasks such as outlier detection, parameter inference, classification, or embedding-based retrieval, while the instrument space allows to find similar systematics and observing conditions.

Application in Astrophysics

The model is demonstrated on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey. The preprocessing pipeline aligns overlapping filters across both surveys into a unified four-channel representation and treats both sources as the same modality. p. 3

Where the paper says it

Despite the shared designations, they differ significantly in their central wavelengths, pixel scales, zero-points, and noise characteristics. Our preprocessing pipeline follows the strategy established by [8], with one key difference: rather than treating all nine bands as distinct channels, we align the overlapping (g, r, i, z) filters across both surveys into a unified four-channel representation and treat both sources as the same modality.

The dual-survey setting involves training on ∼100,000 cross-matched galaxy images from Legacy and HSC as a representative multi-instrument setting. p. 3
Where the paper says it

Here, we show that counterfactual reconstruction improves robustness to instrument systematics while enabling new foundational capabilities, including crossinstrument generation, instrument-independent retrieval, and data-driven denoising. We take a cross-matched dataset from the Multimodal Universe that includes ∼100,000 cross-matched galaxy images from the DESI Legacy Imaging Surveys [Legacy, 15] and the Hyper Suprime-Cam [HSC, 16] survey as a representative multi-instrument setting.

Performance Metrics and Capabilities

The effectiveness is evaluated by probing the latent spaces with a three-layer MLP to predict physical and instrumental properties from embeddings extracted by each encoder. The physics latent space recovers physical properties at a level comparable to AION-1, while the instrument latent space dominates on instrument-related properties. p. 5

Where the paper says it

Details on the properties and source catalogs are provided in Appendix A. The physics latent space recovers physical properties at a level comparable to AION-1.

Additionally, the model enables instrument-independent similarity search in the physics space and counterfactual cross-instrument generation for predicting high-resolution observations from low-resolution ones. p. 3
Where the paper says it

The generative decoder enables generating counterfactual views. In astrophysics, a particularly promising use case is to inform follow-up observations by predicting high-resolution observations from expensive facilities using low resolution observations from a different instrument.

The generative decoder can also act as a data-driven noise model, simulating observations under different conditions while holding the underlying physics fixed. p. 8
Where the paper says it

Data-driven noise model. The instrument encoder and generative decoder together yield a datadriven noise model: by manipulating the instrument conditioning at inference time, we can simulate observations under different conditions while holding the underlying physics fixed.

Limitations and Future Work

Our approach currently requires overlapping observations across instruments, which limits applicability to regions of shared sky coverage. Future work plans include extending the framework to unpaired settings where such overlap is unavailable or limited. p. 9

Where the paper says it

Our approach currently requires overlapping observations across instruments, which limits applicability to regions of shared sky coverage. In future work, we aim to extend the framework to unpaired settings where such overlap is unavailable or limited.

The framework is general and applicable to other domains in astrophysics and beyond, with plans to apply it to tens of millions of light curves from NASA’s Transiting Exoplanet Survey Satellite (TESS) and Kepler missions. p. 9
Where the paper says it

Moreover, while we have focused on galaxy imaging, the framework is general and applicable to other domains in astrophysics and beyond. In particular, we plan to apply it to the tens of millions of light curves from NASA’s Transiting Exoplanet Survey Satellite [TESS, 22] and Kepler [23, 24] missions.

Pruning redundant reasoning tokens

The key insight is that cross-matched observations form natural pairs of same physics, different instrument, providing a training signal for separating the two sets of factors without requiring an explicit forward model of either instrument. This formulation allows the model to leverage all available multi-instrument data: the physics of the target source is informed by its appearances across different instruments, while the instrument response is informed by how other sources appear under similar observing conditions. p. 3

Where the paper says it

The set {zins(xj ′ ,k)}j ′̸=j are the instrument embeddings extracted from observations of other sources j ′ by the same instrument k. This formulation allows the model to leverage all available multi-instrument data: the physics of the target source is informed by its appearances across different instruments, while the instrument response is informed by how other sources appear under similar observing conditions.

How the training signal is built

Training triplets are constructed from an anchor observation (signal s from instrument i), an instrument-augmented observation (same source s, a different instrument i′), and a physics-augmented observation (a different source s′, same instrument i). The physics encoder extracts latents from the instrument-augmented pair to capture invariant physical features, while the instrument encoder processes the physics-augmented observation to isolate measurement artifacts. p. 1

Where the paper says it

Training uses data triplets: an anchor observation (signal s from instrument i), an instrument-augmented observation (same source s, a different instrument i ′ ), and a physics-augmented observation (a different source s ′ , same instrument i). The physics encoder extracts latents from the instrument-augmented pair to capture invariant physical features, while the instrument encoder processes the physics-augmented observation to isolate measurement artifacts.

Where the gains come from

The asymmetry in property prediction shows that our architecture successfully separates physics properties from instrumentspecific information. The physics latents substantially outperform instrument latents at predicting (mostly) physical properties, whereas the pattern reverses for instrument properties. p. 6

Where the paper says it

The full results are shown in Appendix A. The key diagnostic is the asymmetry in the split of properties: the physics latents substantially outperform instrument latents at predicting (mostly) physical properties, whereas the pattern reverses for instrument properties.

This highlights the primary advantage of our architecture: by explicitly disentangling the latent factors, we achieve a more efficient separation of physics and instrumental properties than monolithic foundation models. p. 6
Where the paper says it

On the other hand, we note that AION-1 embeddings retain sensitivity to instrumental properties, but our instrument latent space outperforms both baselines in predicting instrument-related properties. This highlights the primary advantage of our architecture: by explicitly disentangling the latent factors, we achieve a more efficient separation of physics and instrumental properties than monolithic foundation models.

The instrument latent space outperforms both baselines in predicting instrument-related properties. p. 6
Where the paper says it

predicting physics-related properties from the instrument latents), indicating that the model has actively learned to erase instrument-specific information from the physics space and vice versa. On the other hand, we note that AION-1 embeddings retain sensitivity to instrumental properties, but our instrument latent space outperforms both baselines in predicting instrument-related properties.

The training objective is designed to learn representations that disentangle physical properties from measurement systematics. The generative component can also be viewed as a data-driven instrument model, learned implicitly from crossmatched observations. p. 9

Where the paper says it

Compared to previous contrastive approaches, our method enables counterfactual generation by synthesizing how a source would appear under different observing conditions, and it is designed to scale to settings with more than two instruments. The generative component can also be viewed as a data-driven instrument model, learned implicitly from crossmatched observations.

The model successfully removes severe instrumental distortions while preserving the underlying galaxy morphology. p. 8
Where the paper says it

The model successfully generates progressively cleaner images; the average SNR shown is that of the images used to condition the reconstruction. Bottom (Systematic Artifact Removal): By conditioning the physics encoder on an artifact-corrupted HSC image and the instrument encoder on a random set of HSC images, the model successfully removes severe instrumental distortions while preserving the underlying galaxy morphology.

The framework is general and applicable to other domains in astrophysics and beyond. p. 9

Where the paper says it

In future work, we aim to extend the framework to unpaired settings where such overlap is unavailable or limited. Moreover, while we have focused on galaxy imaging, the framework is general and applicable to other domains in astrophysics and beyond.

This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena. p. 7
Where the paper says it

multiple HSC images that exhibit strong morphological evidence to be considered candidate lenses. This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena.

p. 7
Where the paper says it

multiple HSC images that exhibit strong morphological evidence to be considered candidate lenses. This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena.

p. 7
Where the paper says it

multiple HSC images that exhibit strong morphological evidence to be considered candidate lenses. This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena.

The model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising. p. 7
Where the paper says it

HSC’s higher-resolution allows to resolve features, such as faint arcs, that are blurred or undetected in Legacy imaging. Our model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising.

p. 7
Where the paper says it

HSC’s higher-resolution allows to resolve features, such as faint arcs, that are blurred or undetected in Legacy imaging. Our model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising.

p. 7
Where the paper says it

HSC’s higher-resolution allows to resolve features, such as faint arcs, that are blurred or undetected in Legacy imaging. Our model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising.

The model successfully captures realistic morphological structure and noise properties rather than pixel-level artifacts or overly smoothed outputs. The posterior uncertainty bands appropriately expand at higher spatial frequencies, where noise dominates. p. 7

Where the paper says it

Column 3 (Ground Truth): The original target image. Columns 4–5 (Posterior Samples): Independent samples generated by the decoder, demonstrating the model’s ability to capture noise and uncertainty.

The standard deviations slightly above unity indicate mild overconfidence (the model underestimates the true pixel-wise variance by ∼15%) and the effect is more pronounced for HSC-anchored reconstructions. p. 8
Where the paper says it

First, in a controlled SNR traversal, we reconstruct an HSC target galaxy from its Legacy counterpart five times, fixing the initial noise sample x0 but varying the instrument conditioning across the 5th, 25th, 50th, 75th, and 95th percentiles of the HSC SNR distribution (defined as the mean pixel-wise flux divided by the flux error, averaged across channels). Traversing this axis produces progressively cleaner reconstructions, even surpassing the apparent SNR of the ground truth—indicating an exploitable, manipulable noise model rather than memorization.

The framework is general and applicable to other domains in astrophysics and beyond. This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena. The model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising. The generated images are best understood as informed predictions that reduce the search space for follow-up, rather than replacements for real, deeper observations. p. 8

Where the paper says it

Features well below the Legacy detection threshold will not appear in the counterfactual. The generated images are best understood as informed predictions that reduce the search space for follow-up, rather than replacements for real, deeper observations.

p. 8
Where the paper says it

Features well below the Legacy detection threshold will not appear in the counterfactual. The generated images are best understood as informed predictions that reduce the search space for follow-up, rather than replacements for real, deeper observations.

The physics latent space recovers physical properties at a level comparable to AION-1. The instrument latent space dominates on instrument-related properties while the physics latents collapse near the random-baseline level on those same properties. p. 6

Where the paper says it

Notably, our latent spaces often contain less information than the baselines when used for cross-prediction (e. predicting physics-related properties from the instrument latents), indicating that the model has actively learned to erase instrument-specific information from the physics space and vice versa.

The cross-predict baseline outperforms our instrument latents only on extinction-like quantities, where information about the other survey’s footprint is sufficient to recover the target. p. 6
Where the paper says it

This cross-prediction baseline measures the shared spatial component: any predictive power it achieves reflects information that is inherently common to both surveys and therefore accessible to either encoder by construction. We find that predictions from physics latents are always worse than the cross-predict baseline, verifying that the physics latent space is not capturing private instrument information.

The model successfully removes severe instrumental distortions while preserving the underlying galaxy morphology. The framework is general and applicable to other domains in astrophysics and beyond. This confirms that the physics latent space effectively isolates and groups complex physical features, streamlining the discovery of rare astrophysical phenomena. The model can generate the expected HSC-like appearance for any galaxy in the Legacy footprint, effectively acting as a learned survey simulator that performs instrument-aware super-resolution and denoising. The generated images are best understood as informed predictions that reduce the search space for follow-up, rather than replacements for real, deeper observations.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Disentangled Representation Learning enables robust inference by separating intrinsic physical properties from sensor-specific distortions through a dual-encoder architecture. This allows for outlier detection unconfounded by instrument systematics, meaning models can identify physically unusual sources without being biased by instrumental artifacts like saturated pixels or noise patterns.

  2. Counterfactual Generation allows for instrument independent similarity search and novel data generation. The framework enables the model to generate counterfactual views, which can act as a learned survey simulator that performs instrument aware super-resolution and denoising, allowing researchers to predict high resolution observations from low resolution ones for follow up decisions.

  3. Parameter Inference Robustness allows for reliable property estimation despite instrumental variations. The physics latent space recovers physical properties at a level comparable to existing foundation models while being robust to instrumental systematics, meaning the resulting inferred galaxy properties are more trustworthy across different instruments than those from monolithic models.

Abstract

Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the physics underlying the phenomena we observe. Furthermore, it complicates the combination of observations in heterogeneous or multi-instrument settings. We propose a deep learning framework that leverages overlapping observations, a dual-encoder architecture, and a counterfactual generation objective to disentangle these factors of variation. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference unconfounded by measurement distortions, and instrument-independent similarity search. We demonstrate the effectiveness of our approach on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey as a representative multi-instrument setting. This framework provides a general recipe for scientific and multi-modal self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation.

Sources

Related papers