A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology

summary

The gist

A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology demonstrates that a convolutional autoencoder can predict instrument state from simulated…

In short

A self-supervised convolutional autoencoder was developed to predict instrumental systematics in 21cm cosmology data, such as spectral ripples, directly from simulated visibility data without needing explicit models. The model successfully separated information into instrument-only and sky-only latent vectors, achieving high accuracy ($R^2=0.9975$) in predicting the unknown instrument state. This demonstrates a promising machine learning pipeline for cleaning radio interferometric signals.

Key concepts

Baryon Acoustic Oscillations (BAO)
These are primordial sound waves imprinted on the large-scale structure of galaxies. They act as a 'standard ruler' in cosmology, allowing scientists to measure the expansion rate of the universe and constrain dark energy by observing their characteristic scale in galaxy distributions.
Convolutional Autoencoder
This is a type of neural network designed to learn efficient representations (embeddings) of complex data. In this work, it compresses simulated radio visibilities into two separate latent vectors: one representing instrument effects and one representing the sky signal, without needing prior knowledge of what those effects look like.
Latent Vectors ($z_{ ext{inst}}, z_{ ext{sky}}$)
These are compressed mathematical representations of the input data. The goal is to separate them so that $z_{ ext{inst}}$ captures only the instrument's systematic noise, and $z_{ ext{sky}}$ captures only the cosmological sky signal. The success hinges on this separation, as the instrumental latent vector showed no detectable sky signal.
Drift-Scan Observation
This is a specific observational technique where an interferometer observes a fixed patch of the sky while slowly moving or 'drifting' across it. This setup is crucial because it allows the model to exploit the fact that the instrument state remains constant while observing different parts of the sky, enabling the separation of instrumental and sky information.

Terminology used across episodes

This episode discusses

The paper

A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology · Read on arXiv

University of Geneva · Center for Astroparticle Physics

The primordial sound waves in the baryon-photon plasma are imprinted on the large-scale galaxy distribution as Baryon Acoustic Oscillations (BAO), serving as a cosmological standard ruler to constrain dark energy and the Hubble parameter. The upcoming 21cm intensity-mapping arrays are designed to map the 21cm emission across large volumes and measure the BAO scale. However, radio interferometers suffer from instrumental systematics that can hinder detection of the cosmological 21cm signal by leaking bright foregrounds into the signal-dominated modes. This work aims to demonstrate that a self-supervised model can predict the instrument systematic state (here, the amplitude of a spectral ripple in the beam width) in interferometric visibilities and potentially disentangle the instrument from the sky without a forward model. Based on the fact that an instrument stays the same as the sky rotates in a drift-scan observation, and that the same sky is observed under different instrument states, we train a convolutional autoencoder to compress the information in the simulated visibilities and separate this information into instrument-only and sky-only latent vectors. The result shows excellent prediction of the unknown instrument systematic for 15 held-out instrument states with R squared = 0.9975 with a linear probe, against R squared = -3.55 for a random-encoder baseline. The instrument-latent carries no detectable sky signal, as measured with a total-power sky proxy, for any held-out test data. The sky-latent vector and instrument-latent vector show partial separation but not complete disentanglement, identifying sky-latent purity as the primary open problem. The results show the ability of the model to recognize the systematic and show a promising path towards an ML-based foreground cleaning pipeline. Full instrument-sky disentanglement constitutes the natural next step.

Transcript

Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.

Vera: We're looking at the paper: "A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology".

Jocelyn: Here's the gist of the paper. A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology demonstrates that a convolutional autoencoder can predict instrument state from simulated visibilities without needing explicit instrument or sky models, showing a promising path toward ML-based foreground cleaning.

Vera: Pruning redundant reasoning tokens The core aim of this work is to demonstrate that a self-supervised model can predict the instrument systematic state in interferometric visibilities and potentially disentangle the instrument from the sky without a forward model.

Jocelyn: The method trains a convolutional autoencoder to compress simulated visibilities into instrument-only and sky-only latent vectors based on the fact that an instrument remains constant while the sky rotates in a drift-scan observation.

Vera: The model is designed such that the instrumental latent carries no detectable sky signal when measured with a total-power sky proxy for any held-out test data.

Paper summary: Vera: So, Jocelyn, let's talk about this paper, "A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology <ref:2610.37596#pg1,A self-supervised machine learning approach to identify instrumental systematics in 21cm>." Basically, the whole point is that we have these upcoming 21cm intensity-mapping arrays designed to map the large-scale galaxy distribution using Baryon Acoustic Oscillations <ref:2610.37596#pg1,upcoming 21cm intensity-mapping arrays>. But radio interferometers have these instrumental problems where bright foregrounds leak into the signal we want, which messes up our measurements.

Jocelyn: Right, Vera, and what this work does is try to show that a self-supervised machine learning model can predict the instrument systematic state—specifically they are talking about this amplitude of a spectral ripple in the beam width—using just those simulated visibilities. And they claim it can potentially separate the instrument from the sky without needing any kind of forward model for how things work.

Subrahmanyan: That's interesting because we're dealing with these kinds of systematics, like direction-dependent and direction-independent effects, that can be hard to pull out during calibration. If this model works, it means we might be able to clean the data in a way that doesn't rely on building a super detailed physical model of every single instrumental glitch.

Vera: Exactly. The paper is focusing on how these drift-scan observations work, where the telescope stays fixed while the sky rotates and that makes things time-dependent, which is where those instrumental effects show up in the visibilities. They train a convolutional autoencoder on simulated visibilities to compress them into two different latent vectors: one for the instrument and one for the sky.

Jocelyn: And they enforce these constraints using three different loss components: a reconstruction loss, an instrument consistency loss, and a sky consistency loss. The instrument consistency loss specifically checks that the instrumental latent vector stays the same even when you look at two different subsets of Right Ascension data from the same scan, which enforces RA-invariance for that instrumental component.

Subrahmanyan: That idea of enforcing invariance constraints through those losses is key because it’s how you try to force the network to learn what's truly instrumental versus what's just part of the sky signal. But I wonder how robust this separation actually is when you move from simulation to real data, especially given all the noise we deal with in actual 21cm surveys <ref:2610.37596#pg1>.

Paper summary: Vera: That’s a fair question, Subrahmanyan. The results they show are quite strong for testing this idea on held-out data. They found excellent prediction of the unknown instrument systematic state when they tested it on fifteen different held-out instrument states, achieving an R squared of zero point nine nine seven five with a linear probe to recover the true state A.

Jocelyn: And that’s really compelling because their baseline for comparison is much worse; they saw an R squared of negative three point five five for a random encoder baseline. That difference shows the model actually learned something meaningful about the systematic structure, not just memorized the input data.

Subrahmanyan: An R squared of zero point nine nine seven five is certainly high, especially when you compare it to that random encoder baseline which was failing significantly; it suggests the structure they are learning in that instrument latent vector is very strongly correlated with the actual instrumental state they are trying to predict. However, we still have to worry about those residual leaks into the sky component.

Vera: They did look at that leakage too. They measured this by looking at a total-power sky proxy for any held-out test data, and their result is that the instrument latent carries no detectable sky signal whatsoever when measured with that specific proxy. That’s a very clean result for the instrument side.

Jocelyn: So they managed to isolate the instrumental part almost perfectly based on their testing method, which is pretty impressive given how messy those interferometric measurements can be. But they still have this question hanging over them about whether the sky latent vector and the instrument latent vector are fully separate or if there's still some mixing.

Subrahmanyan: That partial separation is exactly where the next challenge lies for me in terms of theory. If there’s a persistent leak, it means that even after we use this ML approach, we might not have a perfectly clean sky signal yet before we can trust it for cosmology. They say identifying sky latent purity is their main open problem right now.

Paper summary: Vera: So the ability to recognize the systematic is clear, and they’ve shown a promising path toward building an ML-based foreground cleaning pipeline for 21cm cosmology <ref:2610.37596#pg1>. It shows that machine learning can be a useful tool here, even without having to write out the exact forward model of every single instrumental effect.

Jocelyn: The implication is that we could start developing pipelines that use this kind of self-supervised learning to clean up the data before we even get to the cosmological signal extraction stage. It’s a step toward making those upcoming 21cm arrays more reliable for measuring dark energy and the Hubble parameter <ref:2610.37596#pg1,dark energy and the Hubble parameter>.

Subrahmanyan: If they can eventually achieve full instrument-sky disentanglement, that would be a big win for structure formation studies because it means we get to measure the underlying physics without being constantly hampered by instrumental artifacts that look like cosmological signals.

Vera: So, to wrap up this overview of "A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology," this paper demonstrates a way to use a convolutional autoencoder to predict instrument state from simulated visibilities by learning separate latent vectors for the instrument and the sky <ref:2610.37596#pg1,A self-supervised machine learning approach to identify instrumental systematics in 21cm>. The success is shown by an R squared of zero point nine nine seven five when testing against fifteen held-out states, and they found that the instrument latent carries no detectable sky signal with their specific proxy measurement.

Jocelyn: It really points toward a future where we can use ML to clean up these tricky instrumental effects in radio data without needing an explicit forward model for every single glitch. The next step, as they’ve identified, is figuring out how much instrumental signal still leaks into that sky latent vector before we can fully trust the sky component.

Subrahmanyan: That leakage issue is what keeps me thinking about the precision needed for our future measurements; if that leak isn't completely gone, it dictates how clean our cosmological constraints on dark energy will be. It’s a promising start to separating the noise from the signal in these ambitious 21cm experiments <ref:2610.37596#pg1>.

Conclusion: Vera: So, this paper is about using machine learning to figure out what's messing up the data from those 21cm experiments <ref:2610.37596#pg1>.

Jocelyn: Yeah, they take simulated visibility data and train an autoencoder to separate the signal from the instrument noise without needing a fancy physical model for every glitch.

Vera: The authors are tackling this problem of instrumental systematics in radio astronomy. They’ve shown that you can predict those specific instrumental distortions—like a ripple in the beam width—just by looking at how the data is structured across different observation states.

Jocelyn: It’s about using self-supervised learning to create two separate latent vectors, one for the instrument and one for the sky, based on how an instrument stays constant while you watch the sky rotate during a drift scan.

Vera: The key result is that this instrumental latent vector actually carries no detectable signal when you test it against real test data using a total power proxy. That’s strong evidence that they learned something meaningful about the instrument itself.

Jocelyn: But there's still this lingering question they bring up, which is how much of the sky signal might still be mixed into that sky latent vector before we can trust it for cosmology.

Vera: The title, "A self-supervised machine learning approach to identify instrumental systematics in 21cm Cosmology," really sums up the ambition here <ref:2610.37596#pg1,A self-supervised machine learning approach to identify instrumental systematics in 21cm>. It’s about using this ML technique to clean up those tricky foregrounds and get a clearer view of the cosmological signal we want.

Jocelyn: And it opens up a path toward building these automated foreground cleaning pipelines for future 21cm intensity mapping arrays <ref:2610.37596#pg1>.

Vera: So, the main implication is that machine learning can be a powerful tool to tackle the messy instrumental side of radio cosmology, even before you get to the physics of dark energy and expansion.

Jocelyn: We have a lot more to unpack about how this separation works and what it means for those actual cosmological measurements next.

More episodes

← Home