MAHA-z: A Mahalanobis distance kNN estimator for photometric redshifts

summary

The gist

MAHA-z presents a photometric-redshift (photo-z) estimator that combines k-nearest neighbour (kNN) regression with a Mahalanobis distance metric learned from the covariance of a spectroscopic…

In short

MAHA-z is a photometric redshift (photo-z) estimator that uses k-nearest neighbour regression combined with a Mahalanobis distance metric learned from spectroscopic data. This method combines features like magnitudes and colours to estimate galaxy redshifts, providing both a point estimate and a full probability distribution for each galaxy. It is practical because it handles correlated features and missing data well, offering competitive accuracy at low computational cost.

Key concepts

k-nearest neighbour (kNN) regression
This technique finds the 'k' closest galaxies in a feature space defined by magnitudes, colours, and optionally morphology. The mean of these neighbors provides the primary redshift estimate for a target galaxy. The spread among these neighbors is used to determine the uncertainty of that estimate.
Mahalanobis distance
This metric generalizes standard distance by accounting for correlations between different features (like different filters) and their varying units. It learns a covariance matrix from spectroscopic data, allowing it to measure distances in a way that respects the actual structure of the galaxy population, which is crucial when dealing with heterogeneous photometric data.
Feature Space
This is a multi-dimensional space where each galaxy is represented by its measurements—magnitudes ($m$), colours ($c$), and optionally morphological parameters ($ ilde{ u}$). The Mahalanobis metric operates within this space, using the learned covariance to understand how these different measurements relate to each other in terms of physical structure.

Terminology used across episodes

This episode discusses

The paper

MAHA-z: A Mahalanobis distance kNN estimator for photometric redshifts · Read on arXiv

Paula S. Ferreira, Ribamar R. R. Reis, Renyue Cen

Center for Cosmology and Computational Astrophysics, Institute for Advanced Study in Physics, Zhejiang University · Institute of Astronomy, School of Physics, Zhejiang University · Instituto de Física, Universidade Federal do Rio de Janeiro

We present MAHA-z, a photometric-redshift (photo- z) estimator that combines k-nearest neighbour (k NN) regression with a Mahalanobis distance metric learned from the covariance of a spectroscopic reference sample. For each target galaxy, MAHA-z performs a neighbour search in a colour-magnitude feature space with optional morphological parameters, and returns both a point estimate and a redshift posterior p(z f) obtained by kernel density estimation over the neighbour redshifts. The Mahalanobis metric accounts for correlated features and heterogeneous units, eliminating ad hoc normalisation, and its subspace restriction allows MAHA-z to handle missing bands on a per-galaxy basis without imputation. We validate the method on four datasets spanning simulations and survey photometry: the Euclid Flagship 2 mock, the Legacy-KiDZ calibration sample, VIPERS, and the DES Year 3 BAO sample. On Euclid Flagship 2 (LSST+NISP) we achieve σ NMAD=0.0148 with a 0.27% outlier fraction, and on DES Y3 we obtain σ NMAD about0.023 with <1% outliers and posterior calibration comparable to the ANNz2 catalogue. MAHA-z processes millions of galaxies in minutes and provides competitive PDFs at substantially lower computational cost than trained machine-learning models, making it a practical option for iterative photo- z production and validation in Stage-IV surveys.

Transcript

Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.

Vera: I'm Vera.

Jocelyn: And I'm Jocelyn, here with Subrahmanyan.

Vera: We're looking at the paper: "MAHA-z: A Mahalanobis distance kNN estimator for photometric redshifts".

Jocelyn: Here's the gist of the paper. MAHA-z presents a photometric-redshift (photo-z) estimator that combines k-nearest neighbour (kNN) regression with a Mahalanobis distance metric learned from the covariance of a spectroscopic reference sample, making it a practical option for iterative photo-z production and validation in Stage-IV surveys<ref:2611.14582#pg2>.

Vera: MAHA-z assigns a photometric redshift to each target galaxy by combining three ingredients: (i) a feature vector built from magnitudes, colours, and optionally morphological parameters; (ii) a Mahalanobis metric whose covariance encodes the structure of the galaxy population; and (iii) a k-nearest neighbour regression whose mean gives 1 point estimate and whose spread gives the per-galaxy uncertainty<ref:2611.14582#pg4>.

Jocelyn: The Mahalanobis distance generalizes the Euclidean distance to a space whose coordinates are correlated and carry different physical units, which is invariant under invertible linear transformations of the coordinates, essential when dealing with magnitudes and colours measured in different bands with very different variances<ref:2611.14582#pg4>.

Vera: Feature Space and Metric Construction The feature vector for each galaxy is defined as f = m1,..., mNb, c1,..., cNc, µ1,..., (3)<ref:2611.14582#pg4>.

Paper summary: Vera: So wrapping up on this paper, MAHA-z is a photometric redshift estimator that uses k nearest neighbour regression combined with a Mahalanobis distance metric learned from the covariance of a spectroscopic training sample.

Jocelyn: The main implication for me is that it gives us a tool that respects the geometry of the galaxy population when we are dealing with different photometric bands and different units.

Subrahmanyan: From my side, I see this as a way to get redshift estimates faster and with better statistical rigor without needing the heavy computational burden of more complex neural network codes.

Vera: It’s a transparent, training light alternative to template fitting or neural network photo-z codes that provides a posterior as informative as that of a neural network at a fraction of the computational cost.

Jocelyn: The combination of properties is not just aesthetic because the covariance matrix C absorbs the units and correlations, allowing for native treatment of heterogeneous photometry without imputation.

Subrahmanyan: We’re talking about a metric that respects how galaxies are actually clustered in their data space, which is what we need when dealing with heterogeneous photometry across different surveys.

Conclusion: Vera: So we’ve looked at MAHAz, this new photometric redshift estimator that uses k nearest neighbour regression with a Mahalanobis distance metric learned from spectroscopic data.

Jocelyn: It boils down to taking the features of a galaxy, like its colours and magnitudes, and finding its neighbours in that feature space.

Subrahmanyan: And instead of just looking at how far away it is using simple Euclidean distance, they use this Mahalanobis thing which accounts for how those different measurements are correlated.

Vera: Exactly. The paper shows this setup works well on four different datasets, including simulations and actual survey data like the Euclid Flagship mock and the DES Year Three sample.

Jocelyn: The numbers look pretty promising too, especially when you compare its performance to other methods like ANNz2 on some of those samples.

Subrahmanyan: And what this means for us is that we get a method that’s fast enough to do lots of photo-z estimations iteratively in big surveys without needing massive training sets for complex neural networks.

Vera: It gives us a way to get good redshift estimates with very low scatter, like the sigma NMAD values they report on the Euclid Flagship data.

Jocelyn: But it's still important to remember that this is a tool, not a perfect oracle; you still need to check those caveats about outliers and how well it handles missing bands.

Subrahmanyan: That’s the key; we need to see if this geometric approach actually captures the underlying structure of galaxy populations in the way they claim.

Vera: So, as we wrap up on MAHAz, it gives us a transparent, relatively simple way to get photometric redshifts that respects how galaxies are naturally arranged in their data space.

Jocelyn: It’s a practical step forward for getting those large catalogs populated with redshift information without the massive computational overhead of other machine learning codes.

Subrahmanyan: The next thing we need to watch is whether this method can handle the full complexity of structure formation and how it connects to the big cosmological picture.

More episodes

← Home