Wide Area VISTA Extragalactic Survey (WAVES): Redshift-limited selection of targets for the Wide survey using decision-tree classification

arXiv:2510.11132 · astro-ph.CO, astro-ph.GA, astro-ph.IM · Submitted 2025-10-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.

Vera: We're looking at the paper: "Wide Area VISTA Extragalactic Survey (WAVES)".

Jocelyn: Here's the gist of the paper. The Wide-Area VISTA Extragalactic Survey (WAVES) uses a decision-tree classification method to select targets for its Wide survey, demonstrating that a machine-learning classifier can be used to select flux- and redshift-limited samples from deep photometric data.

Vera: The selection process involves assigning probabilities of objects being below the redshift limit, rather than estimating individual redshifts for the input photometric objects.

Jocelyn: This is achieved using a supervised machine learning approach of eXtreme Gradient Boosting (XGB), trained on a comprehensive spectroscopic sample overlapping with WAVES fields.

Vera: The feature space for the classifier consists of nine VST+VISTA magnitudes from u to Ks and all the possible colors, but most relevant for the classification are the g-band and the u−g, g−r and J−Ks colors.

Title and authors: Vera: So we're looking at this paper called "Wide Area VISTA Extragalactic Survey (WAVES): Redshift-limited selection of targets for the Wide survey using decision-tree classification". It’s from Kaur and colleagues, and they’re talking about how to select targets for their wide survey in a way that respects both flux limits and redshift limits simultaneously.

Jocelyn: Right, so it sounds like they are using a machine learning approach to filter the massive amount of data coming from VISTA and Wide surveys to pick the best candidates for further study. It’s about making sure we aren't just picking bright, nearby galaxies when we want to look at something deeper or more distant.

Subrahmanyan: From my side, I'm curious how this selection method handles the inherent difficulty in assigning redshifts when you only have photometric data available for all these objects. That’s a tough spot to be in because those surveys are designed to measure those redshifts themselves, right?

Vera: Exactly. The paper tackles that by not trying to get an individual redshift estimate for every single object. Instead, they use a supervised machine learning method called eXtreme Gradient Boosting or XGBoost to assign probabilities of objects being below the redshift limit.

Jocelyn: So instead of giving us a hard number for the distance, they give us a chance—a probability—that an object qualifies as one we should select for the Wide survey. They train this classifier on data from spectroscopic samples that overlap with WAVES fields to teach it what targets look like.

Subrahmanyan: That sounds like a very clever way to handle the uncertainty in redshift estimation. But what does this classification actually depend on? What are the input features they’re feeding into the model?

Vera: The classifier uses nine VISTA plus VISTA magnitudes, from u all the way to K-sub-band, and all those possible colors. But they say that most important for the actual classification are just a few specific colors: g band and u minus g, and also g minus r and J minus K sub-band.

Jocelyn: So they’re focusing on those key color combinations to make the decision about whether an object is a target or not. They are essentially saying that these specific spectral features tell the model a lot about whether it meets the criteria for being selected.

Subrahmanyan: That makes sense, because those colors directly relate to how redshift affects the observed light from different parts of a galaxy’s spectrum. But how robust is this training set? What kind of data are they using to teach this XGBoost model?

Title and authors: Vera: They trained it on a dataset that comes from spectroscopic samples cross-matched with the WAVES imaging. This includes both the main KiDS, VIKING, and WW areas, and also Deep Fields where they have available optical and near infrared photometry.

Jocelyn: And they supplement that by using measurements in the WD fields because key deep spectroscopic surveys are often located only in those deep fields. So it’s a comprehensive look at what their training set represents.

Subrahmanyan: That’s interesting because it means the model is learning from a mix of different environments, which should give it some general understanding of galaxy properties across various redshifts and fluxes. But how did they handle the split between training and testing data?

Vera: They randomly sampled eighty percent of their spectroscopic catalog to be the training set, and the remaining twenty percent was used for testing. That’s standard procedure for a supervised machine learning setup to see how well it performs on unseen data.

Jocelyn: And when they evaluated it, they looked at purity, completeness, and F1 score specifically for class zero—which is the desired WAVES targets—at the limits of redshift less than twenty-one point one magnitudes and a redshift less than zero point two.

Subrahmanyan: Ninety-five percent for those metrics sounds solid. But what does that actually mean for someone who just wants to know if this method is reliable? Is ninety-five percent good enough when you’re talking about selecting objects for a large survey?

Vera: It means that in the specific limits they tested, the classifier was performing very well at assigning class zero correctly. They found that setting the default classification probability threshold at greater than zero point five gave them the optimal result for their test set.

Jocelyn: So if you’re looking at an object, you see its probability score from this model, and if it’s above five point zero, it's a target. It’s a relatively simple way to make the final call based on the machine learning output.

Subrahmanyan: That leads me to think about where this method shines and where it struggles. The paper notes that the classifier performs best at the bright, low-redshift end of their data range, but it gets worse for galaxies near their redshift threshold or magnitude limit.

Vera: That’s a big caveat. So if you have something faint or very distant, this specific model might not be as accurate in its probability assignments compared to what it does at the brighter end.

Title and authors: Jocelyn: And they also point out that the training set itself is a bit biased towards brighter, redder galaxies and lower redshifts when compared to a general sample limited by redshift or magnitude. This means you have to be careful about contamination if you use this model on a totally different set of objects.

Subrahmanyan: If the training data is skewed, then extending it with fainter galaxies at higher redshifts seems like the logical next step for improvement. That would give the model a better grasp of those challenging regions where it currently deteriorates.

Vera: Right, so they’re suggesting that if we want to push this further, we need more training data covering those fainter, more distant objects to make the whole system more reliable across the board.

Jocelyn: And that brings us to how they actually use this in practice for selecting the final targets. They combine this XGBoost approach with other methods using a cost function called C four P, where C stands for completeness and P stands for purity.

Subrahmanyan: So it’s not just one method; it’s a weighted combination of different selection strategies to get the best final set of candidates. That makes sense from a practical standpoint because no single metric is perfect on its own.

Vera: The classification probability distribution shows a really sharp peak around zero for class one, and generally, the classifier seems more confident in assigning non-targets than targets when it’s looking at the entire inference sample.

Jocelyn: And they end up allocating almost two point six million objects as targets when using their fiducial cut, where that probability is over zero point five. It's a very large number of candidates being selected this way.

Subrahmanyan: That’s a lot of potential targets being flagged for follow-up, which is exciting, but it also confirms what they found earlier: the metrics are maximized when you increase the redshift limit and decrease the magnitude limit.

Vera: So in short, this paper shows that using XGBoost to predict redshift limits via color and magnitude information is a viable way to select targets for surveys like WAVES, even if it has some known weaknesses at the extremes.

Jocelyn: It’s a solid tool for preliminary selection, but it’s important to remember that the model is only as good as the data it was trained on, especially when we push into regimes not well represented in that training set.

Subrahmanyan: We're looking forward to seeing those extended training sets mentioned in the paper, because getting better coverage at higher redshifts is exactly what we need to truly constrain structure formation across cosmic time.

The paper's summary: Vera: So this paper is basically showing how you can use machine learning to filter through massive amounts of photometric data from surveys like WAVES to pick targets for follow-up observations, both based on brightness and redshift limits simultaneously.

Jocelyn: It's about using an XGBoost classifier that doesn't try to calculate individual redshifts for every object but instead assigns a probability of whether an object falls below a certain redshift threshold.

Subrahmanyan: That probabilistic approach is key because it lets you select targets based on expected characteristics rather than relying on perfect measurements, which is crucial when you only have photometric data.

Vera: Right. The authors trained this model using spectroscopic samples that overlap with the WAVES fields to teach the classifier what those desired targets look like.

Jocelyn: They fed it nine VISTA magnitudes and all those various colors, but they found that specific combinations like the g band and u minus g color were actually the most important features for making a good prediction.

Subrahmanyan: From a cosmological view, this is about efficiently sampling the universe. If you can reliably select objects that are both flux-limited and redshift-limited using this method, you're not wasting telescope time on things that aren't going to give you new information about structure formation.

Vera: The performance metrics they report show that the classifier achieves purity and completeness levels around ninety five percent when testing at specific limits like a redshift less than zero point two.

Jocelyn: And they found that setting the default probability threshold at greater than zero point five was actually what worked best for their test set, which is a pretty concrete number for how to use the tool.

Subrahmanyan: That makes sense. But they also laid out some important caveats about where this method is strongest and where it starts to struggle.

Vera: The paper says the classifier works really well at the bright, low-redshift end of their data range, but its accuracy drops when you look at galaxies close to their redshift limit or magnitude limit.

Jocelyn: And they also pointed out that because the training set was biased towards brighter, redder galaxies and lower redshifts compared to a general sample, you have to be careful if you want to use this on completely different data.

Subrahmanyan: So what they suggest is that extending the training set with more fainter galaxies at higher redshifts is a way to improve the model's reliability across the whole range.

Vera: That’s the future work mentioned—more training data for those challenging regimes to make this selection method more robust.

Jocelyn: Ultimately, they combine this XGBoost approach with other methods using a cost function called C four P to get a final weighted probability for selecting the WAVES targets.

Subrahmanyan: And when you run the full selection strategy, they end up allocating almost two point six million objects as targets under their preferred cut.

Vera: That confirms that this approach is very effective at identifying a large number of candidates under their specific conditions.

Jocelyn: It’s a solid tool for initial filtering, but you have to remember that the model's performance is tied directly to the quality and range of the data it was originally trained on.

Subrahmanyan: And if you want to push this further into exploring those fainter, more distant galaxies, getting that extended training set is definitely what we need.

The paper's improvements: Tom: So we're looking at how the authors suggest ways to make this classification method even better for future use in WAVES surveys.

Vera: They point out that because their training set was a bit biased towards brighter, lower-redshift galaxies, they need to expand it with more fainter galaxies at higher redshifts.

Jocelyn: That way would give the machine learning model a much better grasp of those challenging regions where it currently performs less reliably.

Subrahmanyan: From a cosmological standpoint, that means the method could become more useful for probing structure formation at very high redshifts, which is really what we want to study.

Vera: The authors also suggest using the classification probability distribution in a different way than just picking objects above zero point five.

Jocelyn: Instead, they propose using a cost function called C four P to combine the target probabilities from this XGBoost approach with other selection strategies.

Subrahmanyan: Combining multiple methods like that is smart because it usually helps you get a more balanced result when dealing with complex astrophysical selection problems.

Vera: The goal of that combined approach is to maximize both completeness and purity simultaneously, which means you're trying to get the best possible set of targets for follow-up.

Jocelyn: When they ran the final selection strategy using that C four P cost function, they ended up allocating nearly two point six million objects as targets under their initial cut.

Subrahmanyan: That number is significant because it shows how many promising candidates you can pull out of a large survey area with this refined selection technique.

Vera: The authors also confirmed that the best results are still seen when you increase the redshift limit and decrease the magnitude limit, which tells us more about where these kinds of surveys yield the most valuable science.

Jocelyn: So even with these improvements, they are still confirming that pushing those limits is key to getting a good sample.

Subrahmanyan: It sounds like their focus is on refining the selection process itself, rather than just finding one perfect object at a time.

Vera: And it leaves us wondering what kind of data or follow-up observations would be needed next to really test if this refined method can push those limits even further.

Conclusion: Vera: So to wrap up, this paper on "Wide Area VISTA Extragalactic Survey (WAVES): Redshift-limited selection of targets for the Wide survey using decision-tree classification" shows a solid way to use machine learning to filter deep photometric data into targeted samples.

Jocelyn: It’s basically giving us a tool that uses color and magnitude information to assign probabilities about whether an object is going to be a target based on its redshift limit.

Subrahmanyan: From my side, this confirms that we can use these sophisticated classification methods to efficiently sample the universe when we have limited spectroscopic follow-up time.

Vera: The main implication is that this classifier gives us a systematic way to select candidates that respect both flux and redshift constraints right from the start.

Jocelyn: It means fewer wasted observations on objects that are too bright or too close by, which is crucial for keeping our survey focused on the most interesting science.

Subrahmanyan: If this method proves reliable across a wider range of redshifts, it helps us build better statistical samples of galaxy populations at different cosmic times.

Vera: The authors did flag that the performance is really best at the bright, low-redshift end, so we can't expect perfect results when looking at fainter or more distant objects right away.

Jocelyn: And they clearly showed that because of their training data bias, you need to be careful about contamination if you apply this to a completely new set of observations.

Subrahmanyan: So the future work they suggest, extending the training set with fainter galaxies, is exactly what we need to push this methodology into those higher redshift regimes where cosmology gets really interesting.

Vera: It’s a good paper because it gives us a concrete, testable method for tackling this kind of selection problem in massive photometric surveys.

Jocelyn: We'll keep an eye on how they use that C four P cost function to combine these different methods for the final target allocation.

Subrahmanyan: It’s important work because it lays out a clear path for how observational constraints translate into usable cosmological samples.

G. Kaur, M. Bilicki, S. Bellstedt, E. Tempel, W. A. Hellwing, I. Baldry, B. Bandi, S. Barsanti, S. Driver, N. Guerra-Varas, B. Holwerda, C. Lagos, J., Loveday, and A., Robotham

Center for Theoretical Physics, Polish Academy of Sciences · ICRAR, The University of Western Australia · Tartu Observatory, University of Tartu · Astrophysics Research Institute, Liverpool John Moores University · RF (Radio Frequency) · Astronomy Centre, University of Sussex · Sydney Institute for Astronomy, School of Physics, University of Sydney · Research School of Astronomy and Astrophysics, Australian National University · European Southern Observatory · Physics & Astronomy Department, University of Louisville

astro-ph.CO, astro-ph.GA, astro-ph.IM

Submitted: 2025-10-13

Updated: 2026-11-04

Comments: Added DESI DR1 as validation; photometric inference catalogue updated to the final WAVES version

Project page: https://wavesurvey.org

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 85/100

The gist: The Wide-Area VISTA Extragalactic Survey (WAVES) uses a decision-tree classification method to select targets for its Wide survey…

Key concepts

XGBoost
An eXtreme Gradient Boosting algorithm used as the machine learning classifier. It was trained on spectroscopic data to learn patterns in photometric features to predict whether an object is a desired target (class 0) or not (class 1). It is highly effective for classification tasks with complex feature sets.
Redshift Probability Assignment
Instead of calculating an exact redshift for every object, the model assigns a probability that an object's redshift ($z$) is less than a chosen limit (e.g., $z < 0.2$). This probabilistic approach allows the selection process to be directly linked to survey limits, making it ideal for selecting flux- and redshift-limited samples.
Feature Importance
Analysis showing which input variables—like specific magnitudes or colors (e.g., g-r, u-g)—contribute most to the classifier's decision. The study found that certain color combinations are the most critical for distinguishing between desired targets and non-targets, guiding future data selection.
Cost Function C4P
A weighted combination method used to combine target probabilities derived from different selection approaches. This function balances two key metrics: completeness (C) and purity (P). By optimizing this function, researchers can derive a final, more robust probability for selecting WAVES-Wide targets.

Terminology

Summary

The Wide-Area VISTA Extragalactic Survey (WAVES) uses a decision-tree classification method to select targets for its Wide survey, demonstrating that a machine-learning classifier can be used to select flux- and redshift-limited samples from deep photometric data. p. 0

Where the paper says it

Our work demonstrates that a machine-learning classifier could be used to select a flux- and redshift-limited sample from deep photometric data. Within WAVES, the probabilities we have derived are combined with a number of other approaches for the final Wide target selection employed for the forthcoming 4MOST observations.

Methodology for Target Selection

The selection process involves assigning probabilities of objects being below the redshift limit, rather than estimating individual redshifts for the input photometric objects. This is achieved using a supervised machine learning approach of eXtreme Gradient Boosting (XGB), trained on a comprehensive spectroscopic sample overlapping with WAVES fields. p. 0

Where the paper says it

2, the redshift limit of the subsurvey. This is done with the supervised machine learning approach of eXtreme Gradient Boosting (XGB), trained on a comprehensive spectroscopic sample overlapping with WAVES fields.

The feature space for the classifier consists of nine VST+VISTA magnitudes from u to Ks and all the possible colors, but most relevant for the classification are the g-band and the u−g, g−r and J−Ks colors. p. 0
Where the paper says it

Our feature space is composed of nine VST+VISTA magnitudes from 𝑢 to 𝐾𝑠 and all the possible colors, but most relevant for the classification are the 𝑔-band and the 𝑢 − 𝑔, 𝑔 − 𝑟 and 𝐽 − 𝐾𝑠 colors. We check the performance of our classifier both for the fiducial WAVES-Wide limits, as well as for a range of neighboring redshift and magnitude thresholds, consistently finding purity and completeness at the level of 94-95%.

The goal is to assign class "0 to desired WW targets (below chosen thresholds) and class 1" otherwise.

Training Data Composition

The ML model is trained on a dataset derived from spectroscopic samples cross-matched with WAVES imaging, which includes both the main KiDS/VIKING/WW area and the WAVES-DDFs where appropriate optical and NIR photometry is available. The input photometric catalog for training supplements WW coverage with measurements in the WD fields because key deep spectroscopic surveys are located only in those Deep fields. p. 2

Where the paper says it

While for the final (inference) dataset we use only the WW coverage, for the training phase we supplement that with photometric measurements in the WD fields. This is necessary because most of the key deep spectroscopic surveys required for our calibration are located only in the Deep fields, while the Wide ones are covered mostly by brighter-end samples.

The model is trained on a randomly sampled 80% of the spec-z-cat as the training set, and the rest as the test set. p. 6
Where the paper says it

To select the best model and test its performance, a random subset of the dataset with true labels is held out, which is referred to as the test set. In this work, we use randomly sampled 80% of the spec-z-cat as the training set and the rest as the test set.

Performance Metrics and Analysis

The performance of the classifier is evaluated using purity, completeness, and F1 score for class 0 (WW targets) at fiducial limits of Z < 21.1 mag and z < 0. p. 0

Where the paper says it

1 and 𝑍 < 21. 25 mag, and additionally redshift-limited, while the true redshifts are not known a priori but will be only measured by 4MOST.

2. The results show that the classification metrics such as completeness, purity or accuracy were at the level of ∼ 95% in all cases. p. 6
Where the paper says it

2), we mainly utilize purity and completeness metrics for class 0, corresponding to galaxies with redshifts below this threshold, as well as their harmonic mean, the F1 score, to evaluate the performance of the classifier. The two-class accuracy is defined as: Accuracy = (0, 0) + (1, 1) (0, 0) + (0, 1) + (1, 0) + (1, 1) (4) Here, (𝑎, 𝑏) refers to ground truth 𝑎, for which class 𝑏 is ascribed by the ML classifier.

The default classification probability threshold of P > 0. p. 8
Where the paper says it

have 𝑧 ≤ 0. The default classification probability threshold, shown with the vertical line, is 0.

5 is found to be optimal for the test set. p. 9
Where the paper says it

In particular, we would like to find out if for our binary classifier there exists a more optimal combination of the redshiftmagnitude thresholds which would still provide a similar number of WW targets as for the default cuts. For this purpose, we ran a set of new XGB classifications where each time the samples with true labels (train, validation, and test) were changed after imposing new 𝑍-band magnitude and/or redshift thresholds.

Implications and Future Directions

The paper notes that the classifier performs best at the bright, low-redshift end, deteriorating for galaxies close to the WW redshift threshold and magnitude limit. The training set is biased towards brighter, redder and lower-redshift galaxies as compared to a general galaxy sample magnitude-limited to Z 0. p. 9

Where the paper says it

6 mag, the averaged metrics are biased towards the brighter end. The trends as a function of redshift and magnitude are likely driven by two aspects: one is the limitations of the training set, whose very complete flux-limited component from GAMA reaches to 𝑧 < 0.

9 or P > 0.91, can be adopted in the final inference set to reduce contamination by false positives and obtain a fixed number of target candidates. p. 0
Where the paper says it

We apply the classifier trained on the full spectroscopic sample to 14 million photometric galaxies from the WAVES input catalog, which have all 9 bands measured. We then check what XGB probability threshold should be adopted to obtain a feasible number of target candidates, finding that 𝑃XGB(𝑧 ≤ 0.

The results suggest that the supervised classification approach could be further improved if the training set with true redshift labels is extended by adding more of fainter galaxies at higher redshifts. p. 4
Where the paper says it

4 and 5 indicate, this mismatch is driven by an underabundance of faint, higher-redshift sources in the spec-z-cat. The lower completeness of the spectroscopic training data at fainter, higher-redshift range, as compared to the photometric catalog, will have implications for our supervised learning approach, as we will show below.

Feature Importance Insights

Feature importance analysis indicates that the g−r and u−g colors are the most important for the classification, followed by the g-band magnitude and some nearIR colors. The fact that the g band appears in the 3 top features is noted. p. 7

Where the paper says it

Table 2 (middle column) shows that out of the total 45 input features, the 𝑔 −𝑟 and 𝑢 − 𝑔 colors are the most important for our classification, followed by the 𝑔-band magnitude and some nearIR colors. The fact that the 𝑔 band appears in the 3 top features Table 2.

The classifier seems more confident for the test set, where there is a clearer separation between dark (targets) and light (non-targets) pixels compared to the entire inference sample. p. 8
Where the paper says it

An ideal classifier would label all the galaxies to the left of this line as targets (giving them 𝑃 > 0. dark color) while those to the right would all have low 𝑃 and light color.

The classifier assigns relatively low probabilities of being a target to objects whose colors are not well represented in the current spectroscopic training. p. 2
Where the paper says it

In particular, if a too permissive cut on classification probability is applied, the number of resulting target candidates becomes too large to be practically observed. This is likely driven by particular contamination from higher-redshift galaxies whose colors are similar to those of true targets, but their color-redshift relation is not well mapped in the training set.

Final Target Selection Strategy

The final selection combines the XGBoost approach with other methods for WAVES-Wide target selection using a cost function of the form C4P, where C is completeness and P purity. The classification probability distribution shows a very pronounced peak at P ∼ 0 and a generally larger ‘confidence’ of the classifier to assign non-targets than targets. p. 8

Where the paper says it

are calculated by propagating the Poissonian errors on the number of (0,0), (0,1), (1,0), and (0,1) objects, where 0 is the target class and 1 are non-targets, and first position in parentheses is the true class while the second is the class assigned by the classifier. The results of classification are also shown as a normalized confusion matrix in Fig.

The allocated targets are to the right of the vertical P = 0. p. 5
Where the paper says it

Comparison of redshift distributions between our spectroscopic training set (yellow) and the SHARK mock catalog (blue), for galaxies at the flux limit of 𝑍 < 21. The vertical line indicates the fiducial WW redshift limit for the targets.

5 line. The final test shows that for the fiducial cut, almost 2. p. 9
Where the paper says it

The final test we provide in this section is to quantify the performance of our classifier as a function of the (true) redshift, 𝑍-band apparent magnitude, and the 𝑔 − 𝑟 color, for the fiducial WW redshift and magnitude thresholds and using the 𝑃 = 0. 5 cut between the classes.

6 million objects are allocated as targets with a probability over 0. p. 8
Where the paper says it

objects with 𝑃XGB > 0. 5 were allocated to class 0 and the others to class 1.

5. The results confirm our earlier findings that the metrics are maximized if the redshift limit is increased and the magnitude limit decreased. p. 9
Where the paper says it

In the plot, each datapoint with error bars is derived from the training and testing at a particular magnitude limit. We notice that all the 1-class statistics consistently deteriorate when the flux limit is decreased, although not by a large margin: from roughly 0.

p. 9
Where the paper says it

In the plot, each datapoint with error bars is derived from the training and testing at a particular magnitude limit. We notice that all the 1-class statistics consistently deteriorate when the flux limit is decreased, although not by a large margin: from roughly 0.

The number of targets changes very rapidly close to these extremes, while it varies much more slowly over the range of 0. p. 8
Where the paper says it

as WW targets. It is important to note that the number of objects identified as a certain class varies with 5-fold training and testing.

1 ≲ Pcut ≲ 0.7. The results confirm our earlier findings that the metrics are maximized if the redshift limit is increased and the magnitude limit decreased.

Error Estimation Details

The uncertainty in purity using standard error propagation is computed using Equations A.2-A.5, which involve calculating means of true positives, false positives, false negatives, and true negatives obtained from the test performance of the 5-fold models. p. 9

Where the paper says it

, those missed out – false negatives, FN, and contaminants – false positives, FP). While such objects are typically fainter and bluer than the correctly assigned WW targets (true positives), we do not see significant differences between FP and FN that would point to a particular sensitivity of our classifier.

The scatter in the purity metric is determined using Equations A. p. 6
Where the paper says it

9) if the size of the targeted sample in WW should not exceed the expected ∼ 1 million. Classification and metrics The model’s performance in classifying galaxies according to their redshifts is evaluated using a two-class accuracy metric.

2-A.5. The results of classification are also shown as a normalized confusion matrix in Fig. p. 8
Where the paper says it

are calculated by propagating the Poissonian errors on the number of (0,0), (0,1), (1,0), and (0,1) objects, where 0 is the target class and 1 are non-targets, and first position in parentheses is the true class while the second is the class assigned by the classifier. The results of classification are also shown as a normalized confusion matrix in Fig.

6. The errors on the metrics are calculated by propagating the Poissonian errors on the number of (0,0), (0,1), (1,0), and (1,1) objects. p. 8
Where the paper says it

Confusion matrix showing the percentages of Class ’0’ (Class ’1’) sources correctly and incorrectly labeled by the XGB along the top (bottom) row. are calculated by propagating the Poissonian errors on the number of (0,0), (0,1), (1,0), and (0,1) objects, where 0 is the target class and 1 are non-targets, and first position in parentheses is the true class while the second is the class assigned by the classifier.

The corresponding uncertainty in purity using standard error propagation is then computed as Equation A. p. 1
Where the paper says it

At the WW limiting redshift of 𝑧 = 0. 2, this amounts to some 10% relative error, which could be crudely translated to ∼ 90% completeness and purity if such photo-zs are employed for target selection.

2.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Develop an XGBoost based classification pipeline that assigns probabilities of being below a redshift threshold instead of estimating individual redshifts for input photometric objects, as described by we assign probabilities of them being below z = 0.2.

  2. Implement a decision tree based target selection pipeline that uses the "XGBoost algorithm (Chen & Guestrin 2016) to classify galaxies into two classes: class 0 for desired WAVES targets and class 1 otherwise, and assign output probabilities using the formula P(z ≤ zlim) = 1 / (1 + exp(-S))".

  3. Utilize feature importance analysis, including permutation feature importance and SHAP analysis, to determine which bands and colors matter most for WW target selection, identifying features like the g-r and u-g colors as the most important for our classification.

  4. Design a probability threshold selection mechanism that allows for optimizing the number of selected targets by varying the probability cut, noting that "we will in particular find that more strict cuts are needed (even P > 0.9) if the size of the targeted sample in WW should not exceed the expected ∼ 1 million."

  5. Integrate a cost function of C4P to combine target probabilities from different approaches, where C is completeness and P purity, to calculate a final weighted probability for selecting WAVES-Wide targets.

Abstract

The Wide-Area VISTA Extragalactic Survey (WAVES) on the 4-metre Multi-Object Spectroscopic Telescope (4MOST) includes two flux-limited subsurveys with very high (95%) completeness requirements: Wide over about!1200 deg squared and Deep over about!65 deg squared. Both are Z-band selected, and additionally redshift-limited, while the true redshifts are not known a priori but will only be measured by 4MOST. Here, we present a classification-based method to select the targets for WAVES-Wide. Rather than estimating individual redshifts for the input photometric galaxies, we assign probabilities of them being below z=0.2, the redshift limit of the subsurvey. This is done with the supervised machine learning approach of eXtreme Gradient Boosting (XGB), trained and tested on a comprehensive spectroscopic sample overlapping with WAVES fields and using nine-band VST+VISTA photometry for the feature space. We check the performance of our classifier both for the fiducial WAVES-Wide limits, as well as for a range of neighboring redshift and magnitude thresholds, finding purity and completeness at the level of 90-95% depending on the test set. The metrics deteriorate for sources close to the selection limits and for test sets selected from DESI, not used in training. We apply the classifier trained on the full spectroscopic sample to the 8.6 million galaxies from the WAVES photometric catalog. This gives us about 1.2 million target candidates, reducing to 1 million if we apply an XGB probability threshold P XGB(z at most0.2) > 0.72. Our work demonstrates that a machine-learning classifier could be used to select a flux- and redshift-limited sample from deep photometric data.

Sources

Related papers