Wide Area VISTA Extragalactic Survey (WAVES): Redshift-limited selection of targets for the Wide survey using decision-tree classification

summary

The gist

The Wide-Area VISTA Extragalactic Survey (WAVES) uses a decision-tree classification method to select targets for its Wide survey…

In short

The Wide Area VISTA Extragalactic Survey (WAVES) used an XGBoost machine learning classifier to select targets for its Wide survey from deep photometric data. The model assigns probabilities of objects being below a specific redshift limit instead of estimating individual redshifts. The classification achieved high accuracy, confirming that this method can effectively filter deep photometric catalogs into flux- and redshift-limited samples.

Key concepts

XGBoost
An eXtreme Gradient Boosting algorithm used as the machine learning classifier. It was trained on spectroscopic data to learn patterns in photometric features to predict whether an object is a desired target (class 0) or not (class 1). It is highly effective for classification tasks with complex feature sets.
Redshift Probability Assignment
Instead of calculating an exact redshift for every object, the model assigns a probability that an object's redshift ($z$) is less than a chosen limit (e.g., $z < 0.2$). This probabilistic approach allows the selection process to be directly linked to survey limits, making it ideal for selecting flux- and redshift-limited samples.
Feature Importance
Analysis showing which input variables—like specific magnitudes or colors (e.g., g-r, u-g)—contribute most to the classifier's decision. The study found that certain color combinations are the most critical for distinguishing between desired targets and non-targets, guiding future data selection.
Cost Function C4P
A weighted combination method used to combine target probabilities derived from different selection approaches. This function balances two key metrics: completeness (C) and purity (P). By optimizing this function, researchers can derive a final, more robust probability for selecting WAVES-Wide targets.

Terminology used across episodes

This episode discusses

The paper

Wide Area VISTA Extragalactic Survey (WAVES): Redshift-limited selection of targets for the Wide survey using decision-tree classification · Read on arXiv

G. Kaur, M. Bilicki, S. Bellstedt, E. Tempel, W. A. Hellwing, I. Baldry, B. Bandi, S. Barsanti, S. Driver, N. Guerra-Varas, B. Holwerda, C. Lagos, J., Loveday, and A., Robotham

Center for Theoretical Physics, Polish Academy of Sciences · ICRAR, The University of Western Australia · Tartu Observatory, University of Tartu · Astrophysics Research Institute, Liverpool John Moores University · RF (Radio Frequency) · Astronomy Centre, University of Sussex · Sydney Institute for Astronomy, School of Physics, University of Sydney · Research School of Astronomy and Astrophysics, Australian National University · European Southern Observatory · Physics & Astronomy Department, University of Louisville

The Wide-Area VISTA Extragalactic Survey (WAVES) on the 4-metre Multi-Object Spectroscopic Telescope (4MOST) includes two flux-limited subsurveys with very high (95%) completeness requirements: Wide over about!1200 deg squared and Deep over about!65 deg squared. Both are Z-band selected, and additionally redshift-limited, while the true redshifts are not known a priori but will only be measured by 4MOST. Here, we present a classification-based method to select the targets for WAVES-Wide. Rather than estimating individual redshifts for the input photometric galaxies, we assign probabilities of them being below z=0.2, the redshift limit of the subsurvey. This is done with the supervised machine learning approach of eXtreme Gradient Boosting (XGB), trained and tested on a comprehensive spectroscopic sample overlapping with WAVES fields and using nine-band VST+VISTA photometry for the feature space. We check the performance of our classifier both for the fiducial WAVES-Wide limits, as well as for a range of neighboring redshift and magnitude thresholds, finding purity and completeness at the level of 90-95% depending on the test set. The metrics deteriorate for sources close to the selection limits and for test sets selected from DESI, not used in training. We apply the classifier trained on the full spectroscopic sample to the 8.6 million galaxies from the WAVES photometric catalog. This gives us about 1.2 million target candidates, reducing to 1 million if we apply an XGB probability threshold P XGB(z at most0.2) > 0.72. Our work demonstrates that a machine-learning classifier could be used to select a flux- and redshift-limited sample from deep photometric data.

Transcript

Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.

Vera: We're looking at the paper: "Wide Area VISTA Extragalactic Survey (WAVES)".

Jocelyn: Here's the gist of the paper. The Wide-Area VISTA Extragalactic Survey (WAVES) uses a decision-tree classification method to select targets for its Wide survey, demonstrating that a machine-learning classifier can be used to select flux- and redshift-limited samples from deep photometric data.

Vera: The selection process involves assigning probabilities of objects being below the redshift limit, rather than estimating individual redshifts for the input photometric objects.

Jocelyn: This is achieved using a supervised machine learning approach of eXtreme Gradient Boosting (XGB), trained on a comprehensive spectroscopic sample overlapping with WAVES fields.

Vera: The feature space for the classifier consists of nine VST+VISTA magnitudes from u to Ks and all the possible colors, but most relevant for the classification are the g-band and the u−g, g−r and J−Ks colors.

Title and authors: Vera: So we're looking at this paper called "Wide Area VISTA Extragalactic Survey (WAVES): Redshift-limited selection of targets for the Wide survey using decision-tree classification". It’s from Kaur and colleagues, and they’re talking about how to select targets for their wide survey in a way that respects both flux limits and redshift limits simultaneously.

Jocelyn: Right, so it sounds like they are using a machine learning approach to filter the massive amount of data coming from VISTA and Wide surveys to pick the best candidates for further study. It’s about making sure we aren't just picking bright, nearby galaxies when we want to look at something deeper or more distant.

Subrahmanyan: From my side, I'm curious how this selection method handles the inherent difficulty in assigning redshifts when you only have photometric data available for all these objects. That’s a tough spot to be in because those surveys are designed to measure those redshifts themselves, right?

Vera: Exactly. The paper tackles that by not trying to get an individual redshift estimate for every single object. Instead, they use a supervised machine learning method called eXtreme Gradient Boosting or XGBoost to assign probabilities of objects being below the redshift limit.

Jocelyn: So instead of giving us a hard number for the distance, they give us a chance—a probability—that an object qualifies as one we should select for the Wide survey. They train this classifier on data from spectroscopic samples that overlap with WAVES fields to teach it what targets look like.

Subrahmanyan: That sounds like a very clever way to handle the uncertainty in redshift estimation. But what does this classification actually depend on? What are the input features they’re feeding into the model?

Vera: The classifier uses nine VISTA plus VISTA magnitudes, from u all the way to K-sub-band, and all those possible colors. But they say that most important for the actual classification are just a few specific colors: g band and u minus g, and also g minus r and J minus K sub-band.

Jocelyn: So they’re focusing on those key color combinations to make the decision about whether an object is a target or not. They are essentially saying that these specific spectral features tell the model a lot about whether it meets the criteria for being selected.

Subrahmanyan: That makes sense, because those colors directly relate to how redshift affects the observed light from different parts of a galaxy’s spectrum. But how robust is this training set? What kind of data are they using to teach this XGBoost model?

Title and authors: Vera: They trained it on a dataset that comes from spectroscopic samples cross-matched with the WAVES imaging. This includes both the main KiDS, VIKING, and WW areas, and also Deep Fields where they have available optical and near infrared photometry.

Jocelyn: And they supplement that by using measurements in the WD fields because key deep spectroscopic surveys are often located only in those deep fields. So it’s a comprehensive look at what their training set represents.

Subrahmanyan: That’s interesting because it means the model is learning from a mix of different environments, which should give it some general understanding of galaxy properties across various redshifts and fluxes. But how did they handle the split between training and testing data?

Vera: They randomly sampled eighty percent of their spectroscopic catalog to be the training set, and the remaining twenty percent was used for testing. That’s standard procedure for a supervised machine learning setup to see how well it performs on unseen data.

Jocelyn: And when they evaluated it, they looked at purity, completeness, and F1 score specifically for class zero—which is the desired WAVES targets—at the limits of redshift less than twenty-one point one magnitudes and a redshift less than zero point two.

Subrahmanyan: Ninety-five percent for those metrics sounds solid. But what does that actually mean for someone who just wants to know if this method is reliable? Is ninety-five percent good enough when you’re talking about selecting objects for a large survey?

Vera: It means that in the specific limits they tested, the classifier was performing very well at assigning class zero correctly. They found that setting the default classification probability threshold at greater than zero point five gave them the optimal result for their test set.

Jocelyn: So if you’re looking at an object, you see its probability score from this model, and if it’s above five point zero, it's a target. It’s a relatively simple way to make the final call based on the machine learning output.

Subrahmanyan: That leads me to think about where this method shines and where it struggles. The paper notes that the classifier performs best at the bright, low-redshift end of their data range, but it gets worse for galaxies near their redshift threshold or magnitude limit.

Vera: That’s a big caveat. So if you have something faint or very distant, this specific model might not be as accurate in its probability assignments compared to what it does at the brighter end.

Title and authors: Jocelyn: And they also point out that the training set itself is a bit biased towards brighter, redder galaxies and lower redshifts when compared to a general sample limited by redshift or magnitude. This means you have to be careful about contamination if you use this model on a totally different set of objects.

Subrahmanyan: If the training data is skewed, then extending it with fainter galaxies at higher redshifts seems like the logical next step for improvement. That would give the model a better grasp of those challenging regions where it currently deteriorates.

Vera: Right, so they’re suggesting that if we want to push this further, we need more training data covering those fainter, more distant objects to make the whole system more reliable across the board.

Jocelyn: And that brings us to how they actually use this in practice for selecting the final targets. They combine this XGBoost approach with other methods using a cost function called C four P, where C stands for completeness and P stands for purity.

Subrahmanyan: So it’s not just one method; it’s a weighted combination of different selection strategies to get the best final set of candidates. That makes sense from a practical standpoint because no single metric is perfect on its own.

Vera: The classification probability distribution shows a really sharp peak around zero for class one, and generally, the classifier seems more confident in assigning non-targets than targets when it’s looking at the entire inference sample.

Jocelyn: And they end up allocating almost two point six million objects as targets when using their fiducial cut, where that probability is over zero point five. It's a very large number of candidates being selected this way.

Subrahmanyan: That’s a lot of potential targets being flagged for follow-up, which is exciting, but it also confirms what they found earlier: the metrics are maximized when you increase the redshift limit and decrease the magnitude limit.

Vera: So in short, this paper shows that using XGBoost to predict redshift limits via color and magnitude information is a viable way to select targets for surveys like WAVES, even if it has some known weaknesses at the extremes.

Jocelyn: It’s a solid tool for preliminary selection, but it’s important to remember that the model is only as good as the data it was trained on, especially when we push into regimes not well represented in that training set.

Subrahmanyan: We're looking forward to seeing those extended training sets mentioned in the paper, because getting better coverage at higher redshifts is exactly what we need to truly constrain structure formation across cosmic time.

The paper's summary: Vera: So this paper is basically showing how you can use machine learning to filter through massive amounts of photometric data from surveys like WAVES to pick targets for follow-up observations, both based on brightness and redshift limits simultaneously.

Jocelyn: It's about using an XGBoost classifier that doesn't try to calculate individual redshifts for every object but instead assigns a probability of whether an object falls below a certain redshift threshold.

Subrahmanyan: That probabilistic approach is key because it lets you select targets based on expected characteristics rather than relying on perfect measurements, which is crucial when you only have photometric data.

Vera: Right. The authors trained this model using spectroscopic samples that overlap with the WAVES fields to teach the classifier what those desired targets look like.

Jocelyn: They fed it nine VISTA magnitudes and all those various colors, but they found that specific combinations like the g band and u minus g color were actually the most important features for making a good prediction.

Subrahmanyan: From a cosmological view, this is about efficiently sampling the universe. If you can reliably select objects that are both flux-limited and redshift-limited using this method, you're not wasting telescope time on things that aren't going to give you new information about structure formation.

Vera: The performance metrics they report show that the classifier achieves purity and completeness levels around ninety five percent when testing at specific limits like a redshift less than zero point two.

Jocelyn: And they found that setting the default probability threshold at greater than zero point five was actually what worked best for their test set, which is a pretty concrete number for how to use the tool.

Subrahmanyan: That makes sense. But they also laid out some important caveats about where this method is strongest and where it starts to struggle.

Vera: The paper says the classifier works really well at the bright, low-redshift end of their data range, but its accuracy drops when you look at galaxies close to their redshift limit or magnitude limit.

Jocelyn: And they also pointed out that because the training set was biased towards brighter, redder galaxies and lower redshifts compared to a general sample, you have to be careful if you want to use this on completely different data.

Subrahmanyan: So what they suggest is that extending the training set with more fainter galaxies at higher redshifts is a way to improve the model's reliability across the whole range.

Vera: That’s the future work mentioned—more training data for those challenging regimes to make this selection method more robust.

Jocelyn: Ultimately, they combine this XGBoost approach with other methods using a cost function called C four P to get a final weighted probability for selecting the WAVES targets.

Subrahmanyan: And when you run the full selection strategy, they end up allocating almost two point six million objects as targets under their preferred cut.

Vera: That confirms that this approach is very effective at identifying a large number of candidates under their specific conditions.

Jocelyn: It’s a solid tool for initial filtering, but you have to remember that the model's performance is tied directly to the quality and range of the data it was originally trained on.

Subrahmanyan: And if you want to push this further into exploring those fainter, more distant galaxies, getting that extended training set is definitely what we need.

The paper's improvements: Tom: So we're looking at how the authors suggest ways to make this classification method even better for future use in WAVES surveys.

Vera: They point out that because their training set was a bit biased towards brighter, lower-redshift galaxies, they need to expand it with more fainter galaxies at higher redshifts.

Jocelyn: That way would give the machine learning model a much better grasp of those challenging regions where it currently performs less reliably.

Subrahmanyan: From a cosmological standpoint, that means the method could become more useful for probing structure formation at very high redshifts, which is really what we want to study.

Vera: The authors also suggest using the classification probability distribution in a different way than just picking objects above zero point five.

Jocelyn: Instead, they propose using a cost function called C four P to combine the target probabilities from this XGBoost approach with other selection strategies.

Subrahmanyan: Combining multiple methods like that is smart because it usually helps you get a more balanced result when dealing with complex astrophysical selection problems.

Vera: The goal of that combined approach is to maximize both completeness and purity simultaneously, which means you're trying to get the best possible set of targets for follow-up.

Jocelyn: When they ran the final selection strategy using that C four P cost function, they ended up allocating nearly two point six million objects as targets under their initial cut.

Subrahmanyan: That number is significant because it shows how many promising candidates you can pull out of a large survey area with this refined selection technique.

Vera: The authors also confirmed that the best results are still seen when you increase the redshift limit and decrease the magnitude limit, which tells us more about where these kinds of surveys yield the most valuable science.

Jocelyn: So even with these improvements, they are still confirming that pushing those limits is key to getting a good sample.

Subrahmanyan: It sounds like their focus is on refining the selection process itself, rather than just finding one perfect object at a time.

Vera: And it leaves us wondering what kind of data or follow-up observations would be needed next to really test if this refined method can push those limits even further.

Conclusion: Vera: So to wrap up, this paper on "Wide Area VISTA Extragalactic Survey (WAVES): Redshift-limited selection of targets for the Wide survey using decision-tree classification" shows a solid way to use machine learning to filter deep photometric data into targeted samples.

Jocelyn: It’s basically giving us a tool that uses color and magnitude information to assign probabilities about whether an object is going to be a target based on its redshift limit.

Subrahmanyan: From my side, this confirms that we can use these sophisticated classification methods to efficiently sample the universe when we have limited spectroscopic follow-up time.

Vera: The main implication is that this classifier gives us a systematic way to select candidates that respect both flux and redshift constraints right from the start.

Jocelyn: It means fewer wasted observations on objects that are too bright or too close by, which is crucial for keeping our survey focused on the most interesting science.

Subrahmanyan: If this method proves reliable across a wider range of redshifts, it helps us build better statistical samples of galaxy populations at different cosmic times.

Vera: The authors did flag that the performance is really best at the bright, low-redshift end, so we can't expect perfect results when looking at fainter or more distant objects right away.

Jocelyn: And they clearly showed that because of their training data bias, you need to be careful about contamination if you apply this to a completely new set of observations.

Subrahmanyan: So the future work they suggest, extending the training set with fainter galaxies, is exactly what we need to push this methodology into those higher redshift regimes where cosmology gets really interesting.

Vera: It’s a good paper because it gives us a concrete, testable method for tackling this kind of selection problem in massive photometric surveys.

Jocelyn: We'll keep an eye on how they use that C four P cost function to combine these different methods for the final target allocation.

Subrahmanyan: It’s important work because it lays out a clear path for how observational constraints translate into usable cosmological samples.

More episodes

← Home