Euclid Quick Data Release (Q1). Refining the strong lens sample with Space Warps crowd-sourced science

summary

The gist

We demonstrate the efficacy of crowd-sourced refinement of strong gravitational lens candidate samples by citizen scientists The volunteer crowd responses correlate with those from researchers…

In short

This research tested using citizen scientists to refine strong gravitational lens candidate samples by assigning them grades (Definite, Probable, Possible, Non-lens). Volunteers' grading correlated with researchers' classifications. Using calibrated volunteer grades could significantly improve lens purity in future searches, potentially yielding 79% completeness for high-grade lenses in the Euclid DR1 data.

Key concepts

Grade Posteriors
This is the probability that a strong lens system is a specific grade (A, B, C, or X), calculated based on volunteer classifications. These posteriors converge fastest for 'definite' (Grade A) and 'non-lens' (Grade X) systems, showing where volunteer grading is most reliable.
Calibration
The Space Warps Analysis Pipeline (SWAP) uses Bayes' Theorem to calibrate volunteer responses against consensus grades from researchers. This process creates a skill matrix showing the probability that a volunteer will assign a specific grade to a system with that true grade, allowing their subjective opinions to be comparable to expert research.
Selection Function
This describes how volunteers identify systems as lenses. Systems with high signal-to-noise ratios (S/N) are typically identified by volunteers and assigned higher grades, especially if they have a large Einstein radius. Lower S/N systems are less likely to be classified as lenses by the crowd.
Completeness and Purity
Purity measures the ratio of true lenses to total systems above a certain threshold. Combining calibrated volunteer grade posteriors (p(A∪B|Cj)) produces the highest purity, offering a factor of 7 improvement over previous samples, which is crucial for identifying thousands of true lenses in Euclid DR1.

Terminology used across episodes

This episode discusses

The paper

Euclid Quick Data Release (Q1). Refining the strong lens sample with Space Warps crowd-sourced science · Read on arXiv

P. Holloway, A. Verma, T. E. Collett, N. E. P. Lines, T. Li, P. J., Marshall, A., More, M., §

European Space Agency (ESA) · University of Oxford · § SLAC National Accelerator Laboratory

We demonstrate the efficacy of crowd-sourced refinement of strong gravitational lens candidate samples by citizen scientists. Binary (lens/non-lens) classification by citizen scientists has been conducted previously, and here we extend such categorisation into `definite', `probable', `possible', and `non-lenses'. This grading is a required stage ahead of any strong lens sample publication and is usually carried out by (small) teams of researchers. We compare the responses of the volunteers to those of strong lens researchers and discuss the optimal strategy for lens classification and grading in future strong lens searches. Volunteers are found to be able to correctly identify systems of each grade, and their responses can be calibrated to be comparable to those of researchers. Grade posteriors, defined as the probability that a strong lens system classified by the volunteers is a certain grade, converge most rapidly for non-lens systems. We find that one researcher classification is equivalent to only 1.9 volunteer classifications when grading such systems. There is significant disagreement in the grading of intermediate-grade lens candidates amongst both volunteers and researchers. Moving towards calibrated lens probabilities for future lens searches would help to account for such subjectivity. If the Q1 lens search was rerun with volunteers performing lens grading (and corresponding calibration), a completeness of 79% for Grades A+B (corresponding to definite + probable lenses respectively) could be achieved with 41% purity prior to researcher inspection - this would correspond to an expected 12500 true lenses (i.e. excluding contaminants) in Euclid DR1.

Transcript

Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.

Vera: We're looking at the paper: "Euclid Quick Data Release (Q1). Refining the strong lens sample with Space Warps crowd-sourced science".

Jocelyn: Here's the gist of the paper. We demonstrate the efficacy of crowd-sourced refinement of strong gravitational lens candidate samples by citizen scientists The volunteer crowd responses correlate with those from researchers; systems ranked highly by researchers receive correspondingly high ranks from the volunteers Volunteer grades are most useful for identifying Grade A and Grade X systems; the grade posteriors p(GiCj) converge the fastest (i.e., with the fewest volunteer classifications) for these grade extremes Volunteer grading can provide significant improvements in lens purity; 76% (76%) completeness of Grade A (A+B) lenses, with a 50% pure sample The volunteers have a qualitatively similar selection function to the researchers; this holds when both grading and performing binary (lens/non-lens) classification Volunteer Grading Workflow The Space Warps project is dedicated to finding strong lenses, asking volunteers to categorise systems into ‘lenses’ and ‘non-lenses’ in a binary classification workflow termed SW-Classify The high lens detection rate of the volunteers in these searches, alongside the Q1 lens search, provided motivation to launch Space Warps Refine (SW-Refine) whereby volunteers instead provided grades according to their confidence a given system was a lens The grades provided here refer to those received from the Galaxy Judges researcher-grading phase (SLDE-A), but without the weighting scheme applied in SLDE-A The volunteers were asked to grade each galaxy system according to the grading scheme Grade A: Definite lens, Grade B: Probable lens, Grade C: Possible lens, Grade X: Not a lens Calibration and Skill Assessment The Space Warps Analysis Pipeline (SWAP) is used to calibrate the volunteers’ responses according to the consensus grade from the researchers The posterior probability that a system is grade Gi (where Gi ∈ A, B,C, X), following classification Cj from volunteer uj (where again, Cj ∈ A, B,C, X) is given via Bayes Theorem The skill is a 4 × 4 matrix representing the probability that a volunteer will assign a grade Cj to a system with a ‘true’ grade Gi This skill is calculated using the leave-one-out method to improve skill calibration The posterior p(GiCj) is estimated by taking samples from the prior, weighted by the likelihood p(NCjGiµCj) Performance and Optimization The grade posteriors converge the fastest (i.e., with the fewest number of classifications) for Grade A and X systems The convergence for Grade X systems, we find one classification from a researcher provides an equivalent posterior to 1.9 classifications from a SW-Refine volunteer For Grade A systems, convergence is much faster when using individual skill A staged approach of conducting a binary classification followed by a refinement stage helps to reduce the number of systems that need the more time-intensive grading step Setting a threshold of p(Gi = XCjSWR) > 0.9 would remove 70% of Grade X systems in the GJ sample, while removing only 1.5% of Grade A+B systems Future Implications for Euclid Combining the grade posteriors p(A∪BCj) produces the highest purity sample (defined as the ratio of the number of lenses to the total number of systems above a given threshold) This represents a factor of 7 improvement in purity compared to the original GJ sample, in which 7% of the systems were Grade A+B The convergence rate for Grade X systems remains rapid for both crowd-level or individual skill, but is slow for intermediate grades In summary, volunteer grades can act as a useful proxy to researcher grades Additionally, citizen scientists can reduce the number of classifications required from researchers in order to confidently classify them as Grade A or Grade X lenses Selection Function Analysis The primary driver behind whether a system is identified as a lens by volunteers is S/N; systems with a sufficiently high S/N are likely to be identified regardless of their Einstein radius Systems with lower S/N are less likely to be identified as lenses by the volunteers, and those that are identified are likely to be assigned a lower grade The selection function of the SW-Refine volunteers shows that systems with a high S/N are typically assigned a high grade by volunteers, especially if they have a large Einstein radius In summary, volunteer grades can act as a useful proxy to researcher grades Furthermore, volunteers can inspect significantly larger numbers of systems (of order 105) than researchers alone Additionally, citizen scientists can reduce the number of classifications required from researchers in order to confidently classify them as Grade A or Grade X lenses The Euclid DR1 will enable the discovery of thousands of lens candidates Acknowledgements We gratefully acknowledge the Zooniverse team for their invaluable work on the host platform for Space Warps This project would not be possible without the immense contributions of citizen scientists Those contributing classifications for this project are listed below, and on ‘The Team’ webpage We greatly appreciate their commitment, enthusiasm, and time The Euclid Consortium acknowledges the European Space Agency and a number of agencies and institutes that have supported the development of Euclid This work has made use of the Euclid Quick Release Q1 data from the Euclid mission of the European Space Agency (ESA), 2025 The research makes use of ESA Datalabs (datalabs.esa.int), an initiative by ESA’s Data Science and Archives Division in the Science and Operations Department, Directorate of Science References Browne, I.

Vera: W.

Jocelyn: A., Wilkinson, P.

Vera: N., Jackson, N.

Paper summary: Vera: So, this paper is about using citizen scientists to help refine the sample of strong gravitational lens candidates from the Euclid Quick Data Release Q1. They’re moving beyond just saying something is either a lens or not a lens.

Jocelyn: Exactly. The main idea here is that they are extending the binary classification into four grades: definite, probable, possible, and non-lenses. They argue this grading step is necessary before any big strong lens sample publication and usually has to be done by small teams of researchers on their own.

Subrahmanyan: I wonder how much subjectivity there is in those initial research classifications if you just have a few people making them? That sounds like it could introduce a lot of noise into the final catalog.

Vera: That’s what they’re comparing the volunteers to, Subrahmanyan. They compare the responses of these volunteers to those from actual strong lens researchers and they discuss what would be the best strategy for classifying and grading lenses in future searches.

Jocelyn: They find that the volunteers are actually able to correctly identify systems for each of these grades, and they can even calibrate their responses so they become comparable to what researchers are doing.

Subrahmanyan: Calibration is key, right? If you can make citizen science responses match expert consensus, it gives you a reliable way to sort through the data.

Vera: It does. And here’s a bit more detail: the grade posteriors—which is basically the probability that a system is a certain grade given what the volunteers classified—they converge most rapidly for systems that are non-lenses.

Jocelyn: So, they find it’s much faster to agree when something isn't a lens. But what about those intermediate grades, like possible or probable?

Vera: That’s where things get trickier. The paper finds significant disagreement in the grading of intermediate-grade lens candidates among both the volunteers and the researchers.

Subrahmanyan: Disagreement on borderline cases is common in science, I suppose, but if you can't agree on whether something is a definite lens or not, it slows down the whole process for building a sample.

Jocelyn: And they suggest that moving towards calibrated lens probabilities for future searches would help account for that kind of subjectivity.

Vera: It does. They show that one researcher classification is equivalent to only one point nine volunteer classifications when it comes to grading those systems. That’s a big reduction in the workload for researchers if volunteers are involved.

Subrahmanyan: One point nine, that’s a significant factor when you’re talking about scaling up data analysis across ten thousand or more systems, isn't it?

Jocelyn: And they look at what happens if you actually run the Q1 lens search again but use volunteers for the grading. They found that with this approach, they could achieve a completeness of seventy-nine percent for Grade A plus B lenses.

Paper summary: Vera: Seventy-nine percent completeness for those definite and probable lenses, which would correspond to a purity of fifty percent in the sample prior to researcher inspection.

Subrahmanyan: Fifty percent purity means half the systems you’re looking at are actually lenses, which is a decent starting point for anything. What does that translate to in terms of the actual number of targets?

Jocelyn: That would correspond to an expected one two five thousand true lenses in the Euclid Data Release one excluding any contaminants <ref:2611.06667#pg2>. The paper shows how these volunteer grades can provide significant improvements in lens purity when you use them.

Vera: They also found that the selection function for these volunteers is driven by the signal to noise ratio, or S over N. Systems with a high S over N are typically assigned a high grade by volunteers, especially if they have a large Einstein radius.

Subrahmanyan: So, the brighter and bigger the lensing effect you see in the data, the more likely it is that a volunteer will grade it highly? That makes sense from an observational standpoint.

Jocelyn: Right. But even with lower S over N systems, volunteers can still identify them but they’re usually assigned a lower grade because of that signal weakness.

Vera: They also found that combining those grade posteriors for definite and probable lenses produces the highest purity sample, which is defined as the ratio of the number of lenses to the total number of systems above a given threshold.

Subrahmanyan: A factor of seven improvement in purity compared to the original Q1 sample, which was only seventy percent pure for those grades. That’s a substantial gain in quality for a preliminary dataset.

Jocelyn: The convergence rate for those non-lens systems is very fast for both the crowd-level grading and the individual volunteer skill, but it slows down quite a bit when you look at those intermediate grades.

Vera: In short, volunteer grades act as a useful proxy for researcher grades, and they can inspect much larger numbers of systems than researchers alone.

Subrahmanyan: So, citizen scientists can reduce the number of classifications that researchers have to do if they want to confidently sort things into definite or non-lens categories.

Jocelyn: The paper ends by saying that this refinement stage will help maximize the number of lenses found in a data release while minimizing the inspection burden on both researchers and volunteers.

Vera: It’s a practical way to handle the sheer volume of systems coming out of surveys like Euclid DR1.

Jocelyn: And it really shows how crowd science can be integrated into the pipeline to make those samples more robust before they get published.

Conclusion: Vera: So, we just looked at how citizen scientists can help sort through those strong lens candidates from Euclid's Quick Release Q1 data using this new crowd-sourced science approach.

Jocelyn: It’s a paper titled "Euclid Quick Data Release (Q1). Refining the strong lens sample with Space Warps crowd-sourced science," and it’s authored by people from the Euclid Collaboration.

Subrahmanyan: I see they're focused on taking those initial classifications, which are often just binary lens or non-lens, and improving them by adding a grading system.

Vera: Exactly. They introduce definite, probable, possible, and non-lenses. The main point is that this grading step is crucial before you publish a strong lens sample because it usually has to be done by small teams of researchers.

Jocelyn: It looks at how the volunteers' responses compare to actual strong lens researchers and figures out the best strategy for classifying these candidates in future searches.

Subrahmanyan: The results show that the volunteer grades are actually quite accurate when calibrated against researcher consensus, especially for identifying those high and low extremes, like definite or non-lens systems.

Vera: They found that one researcher classification is equivalent to about one point nine volunteer classifications when grading those systems, which means volunteers can significantly cut down on the number of classifications researchers have to do.

Jocelyn: And they show that if you use these calibrated volunteer grades, you could achieve a seventy-nine percent completeness for Grade A plus B lenses before any researcher even looks at them.

Subrahmanyan: That translates to an expected one two five thousand true lenses in the Euclid Data Release one excluding any contaminants. It’s a big deal for the sheer number of systems we expect to see.

Vera: They also pointed out that combining those grade probabilities gives you a sample purity factor of seven improvement over the original sample, which was only seventy percent pure for those definite and probable grades.

Jocelyn: So, this whole study is about using crowd science not just to find more lenses, but to make sure the samples we get are much cleaner and more reliable.

Subrahmanyan: It shows that citizen scientists can act as a useful proxy for researcher grades when you're looking at things like definite or non-lens classifications.

Vera: And it suggests that volunteers can inspect systems in much larger numbers than researchers alone, which helps maximize the number of lenses we find in a data release while minimizing the inspection burden.

Jocelyn: This leads us to think about how this kind of refinement process works practically and what it means for future surveys beyond just Euclid.

More episodes

← Home