VIDE 2.0: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.
Vera: We're looking at the paper: "VIDE 2.0: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys".
Jocelyn: Here's the gist of the paper. Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions vide 2.0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on vide (P.
Vera: M.
Jocelyn: Sutter et al.
Vera: 2015).
Paper summary: Vera: So, wrapping up on "VIDE two point zero: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys" and its authors Paul M <ref:2611.14742#pg1,VIDE 2.0: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys>. Sutter and Benjamin D. Wandelt Vera. What we’ve seen is that this upgraded pipeline addresses the computational hurdles posed by the millions of galaxies we’re expecting from next generation spectroscopic and photometric surveys Vera.
Jocelyn: The title itself, "Robust, Scalable Void Finding," speaks to how they solved both the reliability problem and the scale problem simultaneously Jocelyn. It’s not just about finding voids; it's about finding them reliably across huge scales Jocelyn.
Subrahmanyan: And the implication is that we can now move forward with Stage IV void cosmology because we have a method that delivers deterministic, ensemble-level covariance estimation from commodity hardware Subrahmanyan. This means the theoretical predictions for how structure forms in the universe can be tested against real, massive galaxy surveys with much greater confidence Subrahmanyan.
Vera: The key is that they maintained that parameter-free spirit from earlier work while adding deterministic boundary handling, which produces bit-for-bit identical catalogs given the same inputs Vera. It gives researchers maximum freedom to use these void catalogs for whatever cosmological questions they have in mind Vera.
Jocelyn: And for listeners who just want the simple picture, it means that when we look at those giant galaxy surveys, we can trust the void measurements because they aren't being skewed by weird boundary artifacts or random setup choices Jocelyn. They’re ready for the real observations coming down the pipe Jocelyn.
Conclusion: Vera: So we're wrapping up on VIDE two point zero, this new void finding pipeline by Sutter and Wandelt Vera. Essentially, they’ve taken an older algorithm and made it much better for handling the massive galaxy counts coming from next-gen surveys Vera.
Jocelyn: It’s all about making sure that when you look at millions of galaxies, you don't end up with a mess because of the survey edges or missing data Jocelyn. They use this Voronoi tessellation and watershed scheme to find the voids in a way that doesn't rely on just dumping tons of extra boundary points into the system Jocelyn.
Subrahmanyan: The authors are pushing for this because they need a reliable way to get those void statistics for Stage IV cosmology, which means understanding how structure actually builds up across cosmic time Subrahmanyan. They claim this deterministic approach gives them those converged cosmic variance covariances that they need Subrahmanyan.
Vera: That sounds like it tackles the main headache of making sure your results aren't just random noise from the setup itself, right? It’s about getting a catalog that’s truly reproducible Vera.
Jocelyn: Exactly. They even have this mechanism to flag voids that might be contaminated by galaxies right near the survey boundary, which lets them keep more of those edge cases in their analysis Jocelyn.
Subrahmanyan: The implication here is that we can now run these complex void studies on standard hardware and get results that match what we’d expect from massive real surveys Subrahmanyan. It moves the bottleneck away from needing supercomputers just to process the initial data Subrahmanyan.
Vera: So, it’s about making those high-end cosmological insights accessible to a wider range of researchers by making the input pipeline extremely robust and fast Vera.
Jocelyn: And that deterministic nature is huge. It means if you run the exact same input catalog and mask twice, you get the exact same void catalog back every single time Jocelyn.
Subrahmanyan: That reproducibility is what lets us trust the ensemble work for things like measuring how big cosmic structures are evolving Subrahmanyan.
Vera: And it’s a big step for testing those models against the real data we're about to see from these new telescopes Vera.
Paul M. Sutter, Benjamin D. Wandelt
Department of Physics and Astronomy, Johns Hopkins University · Department of Applied Mathematics and Statistics, Johns Hopkins University
astro-ph.IM, astro-ph.CO
Submitted: 2026-11-10
Updated: 2026-11-10
Comments: 19 pages, 16 figures. Submitted to ApJS. Code available at www.cosmicvoids.net
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions vide 2.0…
Key concepts
- Voronoi Tessellation
- This is a mathematical technique used to divide a space into regions based on proximity. In this algorithm, it's used without needing extra particles to define the void boundaries. It helps precisely map out the shape and location of voids in the galaxy distribution.
- Watershed Scheme
- The watershed scheme works with the Voronoi tessellation to further delineate distinct regions, specifically identifying where one void ends and another begins. This process is crucial for accurately separating individual voids within a large survey area.
- Deterministic Boundary Handling
- VIDE 2.0 handles survey edges and incompleteness without relying on random boundary particles or sampling pinholes. Instead, it uses a rule-based mechanism that is fully deterministic, meaning the exact same input always produces the exact same output catalog, ensuring reproducibility.
- Stage IV Void Cosmology Readiness
- The upgraded pipeline successfully produced converged statistical measures—like void number function and cross-correlations—from large mock catalogs. This demonstrates that VIDE 2.0 provides the reliable, non-stochastic inputs needed for advanced cosmological studies like Stage IV void cosmology.
Summary
Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions vide 2. p. 0 Wandelt1, 2
1Department of Physics and Astronomy, Johns Hopkins University, Baltimore, MD 21218, USA
2Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA
ABSTRACT
Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions. To answer the computational challenges
posed by these surveys, in this paper we present vide 2. To answer the computational challenges
posed by these surveys, in this paper we present vide 2. 0, a significantly improved and enhanced
void finding algorithm and analysis pipeline based on vide (P. 0, a significantly improved and enhanced
void finding algorithm and analysis pipeline based on vide (P. Sutter et al. Sutter et al. Leveraging the
parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement
a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles. Leveraging the
parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement
a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles. This new method identifies edge galaxies and flags voids contaminated
by their presence. This new method identifies edge galaxies and flags voids contaminated
by their presence. The mechanism is fully deterministic, producing bit-for-bit reproducible catalogs
from identical inputs. An automatic rule for setting the mask resolution renders the entire pipeline
parameter-free. Combined with other performance enhancements to the underlying vide engine, we
find significant (> 100×) speedups and a dramatic reduction in memory usage compared to previous
versions: a 12-million-galaxy DESI Y1-class mock catalog now completes in roughly 15 minutes on a
single laptop, where vide 1.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The paper introduces vide 2.0, an upgraded and enhanced implementation of the original void-finding algorithm. p. 1 To answer the data volume challenges presented by
modern and future surveys, and to provide a robust,
flexible platform for analysis and comparison, in this
paper we present vide 2. 0, an upgraded and enhanced
implementation of our original void-finding algorithm. 0, an upgraded and enhanced
implementation of our original void-finding algorithm. This upgrade includes several performance enhancements, a new tessellation engine, and most crucially,
a parameter-free, deterministic approach to boundaryWhere the paper says it
Where the paper says it
How the training signal is built
The boundary handling procedure in vide 2.0 replaces the previous scheme by using voro++ instead of qhull. p. 4 5
faces of a given particle’s Voronoi tessellation, if that
boundary intersects its local volume. While this does
not remove the ambiguity of the volumes associated with
galaxies near the edge of the survey region, it provides a
zero-cost, exact geometric determination of which galaxies have Voronoi cells clipped by the domain boundary
— information that was fundamentally unavailable in
the previous scheme, where edge proximity could only
be inferred statistically from the random boundary particle distribution. In addition to significant improvements in performance, scalability, and
portability (to be demonstrated below), voro++ provides a clearer relationship between the point set to be
tessellated and the bounding volume containing those
points. In particular, with qhull if a particle is near
a boundary its tessellation is ambiguous. To identify these pathological cases we drive rays down
the pixel centers defining the mask edges. We implement
a memory-efficient linear search to find all the nearest
neighbor galaxies to any point on the line, down to a
resolution threshold determined by the comoving linear
separation of HEALPix pixels at the midpoint of the
survey’s comoving distance range.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The benefits of vide 2.0 are manifest in its efficiency and reliability. p. 1 This engine, which discovered voids via
a parameter-free tessellation scheme to build a density
field from the given tracers, followed by a watershed
transform to join adjacent regions into void basins, powered a raft of void discoveries in both simulations and
observations, including many of the above references. In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes
and sparse density sampling, a key innovation behind
vide compared to zobov was its handling of survey
boundaries, which involved flooding edges with vast
numbers of boundary particles to constrain the tessellation. For
one, it is extremely lightweight. With no additional
particles added, there is no additional cost to running
vide with larger surveys, aside from the expected cost
of dealing with greater numbers of galaxies. With no additional
particles added, there is no additional cost to running
vide with larger surveys, aside from the expected cost
of dealing with greater numbers of galaxies. Secondly, this method is fully deterministic. Secondly, this method is fully deterministic. 0 had two sources of stochasticity: the random placement of boundary particles to flood the survey edges,
and the variable sampling of pinholes when the mask resolution was insufficient.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Impacts on void catalogs
The new approach allows for a more precise classification of voids. A void is classified as contaminated if it includes even a single edge-flagged galaxy, which is exact and independent of random realizations. p. 9 Unlike the geometric heuristic employed
in vide 1. 0, which compared a void’s maximum extent to the distance to the nearest stochastically-placed
boundary particle, this classification is exact: a void is
contaminated if and only if it contains an edge galaxy,
with no dependence on random realizations. 0, which compared a void’s maximum extent to the distance to the nearest stochastically-placed
boundary particle, this classification is exact: a void is
contaminated if and only if it contains an edge galaxy,
with no dependence on random realizations. This precision allows vide 2. This precision allows vide 2. 0 to retain significantly more voids
near survey boundaries while maintaining rigorous contamination control. The per-tracer edge-flag field now carries a small integer recording which of three independent boundarydetection mechanisms (voro++’s native wall detection, the redshift-cap ray-march, the angular-mask raymarch) flagged each tracer, allowing downstream users
to retain or exclude voids by edge type. A lightweight
process-tree poller records per-stage peak resident memory in the run log alongside wall-clock timing, and a
pytest-based regression harness compares end-to-end
pipeline outputs against locked hash manifests on every commit.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Demonstration of Stage IV readiness
The upgraded pipeline was exercised on two large mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the void ellipticity distribution — key ingredients for Stage IV void cosmology. The lack of stochasticity in vide 2. p. 9 Despite the artificial high densities for any flagged galaxies,
many underdense regions approach the survey boundaries, where they are truncated by the lack of data. Since voids have many uses across many analyses,
vide 2.Where the paper says it
Conclusion
In conclusion, vide 2.0 is now well suited to handle large numbers of galaxies in mock and real observations, and ready to tackle the challenges of future campaigns. p. 0 We exercise the upgraded pipeline on two large
mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged
cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the
void ellipticity distribution — key ingredients for Stage IV void cosmology. 0 is now well suited
to handle large numbers of galaxies in mock and real observations, and ready to tackle the challenges
of future campaigns. The resulting void catalog was thus shaped
not only by the real galaxy distribution, but also by the
particular random realization of boundary particles. Since vide was originally developed in the context of the first void Alcock–Paczy´nski test, during
post-processing the default void catalog produced by
vide implemented additional filters, such as eliminating any void whose maximum extent (as defined relative to its volume-weighted center) was greater than the 0 places no random particles, and given
identical mask, nside, and tracer positions it identifies the same edge galaxies (Algorithm 1) before passing
them to the (deterministic) voro++ tessellator. The
pipeline is therefore reproducible at the bit level: rerunning on the same input yields SHA-256 identical catalog files, and the single-domain and KD-tree-split configurations agree bit-for-bit at the recommended buffer
width. In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes
and sparse density sampling, a key innovation behind
vide compared to zobov was its handling of survey
boundaries, which involved flooding edges with vast
numbers of boundary particles to constrain the tessellation. This approach preserved (as much as possible)
the parameter-free spirit of void analysis, and it became
the de facto method employed to produce other public
catalogs (e. Wandelt1, 2
1Department of Physics and Astronomy, Johns Hopkins University, Baltimore, MD 21218, USA
2Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA
ABSTRACT
Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions. To answer the computational challenges
posed by these surveys, in this paper we present vide 2. The resulting void catalog was thus shaped
not only by the real galaxy distribution, but also by the
particular random realization of boundary particles. Since vide was originally developed in the context of the first void Alcock–Paczy´nski test, during
post-processing the default void catalog produced by
vide implemented additional filters, such as eliminating any void whose maximum extent (as defined relative to its volume-weighted center) was greater than the The classification is deterministic, depending only on
the per-tracer edge flags and the macrocenter geometry. The pipeline produces four standard catalogs:
• all: every watershed basin. In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes
and sparse density sampling, a key innovation behind
vide compared to zobov was its handling of survey
boundaries, which involved flooding edges with vast
numbers of boundary particles to constrain the tessellation. This approach preserved (as much as possible)
the parameter-free spirit of void analysis, and it became
the de facto method employed to produce other public
catalogs (e. In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes
and sparse density sampling, a key innovation behind
vide compared to zobov was its handling of survey
boundaries, which involved flooding edges with vast
numbers of boundary particles to constrain the tessellation. This approach preserved (as much as possible)
the parameter-free spirit of void analysis, and it became
the de facto method employed to produce other public
catalogs (e. 0 could not run a catalog of even 106 galaxies on the same hardware. We
are also able to identify voids existing closer to survey boundaries via deterministic per-void contamination flags, maximizing their statistical potential. Curtis et al. 2025), the Alcock–Paczy´nski test (G. We
are also able to identify voids existing closer to survey boundaries via deterministic per-void contamination flags, maximizing their statistical potential. We exercise the upgraded pipeline on two large
mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged
cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the
void ellipticity distribution — key ingredients for Stage IV void cosmology. 0 could not
run a catalog of even 106 galaxies on this hardware: it
crashed from memory exhaustion after ∼ 30 minutes,
well before the watershed step. CMASS-class samples
(∼ 106 galaxies) and DESI Y1 ELG-class samples (∼
107
) are therefore inaccessible to vide 1.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Automatic nside resolution selection improves pipeline robustness by addressing failure modes A (under-resolution) and B (over-segmentation) by matching volume-weighted mean linear pixel resolution to mean tracer separation, yielding NSIDEauto = min(NSIDEmask, NSIDEdata).
-
Deterministic boundary handling removes stochasticity by replacing random boundary particle flooding with a deterministic ray-based search for nearest galaxies along rays defined by mask edges and redshift caps, ensuring that
vide 2.0 distinguishes among three independent sources of edge contamination.
-
Enhanced catalog outputs provide granular provenance information by storing
per-galaxy edge flags indicating whether each tracer was flagged and, if so, the source of contamination (domain boundary, redshift cap, or angular mask edge),
allowing users toselectively filter based on the source of contamination.
-
Performance scaling is significantly improved with voro++ replacing qhull and utilizing a KD-tree for nearest-neighbor search in the ray-based flagging step, resulting in wall time scaling near-linearly with N and substantial speedups over vide 1.0.
-
Statistical rigor is enhanced by providing four distinct catalog products: all, clean, central, and tree; the clean catalog excludes voids contaminated by edge galaxies (
a void is contaminated if any of its member galaxies is edgeflagged
), ensuring thatclean voids free from edge effects
are available for analyses likevoid-galaxy cross-correlations.
-
Ensemble analysis capabilities are strengthened because vide 2.0 catalogs are fully deterministic, meaning
re-running on the same input yields SHA-256 identical catalog files,
allowing researchers to separate true cosmic variance from finder-specific noise across realizations.
Abstract
Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions. To answer the computational challenges posed by these surveys, in this paper we present VIDE 2.0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on VIDE. Leveraging the parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles. This new method identifies edge galaxies and flags voids contaminated by their presence. The mechanism is fully deterministic, producing bit-for-bit reproducible catalogs from identical inputs. An automatic rule for setting the mask resolution renders the entire pipeline parameter-free. Combined with other performance enhancements to the underlying VIDE engine, we find significant (>100 times) speedups and a dramatic reduction in memory usage compared to previous versions: a 12-million-galaxy DESI Y1-class mock catalog now completes in roughly 15 minutes on a single laptop, where VIDE 1.0 could not run a catalog of even 10 6 galaxies on the same hardware. We are also able to identify voids existing closer to survey boundaries via deterministic per-void contamination flags, maximizing their statistical potential. We exercise the upgraded pipeline on two large mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the void ellipticity distribution -- key ingredients for Stage IV void cosmology. VIDE 2.0 is now well suited to handle large numbers of galaxies in mock and real observations, and ready to tackle the challenges of future campaigns.
Sources
- Data Release 1 of the Dark Energy Spectroscopic Instrument
- Why Cosmic Voids Matter: Pristine Evolution
- From Voids to Clusters: Mergers and Evolutionary Pathways of Star-Forming and Quenched Low-Mass Galaxies
- An Historical View: The Discovery of Voids in the Galaxy Distribution
- Exploring Cosmological Constraints of the Void-Lensing Cross-Correlation in the CSST Photometric Survey
- AVISM: Algorithm for Void Identification in coSMology
Related papers
- On-sky demonstration of a vector Zernike wavefront sensor in a cascaded adaptive optics system
- Euclid Data Release 1 (DR1): Calibration strategy and observations
- The General Antiparticle Spectrometer (GAPS) Antarctic Balloon Payload
- Detection and excision of radio-frequency interference in radio arrays
- Unified matched-filter and excess-power pipeline for gravitational-wave detector characterization
- Ushering in sub-arcsecond angular resolution observatories in the far-infrared: The case for a double-Fourier interferometer