VIDE 2.0: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys

arXiv:2611.14742 · astro-ph.IM, astro-ph.CO · Submitted 2026-11-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Astrophysics. The day in astrophysics: observations, models and what the data forces.

Vera: We're looking at the paper: "VIDE 2.0: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys".

Jocelyn: Here's the gist of the paper. Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions vide 2.0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on vide (P.

Vera: M.

Jocelyn: Sutter et al.

Vera: 2015).

Paper summary: Vera: So, wrapping up on "VIDE two point zero: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys" and its authors Paul M <ref:2611.14742#pg1,VIDE 2.0: Robust, Scalable Void Finding for Extremely Large Galaxy Surveys>. Sutter and Benjamin D. Wandelt Vera. What we’ve seen is that this upgraded pipeline addresses the computational hurdles posed by the millions of galaxies we’re expecting from next generation spectroscopic and photometric surveys Vera.

Jocelyn: The title itself, "Robust, Scalable Void Finding," speaks to how they solved both the reliability problem and the scale problem simultaneously Jocelyn. It’s not just about finding voids; it's about finding them reliably across huge scales Jocelyn.

Subrahmanyan: And the implication is that we can now move forward with Stage IV void cosmology because we have a method that delivers deterministic, ensemble-level covariance estimation from commodity hardware Subrahmanyan. This means the theoretical predictions for how structure forms in the universe can be tested against real, massive galaxy surveys with much greater confidence Subrahmanyan.

Vera: The key is that they maintained that parameter-free spirit from earlier work while adding deterministic boundary handling, which produces bit-for-bit identical catalogs given the same inputs Vera. It gives researchers maximum freedom to use these void catalogs for whatever cosmological questions they have in mind Vera.

Jocelyn: And for listeners who just want the simple picture, it means that when we look at those giant galaxy surveys, we can trust the void measurements because they aren't being skewed by weird boundary artifacts or random setup choices Jocelyn. They’re ready for the real observations coming down the pipe Jocelyn.

Conclusion: Vera: So we're wrapping up on VIDE two point zero, this new void finding pipeline by Sutter and Wandelt Vera. Essentially, they’ve taken an older algorithm and made it much better for handling the massive galaxy counts coming from next-gen surveys Vera.

Jocelyn: It’s all about making sure that when you look at millions of galaxies, you don't end up with a mess because of the survey edges or missing data Jocelyn. They use this Voronoi tessellation and watershed scheme to find the voids in a way that doesn't rely on just dumping tons of extra boundary points into the system Jocelyn.

Subrahmanyan: The authors are pushing for this because they need a reliable way to get those void statistics for Stage IV cosmology, which means understanding how structure actually builds up across cosmic time Subrahmanyan. They claim this deterministic approach gives them those converged cosmic variance covariances that they need Subrahmanyan.

Vera: That sounds like it tackles the main headache of making sure your results aren't just random noise from the setup itself, right? It’s about getting a catalog that’s truly reproducible Vera.

Jocelyn: Exactly. They even have this mechanism to flag voids that might be contaminated by galaxies right near the survey boundary, which lets them keep more of those edge cases in their analysis Jocelyn.

Subrahmanyan: The implication here is that we can now run these complex void studies on standard hardware and get results that match what we’d expect from massive real surveys Subrahmanyan. It moves the bottleneck away from needing supercomputers just to process the initial data Subrahmanyan.

Vera: So, it’s about making those high-end cosmological insights accessible to a wider range of researchers by making the input pipeline extremely robust and fast Vera.

Jocelyn: And that deterministic nature is huge. It means if you run the exact same input catalog and mask twice, you get the exact same void catalog back every single time Jocelyn.

Subrahmanyan: That reproducibility is what lets us trust the ensemble work for things like measuring how big cosmic structures are evolving Subrahmanyan.

Vera: And it’s a big step for testing those models against the real data we're about to see from these new telescopes Vera.

Paul M. Sutter, Benjamin D. Wandelt

Department of Physics and Astronomy, Johns Hopkins University · Department of Applied Mathematics and Statistics, Johns Hopkins University

astro-ph.IM, astro-ph.CO

Submitted: 2026-11-10

Updated: 2026-11-10

Comments: 19 pages, 16 figures. Submitted to ApJS. Code available at www.cosmicvoids.net

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions vide 2.0…

Key concepts

Voronoi Tessellation
This is a mathematical technique used to divide a space into regions based on proximity. In this algorithm, it's used without needing extra particles to define the void boundaries. It helps precisely map out the shape and location of voids in the galaxy distribution.
Watershed Scheme
The watershed scheme works with the Voronoi tessellation to further delineate distinct regions, specifically identifying where one void ends and another begins. This process is crucial for accurately separating individual voids within a large survey area.
Deterministic Boundary Handling
VIDE 2.0 handles survey edges and incompleteness without relying on random boundary particles or sampling pinholes. Instead, it uses a rule-based mechanism that is fully deterministic, meaning the exact same input always produces the exact same output catalog, ensuring reproducibility.
Stage IV Void Cosmology Readiness
The upgraded pipeline successfully produced converged statistical measures—like void number function and cross-correlations—from large mock catalogs. This demonstrates that VIDE 2.0 provides the reliable, non-stochastic inputs needed for advanced cosmological studies like Stage IV void cosmology.

Summary

Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions vide 2. p. 0

Where the paper says it

Wandelt1, 2 1Department of Physics and Astronomy, Johns Hopkins University, Baltimore, MD 21218, USA 2Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA ABSTRACT Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions. To answer the computational challenges posed by these surveys, in this paper we present vide 2.

0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on vide (P. p. 0
Where the paper says it

To answer the computational challenges posed by these surveys, in this paper we present vide 2. 0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on vide (P.

M. Sutter et al. p. 0
Where the paper says it

0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on vide (P. Sutter et al.

2015). Leveraging the parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles. p. 0
Where the paper says it

Sutter et al. Leveraging the parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles.

This new method identifies edge galaxies and flags voids contaminated by their presence. p. 0
Where the paper says it

Leveraging the parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles. This new method identifies edge galaxies and flags voids contaminated by their presence.

The mechanism is fully deterministic, producing bit-for-bit reproducible catalogs from identical inputs. p. 0
Where the paper says it

This new method identifies edge galaxies and flags voids contaminated by their presence. The mechanism is fully deterministic, producing bit-for-bit reproducible catalogs from identical inputs.

Combined with other performance enhancements to the underlying vide engine, we find significant (> 100×) speedups and a dramatic reduction in memory usage compared to previous versions. p. 0
Where the paper says it

An automatic rule for setting the mask resolution renders the entire pipeline parameter-free. Combined with other performance enhancements to the underlying vide engine, we find significant (> 100×) speedups and a dramatic reduction in memory usage compared to previous versions: a 12-million-galaxy DESI Y1-class mock catalog now completes in roughly 15 minutes on a single laptop, where vide 1.

Pruning redundant reasoning tokens

The paper introduces vide 2.0, an upgraded and enhanced implementation of the original void-finding algorithm. p. 1

Where the paper says it

To answer the data volume challenges presented by modern and future surveys, and to provide a robust, flexible platform for analysis and comparison, in this paper we present vide 2. 0, an upgraded and enhanced implementation of our original void-finding algorithm.

This upgrade includes several performance enhancements, a new tessellation engine, and most crucially, a parameter-free, deterministic approach to boundary handling that preserves the scale-agnostic, purely morphological void definition in any survey configuration. p. 1
Where the paper says it

0, an upgraded and enhanced implementation of our original void-finding algorithm. This upgrade includes several performance enhancements, a new tessellation engine, and most crucially, a parameter-free, deterministic approach to boundary

How the training signal is built

The boundary handling procedure in vide 2.0 replaces the previous scheme by using voro++ instead of qhull. p. 4

Where the paper says it

5 faces of a given particle’s Voronoi tessellation, if that boundary intersects its local volume. While this does not remove the ambiguity of the volumes associated with galaxies near the edge of the survey region, it provides a zero-cost, exact geometric determination of which galaxies have Voronoi cells clipped by the domain boundary — information that was fundamentally unavailable in the previous scheme, where edge proximity could only be inferred statistically from the random boundary particle distribution.

This provides a clearer relationship between the point set to be tessellated and the bounding volume, explicitly including boundaries as faces of a given particle’s Voronoi tessellation if that boundary intersects its local volume. p. 3
Where the paper says it

In addition to significant improvements in performance, scalability, and portability (to be demonstrated below), voro++ provides a clearer relationship between the point set to be tessellated and the bounding volume containing those points. In particular, with qhull if a particle is near a boundary its tessellation is ambiguous.

The new approach identifies edge galaxies by driving rays down the pixel centers defining the mask edges and querying for nearest neighbors to flag them. p. 4
Where the paper says it

To identify these pathological cases we drive rays down the pixel centers defining the mask edges. We implement a memory-efficient linear search to find all the nearest neighbor galaxies to any point on the line, down to a resolution threshold determined by the comoving linear separation of HEALPix pixels at the midpoint of the survey’s comoving distance range.

Where the gains come from

The benefits of vide 2.0 are manifest in its efficiency and reliability. p. 1

Where the paper says it

This engine, which discovered voids via a parameter-free tessellation scheme to build a density field from the given tracers, followed by a watershed transform to join adjacent regions into void basins, powered a raft of void discoveries in both simulations and observations, including many of the above references. In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes and sparse density sampling, a key innovation behind vide compared to zobov was its handling of survey boundaries, which involved flooding edges with vast numbers of boundary particles to constrain the tessellation.

Specifically, it is extremely lightweight with no additional particles added, meaning there is no additional cost to running vide with larger surveys aside from the expected cost of dealing with greater numbers of galaxies. p. 5
Where the paper says it

For one, it is extremely lightweight. With no additional particles added, there is no additional cost to running vide with larger surveys, aside from the expected cost of dealing with greater numbers of galaxies.

The method is fully deterministic, unlike vide 1. p. 5
Where the paper says it

With no additional particles added, there is no additional cost to running vide with larger surveys, aside from the expected cost of dealing with greater numbers of galaxies. Secondly, this method is fully deterministic.

0 which had two sources of stochasticity: the random placement of boundary particles and variable sampling of pinholes. p. 5
Where the paper says it

Secondly, this method is fully deterministic. 0 had two sources of stochasticity: the random placement of boundary particles to flood the survey edges, and the variable sampling of pinholes when the mask resolution was insufficient.

Impacts on void catalogs

The new approach allows for a more precise classification of voids. A void is classified as contaminated if it includes even a single edge-flagged galaxy, which is exact and independent of random realizations. p. 9

Where the paper says it

Unlike the geometric heuristic employed in vide 1. 0, which compared a void’s maximum extent to the distance to the nearest stochastically-placed boundary particle, this classification is exact: a void is contaminated if and only if it contains an edge galaxy, with no dependence on random realizations.

This precision allows vide 2. p. 9
Where the paper says it

0, which compared a void’s maximum extent to the distance to the nearest stochastically-placed boundary particle, this classification is exact: a void is contaminated if and only if it contains an edge galaxy, with no dependence on random realizations. This precision allows vide 2.

0 to retain significantly more voids near survey boundaries while maintaining rigorous contamination control. p. 9
Where the paper says it

This precision allows vide 2. 0 to retain significantly more voids near survey boundaries while maintaining rigorous contamination control.

The pipeline produces four standard catalogs: all, clean, central, and tree, allowing users to select voids based on their desired properties. p. 8
Where the paper says it

The per-tracer edge-flag field now carries a small integer recording which of three independent boundarydetection mechanisms (voro++’s native wall detection, the redshift-cap ray-march, the angular-mask raymarch) flagged each tracer, allowing downstream users to retain or exclude voids by edge type. A lightweight process-tree poller records per-stage peak resident memory in the run log alongside wall-clock timing, and a pytest-based regression harness compares end-to-end pipeline outputs against locked hash manifests on every commit.

Demonstration of Stage IV readiness

The upgraded pipeline was exercised on two large mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the void ellipticity distribution — key ingredients for Stage IV void cosmology. The lack of stochasticity in vide 2. p. 9

Where the paper says it

Despite the artificial high densities for any flagged galaxies, many underdense regions approach the survey boundaries, where they are truncated by the lack of data. Since voids have many uses across many analyses, vide 2.

0 means that the scatter across realizations comes entirely from the realizations themselves, which is crucial for ensemble work. The analysis shows that the deterministic void finder yields ensemble covariances that converge, and it delivers the multivariate per-void inputs that void-based cosmological studies can adopt without reprocessing.

Conclusion

In conclusion, vide 2.0 is now well suited to handle large numbers of galaxies in mock and real observations, and ready to tackle the challenges of future campaigns. p. 0

Where the paper says it

We exercise the upgraded pipeline on two large mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the void ellipticity distribution — key ingredients for Stage IV void cosmology. 0 is now well suited to handle large numbers of galaxies in mock and real observations, and ready to tackle the challenges of future campaigns.

Void catalogs produced by vide 2. p. 2
Where the paper says it

The resulting void catalog was thus shaped not only by the real galaxy distribution, but also by the particular random realization of boundary particles. Since vide was originally developed in the context of the first void Alcock–Paczy´nski test, during post-processing the default void catalog produced by vide implemented additional filters, such as eliminating any void whose maximum extent (as defined relative to its volume-weighted center) was greater than the

0 are also fully deterministic: given the same input galaxy catalog and survey mask, the output is bit-forbit identical across runs. p. 6
Where the paper says it

0 places no random particles, and given identical mask, nside, and tracer positions it identifies the same edge galaxies (Algorithm 1) before passing them to the (deterministic) voro++ tessellator. The pipeline is therefore reproducible at the bit level: rerunning on the same input yields SHA-256 identical catalog files, and the single-domain and KD-tree-split configurations agree bit-for-bit at the recommended buffer width.

The public code repository details many other updates, fixes, and quality-of-life improvements. p. 1
Where the paper says it

In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes and sparse density sampling, a key innovation behind vide compared to zobov was its handling of survey boundaries, which involved flooding edges with vast numbers of boundary particles to constrain the tessellation. This approach preserved (as much as possible) the parameter-free spirit of void analysis, and it became the de facto method employed to produce other public catalogs (e.

The goal of vide 2. p. 0
Where the paper says it

Wandelt1, 2 1Department of Physics and Astronomy, Johns Hopkins University, Baltimore, MD 21218, USA 2Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA ABSTRACT Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions. To answer the computational challenges posed by these surveys, in this paper we present vide 2.

0 is to allow the user maximum freedom to use the produced void catalogs for their desired purposes. p. 2
Where the paper says it

The resulting void catalog was thus shaped not only by the real galaxy distribution, but also by the particular random realization of boundary particles. Since vide was originally developed in the context of the first void Alcock–Paczy´nski test, during post-processing the default void catalog produced by vide implemented additional filters, such as eliminating any void whose maximum extent (as defined relative to its volume-weighted center) was greater than the

The pipeline produces four standard catalogs: all, clean, central, and tree. p. 9
Where the paper says it

The classification is deterministic, depending only on the per-tracer edge flags and the macrocenter geometry. The pipeline produces four standard catalogs: • all: every watershed basin.

The visualization routines used to create the new slice plots in this paper are made available in the analysis package. All analysis routines are fully backwards-compatible with void catalogs produced by vide 1. p. 1
Where the paper says it

In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes and sparse density sampling, a key innovation behind vide compared to zobov was its handling of survey boundaries, which involved flooding edges with vast numbers of boundary particles to constrain the tessellation. This approach preserved (as much as possible) the parameter-free spirit of void analysis, and it became the de facto method employed to produce other public catalogs (e.

0. The first major upgrade to vide in over a decade maintains the parameter-free spirit that first animated the conversion of vide into a fully-featured void analysis suite. p. 1
Where the paper says it

In addition to performance and reliability improvements, alongside extensions to handle arbitrary volumes and sparse density sampling, a key innovation behind vide compared to zobov was its handling of survey boundaries, which involved flooding edges with vast numbers of boundary particles to constrain the tessellation. This approach preserved (as much as possible) the parameter-free spirit of void analysis, and it became the de facto method employed to produce other public catalogs (e.

The paper concludes by showing that everything leading up to those measurements, from deterministic catalog production to ensemble-level covariance estimation, can be done on commodity hardware. p. 0
Where the paper says it

0 could not run a catalog of even 106 galaxies on the same hardware. We are also able to identify voids existing closer to survey boundaries via deterministic per-void contamination flags, maximizing their statistical potential.

The Alcock–Paczy´nski test remains a prime target for capturing cosmological parameter information from cosmic voids in Stage IV surveys. p. 0
Where the paper says it

Curtis et al. 2025), the Alcock–Paczy´nski test (G.

The contribution of this section is to show that our deterministic void finder yields ensemble covariances that converge, and that it delivers. p. 0
Where the paper says it

We are also able to identify voids existing closer to survey boundaries via deterministic per-void contamination flags, maximizing their statistical potential. We exercise the upgraded pipeline on two large mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the void ellipticity distribution — key ingredients for Stage IV void cosmology.

The full ensemble work was completed on commodity laptop hardware under modest process parallelism, with no parameter retuning between the BOSS-class and DESI-class samples. p. 7
Where the paper says it

0 could not run a catalog of even 106 galaxies on this hardware: it crashed from memory exhaustion after ∼ 30 minutes, well before the watershed step. CMASS-class samples (∼ 106 galaxies) and DESI Y1 ELG-class samples (∼ 107 ) are therefore inaccessible to vide 1.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Automatic nside resolution selection improves pipeline robustness by addressing failure modes A (under-resolution) and B (over-segmentation) by matching volume-weighted mean linear pixel resolution to mean tracer separation, yielding NSIDEauto = min(NSIDEmask, NSIDEdata).

  2. Deterministic boundary handling removes stochasticity by replacing random boundary particle flooding with a deterministic ray-based search for nearest galaxies along rays defined by mask edges and redshift caps, ensuring that vide 2.0 distinguishes among three independent sources of edge contamination.

  3. Enhanced catalog outputs provide granular provenance information by storing per-galaxy edge flags indicating whether each tracer was flagged and, if so, the source of contamination (domain boundary, redshift cap, or angular mask edge), allowing users to selectively filter based on the source of contamination.

  4. Performance scaling is significantly improved with voro++ replacing qhull and utilizing a KD-tree for nearest-neighbor search in the ray-based flagging step, resulting in wall time scaling near-linearly with N and substantial speedups over vide 1.0.

  5. Statistical rigor is enhanced by providing four distinct catalog products: all, clean, central, and tree; the clean catalog excludes voids contaminated by edge galaxies (a void is contaminated if any of its member galaxies is edgeflagged), ensuring that clean voids free from edge effects are available for analyses like void-galaxy cross-correlations.

  6. Ensemble analysis capabilities are strengthened because vide 2.0 catalogs are fully deterministic, meaning re-running on the same input yields SHA-256 identical catalog files, allowing researchers to separate true cosmic variance from finder-specific noise across realizations.

Abstract

Current and future spectroscopic and photometric galaxy surveys will deliver unprecedented numbers of galaxies, with expected counts in the tens of millions. To answer the computational challenges posed by these surveys, in this paper we present VIDE 2.0, a significantly improved and enhanced void finding algorithm and analysis pipeline based on VIDE. Leveraging the parameter-free identification of voids via a Voronoi tessellation and watershed scheme, we implement a robust mechanism for handling survey boundaries and incompleteness without relying on vast numbers of boundary particles. This new method identifies edge galaxies and flags voids contaminated by their presence. The mechanism is fully deterministic, producing bit-for-bit reproducible catalogs from identical inputs. An automatic rule for setting the mask resolution renders the entire pipeline parameter-free. Combined with other performance enhancements to the underlying VIDE engine, we find significant (>100 times) speedups and a dramatic reduction in memory usage compared to previous versions: a 12-million-galaxy DESI Y1-class mock catalog now completes in roughly 15 minutes on a single laptop, where VIDE 1.0 could not run a catalog of even 10 6 galaxies on the same hardware. We are also able to identify voids existing closer to survey boundaries via deterministic per-void contamination flags, maximizing their statistical potential. We exercise the upgraded pipeline on two large mock suites (MultiDark-Patchy DR12 and AbacusSummit DESI Y1 ELG), demonstrating converged cosmic-variance covariances for the void number function, the void-galaxy cross-correlation, and the void ellipticity distribution -- key ingredients for Stage IV void cosmology. VIDE 2.0 is now well suited to handle large numbers of galaxies in mock and real observations, and ready to tackle the challenges of future campaigns.

Sources

Related papers