ADZE 2.0: rarefaction estimates of allelic richness and private allelic richness for whole-genome data

arXiv:2611.12472 · q-bio.PE · Submitted 2026-11-09 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "ADZE 2.0: rarefaction estimates of allelic richness and private allelic richness for whole-genome data".

Marcus: Here's the gist of the paper. Counting the different versions of a gene, or alleles, found in a population is a simple way to measure its genetic diversity Alleles found in only one population, or shared only by a few populations, also carry information about how populations are related and about their history.

Ines: However, the more individuals are sampled, the more alleles are found, so populations sampled at different sizes cannot be compared directly.

Marcus: The program ADZE corrects for this by estimating how many alleles each population would show if all had been sampled at the same size.

Ines: Here we describe a new version of ADZE that can analyze whole-genome sequencing data from thousands of individuals, and that can also measure diversity region by region along the genome.

Title and authors: Ines: So we’re looking at this paper called "ADZE two point zero: rarefaction estimates of allelic richness and private allelic richness for whole-genome data <ref:2611.12472#pg1,ADZE 2.0: rarefaction estimates of allelic richness and private allelic richness>." It seems like they’re taking the original ADZE program, which was designed for older microsatellite data, and updating it to handle the massive amounts of whole-genome sequencing information we get now.

Marcus: Right. The core idea is that counting alleles in just one population isn't enough because if you sample different numbers of people, you can't really compare those counts directly without a correction factor. This new version aims to estimate what the diversity would look like if everyone had been sampled the same size, which lets us compare populations even when our actual samples are small or uneven.

Yuki: From a population genetics angle, that’s crucial because it lets us look at how alleles are distributed across different groups and sets of populations, not just in isolation. It moves beyond counting what's present to understanding the relationships between those allele counts across a whole species history.

Ines: Exactly. And this version handles whole-genome sequencing data directly in the variant call format, which is a big leap from the old way where they had to read specific genotype files and write everything down in a parameter file, as seen on page two of the paper.

Marcus: That's right. They’re tackling the problem that ADZE one point zero couldn't handle because modern datasets, like the one thousand Genomes Project data, are just too big for the old program to load into memory or process efficiently <ref:2611.12472#pg2>. This version is built to deal with those millions of variant sites and thousands of individuals at once.

Yuki: And they’re not just looking at the whole genome in one go; they can analyze it region by region along the genome, which opens up ways to see if diversity patterns are localized in specific parts of the DNA.

Ines: That regional analysis is a key part of what's new here. They define something called rarefied allelic richness, which is basically how many allele types you expect to find in a small subsample size for a specific population, denoted as alpha sub hat g g.

Marcus: And they also introduce private allelic richness, which counts the alleles that are present in one population and absent from every other set of populations. That’s important for understanding unique genetic markers shared by only one group.

Title and authors: Yuki: It connects this back to the earlier work by Kalinowski on private allelic richness for a single population, showing how they've extended that concept to entire sets of populations, which is a pretty big conceptual step forward in population diversity studies.

Ines: The paper highlights some specific improvements they made to ADZE two point zero <ref:2611.12472#pg1>. They say the main thing is treating blocks of phased single nucleotide polymorphisms as haplotype alleles, instead of just counting individual sites one by one across the genome.

Marcus: That’s a significant methodological change because it allows them to look at haplotypes—those long stretches of linked DNA—as a single unit for analysis, which should give them more meaningful data when dealing with whole-genome sequencing data.

Yuki: When you analyze by haplotype, you are looking at evolutionary blocks, which helps connect the genetic diversity to the history of those specific lineages rather than just random single-point variation.

Ines: Then there’s the computational side. They mention improving memory usage and using a two-pass approach where it reads the data twice to ensure that memory doesn't explode as you add more samples, which is something people in bioinformatics always worry about with large datasets.

Marcus: That improved efficiency is important for practical use. If a program runs out of memory on a big cohort, it’s useless, so making sure it handles that scaling better is essential for real-world genomics work.

Yuki: And they also added these detailed statistical reports, including the mean across all loci used and the variance across loci, and even the standard error of the mean at every sample size level g. That level of detail helps you really understand where your estimates are stable and where they are noisy.

Ines: So, to wrap up on what this means for us: this paper provides a tool that lets us analyze whole-genome sequencing data from thousands of people in a way that accounts for sample size differences, allowing for regional diversity analysis along the genome.

Marcus: It’s about making the comparison of genetic diversity between populations more robust, especially when we aren't sampling every single person available in a population. This ADZE two point zero version is designed to handle the scale of modern genomics data better than its predecessor could, which is a significant practical step for cohort studies <ref:2611.12472#pg1>.

Title and authors: Yuki: For the wider context, this capability helps us map out how genetic diversity has been partitioned across human populations over evolutionary time by giving us these precise estimates of allelic richness and private allelic richness across large genomic regions.

Ines: So, in short, they’ve taken an old tool and made it capable of working on massive whole-genome data while adding features for haplotype analysis and regional statistics that give us a much richer picture of population structure.

Marcus: It’s a solid update for the software because it addresses the limitations of loading huge datasets into memory while still providing those hard statistics we need to make claims about diversity.

Yuki: And I think what this really changes is how we frame our questions about relatedness; instead of just saying population A has more alleles than population B, we can now look at the specific regions where those differences occur and understand the expected variation based on sample size.

Ines: It’s a useful computational biologist tool because it gives us concrete, reproducible estimates for allelic richness that are tailored to how we actually sample things in real research settings.

Marcus: Yeah, so this ADZE two point zero paper is important because it’s making big-scale genetic diversity statistics more accessible and accurate for the kind of whole-genome data we’re generating today <ref:2611.12472#pg1>.

Yuki: It’s a step forward in understanding the deep structure of human genetic variation across populations by providing better ways to partition those alleles, which is really valuable for historical studies.

Ines: Well, that brings us to the end of this look at ADZE two point zero: rarefaction estimates of allelic richness and private allelic richness for whole-genome data <ref:2611.12472#pg1,ADZE 2.0: rarefaction estimates of allelic richness and private allelic richness>. We’ve seen how they improved the tool from an old microsatellite counter to something that handles whole-genome sequencing with haplotype analysis and detailed regional statistics.

Marcus: It’s a practical piece of software that makes it much easier to get those hard diversity numbers out of massive cohorts without losing accuracy due to sample size effects.

Yuki: And for population genetics, it provides a more nuanced way to see how alleles are shared across different groups in a species, especially when we look at the expected richness based on our actual sampling effort.

The paper's summary: Ines: So, we're talking about ADZE two point zero now, and what this paper really boils down to is that they’ve taken that old program and made it work on whole-genome sequencing data from thousands of people without it just crashing or losing its grip on the data.

Marcus: Yeah, so they fixed the main problem with the original ADZE which was how it handled massive amounts of genetic information. The summary says this new version lets you do two things: one, compute those diversity statistics genome-wide or even in specific windows along the DNA sequence.

Yuki: And it’s not just counting single spots anymore; they are treating blocks of phased single nucleotide polymorphisms as haplotype alleles, which gives them a better handle on how genetic variation is structured across the genome.

Ines: That’s key because it moves beyond just looking at random points and starts looking at those longer stretches of DNA that are inherited together. They define something called rarefied allelic richness, which is basically an estimate of how many allele types you expect to find in a small sample size for a specific population.

Marcus: And they also added this idea called private allelic richness, which counts the alleles that are present in one population and are completely missing from every other set of populations you compare it to. That helps us spot truly unique genetic markers.

Yuki: It connects back to what we know about how these allele counts relate across different groups, which is important for understanding the deep history of human populations. It lets us test those hypotheses about relatedness with more statistical power than before.

Ines: The computational part is also tighter now, because they improved the memory management and used a two-pass approach where it reads the data twice to make sure that memory doesn't just explode when you add more samples to your study.

Marcus: So for someone running this on a big cohort, that means they can actually get those statistics without having their computer melt before they finish the job. They also gave us really detailed statistical outputs, like the mean across all the loci used and how much variance there is at every sample size level.

Yuki: It means we can get more rigorous conclusions because we aren't just getting one number; we’re getting a picture of how stable those estimates are for different levels of sampling effort.

Ines: This whole package essentially gives us a way to look at the whole genome, piece by piece, and tell us what the genetic diversity looks like under different hypothetical sampling scenarios.

Marcus: So it’s moving from just counting alleles to estimating expected diversity based on sample size across large datasets. And this is how we can start making more robust claims about population structure using modern sequencing data.

Yuki: What this changes for us is that we can now map out how genetic diversity has been partitioned across human populations over evolutionary time by giving us these precise estimates of allelic richness and private allelic richness across large genomic regions.

Ines: It’s a step forward in understanding the deep structure of human genetic variation by providing better ways to partition those alleles, which is really valuable for historical studies.

Marcus: It’s a practical piece of software that makes it much easier to get those hard diversity numbers out of massive cohorts without losing accuracy due to sample size effects.

Yuki: And for population genetics, it provides a more nuanced way to see how alleles are shared across different groups in a species, especially when we look at the expected variation based on our actual sampling effort.

Ines: Well, that brings us to the end of this look at ADZE two point zero: rarefaction estimates of allelic richness and private allelic richness for whole-genome data. We’ve seen how they improved the tool from an old microsatellite counter to something that handles whole-genome sequencing with haplotype analysis and detailed regional statistics.

Marcus: It’s a practical piece of software that makes it much easier to get those hard diversity numbers out of massive cohorts without losing accuracy due to sample size effects.

Yuki: And for population genetics, it provides a more nuanced way to see how alleles are shared across different groups in a species, especially when we look at the expected variation based on our actual sampling effort.

Ines: So, this ADZE two point zero paper is important because it’s making big-scale genetic diversity statistics more accessible and accurate for the kind of whole-genome data we’re generating today.

Marcus: Yeah, so this ADZE two point zero paper is important because it’s making big-scale genetic diversity statistics more accessible and accurate for the kind of whole-genome data we’re generating today.

Yuki: It’s a step forward in understanding the deep structure of human genetic variation across populations by providing better ways to partition those alleles, which is really valuable for historical studies.

The paper's improvements: Ines: So, we're looking at what ADZE two point zero suggests as improvements to the original version and what that means for how we actually use it on real genomic data. The paper points out a few key changes they made to the core methods of counting allelic richness.

Marcus: They specifically highlight treating those blocks of phased single nucleotide polymorphisms not just as individual sites, but as whole haplotype alleles, which is a big shift in how they analyze the sequence data.

Yuki: That’s important because it means the analysis is looking at evolutionary units rather than just random single-point differences, which should give us a better sense of lineage history.

Ines: They also mentioned improving the way memory is handled, using a two-pass approach where the program reads the data twice to make sure that memory usage doesn't just blow up as you increase your sample size.

Marcus: That sounds practical because it addresses one of the biggest headaches in genomics, which is dealing with massive datasets without running out of RAM. They also introduced much more detailed statistics in their reporting, giving us the mean across all the loci used and a variance calculation at every single sample size level.

Yuki: That level of detail helps us gauge where our diversity estimates are reliable and where they might be noisy because we can see the standard error of the mean for each population or combination.

Ines: The implication here is that researchers get much more confidence in their diversity numbers because they aren't just getting one big number; they are seeing the stability of those estimates across different scales.

Marcus: It means when you try to make claims about population structure using this software, you have better evidence to back up those claims because you know how much variance is inherent in the measurement record.

Yuki: It connects back to what we talked about earlier—the idea that we can get more rigorous conclusions because we’re seeing the standard error for each g value.

Ines: So, these improvements are really about making the tool more robust and giving users a clearer picture of the uncertainty involved in their diversity estimates.

Marcus: This makes it much easier for people working on large cohorts to get those hard diversity numbers out without worrying that they’ve introduced a batch effect just by running out of memory.

Yuki: It’s about making the comparison of genetic diversity between populations more robust, especially when we aren't sampling every single person available in a population.

Ines: It's a solid update for the software because it addresses the limitations of loading huge datasets into memory while still providing those hard statistics we need to make claims about diversity.

Marcus: Yeah, so this ADZE two point zero paper is important because it’s making big-scale genetic diversity statistics more accessible and accurate for the kind of whole-genome data we’re generating today.

Yuki: It’s a step forward in understanding the deep structure of human genetic variation by providing better ways to partition those alleles, which is really valuable for historical studies.

Conclusion: Ines: So we've covered how ADZE two point zero updates its methodology to handle whole genome sequencing data better, focusing on those haplotype blocks and regional analysis for allelic richness.

Marcus: It’s really about making the statistics more robust by adding detailed variance calculations at every sampling size level, which helps us avoid over-claiming diversity.

Yuki: For population genetics, this means we can get a more nuanced view of how alleles are shared across different groups in a species because we aren't just looking at one static count anymore.

Ines: The implication is that researchers using this software will be able to make more defensible claims about population structure when they analyze massive genomic datasets.

Marcus: It’s practical because it addresses the limitations of previous versions by improving memory management and ensuring the calculations stay stable even with huge sample cohorts.

Yuki: We can now use this tool to test historical hypotheses about human relatedness with a much stronger statistical foundation than we had before.

Ines: Overall, ADZE two point zero is a significant step in providing better computational tools for population geneticists working with whole genome data.

Marcus: It’s a solid piece of software that makes it much easier to get those hard diversity numbers out of massive cohorts without losing accuracy due to sample size effects.

Yuki: And for population genetics, it provides a more nuanced way to see how alleles are shared across different groups in a species, especially when we look at the expected variation based on our actual sampling effort.

Department of Biology, The Pennsylvania State University

q-bio.PE

Submitted: 2026-11-09

Updated: 2026-11-11

Comments: 9 pages, 1 figure, 2 tables

Code: https://github.com/szpiech/ADZE

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 77/100

The gist: Counting the different versions of a gene, or alleles, found in a population is a simple way to measure its genetic diversity Alleles found in only one population…

Key concepts

Rarefied Allelic Richness ($α(j)g$)
This estimates the expected number of different allele types you would find in a small subsample (size 'g') for a specific population. It helps researchers understand how many genetic variations are likely present in a smaller group compared to the whole population.
Rarefied Private Allelic Richness ($π(j)g$)
This counts alleles that exist only in one specific population and are completely absent from all other populations being studied. It is used to measure the unique genetic material present in a particular group.
Haplotype Alleles
Instead of counting single SNPs, this method groups consecutive, linked Single Nucleotide Polymorphisms (SNPs) into blocks called haplotypes. This treats these blocks as single alleles for analysis, offering a more nuanced view of genetic variation across the genome.
Genomic Windows/Regions
The software allows users to analyze genetic diversity not just across the entire genome, but in specific sections or 'windows.' This enables researchers to perform regional analyses to see how diversity varies depending on which part of the DNA sequence is examined.

Terminology

Summary

Counting the different versions of a gene, or alleles, found in a population is a simple way to measure its genetic diversity Alleles found in only one population, or shared only by a few populations, also carry information about how populations are related and about their history. p. 0

Where the paper says it

edu; ORCID: 0000-0001-6372-8224 Non-specialist summary Counting the different versions of a gene, or alleles, found in a population is a simple way to measure its genetic diversity. Alleles found in only one population, or shared only by a few populations, also carry information about how populations are related and about their history.

However, the more individuals are sampled, the more alleles are found, so populations sampled at different sizes cannot be compared directly. p. 0
Where the paper says it

Alleles found in only one population, or shared only by a few populations, also carry information about how populations are related and about their history. However, the more individuals are sampled, the more alleles are found, so populations sampled at different sizes cannot be compared directly.

The program ADZE corrects for this by estimating how many alleles each population would show if all had been sampled at the same size. p. 0
Where the paper says it

However, the more individuals are sampled, the more alleles are found, so populations sampled at different sizes cannot be compared directly. The program ADZE corrects for this by estimating how many alleles each population would show if all had been sampled at the same size.

Here we describe a new version of ADZE that can analyze whole-genome sequencing data from thousands of individuals, and that can also measure diversity region by region along the genome. p. 0
Where the paper says it

The program ADZE corrects for this by estimating how many alleles each population would show if all had been sampled at the same size. Here we describe a new version of ADZE that can analyze whole-genome sequencing data from thousands of individuals, and that can also measure diversity region by region along the genome.

Core Methodology

The paper introduces ADZE 2.0, an update to the original ADZE program designed to read VCF files and compute allelic richness statistics genome-wide or in genomic windows. p. 1

Where the paper says it

Here we present an update to ADZE with improved memory usage, computational efficiency, and usability. 0 reads VCF files directly, can compute each statistic genome-wide or in windows along the genome, and can analyze haplotypes by grouping consecutive phased SNPs into alleles.

This new version handles whole-genome sequencing data and can treat blocks of phased SNPs as haplotype alleles. p. 0
Where the paper says it

Here we present ADZE 2. 0, which reads VCF, computes each statistic genome-wide or in genomic windows, and can treat blocks of phased SNPs as haplotype alleles.

The methods define rarefied allelic richness, denoted as αˆ(j)g, which is the expected number of allele types in a subsample of size g for population j. p. 1
Where the paper says it

Summing Pijg over allele types gives the rarefied allelic richness of population j, which is the expected number of allele types in the sub-sample (Hurlbert 1971; Petit et al. 1998; Kalinowski 2004), αˆ (j) g = X I i=1 Pijg.

Furthermore, it defines the rarefied private allelic richness as πˆ(j)g, which counts alleles present in population j and absent from all other populations in a set. p. 1
Where the paper says it

The rarefied private allelic richness is the expected number of such alleles (Kalinowski 2004), πˆ (j) g = X I i=1      Pijg   Y J j ′=1 j ′̸=j Qij′g        , (3) To extend this to sets of populations, write S = {1, 2, . , J} for the full set of populations, and write Ckm, with m = 1, .

Data Input and Processing

ADZE 2.0 is capable of reading VCF files directly, only reading the GT field for input. p. 3

Where the paper says it

sample contributes as many gene copies as its GT field holds, so haploid and diploid records can be in the same file, and an uncalled allele counts as missing under --tolerance in the same way as for structure input. Because a VCF does not contain population information, --samples is required with VCF input.

Because a VCF does not contain population information, the program requires a --samples file containing sample names and grouping names. p. 3
Where the paper says it

Because a VCF does not contain population information, --samples is required with VCF input. It takes a file with two columns, the sample name and the grouping name.

The software supports haplotype analysis by allowing consecutive SNPs to be combined into blocks using options like --hap-snps N or --hap-bp N. p. 3
Where the paper says it

3 Haplotype alleles With phased VCF input, consecutive SNPs can be combined into blocks, and each block is a locus whose alleles are haplotypes of the SNP alleles. --hap-snps N makes each block N consecutive SNPs, and --hap-bp N makes each block a region of N basepairs starting at position 1.

Additionally, statistics can be reported in windows along the genome, allowing for regional analysis with options like --window-bp N or --window-loci N. p. 3
Where the paper says it

MAX_G is determined once from all loci that remain after filtering, and the same value is used for every window. A window in a region with larger sample sizes is therefore not reported at a larger g than the rest of the genome, and all windows along a scan can be compared at the same g.

Performance and Validation

The authors validate ADZE 2.0 by re-analyzing human microsatellite data from the HGDP-CEPH panel, showing that the new version reproduces ADZE 1. p. 3

Where the paper says it

These data are the 783 microsatellite loci of Rosenberg et al. (2005) genotyped in the 952 individuals of the HGDP-CEPH panel with no known first- or second-degree relatives (Rosenberg 2006), which we grouped into the same five geographic regions (Africa, Eurasia, East Asia, Oceania and America).

0 estimates exactly on chromosome 22. p. 0
Where the paper says it

0 estimates for human microsatellite data exactly. On chromosome 22 of the high-coverage 1000 Genomes Project data, private allelic richness for all 325 population pairs took 4.

In a test on chromosome 22 of the high-coverage 1000 Genomes Project data, ADZE 2. p. 1
Where the paper says it

(2008), and the run time and memory use of ADZE 2. 0 on chromosome 22 of the high-coverage 1000 Genomes Project data.

0 achieved run times of up to 5. p. 5
Where the paper says it

5 times on 32 threads), likely because they do less computation per locus, so that more of their time is spent reading the file, which is done on a single thread. Peak memory did not exceed 507 MB in any run.

8 to 7.7 times faster than on one thread when analyzing pairs of populations. p. 4
Where the paper says it

With 32 threads, the analyses that include pairs of populations ran 5. 7 times faster than on one thread, and 5

The memory usage remained manageable, not exceeding 507 MB in any run. p. 5
Where the paper says it

5 times on 32 threads), likely because they do less computation per locus, so that more of their time is spent reading the file, which is done on a single thread. Peak memory did not exceed 507 MB in any run.

Usability and Implementation

ADZE 2.0 is implemented in C++ and requires only a C++17 compiler. p. 5

Where the paper says it

5 Implementation and availability ADZE 2. 0 is written in C++ and requires only a C++17 compiler.

It offers high usability by allowing every parameter to be given on the command line, with options taking precedence over parameter files. p. 2
Where the paper says it

1 Usability Every parameter can be given on the command line, and only the data file is required. Parameter files written for ADZE 1.

Features include reporting statistics genome-wide or in windows, and supporting multi-threading via the --threads N option. p. 0
Where the paper says it

0, which reads VCF, computes each statistic genome-wide or in genomic windows, and can treat blocks of phased SNPs as haplotype alleles. We also implement improved memory handling and multi-threading support.

The software is available as source code on GitHub along with stand-alone binaries for Linux, macOS, and Windows. p. 5
Where the paper says it

com/szpiech/ADZE. Stand-alone binaries for Linux, macOS and Windows are also provided.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved Allelic Richness Estimation for Whole-Genome Data: The ADZE 2.0 system can compute each statistic genome-wide or in genomic windows and treat blocks of phased SNPs as haplotype alleles, allowing for region by region analysis along the genome rather than just locus-by-locus counting.

  2. Enhanced Private Allele Analysis: The system can now analyze haplotypes by grouping consecutive phased SNPs into alleles, which extends the original ADZE approach to a more nuanced understanding of allele distribution across population sets, as defined by the equation for private allelic richness for sets of populations.

  3. Improved Computational Efficiency and Memory Management: ADZE 2.0 addresses limitations of previous versions by improving memory usage and using a two-pass approach where it reads the data twice, ensuring that memory use does not explode with increasing sample size.

  4. Flexible Input Handling: The system can read whole-genome sequencing data directly in the variant call format (VCF) and requires a separate sample file for population information, allowing it to analyze subsets of samples or groups at different levels of grouping.

  5. Detailed Statistical Reporting: The AI system provides comprehensive statistical output including the mean across the L loci used, x̄ = L−1 PLl=1 xl, together with the variance across loci, s2 = (L − 1)−1 PLl=1(xl − x̄)2, and the standard error of the mean, s/√L, at each g and for each population or combination.

Abstract

Allelic richness and private allelic richness describe how alleles are distributed across populations, but both depend strongly on sample size. Rarefaction makes them comparable by computing the expected number of alleles in subsamples of a common size, and ADZE implemented this approach and an extension to alleles shared only by a set of populations. However, ADZE was developed for microsatellite data and cannot read sequencing data in variant call format (VCF). Here we present ADZE 2.0, which reads VCF, computes each statistic genome-wide or in genomic windows, and can treat blocks of phased SNPs as haplotype alleles. We also implement improved memory handling and multi-threading support. ADZE 2.0 reproduces the ADZE 1.0 estimates for human microsatellite data exactly. On chromosome 22 of the high-coverage 1000 Genomes Project data, private allelic richness for all 325 population pairs took 4.4 minutes on 32 threads. ADZE 2.0 is available at https://github.com/szpiech/ADZE.

Related papers