A Unified Unsupervised Framework for Genome-Wide Association Studies in Heterogeneous Populations

Genome-wide association studies (GWAS) have greatly advanced the discovery of genetic variants underlying complex traits and diseases. Yet in heterogeneous populations, existing GWAS strategies typically either pool all individuals under an assumption of population homogeneity or perform meta-analysis across predefined subgroups, both of which are limited when latent genetic heterogeneity attenuates subgroup-specific effects and masks true associations or subgroup labels are imprecise. Here we present UCALM, a unified unsupervised framework that infers genetically homogeneous subgroups directly from the data and integrates subgroup-specific GWAS with a novel layered meta-analysis method to capture both shared and subgroup-specific association signals. Through extensive simulations and analyses of large-scale human and livestock cohorts, including the UK Biobank (n about 487, 000) and a heterogeneous pig cohort (n about 85, 000), we demonstrate that UCALM substantially alleviated the mean genomic inflation across 24 UK Biobank traits to 1.17 compared with 1.43 for GLM and 1.37 for LDAK-KVIK, and further revealed 74 loci in the pig cohort that were previously obscured by conventional approaches. Our results establish a robust and broadly applicable strategy for association mapping in structured populations and improve the resolution of genetic signals across diverse species.