A tree-based kernel for densities and its applications in clustering DNase-seq profiles

arXiv:2509.15480 · stat.ME, stat.AP · Submitted 2025-09-18 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "A tree-based kernel for densities and its applications in clustering DNase-seq profiles".

Jane: Here's the gist of the paper. Modeling multiple sampling densities within a hierarchical framework enables borrowing of information across samples <ref:2509.15480#pg2> This paper proposes a nonparametric density kernel flexible enough to capture diverse covariance structures and adaptive to various spatial patterns of TF footprints <ref:2509.15480#pg7> Extensive simulations show that our kernel substantially improves clustering accuracy <ref:2509.15480#pg8>.

Tom: Problem Motivation The motivating problem is clustering chromatin accessibility profiles from high-throughput DNase-seq experiments to detect transcription factor (TF) binding <ref:2509.15480#pg4> TF binding typically produces footprint profiles with spatial patterns, creating long-range dependency across genomic locations <ref:2509.15480#pg4>.

Jane: Existing nonparametric hierarchical models impose restrictive covariance assumptions and cannot accommodate such dependencies, often leading to biologically uninformative clusters <ref:2509.15480#pg4>.

Tom: The count vectors are usually high-dimensional, overdispersed, and exhibit complex correlation structures driven by biological and spatial factors <ref:2509.15480#pg3>.

Paper summary: Tom: So we're diving into this paper, "A tree-based kernel for densities and its applications in clustering DNase-seq profiles." The main idea here is using a density kernel that's flexible enough to capture the complex spatial dependencies found in transcription factor binding footprints.

Jane: That sounds like it’s tackling a really messy problem, because traditional hierarchical models often impose too many restrictive assumptions about how those groups are related. What exactly are they proposing to solve this issue?

Lu: They propose a nonparametric density kernel that is flexible enough to handle diverse covariance structures and is adaptive to various spatial patterns of transcription factor footprints, which they call CorTree > Page seven. This kernel lets you borrow information across samples within a hierarchical framework, which the abstract mentions <ref:2509.15480#pg2>.

Meng: So they're moving away from those restrictive assumptions to something more adaptive, and the simulations show this kernel substantially improves clustering accuracy > Page eight. That’s good news for deployment because it suggests better results than what we currently run on real traffic <ref:2509.15480#pg8>.

Lalam: It seems like they are using a recursive dyadic partitioning of the sample space to define this sampling model > Page seven and then modeling those conditional splitting probabilities with a prior that accounts for spatial correlation > Page eight. That sounds like the engine behind the whole thing.

Tom: Exactly, and to make that concrete, they define a hierarchical tree model with a sparse covariance structure by assigning a multivariate normal distribution on the vectorized splitting probabilities across different locations A epsilon > Page eight. It’s essentially building this tree structure where the correlation is baked into the model itself rather than just learned from every single data point.

Jane: But I have to ask, what's the actual practical benefit for someone who doesn't read papers like this? Why go through all this complexity instead of just using a simpler mixture model?

Lu: The benefit is that this method allows you to capture those long-range dependencies in transcription factor footprints that standard models often miss > Page one <ref:2509.15480#pg1>. A key feature of these density random effects, which acts as kernels in latent variable models, is the functional covariance they induce, which determines how densities are grouped in mixture models <ref:2509.15480#pg1>.

Tom: So we're talking about moving beyond just assuming independent and identically distributed observations and letting the structure of the tree itself define that correlation across different genomic locations rather than having it learned adaptively from every single measurement > Page three. That’s a big conceptual shift for how we model these profiles.

Meng: If this works well, it means we might be able to deploy a system that clusters these biological profiles with better accuracy than what we currently run on real traffic <ref:2509.15480#pg10>. That’s the deployment angle I care about most right now.

Paper summary: Lalam: I think this approach is robust because it leverages the tree structure to manage the complexity inherent in those spatial patterns of transcription factor footprints, making it adaptive to various spatial patterns > Page seven. It seems like a solid way to handle the biological noise we see in DNase-seq data <ref:2509.15480#pg2>.

Jane: So, if I think about what this means for someone just listening, it means they are looking at data like DNA accessibility profiles and this method gives them a statistically sound way to group those features based on their underlying spatial relationships. Where does the performance comparison fit into this picture?

Tom: The performance evaluation shows that when you compare CorTree against methods like DMM and IndTree, CorTree achieves the highest mean and median ARI score > Page five. This ablation study clearly shows that the incremental flexibility in design, moving from DMM to IndTree to CorTree, leads to increasingly better performance <ref:2509.15480#pg5>.

Lu: That's what I was hoping for; seeing a clear progression of improvement based on increasing complexity of the kernel structure > Page eight. It shows that adding flexibility in the tree definition actually helps, as long as it's structured correctly to capture the spatial correlation <ref:2509.15480#pg9>.

Meng: From a deployment view, they also mention that when you have thousands of samples, the posterior distribution for CorTree can actually have many modes > Page five. This means you could get several distinct cluster configurations that all explain the data just as well, which is something we need to consider at scale <ref:2509.15480#pg5>.

Lalam: I think what this means for people who only listen to the show is that they can use this technique not just for finding clusters in genomics, but potentially extend it to other settings like clustering microbiome count profiles > Page five. It shows versatility beyond just DNase-seq data <ref:2509.15480#pg5>.

Tom: Exactly. But there's a caveat here that the choice of the initial Z value is super important; it has to be something scientifically meaningful to guide the clustering process > Page five. If you pick a random starting point, you’re going to get biologically uninformative clusters <ref:2509.15480#pg4>.

Jane: So, if we think about what this means for someone just listening, it means they need to choose a starting point that reflects real biological context, like using the PWM score which indicates transcription factor binding specificity as that initial informative choice > Page five. That gives you a better shot at finding those meaningful clusters <ref:2509.15480#pg5>.

Lu: The paper does acknowledge its limitations by noting that the posterior of CorTree might have many modes when there are thousands of samples, so you still need a scientific guide to pick which configuration you're looking at > Page five. That’s a fair point about interpretation <ref:2509.15480#pg5>.

Paper summary: Meng: That’s fair, and the authors are clear that they can extend this model, but they also admit that using it effectively still requires scientific guidance on how to interpret those modes > Page five. It’s not just a black box thing yet <ref:2509.15480#pg5>.

Tom: So we've gone through the summary of this paper, "A tree-based kernel for densities and its applications in clustering DNase-seq profiles," and it really boils down to this: they developed a density kernel that’s flexible enough to capture complex spatial dependencies inherent in TF footprints > Page one <ref:2509.15480#pg1,A tree-based kernel for densities and its applications in clustering DNase>.

Jane: And the key finding is that when you apply this CorTree method to real DNase-seq data from ENCODE, it outperforms other methods like DMM and IndTree, achieving the highest accuracy metrics > Page five. It really shows how much better this structure is than what we had before <ref:2509.15480#pg5>.

Lu: The implication is that we can now use this structure to find biologically meaningful clusters corresponding to binding events of two common transcription factors, which they found in the ENCODE data > Page five. That's a tangible biological discovery for researchers <ref:2509.15480#pg5>.

Meng: From a deployment view, they also mention that when you have thousands of samples, the posterior distribution for CorTree can actually have many modes, which means you could get several distinct cluster configurations that all explain the data just as well > Page five. That’s a huge consideration for making this production ready <ref:2509.15480#pg5>.

Tom: Exactly. And the choice of the initial Z value is important there; it has to be something scientifically meaningful to guide the clustering process, like using the PWM score which indicates TF binding specificity > Page five. That’s how you steer the model toward useful results <ref:2509.15480#pg5>.

Jane: It’s about choosing a starting point that reflects real biological context, like using the PWM score as that initial informative choice to get meaningful clustering results > Page five. That's a concrete piece of advice for someone trying to apply this model <ref:2509.15480#pg5>.

Lu: The paper acknowledges its limitations by noting that the posterior of CorTree might have many modes when there are thousands of samples, so you still need a scientific guide to pick which configuration you're looking at > Page five. That’s a fair point about interpretation <ref:2509.15480#pg5>.

Tom: That's the core idea of "A tree-based kernel for densities and its applications in clustering DNase-seq profiles." It’s about building a model that understands the complex spatial relationships in biological data, and showing that this structure leads to better clustering results when compared against existing methods <ref:2509.15480#pg5>.

Jane: And the whole picture is that they didn't just invent a new algorithm, they created a way to model those complex dependencies in a way that is statistically sound for high-throughput sequencing data > Page one <ref:2509.15480#pg1>. That’s what this paper means for us <ref:2509.15480#pg1>.

Conclusion: Tom: So we're wrapping up this paper, "A tree-based kernel for densities and its applications in clustering DNase-seq profiles." It’s about taking those complex spatial patterns in transcription factor binding data and fitting them into a model that actually understands those relationships.

Jane: That sounds like they’re trying to build a better map for where genes are bound, not just counting the dots > Page four.

Tom: Right. They use this density kernel that's flexible enough to capture all sorts of covariance structures, which is way more adaptive than the older models we had <ref:2509.15480#pg7>.

Jane: So, what does this actually mean for the research community? What’s the big picture here?

Tom: It means we can get much better clusters from high-throughput sequencing data like DNase-seq, which is way more useful for finding real biological signals <ref:2509.15480#pg5>.

Jane: And they're showing that this CorTree method beats the other methods they compared it against, like DMM and IndTree, on accuracy metrics > Page five.

Tom: Exactly. It’s not just a theoretical improvement; it’s a practical win for researchers using these large genomic datasets <ref:2509.15480#pg8>.

Jane: So while the results are solid, they also point out that when you have thousands of samples, the model can have multiple possible cluster setups > Page five.

Tom: That’s a fair caveat. It means we still need that scientific guidance on how to interpret which configuration is actually the right one for our biological questions <ref:2509.15480#pg3>.

Jane: So the choice of that starting point, like using a PWM score for TF binding specificity as an initial guide, becomes super important > Page five.

Tom: It’s about steering the model toward meaningful results instead of just getting any random grouping <ref:2509.15480#pg4>.

Jane: We’ve seen how this structure works and how good the accuracy is, but next time we talk about this, we need to look at how they can use it for different kinds of data entirely > Page five.

Yuliang Xu, Kaixuan Luo, Li Ma

Department of Statistics & Data Science Institute, University of Chicago · Department of Human Genetics, University of Chicago

stat.ME, stat.AP

Submitted: 2025-09-18

Updated: 2026-11-14

Code: https://github.com/yuliangxu/CorTree

Project page: https://kevinlkx.github.io/footprint_clustering/index

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 78/100

The gist: Modeling multiple sampling densities within a hierarchical framework enables borrowing of information across samples <ref:2509.15480#pg2> This paper proposes a nonparametric density kernel flexible…

Key concepts

Nonparametric Density Kernel
A flexible mathematical function used in the model that can adapt to various covariance structures and spatial patterns found in TF footprints. It allows the model to capture diverse biological correlations without needing strict, predefined assumptions about how data points are distributed.
Recursive Dyadic Partitioning
The core sampling mechanism where the sample space is repeatedly split into two smaller subsets. This process forms a hierarchical tree structure that models how the data samples are related to each other based on their spatial proximity or correlation.
Hierarchical Tree Model with Sparse Covariance
A statistical framework that organizes samples into a tree structure, where the dependencies between nodes (partitions) are modeled using a sparse covariance structure. This allows the model to effectively capture long-range dependencies inherent in TF binding patterns.
Gibbs Sampling with P´olya–Gamma Augmentation
A Bayesian inference technique used to estimate the unknown parameters of the latent variable model. It is employed here to find the posterior distributions of cluster assignments and mixing proportions, enabling robust statistical inference from complex, high-dimensional data.

Terminology

Summary

Modeling multiple sampling densities within a hierarchical framework enables borrowing of information across samples <ref:2509. p. 0

Where the paper says it

A tree-based kernel for densities and its applications in clustering DNase-seq profiles Yuliang Xu 1 , Kaixuan Luo2 , and Li Ma1 Department of Statistics & Data Science Institute, University of Chicago1 Department of Human Genetics, University of Chicago2 Abstract Modeling multiple sampling densities within a hierarchical framework enables borrowing of information across samples. These “density random effects” can act as kernels in latent variable models to represent exchangeable subgroups or clusters.

15480#pg2> This paper proposes a nonparametric density kernel flexible enough to capture diverse covariance structures and adaptive to various spatial patterns of TF footprints <ref:2509. p. 0
Where the paper says it

Existing nonparametric hierarchical models impose restrictive covariance assumptions and cannot accommodate such dependencies, often leading to biologically uninformative clusters. We propose a nonparametric density kernel flexible enough to capture diverse covariance structures and adaptive to various spatial patterns of TF footprints.

15480#pg7> Extensive simulations show that our kernel substantially improves clustering accuracy <ref:2509. p. 0
Where the paper says it

Bayesian inference for latent variable models using this kernel is implemented through Gibbs sampling with P´olya–Gamma augmentation. Extensive simulations show that our kernel substantially improves clustering accuracy.

15480#pg8>.

Problem Motivation

The motivating problem is clustering chromatin accessibility profiles from high-throughput DNase-seq experiments to detect transcription factor (TF) binding <ref:2509.15480#pg4> TF binding typically produces footprint profiles with spatial patterns, creating long-range dependency across genomic locations <ref:2509. p. 0

Where the paper says it

Our motivating problem is clustering chromatin accessibility profiles from high-throughput DNase-seq experiments to detect transcription factor (TF) binding. TF binding typically produces footprint profiles with spatial patterns, creating long-range dependency across genomic locations.

15480#pg4>. Existing nonparametric hierarchical models impose restrictive covariance assumptions and cannot accommodate such dependencies, often leading to biologically uninformative clusters <ref:2509. p. 0
Where the paper says it

TF binding typically produces footprint profiles with spatial patterns, creating long-range dependency across genomic locations. Existing nonparametric hierarchical models impose restrictive covariance assumptions and cannot accommodate such dependencies, often leading to biologically uninformative clusters.

15480#pg4>. The count vectors are usually high-dimensional, overdispersed, and exhibit complex correlation structures driven by biological and spatial factors <ref:2509. p. 1
Where the paper says it

There are several statistical challenges in clustering such count profiles. The count vectors are usually high-dimensional, overdispersed, and exhibit complex correlation structures driven by biological and spatial factors.

15480#pg3>.

Model Specification

The proposed sampling model is based on recursive dyadic partitioning of the sample space <ref:2509.15480#pg7> The conditional splitting probabilities are modeled with a prior that accounts for the spatial correlation <ref:2509. p. 4

Where the paper says it

At each split, the probability mass is divided between two child nodes, parameterized by the conditional splitting probabilities. We model the conditional splitting probabilities with a prior that accounts for the spatial correlation.

15480#pg8>. The hierarchical tree model with sparse covariance structure is defined by assigning a multivariate normal distribution on the vectorized splitting probabilities across different locations Aϵ <ref:2509. p. 6
Where the paper says it

Our main strategy for incorporating spatial correlation is to transform the conditional splitting probability Vϵ0 to the real line by the logit transform ψϵ = logit(Vϵ) (Jara and Hanson, 2011), and assign a multivariate normal distribution on the vectorized ψϵ across different locations Aϵ . The logit transform in Definition 1 enables us to define a tree model with a covariance structure.

15480#pg8>. The fixed-layer correlated tree distribution specifies that for a fixed integer L > 0, ψL = N(µ, Σ) and ψϵ ind∼ N(µϵ, σ2ϵ) for ϵ ∈ E∗∖Ic <ref:2509. p. 6
Where the paper says it

Definition 2 (Fixed-layer Correlated Tree). ψ is said to have a Fixed-layer Correlated Tree distribution if for a fixed integer L > 0, ψL = {ψϵ : ϵ ∈ Ic} ∈ R 2 L−1 , ψL ∼ N(µ, Σ), ψϵ ind∼ N(µϵ , σ2 ϵ ), ϵ ∈ E ∗ \Ic.

15480#pg9>.

Inference and Computation

Bayesian inference for latent variable models using this kernel is implemented through Gibbs sampling with P´olya–Gamma augmentation <ref:2509.15480#pg8> The posterior of ψi,C given Zi = k follows a normal distribution with precision matrix (Λk + omegai,C)−1(Λkµk,C + κi,C) <ref:2509. p. 6

Where the paper says it

Given the sample space X and a dyadic partition sequence {Tl} ∞ l=0, let ψε = logit(Vε). Then we say that X follows a Dyadic Tree distribution with parameter ψ and write X ∼ Tree(ψ) if starting from ϵ = ∅, X(Aϵ0) ∼ Bin X(Aϵ), logit−1 (ψϵ0) for all ϵ.

15480#pg10>. The posterior of the mixing proportion πk is assigned a stick-breaking prior GEM (α) <ref:2509. p. 8
Where the paper says it

, Zi | π ∼ MultiBern(∞, π), π ∼ GEM(α), (2) where GEM(α) is the Griffiths-Engen-McCloskey stick-breaking model that induces the Dirchlet process with concentration parameter α (Sethuraman, 1994) and MultiBern(∞, π) represents the “infinity multinomial-Bernoulli”, i. , the discrete distribution with probability πk on the value k.

15480#pg10>. The sparse covariance prior on the precision matrix omega can be modeled using the Graphical Horseshoe (GHS) Prior. p. 9
Where the paper says it

The sparse covariance prior is a family of distributions that has support on the symmetric and semi-definite matrices and also induces sparsity on the off-diagonal elements. There are also frequentist alternatives of estimating the precision matrix using graphical lasso (Friedman et al.

Performance Evaluation

The comparison among DMM, IndTree, and CorTree can be seen as an ablation study showing that CorTree achieves the highest mean and median ARI. For DNase-seq data from ENCODE, CorTree achieves improved accuracy relative to benchmark methods and aligns with the reference biological truth validated by the ChIP data <ref:2509. p. 3

Where the paper says it

We apply the model to the Encyclopedia of DNA Elements (ENCODE) Consortium (Consortium et al. , 2012) DNase-seq data to cluster TF footprints, where it achieves improved accuracy relative to benchmark methods and aligns with the reference biological truth validated by the ChIP data.

15480#pg5>. The results demonstrate that the incrementally flexible design in Figure S2 from DMM, to IndTree, to CorTree, leads to increasingly better performance. p. 3
Where the paper says it

In this paper, we propose a general-purpose mixture model for clustering count vectors with flexible within-cluster covariance, allowing the correlation pattern within each cluster to adapt to the data through a Bayesian prior on the covariance structure. We develop an efficient sampling algorithm with R package implementation, and demonstrate its advantages in both simulated and real data.

Application and Validation

The application of the model to DNase-seq data from the ENCODE Consortium results in biologically meaningful clusters corresponding to binding events of two common TFs <ref:2509.15480#pg5>. The posterior of CorTree may have many modes, especially when there are thousands of samples: many distinct cluster configurations can provide similarly good explanations of the data. The choice of the initial Z must be scientifically meaningful to guide meaningful clustering. p. 0

Where the paper says it

Extensive simulations show that our kernel substantially improves clustering accuracy. We apply the proposed mixture model to DNase-seq data from the ENCODE project, which results in biologically meaningful clusters corresponding to binding events of two common TFs.

The most informative choice for TF binding detection is the PWM score, which indicates TF binding specificity. p. 0
Where the paper says it

A key feature of these kernels is the (functional) covariance they induce, which determines how densities are grouped in mixture models. Our motivating problem is clustering chromatin accessibility profiles from high-throughput DNase-seq experiments to detect transcription factor (TF) binding.

CorTree can be extended to other settings, such as clustering microbiome count profiles. p. 1
Where the paper says it

For example, in DNase-seq data, the vector records fragment counts across genomic positions around a candidate regulatory site; in microbiome sequencing data, it records counts of DNA or RNA reads assigned to different taxa. There are several statistical challenges in clustering such count profiles.

The phylogenetic tree structure is used to incorporate contextual knowledge of the taxa and apply CorTree to the American Gut Project data in Supplementary Section S4. p. 3
Where the paper says it

An extension to the microbiome data of the proposed method, featuring a phylogenetic tree structure, is applied to the American Gut Project (McDonald et al. , 2018) data in Supplementary Section S4.

1.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive covariance modeling for count data allows a system to capture diverse covariance structures that are adaptive to various spatial patterns of TF footprints, leading to more biologically meaningful clusters.

  2. The system can perform clustering on high-dimensional, overdispersed count vectors, which is useful for detecting complex biological dependencies like those seen in DNase-seq profiles.

  3. The model can incorporate long-range dependencies by using a nonparametric density kernel flexible enough to capture diverse covariance structures, addressing the limitation of existing models that impose restrictive covariance assumptions.

  4. The system can be applied to clustering microbiome count profiles by leveraging phylogenetic tree structures, as demonstrated in the PhyloTree extension for American Gut Project data.

  5. The system can be initialized using domain-informative variables such as PWM scores or rowsums, allowing for better performance when evaluating clustering results against known biological truths like ChIP-seq data.

Abstract

Modeling multiple sampling densities within a hierarchical framework enables borrowing of information across samples. These density random effects can act as kernels in latent variable models to represent exchangeable subgroups or clusters. A key feature of these kernels is the (functional) covariance they induce, which determines how densities are grouped in mixture models. Our motivating problem is clustering chromatin accessibility profiles from high-throughput DNase-seq experiments to detect transcription factor (TF) binding. TF binding typically produces footprint profiles with spatial patterns, creating long-range dependency across genomic locations. Existing nonparametric hierarchical models impose restrictive covariance assumptions and cannot accommodate such dependencies, often leading to biologically uninformative clusters. We propose a nonparametric density kernel flexible enough to capture diverse covariance structures and adaptive to various spatial patterns of TF footprints. The kernel specifies dyadic tree splitting probabilities via a multivariate logit-normal model with a sparse precision matrix. Bayesian inference for latent variable models using this kernel is implemented through Gibbs sampling with Polya-Gamma augmentation. Extensive simulations show that our kernel substantially improves clustering accuracy. We apply the proposed mixture model to DNase-seq data from the ENCODE project, which results in biologically meaningful clusters corresponding to binding events of two common TFs.

Sources

Related papers