A Geometric Perspective for Marginal R2: Estimation and Inference in Generalized Linear Mixed Models

arXiv:2611.14830 · stat.ME · Submitted 2026-11-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "A Geometric Perspective for Marginal R2".

Jane: Here's the gist of the paper. Generalized linear mixed models (GLMMs) are widely used across scientific disciplines, yet quantifying response variation explained by auxiliary information remains challenging for non-Gaussian outcomes and hierarchical dependence The proposed marginal quasi-R2 method extends the geometric interpretation of classical R2 to non-Gaussian responses through a two-step process involving marginalization and a Quasi-GLM formulation Core Methodology The construction begins by marginalizing the joint distribution of responses and random effects onto the marginal response space, retaining latent heterogeneity in the marginal moment structure Second, a Quasi-GLM formulation equips this marginal space with a locally defined information metric, where orthogonal projection yields a Pythagorean decomposition of explained and unexplained variation This process defines a fixed-design population estimand and its corresponding sample estimator Population Estimands The paper defines the population generalized marginal R2 as the ratio of explained to total working-scale variance, denoted as R2 = SSRw/SSTw This is decomposed into a deterministic signal component F0 and a noise component N0, such that ρ2M = FM + NM The signal F0 represents the total working-weighted variance of the fixed-effect linear predictor across the design space, defined as βT0 C0β0 Asymptotic Theory The asymptotic theory establishes that under regularity conditions, the sample generalized marginal R2 estimator converges in probability to the population estimand ρ2M The asymptotic normality is established by showing that the standardized estimator converges to N(0, σ2ρM), where σ2ρM is determined by the limit of average cluster-level influence functions Inference and Validation The method provides a two-step estimation procedure where the GLMM fits nuisance parameters while the Quasi-GLM evaluates fixed effects and marginal R2 Simulations confirm that the estimator is consistent across eight exponential dispersion families, with relative bias decaying at the expected O(1/M) rate The sandwich-based confidence intervals are shown to be consistent, with the ASE/ESE ratio uniformly close to 1.0 across all specifications Ecological Applications Two ecological applications illustrate the importance of reporting estimation uncertainty alongside marginal quasi-R2 These include analyzing density-dependent size variation in macroalgal recruits and parasite burden in noise-exposed guppies The results show the estimator's responsiveness to signal strength, tracking the true population estimand closely across all eight distributions The paper concludes that marginal quasi-R2 provides a complementary measure of explained variation and its estimation uncertainty, grounding substantive interpretation in the fitted model and the specified auxiliary information The framework connects marginalization, geometric interpretation, population estimands, and statistical inference across different link functions and response families Pruning redundant reasoning tokens Generalized linear mixed models (GLMMs) are widely used across scientific disciplines, yet quantifying response variation explained by auxiliary information remains challenging for non-Gaussian outcomes and hierarchical dependence The proposed marginal quasi-R2 method extends the geometric interpretation of classical R2 to non-Gaussian responses through a two-step process involving marginalization and a Quasi-GLM formulation How the training signal is built The construction begins by marginalizing the joint distribution of responses and random effects onto the marginal response space, retaining latent heterogeneity in the marginal moment structure Second, a Quasi-GLM formulation equips this marginal space with a locally defined information metric, where orthogonal projection yields a Pythagorean decomposition of explained and unexplained variation Where the gains come from The population generalized marginal R2 is defined as the ratio of explained to total working-scale variance, denoted as R2 = SSRw/SSTw This is decomposed into a deterministic signal component F0 and a noise component N0, such that ρ2M = FM + NM The signal F0 represents the total working-weighted variance of the fixed-effect linear predictor across the design space, defined as βT0 C0β0 Pruning redundant reasoning tokens The construction begins by marginalizing the joint distribution of responses and random effects onto the marginal response space, retaining latent heterogeneity in the marginal moment structure Second, a Quasi-GLM formulation equips this marginal space with a locally defined information metric, where orthogonal projection yields a Pythagorean decomposition of explained and unexplained variation How the training signal is built The population generalized marginal R2 is defined as the ratio of explained to total working-scale variance, denoted as R2 = SSRw/SSTw This is decomposed into a deterministic signal component F0 and a noise component N0, such that ρ2M = FM + NM The signal F0 represents the total working-weighted variance of the fixed-effect linear predictor across the design space, defined as βT0 C0β0 Where the gains come from The asymptotic theory establishes that under regularity conditions, the sample generalized marginal R2 estimator converges in probability to the population estimand ρ2M The asymptotic normality is established by showing that the standardized estimator converges to N(0, σ2ρM), where σ2ρM is determined by the limit of average cluster-level influence functions Pruning redundant reasoning tokens The asymptotic theory establishes that under regularity conditions, the sample generalized marginal R2 estimator converges in probability to the population estimand ρ2M The asymptotic normality is established by showing that the standardized estimator converges to N(0, σ2ρM), where σ2ρM is determined by the limit of average cluster-level influence functions Pruning redundant reasoning tokens

Title and authors: Tom: So we're looking at a paper called "A Geometric Perspective for Marginal R2: Estimation and Inference in Generalized Linear Mixed Models." It sounds like they are trying to figure out how to measure the explained variation when you have these complex mixed models, especially when things aren't following a normal distribution.

Jane: Right. It tackles that challenge of quantifying response variation using auxiliary information when the outcomes are non-Gaussian and there’s this hierarchical dependence involved. It suggests a new way to look at the coefficient of determination, R2, by giving it a geometric interpretation that connects it to fixed designs and statistical inference.

Lu: Essentially, they're proposing marginal quasi-R2 through two main steps: first marginalizing over the latent random effects to get a measure on the marginal response space, and then using a Quasi-GLM formulation to define an information metric there. It’s trying to give this R2 something more concrete than just a simple ratio of sums of squares.

Meng: From my side, I'm curious about how this geometric interpretation actually translates into something we can use practically for deployment, you know? Does it just add complexity that slows down the inference time when we need to deploy these models at scale?

Lalam: It seems they are defining a fixed-design population estimand, which they call rho squared M, as the ratio of explained to total working-scale variance in a specific way, and then this is decomposed into a deterministic signal component F zero and a noise component N zero.

Tom: So if I'm seeing this right, they are taking that R2 idea and breaking it down into what's purely determined by the fixed effects versus what’s just noise within the model structure.

Jane: Exactly. And that deterministic signal, F zero, is defined as the total working-weighted variance of the fixed effect linear predictor across the entire design space, which they write as beta T zero C zero beta zero.

The paper's summary: Tom: So how does this marginal quasi-R2 method actually work in practice, Jane? I want to understand the two-step process they laid out.

Jane: Well, the first step involves marginalizing the joint distribution of responses and random effects onto just the marginal response space. This is important because it keeps that latent heterogeneity in mind even though we're no longer looking at it conditionally.

Lu: That marginalization step is crucial because, as they point out on page two of THIS PAPER, this process incorporates latent heterogeneity into the marginal moment structure which doesn't necessarily keep the mean and variance relationships of the original conditional model intact.

Meng: I worry about that uncertainty being retained in the marginal moment structure. If we’re trying to ship a fast prediction model, retaining uncertainty from latent effects adds a lot of overhead for estimation.

Lalam: The second step is where they use the Quasi-GLM formulation on this marginal space to equip it with a locally defined information metric. This metric is what allows them to perform that Pythagorean decomposition of explained and unexplained variation.

Tom: So we're taking the marginalized space, putting an information structure on it, and then using orthogonal projection to get our explained versus unexplained parts. That sounds like a solid mathematical foundation for measuring explanatory power under these tricky conditions.

Jane: It’s about defining that fixed-design population estimand—that rho squared M—and then finding the sample estimator for it, which is what they focus on when they establish the asymptotic theory.

The paper's improvements: Tom: The paper talks about how this method improves upon existing R2 measures or classical R2, and I want to know what those specific improvements are in plain terms.

Lu: The main improvement is that they provide a common geometric interpretation for R2 across different non-Gaussian response families, which existing variance-based constructions don't have. They link it to a fixed-design marginal estimand and corresponding statistical inference.

Meng: What does that mean for me? It means we get a consistent way to measure explained variation regardless of whether we are dealing with a Poisson distribution or something else, as long as the structure fits the GLMM framework they are using.

Jane: They also establish asymptotic theory, showing that under regularity conditions, the sample generalized marginal R2 estimator converges in probability to that population estimand rho squared M. And they show its asymptotic normality is centered around zero with a variance determined by average cluster-level influence functions.

Tom: So they're not just giving us a formula; they're giving us confidence that this estimate gets closer and closer to the true population value as we get more data, and telling us how much error to expect. That’s concrete.

Conclusion: Jane: So wrapping up the paper "A Geometric Perspective for Marginal R2: Estimation and Inference in Generalized Linear Mixed Models," it seems they've proposed a robust way to measure explained variation in complex models by focusing on marginalization and a Quasi-GLM structure.

Tom: They’ve connected the geometric idea of R2 to fixed design estimands, which is pretty significant because it gives us something stable to interpret, even when the response isn't Gaussian.

Lu: The connection between marginalization over latent random effects and that information metric yielding a Pythagorean decomposition of explained and unexplained variation is where the real theoretical power lies. It grounds the estimation in a consistent framework for these hierarchical structures.

Meng: If we look at what they did, it’s providing a consistent way to measure signal strength across all eight exponential dispersion families they tested, and their simulation studies confirm that the relative bias decays at an O of one over M under strong signal conditions. That gives us a good idea of how much data we need before we can trust the estimate.

Lalam: And they show that their sandwich-based confidence intervals are consistent, meaning they actually cover the true value correctly, and it’s interesting that the ASE to ESE ratio stays uniformly close to one point across all of those specifications.

Tom: So ultimately, this paper gives us a new tool for assessing how much auxiliary information explains response variation in GLMMs while providing a solid statistical framework for trusting that measurement. That's what we have on "A Geometric Perspective for Marginal R2: Estimation and Inference in Generalized Linear Mixed Models."

Hao Qin, Ayumi Mizuno, Kyle Morrison, Allison K. White, Shinichi Nakagawa

Department of Biological Sciences, University of Alberta

stat.ME

Submitted: 2026-11-10

Updated: 2026-11-10

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: Generalized linear mixed models (GLMMs) are widely used across scientific disciplines, yet quantifying response variation explained by auxiliary information remains challenging for non-Gaussian…

Key concepts

Marginalization
This step involves collapsing the joint distribution of responses and random effects onto the marginal response space. It retains latent heterogeneity within the structure of the marginal moments, allowing for a focus on the average behavior across different levels without losing information about underlying variation.
Quasi-GLM Formulation
This equips the marginalized response space with a local information metric. By using orthogonal projection in this space, it achieves a Pythagorean decomposition of explained and unexplained variation, providing a geometric way to separate signal from noise in non-Gaussian settings.
Population Estimand ($ ho^2_M$)
This is the true measure of explained variance, defined as the ratio of working-scale explained variance to total working-scale variance. It is decomposed into a deterministic signal component ($F_0$) and a noise component ($N_0$), where $F_0$ represents the total fixed-effect linear predictor across all design points.
Asymptotic Theory
This establishes that the sample estimator of marginal quasi-R2 converges in probability to the true population estimand ($ ho^2_M$) as the number of clusters ($M$) increases. It also defines its asymptotic normality, showing convergence to a normal distribution centered at zero.

Terminology

Summary

Generalized linear mixed models (GLMMs) are widely used across scientific disciplines, yet quantifying response variation explained by auxiliary information remains challenging for non-Gaussian outcomes and hierarchical dependence The proposed marginal quasi-R2 method extends the geometric interpretation of classical R2 to non-Gaussian responses through a two-step process involving marginalization and a Quasi-GLM formulation

Core Methodology

The construction begins by marginalizing the joint distribution of responses and random effects onto the marginal response space, retaining latent heterogeneity in the marginal moment structure Second, a Quasi-GLM formulation equips this marginal space with a locally defined information metric, where orthogonal projection yields a Pythagorean decomposition of explained and unexplained variation This process defines a fixed-design population estimand and its corresponding sample estimator

Population Estimands

The paper defines the population generalized marginal R2 as the ratio of explained to total working-scale variance, denoted as R2 = SSRw/SSTw This is decomposed into a deterministic signal component F0 and a noise component N0, such that ρ2M = FM + NM The signal F0 represents the total working-weighted variance of the fixed-effect linear predictor across the design space, defined as βT0 C0β0

Asymptotic Theory

The asymptotic theory establishes that under regularity conditions, the sample generalized marginal R2 estimator converges in probability to the population estimand ρ2M The asymptotic normality is established by showing that the standardized estimator converges to N(0, σ2ρM), where σ2ρM is determined by the limit of average cluster-level influence functions

Inference and Validation

The method provides a two-step estimation procedure where the GLMM fits nuisance parameters while the Quasi-GLM evaluates fixed effects and marginal R2 Simulations confirm that the estimator is consistent across eight exponential dispersion families, with relative bias decaying at the expected O(1/M) rate The sandwich-based confidence intervals are shown to be consistent, with the ASE/ESE ratio uniformly close to 1.0 across all specifications

Ecological Applications

Two ecological applications illustrate the importance of reporting estimation uncertainty alongside marginal quasi-R2 These include analyzing density-dependent size variation in macroalgal recruits and parasite burden in noise-exposed guppies The results show the estimator's responsiveness to signal strength, tracking the true population estimand closely across all eight distributions

Conclusion

The paper concludes that marginal quasi-R2 provides a complementary measure of explained variation and its estimation uncertainty, grounding substantive interpretation in the fitted model and the specified auxiliary information The framework connects marginalization, geometric interpretation, population estimands, and statistical inference across different link functions and response families

Pruning redundant reasoning tokens

Generalized linear mixed models (GLMMs) are widely used across scientific disciplines, yet quantifying response variation explained by auxiliary information remains challenging for non-Gaussian outcomes and hierarchical dependence The proposed marginal quasi-R2 method extends the geometric interpretation of classical R2 to non-Gaussian responses through a two-step process involving marginalization and a Quasi-GLM formulation

How the training signal is built

The construction begins by marginalizing the joint distribution of responses and random effects onto the marginal response space, retaining latent heterogeneity in the marginal moment structure Second, a Quasi-GLM formulation equips this marginal space with a locally defined information metric, where orthogonal projection yields a Pythagorean decomposition of explained and unexplained variation

Where the gains come from

The population generalized marginal R2 is defined as the ratio of explained to total working-scale variance, denoted as R2 = SSRw/SSTw This is decomposed into a deterministic signal component F0 and a noise component N0, such that ρ2M = FM + NM The signal F0 represents the total working-weighted variance of the fixed-effect linear predictor across the design space, defined as βT0 C0β0

The construction begins by marginalizing the joint distribution of responses and random effects onto the marginal response space, retaining latent heterogeneity in the marginal moment structure Second, a Quasi-GLM formulation equips this marginal space with a locally defined information metric, where orthogonal projection yields a Pythagorean decomposition of explained and unexplained variation

The population generalized marginal R2 is defined as the ratio of explained to total working-scale variance, denoted as R2 = SSRw/SSTw This is decomposed into a deterministic signal component F0 and a noise component N0, such that ρ2M = FM + NM The signal F0 represents the total working-weighted variance of the fixed-effect linear predictor across the design space, defined as βT0 C0β0

The asymptotic theory establishes that under regularity conditions, the sample generalized marginal R2 estimator converges in probability to the population estimand ρ2M The asymptotic normality is established by showing that the standardized estimator converges to N(0, σ2ρM), where σ2ρM is determined by the limit of average cluster-level influence functions

The asymptotic theory establishes that under regularity conditions, the sample generalized marginal R2 estimator converges in probability to the population estimand ρ2M The asymptotic normality is established by showing that the standardized estimator converges to N(0, σ2ρM), where σ2ρM is determined by the limit of average cluster-level influence functions

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved AI systems can quantify explanatory power in generalized linear mixed models using marginal quasi-R2, which is defined through two distinct operations: marginalization over latent random effects and a Quasi-GLM formulation yielding a Pythagorean decomposition of explained and unexplained variation.

  2. The improved system can derive a fixed-design population estimand and its corresponding sample estimator, providing a geometric interpretation of R2 that extends the classical coefficient of determination to non-Gaussian responses using an inner product under the specified working metric.

  3. The AI can perform two-step estimation: first fitting the GLMM to obtain variance components, and second evaluating the Quasi-GLM to obtain a fixed-effect estimate, mirroring Generalized Estimating Equations (GEE) frameworks for inference on primary regression parameters.

  4. The system can establish asymptotic theory, including consistency and asymptotic normality of the marginal quasi-R2 estimator as the number of independent clusters M approaches infinity, with a cluster-robust sandwich standard error structure derived from empirical influence functions.

  5. The AI can construct a confidence interval on the logit scale that enforces domain restrictions [0, 1] by mapping to the logit scale and back via the expit function, using finite-sample corrections like HC1 for better coverage when M is small.

  6. The system can analyze the convergence rate of R2 estimation, showing that relative bias decays at a rate of O(1/M) under strong signal conditions and confirming that confidence interval coverage recovers nominal levels as the number of clusters increases.

Abstract

Generalized linear mixed models (GLMMs) are widely used across scientific disciplines, yet quantifying response variation explained by auxiliary information remains challenging for non-Gaussian outcomes and hierarchical dependence. Existing R squared measures provide useful summaries, but their variance-based constructions are not generally linked to a common geometric interpretation, a fixed-design marginal estimand, and corresponding statistical inference. We propose marginal quasi- R squared through two distinct operations. First, marginalization integrates over latent random effects, retaining their uncertainty in the marginal response moments and specifying the model for explanatory assessment. Second, a Quasi-GLM formulation supplies a local working information metric under which orthogonal projection yields an Pythagorean decomposition of explained and unexplained variation. We define a fixed-design population estimand, establish consistency and asymptotic normality of its estimator, and derive cluster-robust sandwich standard errors and confidence intervals under stated regularity conditions. Simulations assess estimation accuracy and interval coverage across response families and random-effects structures. Two ecological applications illustrate the importance of reporting estimation uncertainty alongside marginal quasi- R squared. The method is implemented in the R package quasiR2.

Related papers