Missing data and cluster graphs: cluster-level missingness vs variable-level missingness
summary
The gist
Missing data is pervasive in many scientific domains such as public health, environmental science, and the social sciences <ref:2605.20943#pg2> Recoverability from missing data is typically studied…
In short
The paper investigates how missing data can be recovered when only coarse structural information is available, such as cluster-level graphs instead of detailed variable-level indicators. It introduces two graph types—m-C-DMG and cm-C-DMG—and shows that the coarser representation (cm) is generally less informative than the finer one (m).
Key concepts
- m-C-DMG
- This graph retains specific missingness indicators for each individual variable while representing causal relationships between variables at the cluster level. It is a more detailed abstraction that keeps track of how each variable is missing data.
- cm-C-DMG
- This is a coarser abstraction where all missingness indicators within a cluster are merged into a single vertex. It aggregates information at the cluster level, sacrificing variable-specific detail for simplicity in structural representation.
- Compatibility
- This defines how well an abstract graph (like m-C-DMG or cm-C-DMG) aligns with the underlying variable-level missingness models. Compatibility determines what kind of probabilistic or causal queries can be reliably recovered from the graph structure.
- Recoverability
- This refers to the ability to successfully reconstruct specific information, such as joint distributions or causal effects, from an abstract representation of missing data. The paper establishes conditions under which these queries can be recovered using either graph type.
Terminology used across episodes
This episode discusses
- Missing data and cluster graphs: cluster-level missingness vs variable-level missingness · Paper Radio
The paper
Missing data and cluster graphs: cluster-level missingness vs variable-level missingness · Read on arXiv
Willow Scott, Eugenio Valdano, Charles K. Assaad
Sorbonne University · Inserm
Missing data is pervasive in many scientific domains such as public health, environmental science, and the social sciences. Recoverability from missing data is typically studied using fully specified variable-level missingness models despite that, in many applications, only coarse structural information is available, for instance when variables are grouped into clusters due to limited knowledge or interpretability reasons. In this paper, we investigate recoverability from such abstract representations. We introduce two classes of cluster-based missingness graphs: the m-C-DMG, which retains variable-specific missingness indicators, and the cm-C-DMG, which aggregates missingness mechanisms at the cluster level. We formalize the notion of compatibility between these abstract graphs and underlying variable-level missingness models, and study how this abstraction affects the recoverability of probabilistic and causal queries. In particular, we give graphical conditions of recovering the joint distribution as well as graphical conditions of recovering a macro causal effect.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Missing data and cluster graphs".
Jane: Here's the gist of the paper. Missing data is pervasive in many scientific domains such as public health, environmental science, and the social sciences <ref:2605.20943#pg2> Recoverability from missing data is typically studied using fully specified variable-level missingness models despite that, in many applications, only coarse structural information is available <ref:2605.20943#pg2>.
Tom: In this paper, we investigate recoverability from such abstract representations <ref:2605.20943#pg2>.
Jane: The authors introduce two classes of cluster-based missingness graphs: the m-C-DMG, which retains variable-specific missingness indicators, and the cm-C-DMG, which aggregates missingness mechanisms at the cluster level <ref:2605.20943#pg2>.
Tom: They formalize the notion of compatibility between these abstract graphs and underlying variable-level missingness models and study how this abstraction affects the recoverability of probabilistic and causal queries <ref:2605.20943#pg2>.
Paper summary: Jane: So we've covered the main points of "Missing data and cluster graphs: cluster-level missingness vs variable-level missingness." We looked at how the m-C-DMG, which keeps variable indicators but clusters them causally, compares to the cm-C-DMG, which merges all indicators within a cluster into one vertex.
Tom: The paper shows that these two abstractions are not interchangeable; they encode different amounts of information, and the cm version is generally less informative for recovery. The main takeaway is that when you have missing data and only coarse structural knowledge, you need to be careful about which abstraction you use because it changes what queries you can answer.
Lu: It provides a principled way to analyze settings where we are forced to deal with uncertainty around relationships between variables because we don't know everything about them. It gives us specific conditions for when you can actually recover the joint distribution or a causal query from these abstract representations, which is really useful theory.
Meng: For deploying this in production, it means if your system relies on cluster summaries, you need to rigorously test whether that coarser representation preserves the necessary details for the queries you care about without introducing unacceptable errors at scale.
Jane: In simpler terms, this work helps us understand how much information we lose by simplifying the structure of our data representation when dealing with missing data in science and social studies. It’s a framework for making informed decisions about which level of structural detail is appropriate for a given problem.
Conclusion: Tom: So we're wrapping up this paper about missing data, and the title itself is pretty direct: "Missing data and cluster graphs: cluster-level missingness vs variable-level missingness."
Jane: It’s a really clear way to put it. Basically, they are comparing two different ways to look at missing information. One way looks at each individual variable separately, and the other way groups those variables into clusters first.
Lu: Exactly. They introduce two models, the m-C-DMG and the cm-C-DMG, which represent these different levels of abstraction—variable level versus cluster level.
Meng: But they’re not just drawing diagrams; they're defining how compatible those abstract graphs are with real missing data models. That compatibility stuff is where it gets interesting for deployment.
Tom: Right, so the core finding seems to be that this abstraction matters a lot for what you can actually figure out from the data later on.
Jane: They show that even when you only have a coarse structural idea of your data, like just cluster information, you can still recover certain things if the abstract graph is compatible in a specific way.
Lu: The paper establishes some strict rules for compatibility. If one cluster has variables with different missingness patterns, it’s much more powerful than just looking at the big cluster summary alone.
Meng: And thecm-C-DMG, which merges everything into one vertex per cluster, is generally less informative than the m-C-DMG because it loses those specific variable details.
Tom: So what does this mean practically? If you’re building a system that relies on these cluster summaries to make decisions, you need to know if you’re using the more detailed m version or the coarser cm version.
Jane: It means we have a principled way to choose our structural simplification based on what kind of query—like figuring out a joint distribution or a causal effect—we actually need to answer.
Lu: They give necessary and sufficient conditions for recovering those queries, which involves checking if certain vertices in the m-C-DMG are connected in a specific way.
Meng: From an engineering standpoint, that condition sounds like it’s going to be tricky to implement because you have to check paths through colliders and specific sets of vertices.
Tom: It sounds like they’re giving us the theoretical blueprint for when we can trust our simplified models, which is a big deal for real-world applications.
Jane: It really moves the conversation from just "how do we handle missing data?" to "what level of structure are we allowed to ignore, and what's the cost?" (SOUND of music slightly increasing in volume and then fading out)
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates