A practical guide to causal discovery with cohort data

arXiv:2108.13395 · stat.AP · Submitted 2021-08-30 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "A practical guide to causal discovery with cohort data".

Jane: Here's the gist of the paper. In this guide, we present how to perform constraint-based causal discovery using three popular software packages: pcalg (with add-ons tpc and micd), bnlearn, and TETRAD.

Tom: The paper focuses on how these packages can be used with observational data and in the presence of mixed data, a known time ordering between variables, and missing data.

Jane: Causal Discovery Terminology A graph consists of nodes and edges, which correspond to the variables in the graph and the arrows between these variables, respectively.

Tom: A causal DAG is a DAG in which the presence of an edge between nodes indicates a belief that there is a direct causal relation; the absence of an edge between nodes indicates the absence of a direct causal relation.

Paper summary: Tom: So this paper is about how to do constraint-based causal discovery using three different software packages: pcalg with some add-ons, bnlearn, and TETRAD. It really focuses on using this stuff with observational data, especially when you have mixed data types or time ordering between the variables.

Jane: Right. So the main idea here is giving people a practical guide on how to perform causal discovery without needing to read all the heavy math themselves. They show us how these tools work together and point out which one is better for specific problems like handling missing data or time-ordered variables in cohort studies.

Lu: What they are claiming, fundamentally, is that constraint-based approaches give you flexibility because you can choose different conditional independence tests to use when building the graph. But they also highlight a big drawback: any mistakes made during the process, like deleting an edge too early in the PC algorithm, those errors get carried forward and you can't always get them back later <ref:2108.13395#pg2>.

Meng: That sounds like a real headache when you're trying to deploy something at scale. If the algorithm propagates errors, how does that affect the reliability of the causal structure we end up with for our actual models? We need to know what breaks in production Meng.

Lalam: I see. So even if we start with good data, a mistake early on might mean the final graph isn't actually a true directed acyclic graph, but a completed partially directed acyclic graph <ref:2108.13395#pg3>.

Tom: Exactly. And they tackle the messy parts of real-world cohort data head-on, like when some variables are continuous and others are categorical at the same time, which is mixed data <ref:2108.13395#pg1>.

Jane: And they address missing data too, showing how you can handle that using techniques like test-wise deletion or multiple imputation within these packages, which is crucial because missing data is super common in cohort studies due to dropout or missed visits <ref:2108.13395#pg2>.

Lu: They also look at time ordering. The paper shows how you can restrict the edges in the graph to only flow in the direction of time, which helps incorporate that known temporal relationship into the discovery process <ref:2108.13395#pg1>.

Meng: That makes sense for clinical studies where timing matters more than just correlation; it grounds the causality in reality, not just abstract statistical relationships Meng. But what about performance? How fast are these algorithms when you throw a massive cohort dataset at them? We need to know if they're feasible for real-world deployment Meng.

Lalam: The paper mentions that TETRAD has a graphical interface, but they also note that it can have substantial run times for large datasets <ref:2108.13395#pg1>. So, while you get a nice picture of the graph, speed might be an issue if you're dealing with huge cohorts.

Paper summary: Tom: And they give some practical advice on which package to pick based on what you need—like bnlearn having straightforward options for time ordering versus tpc relying heavily on pcalg for temporal modifications <ref:2108.13395#pg1>. It's not a one-size-fits-all solution.

Jane: So, the core message is that there isn't just one perfect tool; you have to match the software to the specific challenges of your data—whether it’s mixed types, missing values, or temporal dependencies <ref:2108.13395#pg1>. It’s a guide on practical application rather than just a math derivation.

Lu: And they touch on causal effect estimation by extending these algorithms with the intervention calculus when the DAG is absent, which helps you estimate multiple possible causal effects if you have an incomplete model <ref:2108.13395#pg3>. It shows how they extend the framework beyond just finding a single graph.

Meng: If we use that estimation method, we're not just getting one arrow; we're getting a range of possibilities for what might be causing an effect, which is better for risk assessment but harder to deploy robustly Meng. But does this help us predict what happens in the real world?

Lalam: It helps by giving us more information when the data isn't perfect, showing the uncertainty in the causal links <ref:2108.13395#pg2>. It acknowledges that assuming causal sufficiency—that you know everything about your data—isn't always true, which is a big reality check for researchers <ref:2108.13395#pg3>.

Tom: So, to summarize the paper 'A practical guide to causal discovery with cohort data', it lays out how three popular software packages can be used together to find causal graphs from observational data that has mixed measurements, time ordering, and missing values <ref:2108.13395#pg1>. It’s less about proving a single perfect method and more about giving practitioners the tools to handle these real-world complexities <ref:2108.13395#pg1>.

Jane: And it really matters because observational cohort data is everywhere, but it introduces so many hurdles that standard causal methods often fail—so this guide offers a concrete path forward for researchers and analysts who need to get causal insights from real-world studies <ref:2108.13395#pg2>.

Lu: The implication is that instead of getting stuck with just one theoretical framework, you can compare results across different constraint-based algorithms like PC, which helps you see if your final graph is stable or if the method itself is introducing too much noise <ref:2108.13395#pg3>. It forces a kind of cross-validation on the structure itself.

Meng: I'm worried about the practical overhead here, though; having to manage three different software packages means more maintenance and more potential points of failure in our pipeline Meng. We need something that runs reliably without constant manual tweaking between tools.

Lalam: The paper mentions that constraint-based algorithms build the graph based on conditional independences found through conditional independence testing, and since several DAGs can imply the same set of conditional independences, PC isn't able to distinguish between them <ref:2108.13395#pg3>. This means if you only use PC, you might end up with a structure that is just a completed partially directed acyclic graph <ref:2108.13395#pg3>.

Paper summary: Tom: That's the core limitation they point out: the output isn't always a single DAG, but this CPDAG with both directed and undirected edges, meaning you know something is related but not which way it flows <ref:2108.13395#pg3>. It forces us to interpret that ambiguity carefully.

Jane: It means we have to be very skeptical of the final structure; it's more of a starting point for investigation than a final answer <ref:2108.13395#pg2>. The paper’s focus isn't just on finding *a* graph, but on providing the practical framework for testing and refining those structures across different scenarios <ref:2108.13395#pg1>.

Lu: And they provide recommendations on how to choose your software based on whether you prioritize time ordering, mixed data handling, or simply needing a graphical tool like TETRAD <ref:2108.13395#pg1>. It’s a decision-making guide for anyone starting out with this kind of analysis <ref:2108.13395#pg1>.

Meng: For deployment, I need to know if the computational cost difference between pcalg and bnlearn is significant enough to matter in terms of running time on a server versus a local machine Meng. If the complexity jumps from polynomial to exponential, that's a deployment deal-breaker Meng.

Lalam: The paper notes that for missing data, they use test-wise deletion as one way to handle it, but multiple imputation is another option mentioned <ref:2108.13395#pg2>. So you have choices on how to deal with the incomplete parts of your data <ref:2108.13395#pg2>.

Tom: And they emphasize that using multiple packages to do the analysis and comparing the results adds a layer of plausibility to whatever causal structure you end up with <ref:2108.13395#pg1>. It’s about cross-checking your findings, which is smart when you're dealing with uncertainty <ref:2108.13395#pg1>.

Jane: So the implication for the listener who doesn't read papers is that causal discovery isn't some magic button; it requires a structured approach where you pick your tools carefully and constantly check if your results hold up under different assumptions about the data <ref:2108.13395#pg2>. It’s an iterative process of building, testing, and refining the causal map <ref:2108.13395#pg1>.

Lu: The future work they suggest seems to be extending this framework further, perhaps by incorporating more complex dependencies or different types of observational data that we haven't fully covered in this guide <ref:2108.13395#pg1>. It points toward the next level of complexity in causal inference <ref:2108.13395#pg3>.

Meng: I hope future work addresses the deployment side more directly; moving away from GUI tools like TETRAD and towards something that’s built for high-throughput pipelines is what we need to see Meng.

Lalam: It seems the paper is really pushing researchers toward a more rigorous, comparative approach when dealing with observational data because the inherent noise and assumptions are so high <ref:2108.13395#pg2>. This guide gives you the roadmap for that rigor <ref:2108.13395#pg1>.

Conclusion: Tom: So we're wrapping up this look at "A practical guide to causal discovery with cohort data." Basically, this paper gives us three specific software tools—pcalg, bnlearn, and TETRAD—to figure out how variables cause each other when we have messy real-world observational data.

Jane: It’s about taking that complex idea of a causal graph and showing people exactly how to build one using these programs when the data is mixed up with things like time order or missing values.

Lu: The core contribution is less about inventing a new math trick and more about providing a practical roadmap for researchers who are dealing with messy cohort studies where assumptions are always shaky.

Meng: It’s helpful because it moves the conversation away from just theory to something we can actually run on a computer, but I still have to wonder how much manual tweaking is going to be required across those three different packages.

Lalam: The paper points out that using multiple packages and comparing their results gives you more confidence in the final structure you get, which is important because constraint-based methods can sometimes give you a whole set of possibilities instead of one single answer.

Tom: Right, so it’s less about finding the perfect map and more about having a few different maps to compare so we can see which one is most likely right given the data we have.

Jane: Exactly. It shows that for observational data, you can’t just pick one algorithm and be done with it; you need this kind of toolkit to test your findings across different scenarios like how time ordering affects the results.

Lu: And they also touch on estimating those actual causal effects using something called the intervention calculus when the directed acyclic graph isn't even fully determined yet.

Meng: That estimation part sounds ambitious, but I’m more focused on what happens when you try to run this stuff at scale; if it slows down way too much, it just won’t ship in production.

Lalam: The authors suggest that by using these different approaches together, you get a better picture of the uncertainty around those causal links, which is crucial because real data is never perfect.

Tom: So the big implication here is that causal discovery for observational data isn't a one-shot process; it’s an iterative process of building and testing structures across these different software tools.

Jane: It means researchers can stop guessing about their models and start using a structured method to see if their hypothesized relationships actually hold up under the messy conditions of real-world studies.

Lu: And that points toward future work focusing on incorporating even more complex dependencies into this framework so it handles bigger, messier systems than what’s covered here.

Meng: I’m still hoping for something that moves away from these heavy GUI tools and lets us build pipelines where we can run this discovery process automatically without constant manual intervention.

Boston University School of Public Health · Leibniz Institute for Prevention Research and Epidemiology - BIPS, Bremen · University of Bremen

stat.AP

Submitted: 2021-08-30

Updated: 2023-12-18

Comments: 27 pages, 20 figures

DOI: 10.1093/ije/dyae113

Code: https://github.com/bips-hb/tpc

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 77/100

The gist: In this guide, we present how to perform constraint-based causal discovery using three popular software packages: pcalg (with add-ons tpc and micd), bnlearn, and TETRAD.

Key concepts

Causal DAG
A directed acyclic graph representing a causal structure between variables. Edges show hypothesized direct causal links, while the absence of an edge suggests no direct relationship exists between those variables in the model.
Constraint-based approaches
A method for discovering causal graphs by testing conditional independence hypotheses. Instead of assuming a structure, these algorithms build a graph based on which variables are statistically independent given others in the data.
Mixed measurement scales
This occurs when different variables in the data are measured on different scales (e.g., one variable is continuous and another is categorical). The paper explains how specific tests, like the Conditional Gaussian test, help identify these relationships despite the mixed types of measurements.

Terminology

Summary

In this guide, we present how to perform constraint-based causal discovery using three popular software packages: pcalg (with add-ons tpc and micd), bnlearn, and TETRAD. p. 0

Where the paper says it

A practical guide to causal discovery with cohort data Ryan M. Andrews1 Ronja Foraita2 Vanessa Didelez2,3 Janine Witte2,3 1 Boston University School of Public Health 2 Leibniz Institute for Prevention Research and Epidemiology - BIPS, Bremen 3 University of Bremen December 18, 2023 Abstract In this guide, we present how to perform constraint-based causal discovery using three popular software packages: pcalg (with add-ons tpc and micd), bnlearn, and TETRAD.

The paper focuses on how these packages can be used with observational data and in the presence of mixed data, a known time ordering between variables, and missing data. p. 0
Where the paper says it

We focus on how these packages can be used with observational data and in the presence of mixed data (i. , data where some variables are continuous, while others are categorical), a known time ordering between variables, and missing data.

Causal Discovery Terminology

A graph consists of nodes and edges, which correspond to the variables in the graph and the arrows between these variables, respectively. A causal DAG is a DAG in which the presence of an edge between nodes indicates a belief that there is a direct causal relation; the absence of an edge between nodes indicates the absence of a direct causal relation. p. 1

Where the paper says it

, one cannot start at a node and arrive back at the same node on the graph by following the direction of the edges on the graph) is called a directed acyclic graph (DAG). A causal DAG is a DAG in which the presence of an edge between nodes indicates a belief that there is a direct causal relation; the absence of an edge between nodes indicates the absence of a direct causal relation.

Constraint-based approaches place the problem of causal discovery within the framework of a series of conditional independence hypothesis tests. p. 0
Where the paper says it

LiNGAM (Shimizu et al. Constraint-based approaches, which are the focus of this guide, place the problem of causal discovery within the framework of a series of conditional independence hypothesis tests.

Constraint-based algorithms build the graph based on conditional independences in the data, which are inferred using conditional independence testing. p. 2
Where the paper says it

, 2000) can still be applied, but they produce a less informative output. Constraint-based causal discovery algorithms such as PC, on which we focus in this guide, build the graph based on conditional independences in the data, which are inferred using conditional independence testing.

Software Implementations and Features

The paper introduces three software packages for causal discovery: pcalg (with add-ons tpc and micd), bnlearn, and TETRAD. Each package has specific functions used to implement the PC algorithm, such as pc in pcalg, pc. p. 1

Where the paper says it

In Section 2, we provide a brief introduction to basic causal discovery terminology, followed by introductions to pcalg, bnlearn, and TETRAD. Here, we focus on the basic function(s) within each package that are used to implement the PC algorithm and their required inputs.

stable in bnlearn, and the GUI-based TETRAD. p. 4
Where the paper says it

Unlike pcalg and bnlearn, TETRAD is designed for users who prefer a graphical user interface (GUI), and it is possible to perform causal discovery without writing any code, but R and Python interfaces are available from the CCD website as well. In this guide, we used GUI-based TETRAD version 6.

The comparison table summarizes their capabilities regarding mixed data, time-ordered variables, and missing data. p. 8
Where the paper says it

package mixed data time-ordering missing data pcalg/tpc/micd Conditional Gaussian test (mixCItest from package micd) tiers argument in tpc multiple imputation and test-wise deletion (using test functions from the micd package) bnlearn Conditional Gaussian test (test = "mi-cg") blacklist argument test-wise deletion is automatically applied TETRAD Conditional Gaussian and Degenerate Gaussian test via "tier" specification test-wise deletion is automatically applied 2. 5 Comparison Table 1 provides an overview of the capabilities of pcalg, bnlearn, and TETRAD with respect to common issues with cohort data.

Handling Data Complexities

The guide illustrates how the packages handle specific observational data challenges:

  1. Mixed measurement scales are addressed using tests like the Conditional Gaussian test in pcalg/tpc/micd, or the mi-cg test in bnlearn.

  2. Time ordering is incorporated by restricting edges to be oriented in accordance with the flow of time, as seen with tpc's modifications.

  3. Missing data can be handled via test-wise deletion or multiple imputation; for instance, pcalg uses test-wise deletion through indepTest modifications. p. 8

    Where the paper says it

    GUIDE TO CAUSAL DISCOVERY WITH COHORT DATA Table 1: Overview of pcalg, tpc, micd, bnlearn, and TETRAD with respect to their ability to handle mixed measurement scales, time-ordered variables, and missing data. package mixed data time-ordering missing data pcalg/tpc/micd Conditional Gaussian test (mixCItest from package micd) tiers argument in tpc multiple imputation and test-wise deletion (using test functions from the micd package) bnlearn Conditional Gaussian test (test = "mi-cg") blacklist argument test-wise deletion is automatically applied TETRAD Conditional Gaussian and Degenerate Gaussian test via "tier" specification test-wise deletion is automatically applied 2.

Practical Recommendations

The authors provide practical advice on choosing software, noting that bnlearn has straightforward black and white list options for time ordering but limited conflict resolution. The tpc package relies heavily on pcalg's functionality while adding temporal modifications. For users requiring a graphical interface, TETRAD is the only option available, although it can have substantial run times for large datasets. p. 4

Where the paper says it

edu/tools). Unlike pcalg and bnlearn, TETRAD is designed for users who prefer a graphical user interface (GUI), and it is possible to perform causal discovery without writing any code, but R and Python interfaces are available from the CCD website as well.

The recommendation is to consider performing causal discovery analysis with multiple packages to lend plausibility to the overall results.

Causal Effect Estimation

The paper briefly touches upon estimating causal effects by extending algorithms with the “Intervention calculus when the DAG is Absent” (IDA) algorithm. The IDA algorithm estimates multiple possible causal effects because a CPDAG or MPDAG contains more than one DAG. p. 1

Where the paper says it

A graph that consists only of directed edges and contains no cycles (i. , one cannot start at a node and arrive back at the same node on the graph by following the direction of the edges on the graph) is called a directed acyclic graph (DAG).

In pcalg, the ida function performs this estimation, requiring a valid CPDAG or MPDAG as input. p. 3
Where the paper says it

If an adjacency between nodes A and B is required, an edge between them is forced into the estimated CPDAG or MPDAG, but the orientation is determined by the algorithm. The NAdelete input determines what happens if the indepTest function returns NA.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved causal discovery systems can handle mixed data types by utilizing specific conditional independence tests like mixCItest from pcalg or mi-cg from bnlearn, allowing them to test independence between purely continuous variables using partial correlation tests and discrete variables using G2 or X2 tests.

  2. Improved systems can incorporate time ordering into causal discovery by restricting edges to be oriented in accordance with the flow of time, as described in We restrict edges to be oriented in accordance with the flow of time, which is implemented in tpc.

  3. Improved systems can manage missing data by employing test-wise deletion, where the decision to remove or retain an observation is made for each conditional independence test separately, offering a method that may lead to a higher power for some tests compared to list-wise deletion.

  4. Improved systems can perform multiple imputation for missing data by generating multiply imputed data using e.g., the mice package with the random forest option and then running causal discovery with appropriate tests, as shown in Figure 17.

  5. Improved systems can integrate background knowledge into causal discovery by specifying required and forbidden edges between variables, similar to bnlearn's blacklist and whitelist arguments, allowing users to enforce structural constraints.

Abstract

In this guide, we present how to perform constraint-based causal discovery using three popular software packages: pcalg (with add-ons tpc and micd), bnlearn, and TETRAD. We focus on how these packages can be used with observational data and in the presence of mixed data (i.e., data where some variables are continuous, while others are categorical), a known time ordering between variables, and missing data. Throughout, we point out the relative strengths and limitations of each package, as well as give practical recommendations. We hope this guide helps anyone who is interested in performing constraint-based causal discovery on their data.

Sources

Related papers