Graph-of-Thoughts for LLM Causal Discovery with a Fairness-Pathway Case Study

arXiv:2608.02877 · cs.LG · Submitted 2026-08-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Graph-of-Thoughts for LLM Causal Discovery with a Fairness-Pathway Case Study".

Title and authors: Jane: We're talking about the paper "GraphofThoughts for LLM Causal Discovery with a Fairness-Pathway Case Study." It looks like the authors are tackling a big problem where finding a causal map isn't enough if you need to check for fairness in clinical settings.

Tom: Exactly. The core idea is that even when these new methods find directed structures, that structural accuracy doesn't automatically mean the model is fair on the specific paths we care about for an audit.

Lu: That’s because structural errors can hide critical pathway failures if the discovery process misses exactly which causal link between a protected attribute and an outcome is present.

Meng: So for us who are building models, this means when we chain discovery to fairness analysis, we have to report how well they recovered those specific unfair paths as a basic safeguard against miscertifying models as fair.

Tom: And they are pushing for evaluation based on the edges that the downstream analysis actually depends on instead of just getting an aggregate structural F1 score across all connections.

Jane: That’s a strong call to action for how we should be evaluating these tools going forward in AI research, especially when dealing with fairness in clinical settings.

Lu: They suggest using hybrid scoring methods, combining those data-based statistics with structural rules and task-relative objectives to make the discovery process smarter overall.

Meng: That’s what we need—making sure the discovery process is better fitted to the actual data, not just coherent according to some internal model logic.

Tom: So in short, "GraphofThoughts for LLM Causal Discovery with a Fairness-Pathway Case Study" shows that GoT-CD is structurally strong but needs that extra layer of fairness checking to be useful.

Jane: Right, Tom, it means if you’re doing a fairness audit, you can't just assume getting a high structural score means the model is fair on those specific routes you care about.

Tom: That’s right. The paper shows that GoT-CD yields always-acyclic discovered graphs that are structurally competitive with large language model baselines, but it also shows that this structural fidelity alone doesn't guarantee fairness fidelity >

Lu: They show that structural accuracy doesn't imply fairness fidelity because structural errors can hide critical pathway failures in an audit if the discovery process misses the exact causal link between a protected attribute and an outcome.

Meng: So for us at the startup level, this is a warning that we need to check not just if our graph looks right, but if it actually finds those specific biased paths we are trying to stop when we deploy >

Jane: Exactly. And what they suggest is that when you chain discovery to fairness analysis you must report pathway recovery alongside your fairness estimate as a basic safeguard against miscertifying models as fair >

Tom: They also advocate scoring discovery methods based on the edges that the downstream analysis actually depends on rather than just getting an aggregate structural F1 score >

Lu: They suggest using hybrid scoring methods, combining those data-based statistics with structural rules and task-relative objectives to make the discovery process smarter overall >

Meng: That’s what we need—making sure the discovery process is better fitted to the actual data, not just coherent according to some internal model logic >

Jane: It’s about making sure we aren't just rewarding methods that are good at making dense graphs, but methods that are actually finding those specific causal links relevant for fairness checks >

Tom: So they’re pushing for a way to score discovery methods that focuses on the edges critical for path-specific counterfactual fairness rather than just aggregate structural agreement >

Lu: It emphasizes that the choice of discovery method matters a lot when you're planning on running sensitive counterfactual fairness tests because it determines what you can even test >

Meng: If we adopt this view, then I think the evaluation becomes much more meaningful for production because it ties the score directly to the real risks we are trying to mitigate in deployment >

Jane: That’s a strong call to action for how we should be evaluating these tools going forward, especially when dealing with fairness in clinical settings >

Tom: So in short, "GraphofThoughts for LLM Causal Discovery with a Fairness-Pathway Case Study" shows that GoT-CD is structurally strong but needs that extra layer of fairness checking to be useful >

Lu: It lays out a clear direction for future work suggesting we need those hybrid scoring methods to combine data statistics with structural rules >

Meng: And that’s the next logical step: making the discovery process smarter by incorporating how well it fits the data, not just how coherent the model thinks its graph is >

Jane: We’ll keep an eye on those hybrid scoring methods and see if they help us build more reliable tools for auditing AI models in practice.

Tom: Next time, we're looking at something totally different—a paper about training-free style composition that might change how we control the look of generated text.

The paper's summary: Jane: So, to recap, this paper is about taking those complex causal discovery methods that use large language models and adding a crucial layer: checking if those discovered structures are actually fair for people in real life.

Tom: Right, it’s focusing on this specific case study where they show that even when a method finds a directed graph, that structural accuracy alone doesn't guarantee fairness fidelity.

Lu: They prove that you can have a perfect looking graph—a Directed Acyclic Graph—but if that graph misses the exact causal link between a protected attribute and an outcome, the fairness audit will fail.

Meng: For us in production, this means we can’t just look at the overall structure; we have to make sure our discovery process actually finds those specific biased pathways we are trying to stop when we deploy.

Jane: And what they suggest is a direct action for anyone chaining discovery to fairness analysis: you have to report how well that method recovered those specific unfair paths right alongside your fairness estimate.

Tom: Plus, they’re pushing for a different way to score these methods too, moving away from just getting a big structural F1 score across the whole graph.

Lu: They want evaluation based on the edges that the downstream analysis actually depends on for counterfactual fairness tests. That makes sense because it determines what you can actually test.

Meng: If you adopt that view, then I think the scoring becomes way more meaningful for production because it ties your score directly to the real risks you are trying to mitigate when you put something out there.

Jane: It’s about making sure we aren't just rewarding methods that are good at making dense graphs, but methods that actually find those specific causal links relevant for fairness checks.

Tom: So they’re pushing for a way to score discovery methods that focuses on the edges critical for path-specific counterfactual fairness rather than just aggregate structural agreement across all connections.

Lu: It emphasizes that the choice of discovery method matters a lot when you're planning on running sensitive counterfactual fairness tests because it determines what you can even test.

Meng: If we adopt that view, then I think the evaluation becomes much more meaningful for production because it ties the score directly to the real risks we are trying to mitigate in deployment.

Jane: That’s a strong call to action for how we should be evaluating these tools going forward, especially when dealing with fairness in clinical settings.

Tom: So in short, this paper on GraphofThoughts for LLM Causal Discovery with a Fairness-Pathway Case Study shows that GoT-CD is structurally strong but needs that extra layer of fairness checking to be useful.

Lu: It lays out a clear direction for future work suggesting we need hybrid scoring methods to combine data statistics with structural rules.

Meng: And that’s the next logical step: making the discovery process smarter by incorporating how well it fits the data, not just how coherent the model thinks its graph is.

Jane: We’ll keep an eye on those hybrid scoring methods and see if they help us build more reliable tools for auditing AI models in practice.

The paper's improvements: Tom: So, we've talked about "GraphofThoughts for LLM Causal Discovery with a Fairness-Pathway Case Study," and what they're suggesting now is how to actually make this discovery process better at finding the right structures for fairness work.

Jane: It boils down to adding more statistical grounding to the scoring step of that GraphofThoughts pipeline.

Lu: They mean incorporating things like BIC or conditional independence test statistics alongside their current structural rules. That makes the scoring more informed by actual statistical data instead of just internal model rules.

Meng: That makes sense for making the discovery step more robust when dealing with noisy data, because you want the scoring to reflect how well the graph fits what we actually measure on real traffic or patient records.

Tom: And they also suggest a task-relative evaluation, which is a major shift in how we judge these methods overall.

Jane: Instead of treating every single edge as equally important for that structural F1 score, they suggest that for a fairness audit, the edges on the Sex to MOCA Score pathway carry essentially all the weight.

Lu: This means scoring recovery of those specific edges should be given much more importance than just getting a high overall structural agreement across all connections.

Meng: That’s a huge change because it prioritizes finding what matters for fairness over achieving perfect general graph structure, which is what we usually optimize for in simple metrics.

Tom: So they are basically saying the evaluation protocol needs to change to reward methods that are good at recovering those edges critical for path-specific counterfactual fairness.

Jane: This is a strong point because it suggests that the way we measure success should be directly tied to what downstream analysis actually depends on when you're doing an audit.

Lu: It’s a way to make sure we aren't just rewarding methods that are good at making dense graphs, but methods that are good at finding those specific causal links relevant for fairness checks.

Meng: If you adopt that view, then I think the evaluation becomes much more meaningful for production because it ties the score directly to the real risks you’re trying to mitigate in deployment.

Tom: So they’re pushing for a way to score discovery methods that focuses on those edges critical for path-specific counterfactual fairness rather than just aggregate structural agreement.

Jane: It emphasizes that the choice of discovery method matters a lot when you're planning on running sensitive counterfactual fairness tests because it determines what you can even test.

Lu: They also suggest using hybrid scoring methods, combining those data-based statistics with structural rules and task-relative objectives to make the discovery process smarter overall.

Meng: That’s what we need—making sure the discovery process is better fitted to the actual data, not just coherent according to some internal model logic.

Tom: So in short, they’re pushing for a way to score discovery methods that focuses on the edges critical for path-specific counterfactual fairness rather than just aggregate structural agreement.

Lu: It emphasizes that the choice of discovery method matters a lot when you're planning on running sensitive counterfactual fairness tests because it determines what you can even test.

Conclusion: Tom: So we've talked about "GraphofThoughts for LLM Causal Discovery with a Fairness-Pathway Case Study," and what it boils down to is that structural accuracy isn't enough if you want a fair model in practice.

Jane: Right, Tom, it means GoT-CD is good at making graphs that look right on paper, but those graphs can still be missing the specific bad paths we're trying to find for fairness.

Lu: The authors show that structural errors can hide critical pathway failures in an audit if the discovery process misses the exact causal link between a protected attribute and an outcome.

Meng: For us at the startup level, it’s a warning that we need to check not just if our graph looks right, but if it actually finds those specific biased paths we are trying to stop when we deploy.

Jane: Exactly, and what they suggest is that when you chain discovery to fairness analysis you must report pathway recovery alongside your fairness estimate as a basic safeguard against miscertifying models as fair.

Tom: And they push for evaluation based on the edges that the downstream analysis actually depends on instead of just getting an aggregate structural F1 score.

Lu: They suggest using hybrid scoring methods, combining those data-based statistics with structural rules and task-relative objectives to make the discovery process smarter overall.

Meng: That’s what we need—making sure the discovery process is better fitted to the actual data, not just coherent according to some internal model logic.

Jane: It’s about making sure we aren't just rewarding methods that are good at making dense graphs, but methods that are actually finding those specific causal links relevant for fairness checks.

Tom: So they’re pushing for a way to score discovery methods that focuses on the edges critical for path-specific counterfactual fairness rather than just aggregate structural agreement.

Lu: It emphasizes that the choice of discovery method matters a lot when you're planning on running sensitive counterfactual fairness tests because it determines what you can even test.

Meng: If we adopt that view, then I think the evaluation becomes much more meaningful for production because it ties the score directly to the real risks you are trying to mitigate in deployment.

Jane: That’s a strong call to action for how we should be evaluating these tools going forward, especially when dealing with fairness in clinical settings.

Tom: So in short, the paper on "GraphofThoughts for LLM Causal Discovery with a Fairness-Pathway Case Study" shows that GoT-CD is structurally strong but needs that extra layer of fairness checking to be useful.

Lu: It lays out a clear direction for future work suggesting we need those hybrid scoring methods to combine data statistics with structural rules.

Meng: And that’s the next logical step: making the discovery process smarter by incorporating how well it fits the data, not just how coherent the model thinks its graph is.

Jane: We’ll keep an eye on those hybrid scoring methods and see if they help us build more reliable tools for auditing AI models in practice.

Tom: Next time, we're looking at something totally different—a paper about training-free style composition that might change how we control the look of generated text.

Nitish Nagesh, Elahe Khatibi, Thomas Dean Hughes, Mahdi Bagheri, Pratik Gajane, Amir M. Rahmani

Department of Computer Science, University of California, Irvine · University of Orléans

cs.LG

Submitted: 2026-08-03

Updated: 2026-11-18

License: http://creativecommons.org/licenses/by/4.0/

The gist: GoT-CD yields always-acyclic discovered graphs that are structurally competitive with large language model (LLM) baselines…

Key concepts

GoT-CD Methodology
This is a causal discovery method where the reasoning unit is an entire set of candidate edges instead of just two variables. It uses a multi-step process involving generating several candidate graphs, scoring them based on edge validity and acyclicity, aggregating them under strict constraints, and finally ensuring acyclicity through greedy projection.
Structural Fidelity vs. Fairness Fidelity
Structural fidelity refers to how well the discovered graph matches the true causal structure in terms of edges and DAG properties. Fairness fidelity refers to whether that structure correctly identifies specific unfair paths relevant for fairness audits. The paper demonstrates that high structural accuracy does not guarantee correct identification of these critical fairness pathways.
Task-Relative Evaluation
This evaluation protocol suggests that the importance of graph edges should be weighted by their relevance to the downstream analysis, rather than treating all edges equally. For a fairness audit, edges on the specific path being tested (e.g., Sex -> Brain Volume) carry significantly more weight than aggregate structural agreement metrics like F1.

Terminology

Summary

GoT-CD yields always-acyclic discovered graphs that are structurally competitive with large language model (LLM) baselines, yet that structural fidelity alone does not guarantee fairness-faithful audits. p. 0

Where the paper says it

Here we show that full-graph Graphof-Thoughts reasoning yields always-acyclic discovered graphs that are structurally competitive with large language model (LLM) baselines, yet that structural fidelity alone does not guarantee fairness-faithful audits. We introduce GoT-CD, in which the reasoning unit is a complete candidate edge set: multiple graphs are generated in parallel, scored by a deterministic validity function, and merged under a hard union constraint that forbids invented edges, with greedy projection enforcing a DAG before commitment.

This work introduces GoT-CD to address the knowledge gap between causal discovery and path-specific counterfactual fairness by showing that structural accuracy does not imply fairness fidelity

GoT-CD Methodology

GoT-CD is a causal discovery method in which the reasoning unit is a complete candidate edge set rather than a variable pair or traversal step. The method executes a single pass of a Graph of Operations Generate(k) → Score → KeepBestN(min(2, k)) → Aggregate → Improve → Score → KeepBestN(1), followed by deterministic post-processing. p. 6

Where the paper says it

2 Graph-of-Thoughts Causal Discovery (GoT-CD) GoT-CD treats causal discovery as a full-graph reasoning problem: a thought is a complete candidate edge set over all variables, rather than a single pairwise judgement [16] or the expansion of one node in a traversal [13]. The method executes a single pass of a Graph of Operations [4], Generate(k) → Score → KeepBestN(min(2, k)) → Aggregate → Improve → Score → KeepBestN(1), (1) followed by deterministic post-processing.

The Generate step prompts the model once with the variable names and descriptions and a domain preamble, requesting k distinct candidate graphs. p. 6
Where the paper says it

The method executes a single pass of a Graph of Operations [4], Generate(k) → Score → KeepBestN(min(2, k)) → Aggregate → Improve → Score → KeepBestN(1), (1) followed by deterministic post-processing. Generate prompts the model once with the variable names and descriptions and a domain preamble, requesting k distinct candidate graphs; responses are parsed into edge lists, with multiword variable names matched longest-first so that composite names such as Brain Volume are not truncated.

The Score applies a deterministic local function—not a model call—that penalizes edges referencing undeclared variables (−5 each) and self-loops (−3 each), rewards the count of well-formed edges, adds a bonus for acyclicity (+3) or a penalty otherwise (−4), and applies a soft density prior discouraging graphs with fewer than 0. p. 6
Where the paper says it

Generate prompts the model once with the variable names and descriptions and a domain preamble, requesting k distinct candidate graphs; responses are parsed into edge lists, with multiword variable names matched longest-first so that composite names such as Brain Volume are not truncated. Score applies a deterministic local function—not a model call—that penalizes edges referencing undeclared variables (−5 each) and self-loops (−3 each), rewards the count of well-formed edges, adds a bonus for acyclicity (+3) or a penalty otherwise (−4), and applies a soft density prior discouraging graphs with fewer than 0.

5n or more than 2.5n edges. The Aggregate step merges the retained candidates into a single graph under a hard union constraint: the prompt instructs the model to emit only edges appearing in at least one input candidate, and the parser independently filters the response against that union. p. 6
Where the paper says it

Using a deterministic scorer rather than model-based self-evaluation keeps candidate ranking reproducible and adds no inference cost. Aggregate merges the retained candidates into a single graph under a hard union constraint: the prompt instructs the model to emit only edges appearing in at least one input candidate, and the parser independently filters the response against that union.

Acyclicity is guaranteed by a final greedy projection: edges are traversed in order and each is retained only if it does not create a directed cycle with those already kept. p. 6
Where the paper says it

Where Improve empties the graph entirely, the procedure falls back to the union of the highest-scoring Generate candidates. Acyclicity is guaranteed by a final greedy projection: edges are traversed in order and each is retained only if it does not create a directed cycle with those already kept.

Comparison Against Baselines

GoT-CD produces a valid DAG on all five reported benchmarks and achieves the best DAG-valid F1 among LLM methods on Asia (0.750), Alzheimer’s (0. p. 0

Where the paper says it

Under a locked n=100 protocol with backbone gpt-4o-mini, GoT-CD returns a valid DAG on all five reported benchmarks and achieves the best DAG-valid F1 among LLM methods on Asia (0. 750), Alzheimer’s (0.

757), and COVID-Respiratory (0. p. 0
Where the paper says it

750), Alzheimer’s (0. 757), and COVID-Respiratory (0.

688), winning against LLM-BFS on four of five datasets. p. 0
Where the paper says it

757), and COVID-Respiratory (0. 688), winning against LLM-BFS on four of five datasets.

On an Alzheimer’s benchmark with known unfair path Sex → Brain Volume → MOCA Score, GoT-CD recovers the groundtruth unfair pathway with correct effect sign (TElin=−0. p. 2
Where the paper says it

On Alzheimer’s it additionally recovers the groundtruth unfair pathway with correct effect sign (TElin=−0. benchmark −0. p. 0

Where the paper says it

688), winning against LLM-BFS on four of five datasets. On an Alzheimer’s benchmark with known unfair path Sex → Brain Volume → MOCA Score, a posthoc path-specific audit shows that five of eight discovered graphs— including LLM-BFS despite competitive structural F1 (0.

580 vs. benchmark −0.572), whereas LLMBFS—despite competitive structural F1 (0. p. 2
Where the paper says it

benchmark −0. 572), whereas LLMBFS—despite competitive structural F1 (0.

649)—recovers no S→Y path and consequently reports PSE·=0. p. 2
Where the paper says it

572), whereas LLMBFS—despite competitive structural F1 (0. 649)—recovers no S→Y path and consequently reports PSE|·|=0.

Structural Fidelity vs. p. 0

Where the paper says it

Discovery methods are routinely scored on aggregate structural metrics that weight all edges equally, and no established evaluation asks whether the specific pathway an audit depends on survives discovery—or what the audit reports when that pathway is missing. Here we show that full-graph Graphof-Thoughts reasoning yields always-acyclic discovered graphs that are structurally competitive with large language model (LLM) baselines, yet that structural fidelity alone does not guarantee fairness-faithful audits.

Fairness Fidelity

Structural fidelity does not imply fairness fidelity, as demonstrated on the Alzheimer’s benchmark where GoT-CD is both the structural F1 leader (0.757) and recovers the groundtruth unfair path Sex → Brain Volume → MOCA Score. p. 7

Where the paper says it

A companion discrete estimate binarizes the protected attribute, mediators, and outcome at their medians, fits logistic regressions on the induced S–M–Y subgraph, and computes the natural direct and indirect effects [7, 23]. Alongside these quantities we report whether each discovered graph recovers the reference unfair path Sex → Brain Volume → MOCA Score verbatim.

Conversely, LLM-BFS, despite competitive structural F1 (0. p. 0
Where the paper says it

688), winning against LLM-BFS on four of five datasets. On an Alzheimer’s benchmark with known unfair path Sex → Brain Volume → MOCA Score, a posthoc path-specific audit shows that five of eight discovered graphs— including LLM-BFS despite competitive structural F1 (0.

649), recovers no S→Y path and therefore reports PSE·=0. p. 2
Where the paper says it

572), whereas LLMBFS—despite competitive structural F1 (0. 649)—recovers no S→Y path and consequently reports PSE|·|=0.

GES recovers the same pathway but inflates total path mass more than sevenfold via eight additional spurious routes (PSE·=4. p. 2
Where the paper says it

649)—recovers no S→Y path and consequently reports PSE|·|=0. GES recovers the same pathway but inflates total path mass more than sevenfold via eight additional spurious routes.

034).

Implications for Clinical Auditing

The practical consequence is that pipelines chaining discovery to fairness analysis inherit discovery errors in a form that the fairness output may not flag, as a graph missing the protected attribute pathway yields a confident zero rather than an uncertain estimate. Therefore, reporting pathway recovery alongside the fairness estimate is considered a minimal safeguard. p. 2

Where the paper says it

6 Implications The practical consequence for clinical model auditing is that pipelines chaining discovery to pathspecific fairness inherit discovery errors in a form that the fairness output may not flag: a graph missing the protected attribute pathway yields a confident zero rather than an uncertain estimate. Reporting pathway recovery alongside the fairness estimate is therefore a minimal safeguard, and evaluation of discovery methods intended for such pipelines should be scored on the edges the downstream analysis depends on—not solely on aggregate structural agreement.

Future Directions

Future work suggests a hybrid in which the scoring step of the Graph-of-Thoughts pipeline incorporates a data-based score (BIC, or conditional independence test statistics) alongside the current structural heuristics. Additionally, task-relative evaluation suggests a fairness-aware discovery objective where the procedure is scored partly on the structural determinacy of the paths between them. p. 2

Where the paper says it

Reporting pathway recovery alongside the fairness estimate is therefore a minimal safeguard, and evaluation of discovery methods intended for such pipelines should be scored on the edges the downstream analysis depends on—not solely on aggregate structural agreement. A fuller interpretation of these findings, including differentiation from prior LLM discovery families, limitations of the present protocol, and directions for score-guided and fairness-aware variants, is developed in Section 6.

Complementarity of Priors

LLM priors and statistical signal are complementary in that LLM-based methods outperform classical baselines on compact, knowledge-rich benchmarks (Asia, Alzheimer’s), while continuous-relaxation methods dominate when the SEM assumptions match the data (Sweden-Traffic). This complementarity motivates future score-guided GoT variants that rank candidate graphs by data fit as well as by internal coherence. p. 4

Where the paper says it

Its consistent underperformance relative to full-graph GoT-CD—on structural F1 on four of five benchmarks, and on Alzheimer’s unfair-path recovery—suggests that the benefit of the reasoning topology depends on operating over the global structure rather than local expansions. This finding also motivates, but does not yet implement, a score-guided GoT variant in which the deterministic scorer is replaced or augmented by a data-based score (BIC or CI statistics), closing the loop between the hybrid literature and the global-thought design.

Evaluation Protocol Shift

The work advocates for task-relative evaluation, suggesting that structural F1 treats all edges as equally important, but for a fairness audit the edges on the S→Y pathway carry essentially all the weight. This suggests that scoring recovery of the edges the downstream analysis actually depends on gives a more informative ranking than aggregate structural agreement alone. p. 0

Where the paper says it

572), a false-clean certificate indistinguishable from genuine fairness in the audit output alone; GoT-CD recovers the pathway with correct effect sign while leading structural recovery, whereas GES recovers the pathway but inflates path mass more than sevenfold. Pipelines that chain discovery to fairness analysis should therefore report pathway recovery alongside the fairness estimate, and discovery methods intended for such pipelines should be evaluated on the edges the downstream analysis depends on rather than on aggregate structural agreement alone.

Limitations and Constraints

Limitations include using a single LLM backbone (gpt-4o-mini) at a single main branch factor (k=3; ablated at k=1, 5) and sample size (n=100). Furthermore, the fairness case study rests on a single protected attribute (Sex), a single outcome (MOCA Score), and a single designated unfair path. p. 6

Where the paper says it

Sex is designated the protected attribute and MOCA Score the outcome; the reference graph contains exactly one directed path between them, Sex → Brain Volume → MOCA Score, which is treated as the unfair pathway. 2 Graph-of-Thoughts Causal Discovery (GoT-CD) GoT-CD treats causal discovery as a full-graph reasoning problem: a thought is a complete candidate edge set over all variables, rather than a single pairwise judgement [16] or the expansion of one node in a traversal [13].

The computation of path-specific effects makes additional assumptions worth naming, such as assuming linearity and no unobserved confounding beyond what the graph encodes. p. 3
Where the paper says it

GES [8] performs greedy forward and backward edge search over equivalence classes under a Bayesian information criterion; generalized scores relax parametric assumptions within the same search skeleton [12], while exact dynamic-programming and shortest-path formulations target globally optimal Bayesian networks [34, 44] at higher computational cost. Functional-causal models such as LiNGAM exploit non-Gaussianity of noise to identify orientation beyond Markov equivalence under linearity assumptions [32, 33].

Conclusion on Discovery

GoT-CD is not uniformly dominant on structural F1 (Child remains difficult for all LLM methods), but when it errs it does so with conservative edge sets rather than dense correlation graphs. The union constraint on aggregation is the mechanism that makes this merge safe, preventing the model from synthesizing plausible-sounding edges that no reasoning branch proposed. p. 6

Where the paper says it

Aggregate merges the retained candidates into a single graph under a hard union constraint: the prompt instructs the model to emit only edges appearing in at least one input candidate, and the parser independently filters the response against that union. This constraint prevents the model from synthesizing plausible-sounding edges that no reasoning branch proposed—a failure mode we observed when aggregation was left unconstrained.

This indicates that the gain comes from the global thought representation rather than from Graphof-Thoughts machinery per se. p. 7
Where the paper says it

Three LLM baselines share the same gpt-4o-mini backbone: pairwise querying without correlation statistics [16], LLM-BFS without statistics [13], and GoT-CD-BFS, a hybrid that retains the breadth-first traversal but replaces each single-shot query with a Graph-of-Thoughts sub-pipeline over candidate variable sets rather than full graphs. Comparing GoT-CD against GoT-CD-BFS isolates the contribution of the global thought representation from that of Graph-ofThoughts reasoning as such.

Final Recommendation

Pipelines that chain causal discovery to path-specific fairness should report pathway recovery alongside the fairness estimate, and should score discovery methods on the edges the audit depends on—not solely on aggregate structural F1. This is presented as a minimal safeguard against certifying unfair models as fair. p. 2

Where the paper says it

6 Implications The practical consequence for clinical model auditing is that pipelines chaining discovery to pathspecific fairness inherit discovery errors in a form that the fairness output may not flag: a graph missing the protected attribute pathway yields a confident zero rather than an uncertain estimate. Reporting pathway recovery alongside the fairness estimate is therefore a minimal safeguard, and evaluation of discovery methods intended for such pipelines should be scored on the edges the downstream analysis depends on—not solely on aggregate structural agreement.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Full-graph Graph of Thoughts reasoning yields always-acyclic discovered graphs that are structurally competitive with large language model baselines, yet that structural fidelity alone does not guarantee fairness-faithful audits. This allows for the creation of discovery methods that produce DAGs, which are necessary for path-specific fairness analysis, avoiding the need for heuristic cycle breaking.

  2. A post-hoc path-specific fairness audit should be reported alongside structural metrics to ensure downstream analysis is grounded in the specific structure relevant to the audit. The paper suggests: Pipelines that chain discovery to fairness analysis should therefore report pathway recovery alongside the fairness estimate.

  3. Discovery methods intended for pipelines that chain discovery to fairness analysis should be evaluated on the edges the downstream analysis depends on rather than on aggregate structural agreement alone. This shifts the evaluation metric from aggregate structural metrics like F1 to task-relative evaluation based on the edges critical for path-specific counterfactual fairness.

Abstract

Methods that use large language models (LLMs) for causal discovery reason locally, deciding one variable pair or one node at a time, so they never compare alternative whole graphs. We introduce GoT-CD, a Graph-of-Thoughts method that asks an LLM for several complete candidate graphs from variable metadata alone, ranks them with a deterministic validity score, merges them under a hard union constraint, refines the merge, and projects the result to a DAG. We evaluate on Asia, Child, and an Alzheimer's disease graph with 100 samples and GPT-4o-mini. Over five seeds on Alzheimer's and Child, GoT-CD gives the highest mean F1 score and the lowest mean structural Hamming distance of the LLM methods, with 95% intervals of the difference excluding zero in seven of eight comparisons, and it returns a DAG in every run while using three LLM requests per graph. Replaying the logged runs without individual components attributes most of the gain to generating and merging several complete candidates. A case study on the effect of Sex on MOCA Score in the Alzheimer's graph shows the limits of structural scores: the fairness verdict drawn from the discovered graph disagrees with the reference verdict in twelve of fifteen LLM runs, including all five GoT-CD runs, which mostly drop the edge out of Sex. Graphs meant for a specific downstream analysis should therefore also be checked on the pathways that analysis depends on.

Sources

Related papers