Shaking the causal tree: On the faithfulness and minimality assumptions beyond pairwise interactions
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Condensed Matter. The day in condensed matter: materials, phases and the measurements under them.
Kai: We're looking at the paper: "Shaking the causal tree".
Mira: Here's the gist of the paper. Shaking the causal tree: On the faithfulness and minimality assumptions beyond pairwise interactions shows how genuine causal synergism violates faithfulness in non-pairwise systems, necessitating an update to causal discovery algorithms Here, we show that causal emergent information is necessarily contained in CMIs and finally we show how faithfulness can be wrongly assumed only because of the appearance of spurious correlations by providing an example of a non-pairwise system which should violate faithfulness, in principle, but it does not.
Kai: The Problem with Standard Causal Discovery Causal discovery algorithms propose the breakdown of mutual information (MI) and conditional mutual information (CMI) into sets of variables to reveal causal influences.
Mira: These algorithms suffer from the lack of accounting emergent causes when connecting links, resulting in a spuriously embellished view of the organization of complex systems.
Kai: The emerging field of non-pairwise network modeling is elucidating how standard networks embed pairwise relationships into our structural interpretation.
Paper summary: Kai: To get into the details of "Shaking the causal tree: On the faithfulness and minimality assumptions beyond pairwise interactions", they start by pointing out a core limitation in how causal discovery algorithms work. They say that breaking down mutual information or conditional mutual information into simple sets of variables doesn't account for emergent causes when those links are connected.
Mira: The paper claims that this leads to a spuriously embellished view of the organization of complex systems, and they introduce non-pairwise network modeling to show how standard networks embed these pairwise relationships in a way that’s misleading.
Lev: It sounds like they are saying that transfer-like analysis, which often relies on pairwise interactions, gets blinded when you have synergistic dependencies because those concepts of faithfulness can fail completely. That's a concern for any error correction researcher because we deal with systems where those synergies are crucial.
Kai: They formalize synergism using something called Partial Information Decomposition, or PID. They realize that mutual information isn't enough to understand how information is distributed across multiple interacting variables simultaneously.
Mira: The PID framework helps by identifying what they call partial informational atoms, or PI’s, defined by a specific mapping. This allows them to define the causal link from a set of sources to the target as being due to a synergic term called Syn.
Lev: So, when you look at how information flows through multiple variables at once, instead of just looking at I(X one;Y) and I(X two;Y), you have to consider this structure where the influence is distributed across sets <ref:2203.10665#pg2>. That sounds computationally heavy for real-world systems.
Kai: They then define faithfulness and minimality using formal definitions that link them directly to this PID language, which helps isolate when a causal influence actually satisfies faithfulness, specifically when that source is independent of the others' parents.
Mira: The paper shows that conditioning on a set of variables changes the information between two others in a specific way, and this decomposition relates back to terms in the informational lattice. It confirms that to get information from an influence of order k, you have to account for conditioned sets of proportional size.
Lev: That suggests that maybe the pairwise interaction regime—where you only look at simple conditional mutual information—is actually the only case where those standard MI values are nonzero, which is a serious challenge if we're trying to model real-world complexity.
Conclusion: Kai: So, looking back at "Shaking the causal tree: On the faithfulness and minimality assumptions beyond pairwise interactions", the authors are really pushing back on how we define causality in complex systems that aren't just simple pairs. They show that when you go beyond pairwise interactions, faithfulness can fail because of things called spurious correlations.
Mira: That failure isn't just a technical glitch; it stems from the functional properties of the system itself, which induces this specific type of redundancy. The paper suggests we need to incorporate context-specific independence to make probabilistic inference more efficient when dealing with synergism.
Lev: From an engineering standpoint, if this is true, it means our current causal discovery algorithms are missing genuine causal influences because they are being fooled by these induced regularities instead of finding the real cause. That points toward a necessary update for how we model complex systems that have deep dependencies.
Kai: The paper proposes that the separation of the informational lattice into sets R(X) and S(X), which they call the strong synergism condition, is something missing from this field of causal synergism research. It's a new concept we need to formalize.
Mira: It’s about moving beyond just checking if a link exists, to understanding how that link is structured across higher orders of interaction. This whole discussion strengthens the connection between causality and information theory in ways the previous work didn't fully explore.
Lev: So what this means for me working on error correction, it suggests that when we look at high-order interactions, we need a framework that explicitly handles this type of redundancy to ensure our assumptions about causal minimality hold true.
Kai: Ultimately, the paper suggests an update to causal discovery algorithms so they can actually detect and isolate emergent causal influences that are currently being missed by relying only on pairwise information.
Tiago Martinelli, Diogo O. Soares-Pinto, Francisco A. Rodrigues
Instituto de Física de Sãao Carlos, Universidade de Sãao Paulo · Institute of Mathematics and Computer Science, University of Sãao Paulo
cond-mat.stat-mech, physics.data-an
Submitted: 2022-03-20
Updated: 2026-11-08
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 92/100
The gist: Shaking the causal tree: On the faithfulness and minimality assumptions beyond pairwise interactions shows how genuine causal synergism violates faithfulness in non-pairwise systems…
Key concepts
- Faithfulness (F)
- Faithfulness means a structural model accurately reflects the true causal relationships. Formally, it requires that if you condition on all parents of a node in the graph, that node becomes conditionally independent of its other parents. The paper shows this assumption fails when complex, non-pairwise interactions are present.
- Causal Minimality (M)
- Minimality requires that no node in the system is conditionally independent of any of its parents given the remaining parents. This condition ensures that every causal influence is explicitly represented in the structure. The paper explores how this property relates to faithfulness and how it can be bypassed by certain types of correlations.
- Partial Information Decomposition (PID)
- PID is a framework used to understand how information is distributed across multiple interacting variables, going beyond simple mutual information. It breaks down total information into 'partial informational atoms' (PIs) that capture synergistic dependencies, which standard methods often miss.
- Strong Synergism Condition
- This condition separates the informational lattice into two subsets, R(X) and S(X). This separation is key to understanding when faithfulness holds versus when it fails in non-pairwise systems. It helps distinguish genuine causal influence from spurious regularities.
Terminology
Summary
Shaking the causal tree: On the faithfulness and minimality assumptions beyond pairwise interactions shows how genuine causal synergism violates faithfulness in non-pairwise systems, necessitating an update to causal discovery algorithms Here, we show that causal emergent information is necessarily contained in CMIs and finally we show how faithfulness can be wrongly assumed only because of the appearance of spurious correlations by providing an example of a non-pairwise system which should violate faithfulness, in principle, but it does not. p. 0 Finally, we show how faithfulness can be wrongly assumed only because of the appearance of spurious correlations by providing an example of a non-pairwise system
which should violate faithfulness, in principle, but it does not. The net result proposes an update to
causal discovery algorithms, which can, in principle, detect and isolate emergent causal influences in
the network reconstruction problems undetected so far.Where the paper says it
The Problem with Standard Causal Discovery
Causal discovery algorithms propose the breakdown of mutual information (MI) and conditional mutual information (CMI) into sets of variables to reveal causal influences. These algorithms suffer from the lack of accounting emergent causes when connecting links, resulting in a spuriously embellished view of the organization of complex systems. p. 0 Rodrigues
Institute of Mathematics and Computer Science, University of S˜ao Paulo, 13566-590, S˜ao Carlos, SP, Brazil
(Dated: August 9, 2022)
Built upon the concept of causal faithfulness, the so-called causal discovery algorithms propose
the breakdown of mutual information (MI) and conditional mutual information (CMI) into sets of
variables to reveal causal influences. These algorithms suffer from the lack of accounting emergent
causes when connecting links, resulting in a spuriously embellished view of the organization of complex systems. To infer correct causal links from
multivariate time-dependent data, the recent discovery
algorithms use transfer-like quantifiers relying on concepts such as Granger causality and directed information theory [13–15], which are already formally justified
by causal concepts such as d-separation and faithfulness
[16–18]. The emerging field of non-pairwise network modeling [19, 20] is elucidating how standard networks embed
pairwise relationships into our structural interpretation
of the organization and behavior of complex systems
∗
tiago. As elucidated in Ref. [10], but
not taken forward, the point is that for synergic dependencies the concept of faithfulness can be violated by
provoking the failure of all the algorithms above.Where the paper says it
Where the paper says it
Where the paper says it
Formalizing Synergism via Partial Information Decomposition
The PID framework addresses the issue where MI gives little insight into how information is distributed over sets of multiple interacting variables. The work of Williams & Beer was to realize that such combinations of information are well structured into a lattice of antichains. p. 1 The PID framework addresses the issue above by a
formal method to decompose the contribution of all informational combinations that a set of multiple sources
variables provides from a single target variable. The
work of Williams & Beer [28] was to realize that such
combinations of information are well structured into a
lattice of antichains,
A(X) = {α ∈ P+(P
+(X)) : a1 6⊂ a2, ∀a1, a2 ∈ α}, (2)
where P
+(S) = P(S) \ {∅} denotes the set of
nonempty subsets of S. The
work of Williams & Beer [28] was to realize that such
combinations of information are well structured into a
lattice of antichains,
A(X) = {α ∈ P+(P
+(X)) : a1 6⊂ a2, ∀a1, a2 ∈ α}, (2)
where P
+(S) = P(S) \ {∅} denotes the set of
nonempty subsets of S. These possible combinations of information are called
partial informational atoms (PI’s) and are defined by the
mapping: α 7→ I∂
(α;Y). where Red(X1, X2;Y) is the PI about Y that is redundantly shared between X1 and X2, Uni(X1;Y) refers to
the PI about Y that is uniquely present in X1 and not
in X2, and Syn(X1, X2;Y) is the PI about Y that is only
revealed by the joint states of X1 and X2 considered together. 5 we can see that the causal link from
{X1, X2} to Y is due to the synergic term Syn(X1, X2;Y).Where the paper says it
Where the paper says it
Where the paper says it
Faithfulness and Minimality in the PID Language
Definition A3 defines faithfulness (F) as P being faithful with respect to a DAG G if (XK =PYXL) whenever (K ∩ PAY) 6= (L ∩ PAY). Causal minimality (M) is satisfied if and only if there is no node that is conditionally independent of any of its parents, given the remaining parents. p. 7 8
Condition (F) ensures that the set of causal parents is unique and that every causal parent presents an observable
effect regardless of the information about other causal parents [3, 9]. On the other hand, (M) says that a distribution
is minimal with respect to a causal graph if and only if there is no node that is conditionally independent of any of
its parents, given the remaining parents. This is in
conflict with the definition of (F), then
Proposition A1 (Faithfulness implies causal minimality). If P is faithful and Markovian with respect to G, then causal
minimality is satisfied. Proposition 2 (Faithfulness & Minimality). Let α ∈
A(PAY) be, then the causal influence of α in Y satisfies,
(a) faithfulness if α, necessarily, belongs to S
(2)
, i. Let α ∈
A(PAY) be, then the causal influence of α in Y satisfies,
(a) faithfulness if α, necessarily, belongs to S
(2)
, i. , is independent of the others parents; and,
(b) minimality/contextual-dependency if α belongs to
S
(k≥2) ∀k, i.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The Role of Conditioning Sets in Capturing High-Order Effects
Proposition B2 shows that the operation of conditioning on Z the information between X and Y is given by I(X;Y Z) = I(X, Z;Y) − I(Z;Y). This formula decomposes into a sum over terms in the complementary set of the informational lattice. p. 2 The calculation of the CMI
was made over all permutations of the set {Xi}
4
i=1
in the CMI
formula. Note that, by viewing A(X) as a causal informational
lattice, A(PAY), where PAY ≡ X emphasizes that X is
the set of parents of Y, and using the terminology of
Def. For these systems, we calculate the average normalized CMI, CMI, to measure the strength of the highorder statistical effects beyond pairwise interactions
(Fig. Our results confirm that to get information from
a causal influence of order k we have to account for
conditioned sets of proportional size. Our results confirm that to get information from
a causal influence of order k we have to account for
conditioned sets of proportional size. Furthermore, the
pairwise interaction regime is the only case where MIs
(CMIs with |Z| = ∅) are nonzero showing the violation
of faithfulness for non-pairwise interactions.Where the paper says it
Where the paper says it
Where the paper says it
Consequences for Causal Discovery
The results show that while well accepted, faithfulness fails to capture synergism when the latter is properly defined. This failure is related to the appearance of spurious regularities instead of genuine causal influences. p. 3 We showed that, while well accepted, faithfulness fails to capture synergism when the
latter is properly defined. And, its assumption is more
related to the appearance of spurious regularities than
genuine causal influences. More intriguing, faithfulness beats minimality and the latter does not seem to be optimal in capturing synergism anymore, Fig. We argue that such behavior is due to the appearance
of a specific type of redundancy, induced by the functional properties of the system. , the context. It has been shown that the presence and knowledge
of such independence lead to more efficient probabilistic
inference by exploiting the local structure of the causal
models [36]. Further investigations of
this approach in synergic causal discovery from large
data sets we leave for future work. Also, our separation of the informational antichain
in the subsets
R
(
X
) and
S
(
X
), called strong synergism
condition [39], has not been incorporated in the field of
causal synergism/emergence [40
–42].Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Simulation Insights on Redundancy Dominance
The simulation results show that when the Hamiltonian only possesses exclusive non-pairwise interactions, faithfulness still fails in capturing causal influence. However, when we relax the exclusivity condition, even without pairwise interactions, the MI’s are nonzero anymore. p. 3 Also, the monotonic increase of the CMI according to the conditioning set size is preserved showing how minimality remains robust to identify high-order of synergism, Fig. However, when we relax the exclusivity condition,
even without pairwise interactions, the MI’s are nonzero
anymore. However, when we relax the exclusivity condition,
even without pairwise interactions, the MI’s are nonzero
anymore. More intriguing, faithfulness beats minimality and the latter does not seem to be optimal in capturing synergism anymore, Fig. While its importance has been
recognized, how to define or quantify this type of redundancy inside PID is an open question [32]. In our
case, it is due to the existence of correlations between
atoms not explored in the informational antichain, e. In what follows
we consider basic concepts from information theory and
causality theory1
[3]. The amount of information the
sources X (also called parents, PAY)
2
, carry about the
target Y is quantified by the mutual information (MI),
I(X;Y) = ∑
x,y∈X×Y
p(X,Y)log p(X,Y)
p(X)p(Y)
. 3-(B), we have MI(Xi
, Xj) 6= 0. The existence
of these correlations among the sources spuriously inflates the calculation of MIs while it provokes the decrease of CMIs according to the conditioning set size. Such redundancy is similar to the concept of mechanistic nature [30, 31]. In its simplest form, this type of
redundancy occurs when the sources are generated independently and are related to the functional properties
of the system as well.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Final Interpretation and Future Directions
The paper concludes that this specific spuriousness can be linked with the concept of (mechanistic) redundancy. A qualitative approach is missing in the literature to formally define or quantify this type of redundancy inside PID. p. 3 In its simplest form, this type of
redundancy occurs when the sources are generated independently and are related to the functional properties
of the system as well. While its importance has been
recognized, how to define or quantify this type of redundancy inside PID is an open question [32]. This would be important for a better understating of some phenomena from multivariate information theory. A possible path in this domain could be to incorporate
the so-called context-specific independence (CSI) [35],
which is the independence that holds in a certain value
of conditioned variables, i. Also, it allows the identification of causal effects,
which would not be possible without any information
about CSI relationships [38]. Further investigations of
this approach in synergic causal discovery from large
data sets we leave for future work. Further investigations of
this approach in synergic causal discovery from large
data sets we leave for future work. Also, our separation of the informational antichain
in the subsets
R
(
X
) and
S
(
X
), called strong synergism
condition [39], has not been incorporated in the field of
causal synergism/emergence [40
–42]. Conclusions and Consequences. The XOR problem is a
classical problem in the domain of AI which was one
of the reasons for the AI winter during the 70s in the
neural networks context [34].Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Simulation Details and Formal Structures
For all Figs.2, 3 the systems were of n + 1 spins, where Xi = [-1, 1] for i = 1,. p. 2 To do so, we consider a system of n + 1 spins, with Hamiltonians having
interactions only of order k,
Hk
(X) = − ∑
|α|=k
Jα ∏
i∈α
XiXn+1, (6)
where Jα are the interaction coefficients the and the sum
runs over all collections of indices α ⊆ [n] of cardinality
|α| = k. For these systems, we calculate the average normalized CMI, CMI, to measure the strength of the highorder statistical effects beyond pairwise interactions
(Fig. 2, 3 the systems were of n + 1 spins, where Xi = {−1, 1} for i = 1, . , n + 1 whose joint probability
distributions can be expressed in the form pXn+1
(xn+1) = exp{Hk
(Xn+1)}/Z, with β the inverse temperature choose
as 1, Z the normalisation constant to make sure that the pXn+1
’s are probabilities, and Hk
(Xn+1) the Hamiltonian
function. , n + 1 whose joint probability
distributions can be expressed in the form pXn+1
(xn+1) = exp{Hk
(Xn+1)}/Z, with β the inverse temperature choose
as 1, Z the normalisation constant to make sure that the pXn+1
’s are probabilities, and Hk
(Xn+1) the Hamiltonian
function. In all simulations, all interaction coefficients J in the Hamiltonians were generated i. In all simulations, all interaction coefficients J in the Hamiltonians were generated i. from a uniform
distribution weighted by the coefficient 0. For a causal graph, there is the additional requirement that the edges E capture causal dependencies (instead of only correlations) between nodes. This means that the decomposition P(x) = ∏i P(xi
|pai
) holds, even if the parent variables are actively
set into their state as opposed to passively observed in that state (causal Markov condition (CMC) [3, 43]),
P(xi
|pai
) = ∏
i
P(xi
|pai
) = ∏
i
P(xi
|do(pai
)) (A2)
Definition A2 (Spatio-temporal Causal Models). B2 ∀Z with |Z| = k − 2
we have that α ∈ S(k≥2)
. Graphical illustration of Def. 1 for the informational lattice
A(X) with X = {X1
, X2, X3}. Coloured boxes explanation:
atoms inside S
(3)
(X); + atoms inside R(3)
(X);
atoms inside R
(3)
E
(X). Illustration of the effect of conditioning the information between X = {X1
, X2, X3} and target node Y
on the element {X2X3} ∈ X
− against {X3} ∈ for informational lattice A(X). Coloured boxes explanation:
goes away when X1
| {X3}; + goes away when
X1
| {X2, X3}; remains only when X1
| {X2, X3}.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Summary of Key Findings
The XOR problem is a classical problem in the domain of AI which was one of the reasons for the AI winter during the 70s in the neural networks context. The paper shows how the difficulties with concepts of statistical independence for the XOR operation raise a problem in the causal discovery domain, as well as when dealing with nonpairwise/synergic relationships. p. 3 The XOR problem is a
classical problem in the domain of AI which was one
of the reasons for the AI winter during the 70s in the
neural networks context [34]. Here, we showed how
the difficulties with concepts of statistical independence
for the XOR operation raise a problem in the causal
discovery domain, as well as when dealing with nonpairwise/synergic relationships. Here, we showed how
the difficulties with concepts of statistical independence
for the XOR operation raise a problem in the causal
discovery domain, as well as when dealing with nonpairwise/synergic relationships. By doing so, we raised
the importance of incorporating concepts, namely, partial information theory, beyond the classical Shannon
approach when dealing with the concept of synergism. By doing so, we raised
the importance of incorporating concepts, namely, partial information theory, beyond the classical Shannon
approach when dealing with the concept of synergism. This strengthens the already well-established link between causality and information theory. This strengthens the already well-established link between causality and information theory. Specifically, we have elucidated a new way to interpret the concepts of faithfulness and minimality from
the causality domain clarifying the failure and importance of these conditions for network discovery beyond
pairwise interaction. Specifically, we have elucidated a new way to interpret the concepts of faithfulness and minimality from
the causality domain clarifying the failure and importance of these conditions for network discovery beyond
pairwise interaction. We showed that, while well accepted, faithfulness fails to capture synergism when the
latter is properly defined. We showed that, while well accepted, faithfulness fails to capture synergism when the
latter is properly defined. And, its assumption is more
related to the appearance of spurious regularities than
genuine causal influences. Finally, we show how faithfulness can be wrongly assumed only because of the appearance of spurious correlations by providing an example of a non-pairwise system
which should violate faithfulness, in principle, but it does not. The net result proposes an update to
causal discovery algorithms, which can, in principle, detect and isolate emergent causal influences in
the network reconstruction problems undetected so far.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Incorporate partial information theory into causal discovery algorithms to detect emergent causal influences in network reconstruction problems undetected so far. This allows for a more accurate understanding of complex system organization beyond pairwise interactions by connecting conditional mutual information to partial information decomposition theory (PID).
-
Update causal discovery algorithms by using the concept of conditioning operation in mutual information to capture pure synergism by formally connecting such concepts with partial information decomposition theory (PID). This update helps in isolating emergent causal influences that are otherwise missed when only considering pairwise interactions.
-
Develop a method to detect and isolate emergent causal influences in the network reconstruction problems undetected so far by analyzing the structure of the informational antichain A(X) and distinguishing between sets S(X) (synergic sets) and R(X) (redundant sets). This provides a framework to clarify why faithfulness fails for non-pairwise interactions while minimality remains robust.
-
Implement a system that leverages the causal minimality condition, defined as
no node that is conditionally independent of any of its parents, given the remaining parents,
to ensure all causal parents areactive.
This ensures that every edge in the causal graph represents an observable effect regardless of information about other causal parents.
Abstract
Higher-order mechanisms can make genuine causes invisible to causal discovery, but whether this reflects fine tuning or a structural limitation remains unclear. We prove, for a binary causal family, that the failure of faithfulness is not a measure-zero pathology: it persists for every nonzero coupling vector, even though every active edge remains causally minimal. Experiments with established causal-discovery methods show how this separation propagates into algorithm-dependent failures. A Partial Information Decomposition then identifies the lattice supports probed by faithfulness and minimality, placing the two premises at opposite ends of a single conditioning hierarchy. This exposes a two-sided obstruction: synergy can erase faithfulness, while redundancy can make the positive margin protecting minimality arbitrarily weak. Exact causal premises and practical discoverability can therefore point in different directions
Sources
- Nonnegative Decomposition of Multivariate Information
- Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic
- Causal network discovery by iterative conditioning: comparison of algorithms
Related papers
- Emergent long-tail dynamics in driven magnets with dynamical frustration
- Statistical Mechanics of Dynamical System Identification
- Second law of thermodynamics in closed quantum many-body systems
- Chemical potentials from structure factors: II. Charged multi-component mixtures
- Genuine quantum scars in Floquet chaotic many-body systems
- Stochastic Thermodynamics for Autoregressive Generative Models: A Non-Markovian Perspective