PISA: Prioritized Invariant Subgraph Aggregation for Out-of-Distribution Generalization on Graphs

The gist: PISA introduces Prioritized Invariant Subgraph Aggregation for Out-of-Distribution Generalization on Graphs, which shows that learning an instance-conditional MLP over frozen branch logits is a safe and effective way to combine multiple invariant predictors, improving OOD generalization by 2.96 points on synthetic shifts and 1.74 points on realistic shifts

The core problem addressed is the combination step in multi-subgraph invariant learning, which existing methods have largely ignored <ref:2511.22435#pg3>. The paper demonstrates that uniform averaging is suboptimal because the diversity regularizer used to obtain multiple subgraphs explicitly breaks the exchangeability required for it to be optimal <ref:2511.22435#pg3>. PISA solves this by learning an instance-conditional MLP over stacked branch logits, which allows for prioritization based on input data, rather than relying on fixed aggregation rules like uniform averaging or greedy selection <ref:2511.22435#pg3>. This learned aggregator is shown to be safe because the branches are frozen and read the graph only through their invariant subgraphs, meaning any measurable aggregator of their outputs remains invariant <ref:2511.22435#pg3>.

How it works involves a two-phase process. Phase I focuses on diverse invariant subgraph extraction, where multiple GNNs are trained in parallel with a diversity regularizer that pushes their masks apart to lock onto different causal substructures <ref:2511.22435#pg6>. The objective function for this phase includes a supervised loss on the branch's subgraph and a contrastive term that encourages embeddings of same-label graphs to be close, while the diversity term penalizes pairwise cosine overlap between masks <ref:2511.22435#pg6>. Phase II involves prioritized dynamic aggregation, where the extracted branches are frozen and their logits are stacked into a tensor P <ref:2511.22435#pg5>. A lightweight MLP Aθ maps this stack to the final prediction Yb = Aθ(vec(P)), trained using empirical risk minimization on the cross-entropy loss LII <ref:2511.22435#pg5>. This learned aggregator is instance-conditional, meaning it places importance on each branch based on the input graph G <ref:2511.22435#pg5>.

Theoretical guarantees establish the safety of this approach. Proposition 4 proves that invariance is closed under arbitrary measurable aggregation of frozen invariant branches, showing that the aggregate's environment gap is bounded by the branches’ residual invariance violation uniformly over all aggregators <ref:2511.22435#pg7>. Furthermore, Proposition 5 provides a uniform environment-gap bound, showing that the worst-case environment gap for any measurable aggregator A is bounded by 2Bδ, where δ measures the residual dependence of branch outputs on the environment <ref:2511.22435#pg7>. This implies that capacity in Aθ trades off against estimation error rather than robustness <ref:2511.22435#pg3>.

The performance evaluation across 15 synthetic and real benchmarks shows significant gains over prior methods. PISA improves the average over the strongest prior method by 2.96 points on synthetic shifts and 1.74 points on realistic shifts, achieving nearly double SuGAr’s improvement over ERM on the six DrugOOD tasks <ref:2511.22435#pg2>. The study also investigates how performance scales with the number of invariant branches (n), finding that performance peaks at n = 10 among tested values, and confirms that PISA outperforms SuGAr at every tested n <ref:2511.22435#pg6>. Finally, the paper notes that while increased model count alone does not explain PISA’s gains, it suggests extraction is the next bottleneck <ref:2511.22435#pg9>.

The methodology relies on formal structural causal models (SCMs) to delineate invariant content (C) from environment-varying style (S). The authors examine three SCM types: Fully Informative Invariant Features (FIIF), Partially Informative Invariant Features (PIIF), and Mixed Interactions (MIIF) <ref:2511.22435#pg17>. Assumption 1 formalizes the intuition that invariant content C is independent of the environment E, which is only allowed to act on the spurious branch S <ref:2511.22435#pg17>. The analysis also incorporates a latent separability condition, Assumption 6, which posits that H(C Y) ≤ H(S Y), capturing the idea that invariant content clusters more tightly than the spurious component conditional on the label <ref:2511.22435#pg17>.

The paper concludes that PISA replaces fixed combination rules with a small MLP over cached branch logits and improves OOD generalization by 2.96 and 1.74 points at negligible cost <ref:2511.22435#pg10>. The residual multi-subgraph gap suggests extraction as the next bottleneck <ref:2511.22435#pg11>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>. This work shows that learning the combination rule is safe because frozen invariant branches expose no environment-dependent information <ref:2511.22435#pg3>. This approach provides a principled response to distribution shift on graphs by focusing on causal substructures rather than environment-specific shortcuts <ref:2511.22435#pg2>. The paper's findings are highly relevant to the field of invariant and graph OOD learning <ref:2511.22435#pg7>. The results suggest that PISA improves aggregation rather than extraction and therefore does not eliminate the harder multi-subgraph extraction problem <ref:2511.22435#pg6>.

1. Bold Header: Prioritized dynamic aggregation of frozen branch outputs improves OOD generalization by learning instance-specific importance weights for each invariant predictor. This allows PISA to improve synthetic and realistic OOD averages by 2.96 and 1.74 points over the strongest prior method.

2. Bold Header: The system can utilize a lightweight MLP, trained on frozen branches at negligible cost, to assign instance-specific importance to stacked branch logits, which is shown to be the right object to fit for approximating the optimal aggregation rule.

3. Bold Header: PISA enables robust generalization across different environments by leveraging the principle that invariance is closed under post composition, ensuring that the aggregate’s environment gap is bounded by the branches’ residual invariance violation uniformly over aggregators.

4. Bold Header: The improved system can achieve performance gains by identifying and exploiting a bottleneck in existing methods, specifically proving that uniform averaging is optimal only when every branch carries the same total error covariance with the ensemble—a knife edge balance that is destroyed by diversity regularizers.

5. Bold Header: The system can be designed to scale its aggregation capacity optimally by utilizing a learned aggregator whose depth peaks at 3–4 layers for a 30-dimensional input, balancing estimation error against capacity as predicted by the analysis in Equation (10).

Invariant learning on graphs aims to build predictors that rely on causal substructures rather than on environment-specific shortcuts. Current methods extract either a single invariant subgraph (CIGA) or, more recently, a set of diverse invariant subgraphs (SuGAr). We observe that the second half of the multi-subgraph pipeline, "how the extracted subgraphs are combined" has gone essentially unexamined; existing methods average branch predictions uniformly or select one greedily. We show that this is precisely where accuracy is lost. A minimum-variance combination argument establishes that uniform averaging is optimal only when every branch carries the same total error covariance with the ensemble---a knife edge balance that second order exchangeability guarantees and that the diversity regularizer used to obtain multiple subgraphs is explicitly designed to break. We then show that invariance is closed under post composition; because the branches are frozen and read the graph only through their invariant subgraphs, any measurable aggregator of their outputs remains invariant, and the aggregate's environment gap is bounded by the branches' residual invariance violation uniformly over aggregators. Learning the combination rule by plain empirical risk minimization is therefore safe, and aggregator capacity trades off against estimation error rather than robustness. PISA instantiates this with a lightweight instance-conditional MLP over stacked branch logits, trained on frozen branches at negligible cost. Across 15 synthetic and real benchmarks, PISA improves the average over the strongest prior method by 2.96 points on synthetic shifts and 1.74 points on realistic shifts, with per-dataset gains up to 5.90 points, and nearly doubles SuGAr's improvement over ERM on the six DrugOOD tasks (from +2.76 to +5.07 ROC-AUC).