Fairness in Link Prediction Beyond Demographic Parity: A Reproducibility Study

This paper, Fairness in Link Prediction Beyond Demographic Parity: A Reproducibility Study by Valentijn Oldenburg, Floris de Kam, Stef de Wildt, and Jarno Balk, reproduces and extends the work of Mattos et al. (2025) on fairness in ranked link prediction. The study validates three core claims (C1, C2, C3) and introduces extensions to test robustness.

Core Claims Reproduced:

• Claim C1 (Demographic parity hides within-group exposure bias): The study confirms that the dyadic demographic parity metric (∆DP) can indicate aggregate parity even when specific subgroup-pair links are systematically ranked lower. The paper states: we reproduce this claim by showing that ∆DP can indicate aggregate parity even when some subgroup-pair links are systematically ranked lower than others. This is empirically shown in Figure 2, where ∆DP remains largely invariant across bins of exposure ratios, while NDKL varies substantially.

• Claim C2 (NDKL overcomes demographic parity limitations): The rank-aware Normalized Discounted KL-divergence (NDKL) metric is shown to detect exposure disparities that ∆DP misses. The paper confirms: The proposed rank-aware Normalized Discounted KL-divergence (NDKL), however, does detect such disparities. Figure 2 demonstrates NDKL's sensitivity to shifts in exposure within intra-group links, unlike ∆DP.

• Claim C3 (MORAL achieves strong fairness-utility trade-offs): The MORAL post-processing method is reproduced as effective in improving exposure-based fairness while maintaining competitive utility. The results show: MORAL consistently achieves (tied-)lowest, near-zero NDKL while also achieving the (near-)highest Precision on all datasets. Tables 2 and 3 report these results across six datasets, with MORAL showing minimal utility loss.

Extensions Beyond Original Work:

1. Asymmetric homophily stress-test: Using synthetic graphs generated with the Directed Preferential Attachment with Homophily (DPAH) model, the study varies majority/minority homophily (hMM, hmm) and minority fraction (fm). Results show that exposure-based fairness metrics (NDKL, AWRF) are most strained at intermediate absolute homophily (hedge ≈ 0.6), while Precision tends to drop with increasing homophily due to weaker training signals for decoupled predictors.

2. Metric robustness analysis: The study evaluates MORAL using alternative metrics—Attention-Weighted Rank Fairness (AWRF) for fairness and NDCG for utility. Results show MORAL maintains low AWRF scores (0.001–0.003 range) across datasets, and the separation between MORAL and baselines is even larger under AWRF, suggesting MORAL is not merely 'gaming' the metric. NDCG results indicate MORAL preserves ranking quality, especially at higher ranks.

3. Categorical sensitive attributes: The framework is extended to handle sensitive attributes with cardinality m > 2, tested up to m = 10 (T(10) = 55 subgroup-pair types) on the Credit dataset. Results show ∆DP remains insensitive to growing T, while NDKL and AWRF degrade monotonically, indicating that aligning exposure with π becomes more challenging as T increases. Utility decreases modestly but remains high (0.97–1.00 range).

Key Findings and Conclusions:

• The study confirms that dyadic demographic parity is insufficient for ranked link prediction, as it can obscure subgroup-pair exposure disparities.

• MORAL effectively reduces exposure bias with minimal utility loss across diverse settings and datasets.

• The paper recommends prioritizing exposure-based metrics (NDKL, AWRF) over demographic parity in ranked link prediction.

• MORAL's fairness-utility trade-offs are most evident at intermediate absolute homophily, and the framework scales to categorical attributes, though with a fairness-scalability trade-off.

The authors release a corrected, reproducible implementation at https://github.com/Floris93100/reproducing-MORAL.

Improvements to AI Systems:

1. Fairness-Aware Link Prediction with Rank-Sensitive Metrics: Replace demographic parity (∆DP) with exposure-based metrics (NDKL, AWRF) in the training objective or post-processing of link prediction models. This ensures the AI system does not systematically rank certain subgroup-pair links (e.g., minority–minority) lower, even when aggregate parity appears satisfied. The improved system can detect and correct hidden within-group exposure bias in recommendation, social network, or knowledge graph link prediction.

2. Robust Post-Processing Fairness Module (MORAL-Enhanced): Integrate the MORAL algorithm as a plug-in post-processing layer for any existing link predictor. The improved system can adjust predicted link scores to align exposure with the desired distribution π, achieving near-zero NDKL/AWRF while preserving precision (utility loss < 3% across six datasets). This makes it deployable in production systems without retraining the base model.

3. Homophily-Aware Fairness Calibration: Use the DPAH stress-test findings to dynamically adjust fairness constraints based on the graph's homophily level. At intermediate absolute homophily (hedge ≈ 0.6), the system can increase fairness regularization to prevent precision collapse, while at extreme homophily, it can relax constraints to maintain training signal. The improved system can self-tune its fairness–utility trade-off based on measured graph topology.

4. Scalable Fairness for Categorical Sensitive Attributes: Extend the fairness framework to handle sensitive attributes with cardinality >2 (up to m=10). The improved system can compute NDKL/AWRF for all T = m(m+1)/2 subgroup-pair types and apply MORAL to reduce exposure disparities, even though fairness degrades monotonically with T. This enables fair link prediction in domains with multi-class demographics (e.g., ethnicity, age groups, or multi-label attributes).

5. Metric-Agnostic Fairness Verification: Incorporate AWRF and NDCG as secondary evaluation metrics to prevent gaming of a single fairness metric. The improved system can verify that fairness improvements are genuine (not metric-specific) by checking that low NDKL also corresponds to low AWRF, and that ranking quality (NDCG) is preserved at higher ranks—ensuring the system's fairness is robust across different definitions.

6. Exposure-Disparity Auditing Tool: Build an auditing module that reports both ∆DP and NDKL/AWRF per subgroup-pair, flagging cases where aggregate parity masks systematic rank suppression. The improved system can generate detailed fairness reports for stakeholders, highlighting specific subgroup-pair biases that demographic parity would miss, enabling targeted interventions.

In fair ranked link prediction, demographic parity (DP) is a common fairness metric. Yet, Mattos et al. (2025) argue that it fails to detect exposure bias because it ignores where links appear in the ranking. In this study, we reproduce this claim by showing that DP can indicate aggregate parity even when some subgroup-pair links are systematically ranked lower than others. The proposed rank-aware Normalized Discounted KL-divergence (NDKL), however, does detect such disparities. We also reproduce the effectiveness of MORAL, a post-processing method that improves exposure-based fairness while maintaining competitive utility. Beyond reproduction, we assess robustness using synthetic homophily settings, categorical sensitive attributes, and additional fairness and utility metrics, including subgroup-pair-adapted Attention-Weighted Rank Fairness (AWRF). Overall, our results show that exposure-based metrics uncover biases hidden by DP and that MORAL reduces these biases with minimal utility loss across diverse settings and datasets. We release a corrected, reproducible implementation at https://github.com/Floris93100/reproducing-MORAL.