A Model Ensemble-Based Post-Processing Framework for Fairness-Aware Prediction

The gist: A post-processing framework leveraging model ensembling facilitates fairness-aware prediction by blending predictions from a pre-trained complex model and a simpler model, effectively enhancing fairness across classification, regression, and survival analysis while maintaining or minimally affecting predictive accuracy. <ref:2603.18838#pg2>

Introduction to Fairness and Mitigation Strategies

Fairness in machine learning refers to the idea that a model’s predictions should not systematically favor or disadvantage certain individuals or groups defined by sensitive attributes such as gender, race, or age <ref:2603.18838#pg3>. Various fairness metrics and strategies to mitigate unfairness can be broadly categorized into three main approaches: pre-processing, in-processing, and post-processing <ref:2603.18838#pg5>. Pre-processing methods modify the input data to reduce bias before model training <ref:2603.18838#pg6>, but they can lead to a loss of valuable information or introduce unintended biases. In-processing techniques embed fairness objectives directly into model optimization by incorporating additional fairness constraints or regularization terms but often lead to increased computational complexity and are typically model-specific. Post-processing methods adjust the model outputs after training without modifying the model or training data, making them model-agnostic and relatively easy to apply, though many are tailored to specific tasks or fairness notions.

The Proposed Ensemble Framework

The proposed framework introduces a general post-processing framework for fairness-aware prediction based on model ensembling, which is model-agnostic and applicable across a wide range of machine learning tasks. It is inspired by the principles of mixture modeling and the mixture-of-experts paradigm. The core idea involves taking a weighted average of a pre-trained complex model and a simpler model, blending their predictions while tuning ensemble weights and the parameters of the simpler model to promote fairness. This design allows for flexible model combination while preserving interpretability and modularity. The framework is designed to operate independently of any specific model internals, supporting a wide range of fairness definitions and not requiring access to sensitive attributes at inference time.

Model Combination and Optimization

The combined prediction function is defined as fcombined(x) = α(x)fperf(x) + (1 − α(x))fsimp(x), where fperf is the pre-trained model, fsimp is the simpler model, and α(x) denotes the input-dependent gating weight. The optimization objective follows a formulation similar to FRAPPE, aiming to balance fidelity to the base model with fairness constraints. For binary classification, this involves minimizing the cross-entropy loss dCE and incorporating the statistical parity penalty LSP(fcombined). The parameter λ governs the trade-off between fidelity to the base model and fairness.

Task-Specific Applications

The framework is demonstrated across three representative tasks:

1. Binary Classification: The objective function minimizes dCE and uses the LSP penalty to promote demographic parity.

2. Regression: The dissimilarity measure dpred is defined as the squared error loss, and fairness regularization involves the Statistical Parity AUC penalty.

3. Survival Analysis: For survival analysis, the dissimilarity is quantified by the integrated squared difference of survival functions over time [0, τ], and fairness is addressed via a time-averaged group fairness regularization term LGF(Scombined)(t).

Experimental Results

Empirical evaluations across seven real-world datasets—including COMPAS, Heart, Adult, German Credit, Insurance, WHAS, and Employee turnover—demonstrate that the framework effectively enhances fairness while maintaining or only minimally affecting predictive accuracy. In binary classification on the Adult dataset with a 1-pretrained MoE with MLP model under single-sensitive (gender) settings, the DP is achieved at 0.0100 with accuracy above 0.76. Across all configurations on the COMPAS dataset, the framework demonstrates superior fairness–utility trade-offs compared to both post- and in-processing baselines.

Conclusion

In summary, the proposed ensemble approach offers greater flexibility than FRAPPE by using a mixture-of-experts formulation that combines a base model with a simpler, fairness-adjusted model, enabling instancedependent trade-offs between fairness and performance. This framework is model-agnostic, supports a wide range of tasks including survival analysis, and provides effective and tunable control over the fairness–performance balance.

Future Work

Future work could explore extending the framework to incorporate additional ensemble strategies or adaptive feature selection methods tailored to dataset characteristics. One promising direction is to expand the mixture-of-experts approach by including multiple auxiliary models with distinct inductive biases or fairness objectives. Investigating whether the proposed approach can be extended to settings such as individual fairness is an important direction for future work.

References

[1] Barocas, S., Hardt, M., Narayanan, A.: Fairness and machine learning. Recommender systems handbook 1, 453–459 (2020)

[2] Zafar, M.B., Valera, I., Rogriguez, M.G., Gummadi, K.P.: Fairness constraints: Mechanisms for fair classification. In: Artificial Intelligence and Statistics, pp. 962–970 (2017)

[3] Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226 (2012)

[4] Kamishima, T., Akaho, S., Asoh, H., Sakuma, J.: Fairness-aware classifier with prejudice remover regularizer. In: Machine Learning and Knowledge Discovery in Databases: European Conference, Proceedings, Part II 23, pp. 35–50 (2012)

[5] Filippi, G., Zannone, S., Hilliard, A., Koshiyama, A.: Local law 144: A critical analysis of regression metrics. arXiv:2302.04119 (2023)

[6] Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Neel, S., Roth, A.: A convex framework for fair regression. arXiv:1706.02409 (2017)

[7] Agarwal, A., Dud´ık, M., Wu, Z.S.: Fair regression: Quantitative definitions and reduction-based algorithms. In: International Conference on Machine Learning, pp. 120–129 (2019)

[8] Chierichetti, F., Kumar, R., Lattanzi, S., Vassilvitskii, S.: Fair clustering through fairlets. Advances in neural information processing systems 30 (2017)

[9] Kleindessner, M., Awasthi, P., Morgenstern, J.: A notion of individual fairness for clustering. arXiv:2006.04960 (2020)

[10] Li, B., Li, L., Sun, A., Wang, C., Wang, Y.: Approximate group fairness for clustering. In: International Conference on Machine Learning, pp.

1. Bold Header: Model Ensemble-Based Post-Processing Framework

This framework facilitates fairness-aware prediction by leveraging model ensembling, allowing it to be model-agnostic and applicable across various learning tasks, model architectures, and fairness definitions. The improved system can apply this method to classification, regression, and survival analysis without needing internal model details.

2. Bold Header: Flexible Trade-off Control via Learned Gating

The system utilizes a learned gating function α(x) in the Mixture-of-Experts (MoE) formulation, which allows for instancedependent trade-offs between fairness and performance by dynamically modulating each model’s contribution based on input features. This offers a more interpretable and adaptive mechanism than FRAPPE’s global perturbation.

3. Bold Header: Extension to Survival Analysis Fairness

The ensemble approach naturally extends to survival analysis, overcoming limitations where other post-processing methods are restricted, offering a more flexible and principled solution for fairness in this critical setting by using the Cox proportional hazards model as the simpler model fsimp.

4. Bold Header: Task-Specific Simpler Models

The framework allows the choice of the simpler model fsimp to be task-dependent, such as using logistic regression for binary classification or a linear model for regression, ensuring that any model appropriate for the task at hand can be used.

5. Bold Header: Robust Fairness Evaluation Metrics

The system supports evaluation across multiple fairness metrics, including Demographic Parity (DP) for classification and Statistical Parity AUC (SPAUC) for regression, ensuring group fairness is assessed through criteria like Demographic Parity [30] or Statistical Parity AUC [5].

Striking an optimal balance between predictive performance and fairness continues to be a fundamental challenge in machine learning. In this work, we propose a post-processing framework that facilitates fairness-aware prediction by leveraging model ensembling. Designed to operate independently of any specific model internals, our approach is widely applicable across various learning tasks, model architectures, and fairness definitions. Through extensive experiments spanning classification, regression, and survival analysis, we demonstrate that the framework effectively enhances fairness while maintaining, or only minimally affecting, predictive accuracy.