RecourseBench: A Modular Framework for Reproducible Algorithmic Recourse Evaluation
cs.AI, cs.LG
Submitted: 2026-06-15
Updated: 2026-08-28
License: http://creativecommons.org/licenses/by/4.0/
The gist: Algorithmic recourse methods provide counterfactual explanations that inform individuals of the actions required to overturn an unfavorable model decision.
Terminology
Abstract
Algorithmic recourse methods provide counterfactual explanations that inform individuals of the actions required to overturn an unfavorable model decision. Despite rapid methodological progress, principled comparison remains elusive; existing frameworks are often difficult to extend and lack both interoperability and systematic verification that integrated methods faithfully reproduce their originally reported claims. We introduce RecourseBench, a unified evaluation framework built around three commitments: modularity, reproducibility, and interactivity. The framework decomposes the pipeline into five fully decoupled layers---Data, Preprocessing, Model, Recourse Method, and Evaluation---governed by abstract interfaces and a dynamic registry. Every integrated method is classified into a four-tier reproducibility taxonomy based on artifact availability, followed by a systematic verification of its core empirical claims. We further provide an interactive web interface for flexible, configuration-driven exploration across datasets, model architectures, methods, and evaluations. To our knowledge, RecourseBench is the first benchmark to explicitly ground recourse evaluation in structured claim verification and mathematically rigorous reproducibility standards, all while featuring the largest collection of state-of-the-art recourse algorithms (27 in total).
Sources
- Getting a CLUE: A Method for Explaining Uncertainty Estimates
- Consistent Counterfactuals for Deep Models
- What is Reproducibility in Artificial Intelligence and Machine Learning Research?
- RobustX: Robust Counterfactual Explanations Made Easy
- Argumentative Ensembling for Robust Recourse under Model Multiplicity
- Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
- Learning-Augmented Robust Algorithmic Recourse
- Inverse Classification for Comparison-based Interpretability in Machine Learning
- Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers
- Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse
- Towards Robust and Reliable Algorithmic Recourse
- Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems