Hide&Seek: Learning to Explain in an End-to-End Differentiable Network

arXiv:2608.16689 · stat.ML, cs.LG · Submitted 2026-08-17 · Read on arXiv

stat.ML, cs.LG

Submitted: 2026-08-17

Updated: 2026-08-17

Comments: 27 pages, 12 figures, 16 tables. Accepted at ICML 2026 (PMLR 306). Code at https://github.com/talellinson/hide-and-seek-icml2026

Journal ref: Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:27939-27965, 2026

Code: https://github.com/talellinson/hide-and-seek-icml2026

License: http://creativecommons.org/licenses/by/4.0/

The gist: Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models.

Terminology

Abstract

Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.

Sources

Related papers