Conformal Prediction Sets Quantify Information Gain: A Theoretical Perspective
The research connects conformal prediction set size reduction to classical information theory using generalized entropy measures. The authors developed a family of measures that provide exact integral representations for Shannon mutual information, formally justifying using set-size reduction as a principled metric for quantifying feature information gain in classification tasks.