TabReliBench: Evaluating Tabular Foundation Models Beyond Predictive Accuracy
The research introduces TABRELIBENCH, a benchmark to evaluate Tabular Foundation Models across five reliability dimensions: point accuracy, distributional quality, marginal calibration, subgroup calibration, and interval sharpness. Analysis shows that optimizing for one metric often harms another. The conclusion is that reliable model selection requires a multidimensional evaluation rather than relying on single predictive accuracy.