Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?
Deep ensembles, a method for quantifying uncertainty by averaging models trained with different initializations, largely fail to capture meaningful uncertainty in Graph Neural Networks (GNNs). The study found that these ensembles suffer from 'epistemic collapse,' where independently trained models become too similar. This means the ensemble gains are mostly just stabilizing optimization noise, not providing better uncertainty estimates.