Tracing sources of epistemic uncertainty in deep learning predictions: homo- and hetero-scedastic linearized estimators
Pierre Nodet, Thomas George
cs.LG, stat.ML
Submitted: 2026-08-07
Updated: 2026-08-11
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: We adapt two classical statistical estimators for quantifying uncertainty to modern deep learning, in order to provide clearer insights into uncertainty attributable to two sources: aleatoric
Terminology
Abstract
We adapt two classical statistical estimators for quantifying uncertainty to modern deep learning, in order to provide clearer insights into uncertainty attributable to two sources : aleatoric uncertainty, or locally scarce data. Our approach leverages recent advances in approximate Fisher Information Matrices, to enable scaling to actual architectures. Experimental results demonstrate how each test points is differentially impacted by both sources, highlighting the practical utility of our estimators in improving the robustness of real-world applications.
Sources
- Studying Large Language Model Generalization with Influence Functions
- Improving predictions of Bayesian neural nets via local linearization
- Adam: A Method for Stochastic Optimization
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- Six Lectures on Linearized Neural Networks
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks