From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models
The study analyzed probe-based uncertainty estimation by testing different feature designs and settings. It found that while raw hidden states work well in-domain, structured and compressed features are more robust when models face distribution shifts. This suggests that using structured features provides a more reliable starting point for practical, deployment-oriented applications.