An Assessment of Human vs. Model Uncertainty in Soft-Label Learning and Calibration
Researchers tested human-elicited soft-labels against synthetic labels to see which improves model calibration and robustness. They found that human soft-labels act as a regularizer, aligning model uncertainty with real human uncertainty, unlike synthetic labels which fail this alignment.