How many labelers do you have? A closer look at gold-standard labels

arXiv:2206.12041 · math.ST, cs.HC, cs.LG, stat.TH · Submitted 2022-06-24 · Read on arXiv

math.ST, cs.HC, cs.LG, stat.TH

Submitted: 2022-06-24

Updated: 2026-10-01

Terminology

• Fidelity — 1× in this paper · explained in 2 episodes

Sources

• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• Learning from Imperfect Annotations

Related papers

• Conformal Prediction for Dyadic Regression Under Complex Missingness
• Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel
• Bentkus-type asymptotic e-values
• High-Dimensional Asymptotics of Differentially Private PCA
• A proximal gradient algorithm for composite log-concave sampling
• KL Convergence Guarantees for Score diffusion models under minimal data assumptions