From Theory to Decision Rule: Calibrating the Noisy-Label Crossover for Vision-Language Model Weak Supervision Across Three Medical-Imaging Benchmarks
The study calibrates a theoretical limit for using Vision-Language Model (VLM) pseudo-labels in medical imaging tasks like PatchCamelyon and ISIC. It determines where weak labels stop helping and start hurting, finding task-specific crossover points. The result is a simple decision rule: compare gold-only accuracy to VLM accuracy on a small set of gold labels to predict whether adding pseudo-labels will improve performance.