Visual Contrastive Self-Distillation
Visual Contrastive Self-Distillation (VCSD) creates an on-policy self-distillation signal by contrasting teacher predictions under the original image versus a content-erased control. This contrast highlights tokens whose likelihood is specifically boosted by the visual content, allowing the student to sharpen its target distribution without needing external teachers or privileged answers.