A Glance Is All You Need: Single-Pass Fine-Grained Image Captioning with SimLoss
SimLoss is a new method for training single-pass image captioning models to produce detailed captions by aligning the model's internal representation with a frozen image embedding. It uses contrastive loss to provide dense visual supervision before text decoding, allowing fast, high-quality captioning without needing expensive human labels or multi-stage pipelines.