LLaVA-CKD: Bottom-Up Cascaded Knowledge Distillation for Vision-Language Models

Name: LLaVA-CKD: Bottom-Up Cascaded Knowledge Distillation for Vision-Language Models
Uploaded: 2026-10-22T12:37:37.293412+00:00
Duration: 507 s
Description: The LLaVA-CKD framework addresses deployment concerns for large Vision-Language Models by using cascaded knowledge distillation. Instead of one teacher, it introduces intermediate teachers to gradually improve a smaller student model. This bottom-up approach overcomes the limitations of large capacity gaps in knowledge transfer, leading to competitive performance on seven standard VQA benchmarks.

The LLaVA-CKD framework addresses deployment concerns for large Vision-Language Models by using cascaded knowledge distillation. Instead of one teacher, it introduces intermediate teachers to gradually improve a smaller student model. This bottom-up approach overcomes the limitations of large capacity gaps in knowledge transfer, leading to competitive performance on seven standard VQA benchmarks.

Open the full episode