Token-Budget Distillation: Transferring Full-Token Semantics to Compressed Video Vision-Language Models
The Token-Budget Distillation framework fine-tunes video vision-language models under strict token limits by using a dual teacher and student approach. It transfers full-token semantics to compressed models through reliable distillation techniques, ensuring the student reconstructs high-fidelity reasoning from scarce visual evidence. This method preserves semantic accuracy even when the model is heavily compressed.