ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance
ToDRE is a training-free method to compress large vision-language models by pruning redundant visual tokens. It achieves this by combining two strategies: first, selecting a diverse set of tokens using a greedy algorithm; second, removing irrelevant tokens in deep model layers where cross-modal attention has faded. This results in up to 90% token pruning with significant speed improvements while maintaining high performance.