What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs
The episode discusses the paper 'What Transfers from Text to Vision?' which introduces the Capability-Driven Multimodal Scaling Law. Hosts analyze findings like 'transfer tax' in benchmarks and conclude that base LLMs often outperform instruction-tuned versions due to an 'alignment tax.' This provides a principled, quantitative roadmap for selecting VLM backbones, moving beyond guesswork.