Scaling Laws for Task-Specific LLM Distillation
The episode discusses the paper "Scaling Laws for Task-Specific LLM Distillation," which maps how to compress large language models for specific tasks while maintaining general intelligence. Hosts discuss how in-domain performance degrades predictably, and that blended chain-of-thought supervision is key to recovering lost general knowledge during compression.