Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities
The episode discusses a paper proposing BIDS (Balanced and Influential Data Selection), a method to improve how data is selected for training large language models. The hosts explain that standard methods are biased, leading to unbalanced models. BIDS fixes this by normalizing scores and iteratively selecting data to ensure balanced learning across diverse capabilities.