Aligning the Query Space: Greedy Information Projection for Language Model Data Selection

Name: Aligning the Query Space: Greedy Information Projection for Language Model Data Selection
Uploaded: 2026-10-10T11:45:32.809094+00:00
Duration: 795 s
Description: Greedy Information Projection (GIP) selects optimal training data for LLM fine-tuning by maximizing mutual information between selected examples and task queries. This method balances data quality and diversity using geometric projections of embeddings, achieving near full-dataset performance with only 1–20% of the required data.

Greedy Information Projection (GIP) selects optimal training data for LLM fine-tuning by maximizing mutual information between selected examples and task queries. This method balances data quality and diversity using geometric projections of embeddings, achieving near full-dataset performance with only 1–20% of the required data.

Open the full episode