DataSense-Bench: The First Step Toward an AI Scientist
DataSense-Bench tests if frontier AI models can judge data value for fine-tuning by asking agents to select and rank training subsets. Agents choose groups of interactions for terminal problem solving or tool use tasks, aiming to maximize post-training performance. Findings show limited gains over random selection and inconsistent ranking accuracy across different models and tasks.