NodeSynth: Socially Aligned Synthetic Data for AI Evaluation
NodeSynth is a three-step framework that creates high-quality synthetic prompts for AI safety evaluation by using an expert-curated taxonomy. It transforms abstract ideas into grounded, evidence-based queries that capture real-world social contexts. This method significantly outperforms human data in eliciting failures against mainstream models and helps identify weaknesses in safety guard models.