Human or Synthetic? What a Fixed Budget Buys for Post-Training
cs.CL, cs.LG
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/shourikb/human-or-synthetic
Terminology
Sources
- When Weak LLMs Speak with Confidence, Preference Alignment Gets Stronger
- Constitutional AI: Harmlessness from AI Feedback
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information
- Olmo 3
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- LoRA: Low-Rank Adaptation of Large Language Models
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
- gpt-oss-120b & gpt-oss-20b Model Card
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Balancing the Budget: Understanding Trade-offs Between Supervised and Preference-Based Finetuning
- HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data