Mass Producing Training Data with Surprising Side Effects
cs.CL, cs.AI, cs.LG
Submitted: 2026-11-04
Updated: 2026-11-04
Code: https://github.com/tatsu-lab/stanford_alpaca
Terminology
Sources
- Subliminal Effects in Your Data: A General Mechanism via Log-Linearity
- Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal
- Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs
- Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
- Subliminal Learning Is Steering Vector Distillation
- Subliminal Learning: Language models transmit behavioral traits via hidden signals in data
- Scaling Laws for Reward Model Overoptimization
- Datamodels: Predicting Predictions from Training Data
- Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback
- Emergent Misalignment Is Not Magical
- Eliciting Language Model Behaviors with Investigator Agents
- Natural Emergent Misalignment from Reward Hacking in Production RL
- Chunky Post-Training: Data Driven Failures of Generalization
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- Red Teaming Language Models with Language Models
- Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Introspection Adapters: Training LLMs to Report Their Learned Behaviors
- AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors
- Synthetic Data for any Differentiable Target
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data