Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data
cs.LG, cs.AI
Submitted: 2026-11-11
Updated: 2026-11-17
Terminology
Sources
- SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
- In-Context Convergence of Transformers
- How Transformers Learn Causal Structure with Gradient Descent
- Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
- Mass-Editing Memory in a Transformer
- On the Emergence of Implicit Curriculum in RLVR Learning Dynamics
- Muon Outperforms Adam in Tail-End Associative Memory Learning
- Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
- SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
- From System 1 to System 2: A Survey of Reasoning Large Language Models
- Orca: Progressive Learning from Complex Explanation Traces of GPT-4
- Orca 2: Teaching Small Language Models How to Reason
- MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
- LIMO: Less is More for Reasoning
- LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
- O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
- Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models
- Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Them
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis