SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs
Kejian Zhu, Zhuoran Jin, Shangqing Tu, Hongbang Yuan, Yushi Bai, Kang Liu, Juanzi Li, Jun Zhao
cs.CL, cs.LG
Submitted: 2026-08-04
Comments: Code: https://github.com/GaryStack/Parallel-RL
Code: https://github.com/GaryStack/Parallel-RL
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
- SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
- Distributions of Angles in Random Packing on Spheres
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- On Predictability of Reinforcement Learning Dynamics for Large Language Models
- Evaluating Large Language Models Trained on Code
- Measuring Massive Multitask Language Understanding
- Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
- Editing Models with Task Arithmetic
- Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application
- OpenAI o1 System Card
- When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning
- RL Is Neither a Panacea Nor a Mirage: Understanding Supervised vs. Reinforcement Learning Fine-Tuning for LLMs
- Dataless Knowledge Fusion by Merging Weights of Language Models
- Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
- Let's Verify Step by Step
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering