Progressive Alignment of Recommender Foundation Model through Multi-Phase Post-Training
Oseong Choi, Hoeinn Kim, Jihoon Lee, Byungsoo Kang, Taeyeong Jang
cs.IR, cs.AI
Submitted: 2026-08-07
Updated: 2026-08-10
Comments: 9 pages, 3 figures. Accepted to the 20th ACM Conference on Recommender Systems (RecSys '26), Industry Track
Code: https://github.com/webtoon/rec-fm-progressive-alignment
License: http://creativecommons.org/licenses/by/4.0/
The gist: Foundation model(FM) for recommendation has shown strong ability to model long-horizon sequential user behavior.
Terminology
Abstract
Foundation model(FM) for recommendation has shown strong ability to model long-horizon sequential user behavior. In practice, a single pretrained foundation model is often adapted to diverse downstream serving surfaces through Supervised Fine-Tuning(SFT). However, optimizing task-specific objectives such as clicks or likes does not necessarily align the serving policy with the business metrics that determine recommendation quality. We propose a three-phase progressive post-training framework that explicitly separates downstream adaptation from business-metric alignment. The adaptation stage is decomposed into Linear Probing(LP) and Full Fine-Tuning(FFT): LP first stabilizes randomly initialized downstream heads within a frozen pretrained representation space, and FFT then jointly specializes the full model for the target task. On top of this stabilized policy, Reinforcement Fine-Tuning(RFT) aligns the model with practical business objectives using a learned reward model. Rather than directly optimizing the serving policy on sparse business targets, we train the policy on dense implicit feedback and use business-metric supervision only for reward modeling. Offline experiments show that the progressive LP-FFT-RFT framework outperforms single-phase alternatives, and that reward-based alignment yields a stronger serving policy than directly using the reward model itself for ranking. Large-scale online A/B tests further show that the proposed framework improves production recommendation quality over a conventional non-foundation baseline. A reference implementation is available at https://github.com/webtoon/rec-fm-progressive-alignment
Sources
- Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
- Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Large Foundation Model for Ads Recommendation
Related papers
- The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
- SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction
- DualView: Adaptive Local-Global Fusion for Multi-Hop Document Reranking
- How Do LLM-Generated Texts Impact Term-Based Retrieval Models?