Slim Foundation Models for Latency-Critical Recommendation Serving
cs.IR, cs.AI
Submitted: 2026-11-07
Updated: 2026-11-07
Journal ref: NeurIPS 2026 Workshop - Long Context Foundation Model
License: http://creativecommons.org/licenses/by/4.0/
The gist: Click-through rate (CTR) prediction is a core component of recommendation and advertising systems.
Terminology
Abstract
Click-through rate (CTR) prediction is a core component of recommendation and advertising systems. Large language models (LLMs) offer rich semantic priors that improve prediction quality, particularly in cold-start and long-tail settings. Injecting a full-scale LLM into a CTR pipeline is, however, prohibitively expensive at inference time: existing LLM-enhanced CTR methods avoid this cost by caching LLM outputs offline rather than serving the LLM online. We propose a resource-efficient pipeline, SlimFM, that makes online LLM-based semantic modeling practical: (1) dependency-aware structured pruning compresses LLaMA3-8B by 44% to a 4.5B-parameter backbone; (2) LoRA-based field-aware fine-tuning trains the pruned model to produce discriminative field-level semantic embeddings from lightweight textual prompts; (3) a semantic-guided fusion module injects these embeddings into the feature interactions of a conventional CTR model. Across four datasets and six CTR backbones, SlimFM retains over 99% of uncompressed-model accuracy while reducing inference latency by up to 3.4x relative to existing LLM-enhanced baselines.
Sources
- The Llama 3 Herd of Models
- DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
- CELA: Cost-Efficient Language Model Alignment for CTR Prediction
- Wukong: Towards a Scaling Law for Large-Scale Recommendation
Related papers
- UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction
- Revisiting Text Ranking in Deep Research
- GateSID: One Gate to Balance Semantic and Collaborative Signals across the Item Lifecycle
- SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce
- GINGER: Spicing Up Graph-Based Vector Search
- Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA