Is Serving-Time Language-Model Inference Necessary for Recommendation? Evidence from Generative and Agentic Settings

The gist The investigation into attribution ambiguity reveals that performance gains over ID-only baselines in semantic sequential recommendation do not uniquely identify whether they arise from language-model reasoning, semantic-ID generation, end-to-end architectures, or complementary signals.

LIME-Rec Recovery Test

LIME-Rec is introduced as a lightweight and auditable recovery test designed to evaluate whether reported semantic improvements can be explained by simple, transparent, and offline components rather than complex serving-time mechanisms The test combines three independent experts: a SASRec sequential expert, an ItemCF co-occurrence expert, and a semantic expert based on frozen BAAI/bge-base-en-v1.5 item embeddings Each expert produces full-catalog scores, which are normalized per user and combined through auditable scorelevel fusion, followed by bounded history calibration The fusion gate and calibration head are fitted using validation data only, require no serving-time language-model inference, and keep each expert contribution separately inspectable.

Expert Components

The LIME-Rec architecture utilizes three independent experts to generate scores for a candidate item i ∈ I.

1 Sequential Expert We use SASRec (Kang and McAuley 2018) as the sequential expert, which models user preference dynamics and produces a score for every candidate item i ∈ I: sseq(u, i).

2 Collaborative Expert We construct an ItemCF expert to provide explicit collaborative co-occurrence evidence by calculating scf(u, i) = X L sum t=1 to L cooc(ht, i) p NhtNi.

3 Semantic Expert The semantic expert provides itemtext evidence through frozen offline representations by computing ssem(u, i) = ⟨e¯u, ei⟩.

Score Fusion and Calibration

The three independent experts produce scores on different numerical scales, which LIME-Rec applies min–max normalization separately for each user and expert over the full item catalog to get s˜x(u, i). The normalized expert scores are combined through a lightweight per-user fusion gate where the fused score is sfuse(u, i) = X x∈3 w x(u)˜sx(u, i). This fusion gate consumes only summary statistics derived from the expert score distributions to produce expert weights w(u) = softmax a(u), where a(u) = W ϕ(u) + c. Finally, LIME-Rec applies bounded history calibration to adjust the fused scores of previously interacted items: s(u, i) = sfuse(u, i) − λ(u)[i ∈ H(u)], where λ(u) = λmax σ b(u), 0 ≤ λ(u) ≤ λmax.

Experimental Findings and Attribution Analysis

LIME-Rec achieves R@10 scores of 0.0996, 0.1105, and 0.0593 on the text-rich Amazon Beauty, Toys, and Sports benchmarks. A factorial isolation ablation further shows that three-expert fusion without history calibration consistently outperforms calibrated SASRec. Randomly permuting item-text embeddings across item IDs while preserving model capacity reduces R@10 by 13.6%–17.5%, showing that the gains depend on genuine item-text correspondence rather than additional representation capacity alone.

Component Contribution

The analysis of component contributions shows that adding the semantic expert to SASRec improves R@10 from 0.0794 to 0.1000 on Toys. The full three-expert fusion further outperforms every singleton and pairwise configuration across all twelve reported metrics. Candidate-level non-redundancy diagnostics show consistently low top-10 overlap and weak to moderate full-catalog rank association between the three experts.

Conclusion

The results show that competitive semantic-recommendation performance can be recovered through frozen item-text representations, standard sequential and collaborative signals, and lightweight scorelevel fusion. The study concludes that the recovery cannot be explained by repeat-aware history calibration alone; multi-source expert fusion provides the primary standalone gain, while bounded calibration supplies a complementary improvement to the final ranking.

Future Work

Future work should compare against this recovery baseline before attributing gains to serving-time language modeling, semantic-ID generation, or heavier architectures.

--- Page 1 ---

Auditing Semantic Gains in Sequential Recommendation: A Lightweight Recovery Test Kong Wang1∗, Zhongke He2, Xiang Chen3, Hongwei Zeng4, Kai Deng5, Long Wang6, Kehua Yang1 1Hunan University 2Dalian University of Technology 3Tongji University 4University of Chinese Academy of Sciences 6Xinjiang College of Science & Technology 5Beihang University Abstract Recent semantic and generative-retrieval recommenders report substantial improvements over ID-only sequential baselines. However, it remains unclear whether these gains arise from language-model reasoning, semantic-ID generation, end-to-end semantic architectures, stronger offline item representations, or the complementarity of semantic and collaborative signals We investigate this attribution ambiguity through LIME-Rec, a lightweight and auditable recovery test. LIMERec combines three independent experts: a SASRec sequential expert, an ItemCF co-occurrence expert, and a semantic expert based on frozen BAAI/bge-base-en-v1.5 item embeddings Each expert produces full-catalog scores, which are normalized per user and combined through auditable scorelevel fusion, followed by bounded history calibration. The fusion gate and calibration head are fitted using validation data only, require no serving-time language-model inference, and keep each expert contribution separately inspectable. On the text-rich Amazon Beauty, Toys, and Sports benchmarks, LIME-Rec achieves R@10 scores of 0.0996, 0.1105, and 0.0593, exceeding the strongest comparison method in the table by 7.0%–12.0%. A factorial isolation ablation further shows that three-expert fusion without history calibration consistently outperforms calibrated SASRec, ruling out bounded history calibration as the sole explanation for the recovery. Randomly permuting item-text embeddings across item IDs while preserving model capacity reduces R@10 by 13.6%–17.5%, showing that the gains depend on genuine item-text correspondence rather than additional representation capacity alone. These results do not imply that complex semantic models are universally unnecessary. Instead, they show that lightweight recovery from offline item representations and transparent fusion should be ruled out before improvements are attributed to serving-time language modeling, semanticID generation, or heavier semantic machinery.

--- Page 2 ---

Conventional evaluation ID-only sequential baseline interaction signals only Complex semantic recommender LM reasoning, semantic-ID generation, end-to-end semantic modeling Observed gain Δobs over ID-only baseline? Attributed to complex semantic mechanisms?? Observed gains do not uniquely identify their source. Attribution ambiguity Offline item representations? Complementary signals collaborative + semantic Heavier semantic mechanisms LM reasoning + semantic generation Proposed recovery audit ID-only sequential baseline interaction signals only Lightweight recovery baseline Lightweight Auditable No serving-time LM Observed gain Recoverable gain matched by the lightweight baseline Residual gain requires additional mechanism-specific evidence = + Conceptual recovery decomposition, not causal attribution. Figure 1: Motivation for LIME-Rec. Performance gains over an IDonly sequential baseline do not uniquely identify their source, motivating a lightweight recovery audit As illustrated in Figure 1, performance gains over an IDonly sequential baseline do not uniquely identify whether the improvement originates from lightweight offline signals or heavier semantic mechanisms To investigate this problem, we propose LIME-Rec, a lightweight recovery test for auditing semantic gains in sequential recommendation <ref:2608.

1. textbfImprove performance attribution via LIME-Rec auditability by isolating signal sources from serving path requirements. By evaluating whether lightweight system that excludes servingtime language modeling, semantic-ID decoding, and end-toend text optimization reaches the performance of a heavier method, the system can definitively rule out complex semantic models if simple offline representations suffice. This allows for justified architectural simplification without sacrificing gains in R@10 scores across Amazon benchmarks by showing "lightweight recovery from offline item representations and transparent fusion should be ruled out before improvements are attributed to serving-time language modeling, semanticID generation, or heavier semantic machinery."

2. textbfImplement multi-expert fusion for synergistic evidence combination. The system can combine independent expert scores through a lightweight per-user fusion gate that consumes summary statistics derived from the expert score distributions, allowing for dynamic weighting of sequential (SASRec), collaborative (ItemCF), and semantic (BGE) signals based on user context, as shown by the softmax transformation of logits. This provides a mechanism where fusion gate consumes only summary statistics derived from the expert score distributions, while the weighted sum operates on the complete normalized full-catalog score vectors.

3. textbfEmploy bounded history calibration for refined ranking adjustment. The system can adjust scores of previously interacted items using bounded history calibration by applying a penalty term: s(u, i) = sfuse(u, i) − λ(u)[i ∈ H(u)], where the magnitude is constrained by λmax, to ensure that the final ranking reflects both new recommendations and relevant previously seen items. This refinement offers a measurable gain of +0.0115 on Beauty, +0.0115 on Toys, and +0.0074 on Sports compared to fusion without calibration alone.

4. textbfEstablish item-text correspondence control for robust semantic reliance. The system can verify the necessity of item descriptions by testing Randomly permuting item-text embeddings across item IDs while preserving model capacity, which reduces R@10 by 13.6%–17.5%, showing that the gains depend on genuine item-text correspondence rather than additional representation capacity alone. This ensures that the semantic expert's contribution is tied to meaningful content alignment and not merely increased representation capacity.

5. textbfEmploy candidate-level non-redundancy diagnostics for expert selection. The system can assess expert value by analyzing Target-level complementarity metrics, revealing that while the semantic expert is the weakest singleton, it still contributes a non-redundant signal and that the full three-expert fusion further outperforms every singleton and pairwise configuration across all twelve reported metrics. This guides system design toward integrating all three signals for maximal performance.