LLMRec: Benchmarking Large Language Models on Recommendation Task

arXiv:2308.12241 · cs.IR, cs.AI · Submitted 2023-08-23 · Read on arXiv

cs.IR, cs.AI

Submitted: 2023-08-23

Updated: 2026-10-13

Code: https://github.com/williamliujl/LLMRec

Terminology

• Benchmark — 4× in this paper · explained in 2 episodes
• Benchmarking — 2× in this paper · explained in 4 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Explainability — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes

Sources

• Massively Multilingual Neural Machine Translation
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
• The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
• M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
• AugGPT: Leveraging ChatGPT for Text Data Augmentation
• Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System
• Session-based Recommendations with Recurrent Neural Networks
• LoRA: Low-Rank Adaptation of Large Language Models
• Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
• Personalized Transformer for Explainable Recommendation
• Fairness of ChatGPT
• Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4
• Is ChatGPT a Good Recommender? A Preliminary Study
• P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
• GPT-4 Technical Report
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• An Overview of Multi-Task Learning in Deep Neural Networks
• Multitask Prompted Training Enables Zero-Shot Task Generalization
• GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Cost
• LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention

Related papers

• The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
• SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
• MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
• How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
• MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
• Training with Missed Targets in Generative Recommendation: Separating Supervision from Probability Competition