UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction
summary
The gist
Ranking is a core stage in online advertising and recommender systems This paper proposes UniRank, an open benchmark for ranking models that unify sequential modeling and feature interaction UniRank…
In short
UniRank introduces an open benchmark for ranking models that combines sequential modeling and feature interaction. It provides standardized protocols, tools, and datasets to systematically compare diverse architectures across different platforms. The research shows that both unified paradigms are competitive, with performance heavily depending on the specific dataset used.
Key concepts
- Unified Sequential Modeling and Feature Interaction
- This is a new approach for ranking models that tries to combine two distinct methods: sequential modeling, which understands user behavior over time, and feature interaction, which learns how different user or item characteristics influence the prediction. UniRank benchmarks models that successfully integrate both of these capabilities.
- Traditional Single-Task Paradigm
- This training method uses a fixed positive feedback sequence shared by many samples to predict a new impression. The goal is to minimize the error when predicting a new item based on this shared positive history, focusing on single prediction tasks.
- Chronological Pointwise Autoregressive Paradigm
- This paradigm builds a long sequence where the model predicts each current target label based only on the strictly preceding behaviors. It creates a full chronological feedback loop, allowing the model to learn temporal dependencies by predicting one step at a time.
Terminology used across episodes
This episode discusses
- UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction · Paper Radio
- LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
- RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
- Beyond Positive Signals: Unlocking Implicit Negative Behaviors for Enhanced Sequential User Modeling
- Bending the Scaling Law Curve in Large-Scale Recommendation Systems
- On Ranking Consistency of Pre-ranking Stage
- Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems
- UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems
- Gaussian Error Linear Units (GELUs)
- MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
- HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
- TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
- Aggregate and Broadcast: Scalable and Efficient Feature Interaction for Recommender Systems
- MerRec: A Large-scale Multipurpose Mercari Dataset for Consumer-to-Consumer Recommendation Systems
- GraphFM: Graph Factorization Machines for Feature Interaction Modeling
- Muon is Scalable for LLM Training
- EST: Towards Efficient Scaling Laws in Click-Through Rate Prediction via Unified Modeling · Paper Radio
- LaProp: Separating Momentum and Adaptivity in Adam
- Decoupled Weight Decay Regularization
- Mish: A Self Regularized Non-Monotonic Activation Function
- Deep Learning Recommendation Model for Personalization and Recommendation Systems
The paper
UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction · Read on arXiv
Honghao Li, Xianquan Wang, Zibin Zhang, Yi Zhang, Kangyi Lin, Yiwen Zhang
Anhui University · University of Science and Technology of China · Tencent Inc.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction".
Paper summary: Jane: So we’ve got the idea that this paper, UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction, aims to address the challenge of comparing models that combine sequential modeling and feature interaction. The thesis is that current advances are often locked behind proprietary data and infrastructure, which limits academic study of scaling laws or long-sequence behavior >
Tom: They propose UniRank as an open benchmark to tackle this, offering standardized evaluation protocols and an optimized PyTorch toolkit for systematic comparison >
Lu: What’s interesting is how they categorize the ranking model approaches into three families: shallow learning methods, deep feature interaction methods, and sequence modeling methods like DIN or DIEN >
Meng: I’m interested in the training signals. They contrast two main ways to handle sequential data: a traditional single-task new impression only paradigm and a chronological pointwise autoregressive paradigm >
Jane: The traditional one uses a fixed positive-feedback sequence to predict a new impression, aiming for the loss LNIO which is the sum of binary cross entropy terms across all new impressions >
Tom: And the second one, the chronological pointwise autoregressive paradigm, constructs a full feedback sequence where the model predicts current target labels from strictly preceding behavior >
Lu: This second paradigm creates a chronological full-feedback sequence where the target loss LAR sums binary cross entropy terms for every time step t and every item k in that sequence >
Meng: So they’re essentially comparing how you structure the feedback—whether you fix it as context or build it step by step chronologically >
Jane: The evaluation shows that both unified interaction paradigms are competitive, and no single model is universally superior across all data distributions, though performance varies by dataset >
Tom: For instance, EST and HeMix are competitive on Taobao and MerRec, but the paper stresses that performance is highly dependent on the specific data distribution you're using >
Conclusion: Tom: So wrapping up UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction. The authors have given us a way to compare these complex ranking architectures systematically, even if we don't have all the private data anymore >
Jane: It really gives researchers a common language and a standardized setup to study how these different modeling families actually perform under realistic conditions across short video, advertising, and e-commerce >
Lu: The implication is that by providing this open benchmark, we can move past comparing closed implementations and start studying the underlying scaling laws more openly >
Meng: From my side, it means we have a standardized way to test what actually runs at scale without needing access to the massive proprietary infrastructure >
Lalam: UniRank gives us a standardized evaluation protocol and an optimized toolkit, which is what makes this work valuable for moving ranking research forward in an open way >
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game