Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach
summary
The gist
Large-scale e-commerce search must surface a broad set of items from a vast catalog, ranging from bestselling products to new, trending…
In short
The approach addresses large-scale e-commerce search by combining results from multiple independent retrieval channels into a single unified ranking model. By treating this as a query-dependent learning-to-rank problem, the system significantly improves user conversion metrics while maintaining low latency.
Key concepts
- Query-Dependent Learning-to-Rank
- This is a method where the ranking model is trained to optimize the order of items specifically based on the search query. Instead of using a fixed ranking rule, it learns how to best rank items for every unique user query.
- Heterogeneous Candidate Sources
- This refers to different ways items are retrieved from the catalog, such as different search channels (e.g., popular products vs. new arrivals). The model must learn how to effectively combine and re-rank results coming from all these distinct sources.
- Conversion-Weighted Label
- The system uses a single score to measure user engagement across several actions (impression, click, purchase). This score is weighted differently based on the value of the action; for example, a purchase has a higher weight than just an impression.
Terminology used across episodes
This episode discusses
- Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach · Paper Radio
- A Survey on E-Commerce Learning to Rank
- Long or Short or Both? An Exploration on Lookback Time Windows of Behavioral Features in Product Search Ranking
- Beyond Text: Text-Image Fusion for Scalable Two-Tower Retrieval · Paper Radio
The paper
Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach · Read on arXiv
Aditya Gaydhani, Guangyue Xu, Dhanush Kamath, Ankit Singh, Alex Li
Target Corporation
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Unified Ranking Across Retrieval Channels".
Jane: Here's the gist of the paper. Large-scale e-commerce search must surface a broad set of items from a vast catalog, ranging from bestselling products to new, trending, or seasonal items The proposed approach outperforms rank-based fusion methods, leading to a +2.85% improvement in user conversion while meeting a p95 latency under 50 ms.
Tom: Problem Formulation The core problem is reformulated as a query-dependent learning-to-rank problem over heterogeneous candidate sources The system considers a large-scale e-commerce search system that serves user queries in real time For a given query q, each channel ck independently produces a ranked list Rk (q) = (i, sk(q,i)) i ∈ I The final candidate pool for re-ranking is defined as the union of these truncated lists R(q) = Øk=1 R(nk)k(q).
Jane: Data Representation and Label Construction All training instances are defined at the query–item–week level to explicitly incorporate temporal dynamics into the ranking process The final candidate pool for re-ranking is defined as the union of these truncated lists R(q) = Øk=1 R(nk)k(q).
Tom: User interactions follow a conversion hierarchy: Impression → Click → AddToCart → Purchase.
Paper summary: Tom: So this paper is about unifying ranking across different search channels in a large e-commerce system. They claim that their approach outperforms rank based fusion methods like Reciprocal Rank Fusion and Weighted Interleaving by getting a two point eight five percent improvement in user conversion while keeping the latency under fifty milliseconds.
Jane: That’s a solid start for what they’re saying, Tom. So, what is the core problem they are tackling here? Why do we need this unified approach if we already have different channels like product search or personalized recommendations?
Tom: The problem is that these different channels each produce their own ranked list of items for a given query. The system has to figure out how to merge those lists into one final candidate pool for re-ranking, and they say existing rank based fusion methods treat the channels in a way that doesn't account for how useful each channel is depending on the specific query.
Lu: What they are proposing is a learning to rank problem that depends on the query itself, over these different sources. They are looking for a model that can learn this query dependent channel utility and crosschannel interactions. That moves beyond just merging lists with fixed weights or simple rules like Reciprocal Rank Fusion, which they say don't account for that.
Meng: But learning something new at scale is always tricky because of the time constraints. What kind of data are they training this learning model on? Is it just static item features?
Tom: They define their training instances at the query week level, meaning every item is ranked against a specific query within a weekly window. And they build a label based on user interactions following this conversion hierarchy: Impression, Click, AddToCart, and Purchase.
Jane: That sounds like they are trying to capture the user journey really well. How do they actually turn those interactions into a single number for the ranking model? I'm picturing some kind of weighted sum of those actions.
Tom: They define a scalar engagement label, L(q,i,w), which is a weighted aggregation of view only counts, click counts, add to cart counts, and purchase counts for each query item within that week. They calibrate the weights using corpus level conversion statistics to give more importance to rarer high value actions like purchases.
Lu: Specifically they set alpha equal one, beta equal the absolute value of purchase count divided by add-to-cart count, and gamma equal the absolute value of purchase count divided by click count. This means they are explicitly weighting those downstream actions heavily during optimization.
Paper summary: Meng: If you're optimizing for purchases but your system is running on real traffic, how do you handle the fact that some queries might have very few purchases in a short window? That sounds like it could lead to noisy signals if not handled right.
Tom: They address that by applying a per query max normalization to this engagement label, which they call Ltilde(q,i,w). This normalizes the label so it remains comparable across different queries and time windows. The normalization factor is four times the original label value.
Jane: So they’re trying to make sure that even if one query has almost no purchases, its signal isn't completely drowned out compared to a query with many purchases. That’s a smart move for handling the variability in real user behavior.
Tom: Exactly. And they use Gradient Boosted Decision Trees, or GBDT, as the ranking model itself because it performs well on these structured feature sets and is computationally efficient for production systems. The model learns how to combine the query features and item features to predict that engagement label Ltilde(q,i,w).
Lu: I’m interested in the objective function they use. They are training this GBDT using the LambdaMART objective, which uses pairwise gradient updates derived from NDCG to optimize for ranking quality. This directly optimizes how well the model ranks items according to what we consider good ranking metrics.
Meng: Optimizing for NDCG is standard, but making sure that the learning process actually reflects real user behavior under strict latency constraints is where I get nervous. How do you ensure this training pipeline stays fast enough for online serving?
Tom: They focus on building the model structure efficiently, and they say their approach meets a p95 latency of under fifty milliseconds. The GBDT architecture itself is chosen because it's strong in production settings and handles the scale required for large e-commerce systems.
Jane: So, we’ve seen how they define the problem and how they’re training it, but what are the actual results? What did this unified ranking model actually achieve compared to what was out there before?
Tom: Offline, their unified ranking model improved NDCG at eight from zero point six six two zero to zero point seven one six nine. That’s a significant jump in the quality of the rankings themselves based on that metric.
Lu: And they also showed that adding query item engagement features—the label Ltilde(q,i,w)—increased their offline NDCG at eight from zero point seven one six nine to zero point seven seven nine nine. That suggests the features they engineered are actually useful for improving the ranking quality beyond what was already in place.
Meng: But I need to know how that translates online, because offline metrics don't always predict what happens when a user is actually clicking around. What about business impact?
Paper summary: Tom: Online A/B test results show consistent gains across all their business metrics. They found statistically significant improvements in add to cart at one point, which was a one point two one percent increase, and conversion at another point, which was a one point two eight percent increase.
Jane: That’s interesting because they also mentioned that by incorporating the conversion weighted label, they improved add to cart by two point eight one percent and conversion by two point eight five percent while keeping the click through rate about the same. They are showing that you can optimize for different business goals simultaneously without hurting other metrics much.
Tom: So, if we look at how this paper, "Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach," is positioned in the bigger picture, it’s about taking a complex problem—merging heterogeneous search results—and solving it by learning a query dependent utility function rather than using fixed fusion rules.
Lu: It’s about showing that learning to rank models can be adapted to model channel utility directly, which is something existing LTR methods typically don't do when dealing with multi channel sources. It shows the power of GBDT when you feed it rich, temporally aware features derived from user behavior.
Meng: For someone like me looking at production deployment, the fact that they used GBDT instead of a massive neural network is actually pretty important for us. It suggests there’s a practical path to deploying these complex ranking models without needing enormous computational resources for inference.
Jane: Thinking about what this means for the world of search, it seems like we move away from having separate, siloed ranking systems that just plug their results together with a simple formula and toward one more intelligent system that understands context and query intent dynamically.
Tom: The authors are looking ahead too. Their future work mentions three main areas: handling sparse queries better with things like importance sampling or label smoothing, systematically evaluating bias across channels for fairness, and incorporating personalization signals to boost effectiveness further.
Lu: Incorporating personalization signals is definitely the next logical step for improving ranking effectiveness beyond just query and item features. It moves it toward a more holistic user model.
Meng: I agree with Lu on that. But I still wonder about the data quality for those personalization signals when you're dealing with millions of items and fast serving times. It’s a big lift to get that right without hurting performance.
Jane: Well, so the main thing here is this unified approach, which gives them a solid baseline by showing how much better they can do by moving away from simple rank based fusion towards this learning to rank framework with query dependent labels. That's what we saw in "Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach."
Conclusion: Tom: So we’ve been digging into this paper about unifying ranking across different search channels for large e-commerce systems.
Jane: Right, so basically, they took all those separate lists from different channels—like product search and recommendations—and figured out a way to combine them intelligently instead of just sticking them together with a simple formula.
Tom: Exactly. They’re using this learning-to-rank approach to figure out how useful each channel is for any specific user query, which is pretty smart because it stops you from treating every search result the same way.
Jane: I wonder what the authors actually did to make sure this works so fast in a real system, not just on a computer?
Lu: They used Gradient Boosted Decision Trees, or GBDT. That’s a specific model architecture they picked because it handles structured data really well and it runs pretty quickly for production serving.
Meng: Quick question on the speed—how fast is this whole thing supposed to be when we're serving millions of requests? What breaks if the features get too big?
Tom: They claim a p95 latency under fifty milliseconds, which they say is achievable with their setup. It’s designed for real-time use.
Jane: And what did they find when they ran these experiments? Did this unified model actually beat the older ways of combining rankings?
Tom: Offline, it improved their ranking quality metric called NDCG eight from zero point six six two zero to zero point seven one six nine. That’s a measurable jump in how good the results are.
Lu: And they showed that adding those query and item engagement features—that label they built based on clicks and purchases—actually boosted that offline NDCG eight even higher, up to zero point seven seven nine nine.
Jane: So it seems like this unified method gives a solid performance boost both in terms of how good the results look on paper and what happens when users actually start clicking around online.
Tom: Online A/B tests confirmed those gains, showing statistically significant improvements in things like adding items to the cart and overall conversion rates. It’s not just theoretical stuff; it’s working in practice.
Meng: That’s good, but I still have to ask about the personalization part. How do we make this work when we need to add personalized signals without slowing down the entire pipeline?
Jane: That leads us into the next part of their work, which is looking at how they can improve things further by adding personalization signals.
Tom: So, what's the big picture here? They’re moving away from siloed ranking systems toward one intelligent system that understands context and query intent dynamically.
Jane: It shows that learning to rank models can be adapted to model channel utility directly, which is a new way to handle multi-channel search results.
Tom: And they’re looking ahead too, focusing on making sure it works better for tricky queries, like sparse ones, and making sure it's fair across all the different channels.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game