Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Unified Ranking Across Retrieval Channels".
Jane: Here's the gist of the paper. Large-scale e-commerce search must surface a broad set of items from a vast catalog, ranging from bestselling products to new, trending, or seasonal items The proposed approach outperforms rank-based fusion methods, leading to a +2.85% improvement in user conversion while meeting a p95 latency under 50 ms.
Tom: Problem Formulation The core problem is reformulated as a query-dependent learning-to-rank problem over heterogeneous candidate sources The system considers a large-scale e-commerce search system that serves user queries in real time For a given query q, each channel ck independently produces a ranked list Rk (q) = (i, sk(q,i)) i ∈ I The final candidate pool for re-ranking is defined as the union of these truncated lists R(q) = Øk=1 R(nk)k(q).
Jane: Data Representation and Label Construction All training instances are defined at the query–item–week level to explicitly incorporate temporal dynamics into the ranking process The final candidate pool for re-ranking is defined as the union of these truncated lists R(q) = Øk=1 R(nk)k(q).
Tom: User interactions follow a conversion hierarchy: Impression → Click → AddToCart → Purchase.
Paper summary: Tom: So this paper is about unifying ranking across different search channels in a large e-commerce system. They claim that their approach outperforms rank based fusion methods like Reciprocal Rank Fusion and Weighted Interleaving by getting a two point eight five percent improvement in user conversion while keeping the latency under fifty milliseconds.
Jane: That’s a solid start for what they’re saying, Tom. So, what is the core problem they are tackling here? Why do we need this unified approach if we already have different channels like product search or personalized recommendations?
Tom: The problem is that these different channels each produce their own ranked list of items for a given query. The system has to figure out how to merge those lists into one final candidate pool for re-ranking, and they say existing rank based fusion methods treat the channels in a way that doesn't account for how useful each channel is depending on the specific query.
Lu: What they are proposing is a learning to rank problem that depends on the query itself, over these different sources. They are looking for a model that can learn this query dependent channel utility and crosschannel interactions. That moves beyond just merging lists with fixed weights or simple rules like Reciprocal Rank Fusion, which they say don't account for that.
Meng: But learning something new at scale is always tricky because of the time constraints. What kind of data are they training this learning model on? Is it just static item features?
Tom: They define their training instances at the query week level, meaning every item is ranked against a specific query within a weekly window. And they build a label based on user interactions following this conversion hierarchy: Impression, Click, AddToCart, and Purchase.
Jane: That sounds like they are trying to capture the user journey really well. How do they actually turn those interactions into a single number for the ranking model? I'm picturing some kind of weighted sum of those actions.
Tom: They define a scalar engagement label, L(q,i,w), which is a weighted aggregation of view only counts, click counts, add to cart counts, and purchase counts for each query item within that week. They calibrate the weights using corpus level conversion statistics to give more importance to rarer high value actions like purchases.
Lu: Specifically they set alpha equal one, beta equal the absolute value of purchase count divided by add-to-cart count, and gamma equal the absolute value of purchase count divided by click count. This means they are explicitly weighting those downstream actions heavily during optimization.
Paper summary: Meng: If you're optimizing for purchases but your system is running on real traffic, how do you handle the fact that some queries might have very few purchases in a short window? That sounds like it could lead to noisy signals if not handled right.
Tom: They address that by applying a per query max normalization to this engagement label, which they call Ltilde(q,i,w). This normalizes the label so it remains comparable across different queries and time windows. The normalization factor is four times the original label value.
Jane: So they’re trying to make sure that even if one query has almost no purchases, its signal isn't completely drowned out compared to a query with many purchases. That’s a smart move for handling the variability in real user behavior.
Tom: Exactly. And they use Gradient Boosted Decision Trees, or GBDT, as the ranking model itself because it performs well on these structured feature sets and is computationally efficient for production systems. The model learns how to combine the query features and item features to predict that engagement label Ltilde(q,i,w).
Lu: I’m interested in the objective function they use. They are training this GBDT using the LambdaMART objective, which uses pairwise gradient updates derived from NDCG to optimize for ranking quality. This directly optimizes how well the model ranks items according to what we consider good ranking metrics.
Meng: Optimizing for NDCG is standard, but making sure that the learning process actually reflects real user behavior under strict latency constraints is where I get nervous. How do you ensure this training pipeline stays fast enough for online serving?
Tom: They focus on building the model structure efficiently, and they say their approach meets a p95 latency of under fifty milliseconds. The GBDT architecture itself is chosen because it's strong in production settings and handles the scale required for large e-commerce systems.
Jane: So, we’ve seen how they define the problem and how they’re training it, but what are the actual results? What did this unified ranking model actually achieve compared to what was out there before?
Tom: Offline, their unified ranking model improved NDCG at eight from zero point six six two zero to zero point seven one six nine. That’s a significant jump in the quality of the rankings themselves based on that metric.
Lu: And they also showed that adding query item engagement features—the label Ltilde(q,i,w)—increased their offline NDCG at eight from zero point seven one six nine to zero point seven seven nine nine. That suggests the features they engineered are actually useful for improving the ranking quality beyond what was already in place.
Meng: But I need to know how that translates online, because offline metrics don't always predict what happens when a user is actually clicking around. What about business impact?
Paper summary: Tom: Online A/B test results show consistent gains across all their business metrics. They found statistically significant improvements in add to cart at one point, which was a one point two one percent increase, and conversion at another point, which was a one point two eight percent increase.
Jane: That’s interesting because they also mentioned that by incorporating the conversion weighted label, they improved add to cart by two point eight one percent and conversion by two point eight five percent while keeping the click through rate about the same. They are showing that you can optimize for different business goals simultaneously without hurting other metrics much.
Tom: So, if we look at how this paper, "Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach," is positioned in the bigger picture, it’s about taking a complex problem—merging heterogeneous search results—and solving it by learning a query dependent utility function rather than using fixed fusion rules.
Lu: It’s about showing that learning to rank models can be adapted to model channel utility directly, which is something existing LTR methods typically don't do when dealing with multi channel sources. It shows the power of GBDT when you feed it rich, temporally aware features derived from user behavior.
Meng: For someone like me looking at production deployment, the fact that they used GBDT instead of a massive neural network is actually pretty important for us. It suggests there’s a practical path to deploying these complex ranking models without needing enormous computational resources for inference.
Jane: Thinking about what this means for the world of search, it seems like we move away from having separate, siloed ranking systems that just plug their results together with a simple formula and toward one more intelligent system that understands context and query intent dynamically.
Tom: The authors are looking ahead too. Their future work mentions three main areas: handling sparse queries better with things like importance sampling or label smoothing, systematically evaluating bias across channels for fairness, and incorporating personalization signals to boost effectiveness further.
Lu: Incorporating personalization signals is definitely the next logical step for improving ranking effectiveness beyond just query and item features. It moves it toward a more holistic user model.
Meng: I agree with Lu on that. But I still wonder about the data quality for those personalization signals when you're dealing with millions of items and fast serving times. It’s a big lift to get that right without hurting performance.
Jane: Well, so the main thing here is this unified approach, which gives them a solid baseline by showing how much better they can do by moving away from simple rank based fusion towards this learning to rank framework with query dependent labels. That's what we saw in "Unified Ranking Across Retrieval Channels: A Large-Scale E-Commerce Approach."
Conclusion: Tom: So we’ve been digging into this paper about unifying ranking across different search channels for large e-commerce systems.
Jane: Right, so basically, they took all those separate lists from different channels—like product search and recommendations—and figured out a way to combine them intelligently instead of just sticking them together with a simple formula.
Tom: Exactly. They’re using this learning-to-rank approach to figure out how useful each channel is for any specific user query, which is pretty smart because it stops you from treating every search result the same way.
Jane: I wonder what the authors actually did to make sure this works so fast in a real system, not just on a computer?
Lu: They used Gradient Boosted Decision Trees, or GBDT. That’s a specific model architecture they picked because it handles structured data really well and it runs pretty quickly for production serving.
Meng: Quick question on the speed—how fast is this whole thing supposed to be when we're serving millions of requests? What breaks if the features get too big?
Tom: They claim a p95 latency under fifty milliseconds, which they say is achievable with their setup. It’s designed for real-time use.
Jane: And what did they find when they ran these experiments? Did this unified model actually beat the older ways of combining rankings?
Tom: Offline, it improved their ranking quality metric called NDCG eight from zero point six six two zero to zero point seven one six nine. That’s a measurable jump in how good the results are.
Lu: And they showed that adding those query and item engagement features—that label they built based on clicks and purchases—actually boosted that offline NDCG eight even higher, up to zero point seven seven nine nine.
Jane: So it seems like this unified method gives a solid performance boost both in terms of how good the results look on paper and what happens when users actually start clicking around online.
Tom: Online A/B tests confirmed those gains, showing statistically significant improvements in things like adding items to the cart and overall conversion rates. It’s not just theoretical stuff; it’s working in practice.
Meng: That’s good, but I still have to ask about the personalization part. How do we make this work when we need to add personalized signals without slowing down the entire pipeline?
Jane: That leads us into the next part of their work, which is looking at how they can improve things further by adding personalization signals.
Tom: So, what's the big picture here? They’re moving away from siloed ranking systems toward one intelligent system that understands context and query intent dynamically.
Jane: It shows that learning to rank models can be adapted to model channel utility directly, which is a new way to handle multi-channel search results.
Tom: And they’re looking ahead too, focusing on making sure it works better for tricky queries, like sparse ones, and making sure it's fair across all the different channels.
Aditya Gaydhani, Guangyue Xu, Dhanush Kamath, Ankit Singh, Alex Li
Target Corporation
cs.IR
Submitted: 2026-02-26
Updated: 2026-11-16
Comments: Accepted as a regular paper to the Industry & Government Program of the 2026 IEEE International Conference on Big Data (IEEE BigData 2026), Phoenix, AZ, December 14-17, 2026. 10 pages, 4 figures, 2 tables. v4: accepted version (revised title and text)
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 92/100
The gist: Large-scale e-commerce search must surface a broad set of items from a vast catalog, ranging from bestselling products to new, trending…
Key concepts
- Query-Dependent Learning-to-Rank
- This is a method where the ranking model is trained to optimize the order of items specifically based on the search query. Instead of using a fixed ranking rule, it learns how to best rank items for every unique user query.
- Heterogeneous Candidate Sources
- This refers to different ways items are retrieved from the catalog, such as different search channels (e.g., popular products vs. new arrivals). The model must learn how to effectively combine and re-rank results coming from all these distinct sources.
- Conversion-Weighted Label
- The system uses a single score to measure user engagement across several actions (impression, click, purchase). This score is weighted differently based on the value of the action; for example, a purchase has a higher weight than just an impression.
Terminology
Summary
Large-scale e-commerce search must surface a broad set of items from a vast catalog, ranging from bestselling products to new, trending, or seasonal items The proposed approach outperforms rank-based fusion methods, leading to a +2. p. 085% improvement in user conversion while meeting a p95 latency under 50 ms. p. 1
Problem Formulation
The core problem is reformulated as a query-dependent learning-to-rank problem over heterogeneous candidate sources The system considers a large-scale e-commerce search system that serves user queries in real time For a given query q, each channel ck independently produces a ranked list Rk (q) = (i, sk(q,i)) i ∈ I The final candidate pool for re-ranking is defined as the union of these truncated lists R(q) = Øk=1 R(nk)k(q).
Data Representation and Label Construction
All training instances are defined at the query–item–week level to explicitly incorporate temporal dynamics into the ranking process The final candidate pool for re-ranking is defined as the union of these truncated lists R(q) = Øk=1 R(nk)k(q). User interactions follow a conversion hierarchy: Impression → Click → AddToCart → Purchase. p. 2 A scalar engagement label is defined as L(q,i,w) = aPq,i,w + bAq,i,w + cCq,i,w + dVq,i,w <ref:2602. p. 223530#pg2> Weights are calibrated using corpus-level conversion statistics to assign larger weights to rarer highvalue downstream actions <ref:2602. p. 223530#pg2>.
Model Architecture and Training Strategy
The model adopts Gradient Boosted Decision Trees (GBDT) as the ranking model due to their strong performance on structured and heterogeneous feature sets GBDT constructs an additive ensemble of decision trees: f(q,i; θ) = ∑T t=1 η · ht(q,i), where ht is the t-th tree <ref:2602.23530#pg2>. The model is trained using the LambdaMART objective which directly optimizes ranking quality through pairwise gradient updates derived from NDCG <ref:2602. p. 223530#pg2>.
Performance Gains and Contributions
The unified ranking model improves offline NDCG@8 from 0.6620 to 0.7169 Online A/B results show consistent gains across all business metrics, including statistically significant improvements in add-to-cart (+1. p. 321%) and conversion (+1. p. 028%). Adding query–item engagement features increases offline NDCG@8 from 0. p. 37169 to 0.7799 Incorporating the conversion-weighted label improves add-to-cart (+2. p. 381%) and conversion (+2. p. 085%) while maintaining comparable CTR. p. 3
Future Directions
Future work will focus on three directions: First, current data filtering may under-represent tail queries; techniques such as importance sampling, label smoothing, and semi-supervised learning could improve signal quality for sparse queries. Second, we plan to systematically evaluate potential bias and fairness across channels to ensure equitable exposure. p. 3 Finally, incorporating personalization signals represents a promising avenue for further improving ranking effectiveness and user conversion. p. 3
Acknowledgments
We thank Search — Data Science leadership at Target for their support in enabling this initiative. We are grateful to the Search Product team for their collaboration in formulating the case for incorporating user engagement features into the ranking framework, contributing to the A/B test design, and conducting the postexperiment analysis. p. 3 We also thank the Query Understanding team for their work on query normalization, which improved the quality of the engagement features used in this study. p. 3 We also acknowledge the foundational contributions of the Search Retrieval and Engineering groups in developing the retrieval channels and ranker service that made this work possible. p. 3 We also thank the Content Understanding and Search Platform teams. p. 3
References
The references listed are [1] through [19] The references listed are [4] through [20] The references listed are [8] through [20].
Keywords
The keywords include Rank Fusion, Learning-to-Rank, Multi-Channel Retrieval.
ACM Reference Format
Aditya Gaydhani, Guangyue Xu, Dhanush Kamath, Ankit Singh, and Alex Li. 2026. Unified Learning-to-Rank for Multi-Channel Retrieval in Large-Scale E-Commerce Search. p. 0 In Proceedings of arXiv (Preprint). p. 0 ACM, New York, NY, USA. p. 0
DOI
https://doi.org/XXXXXXX. p. 0XXXXXXX.
CCS Concepts
• Information systems → Combination, fusion and federated search; Rank aggregation; Learning to rank.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Unified Ranking Model Implementation: The system can implement a unified ranking model that
learn[s] to merge and rank documents from multiple retrieval channels
by treating channel information as a signal within a single framework thatjointly optimizes multiple business objectives, including clicks, add-to-carts, and purchases.
-
Query-Dependent Channel Utility: The improved system can capture query specific channel utility by moving away from
fixed global weights to retrieval channels
and instead learning a scoring function f (q,i; theta) thatdynamically captures short-term intent shifts that influence channel utility and conversion likelihood.
-
Enhanced Label Construction: By adopting the proposed
conversion-weighted engagement label,
the system can prioritize high value business outcomes, as this approach shows thatstructured weighting of engagement signals across user journey stages better aligns model training with high-value business outcomes, particularly downstream conversion events.
-
Feature Engineering for Intent Shifts: The system can incorporate query item engagement features to model short-term intent by using
clicks, addto-cart events, and purchases as engagement features
combined withdistinct aggregation and temporal decay to capture short-term intent.
-
Latency Constraint Adherence: The proposed approach ensures system viability by satisfying production requirements, achieving a
p95 latency of under 50 ms,
which allows for deployment on high-traffic environments like Target.com.
Sources
- A Survey on E-Commerce Learning to Rank
- Long or Short or Both? An Exploration on Lookback Time Windows of Behavioral Features in Product Search Ranking
- Beyond Text: Text-Image Fusion for Scalable Two-Tower Retrieval
Related papers
- Revisiting Text Ranking in Deep Research
- GateSID: One Gate to Balance Semantic and Collaborative Signals across the Item Lifecycle
- SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce
- GINGER: Spicing Up Graph-Based Vector Search
- Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA
- One-Step Retrieval Framework for Real-Time Sponsored Search Ads Using Hierarchical Text Representations