Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: We're looking at the paper: "Recommendation Quality and the Concentration of Consumption".
Paper summary: Ruth: So we’re going into a bit more detail on this study. The core thesis of "Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix" is that recommendation quality improvements don't necessarily lead to a concentration of consumption at the top, as previously thought.
Warren: They argue that instead, better algorithms diffuse consumption away from superstars and toward a larger number of moderately popular titles. This is what they call the middle-tail.
Iris: The paper claims that this shift happens because algorithmic improvements allow recommendation systems to better infer consumer preferences from the limited data they have.
Ruth: That’s the mechanism they are pointing to. They suggest that existing algorithms have a popularity bias, and as technology gets smarter, it starts overcoming that bias in a way that favors those in the middle of the distribution.
Warren: It challenges older literature that said recommendation systems inherently polarize consumption—meaning they just concentrate consumption among either the head or tail of products.
Iris: They are providing experimental evidence to show this distributional effect isn't fixed; it’s dynamic and depends on things like algorithm maturity and how much title-level interaction data is available.
Ruth: So why does that matter for the real world? It suggests that if we keep investing in these recommendation systems, the payoff isn't just making the top items slightly better; it’s unlocking a whole new segment of content consumption.
Warren: That means platforms can get bigger engagement and users are more likely to find something to play from their system, but they also have to structure their investment strategy around improving those middle-tail titles.
Conclusion: Ruth: To wrap up, this work by Aridor, Winston Chou, Kallus, Scheid, Tran, et al., is about how the technology we build for suggesting content affects consumer behavior in a very specific way.
Warren: The main implication is that as recommendation technology evolves, the system doesn't just get better at predicting what people will watch; it changes *what* they watch by shifting their focus to a more diverse set of titles.
Iris: Essentially, it suggests that the value of recommendation systems isn't solely in optimizing for extreme popularity; it’s in their ability to intelligently explore and serve the middle ground.
Ruth: It’s about recognizing that the returns on investing in middle-tail content are tied directly to how mature and data-rich your platform's recommender system is becoming.
Warren: So, if you’re running a service, you need to think about scaling not just toward more superstars, but how your algorithms handle the trade-off between getting better at matching the middle versus getting better at matching the extremes.
Netflix
econ.GN, cs.IR, q-fin.EC
Submitted: 2026-08-21
Updated: 2026-11-18
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 91/100
The gist: We study an experiment with 8.5 million users on Netflix’s recommender system to measure how improvements in recommendation technology affect the set of products that get consumed Improvements…
Key concepts
- Superstars
- These are the most popular titles on Netflix. The study found that as recommendation quality improves, users stop consuming these highly popular titles and start playing more moderately popular ones instead.
- Middle-tail Titles
- These are titles with moderate popularity. The research shows that better algorithms favor these middle-tail items over superstars because they offer a larger marginal gain in user satisfaction when the algorithm gets smarter.
- Long-tail Titles
- These are the least popular, most niche titles. The study found that recommendation quality improvements have minimal effect on these titles, suggesting their consumption patterns are less sensitive to algorithmic advancements.
- Algorithmic Maturity ($\delta$)
- This parameter represents the quality of the recommendation system's technology. As this maturity increases, the system becomes better at inferring user preferences from existing data, leading to more accurate and beneficial recommendations.
Terminology
Summary
We study an experiment with 8. p. 0 Recommendation Quality and the Concentration of Consumption:
Experimental Evidence from Netflix∗
Guy Aridor Winston Chou Nathan Kallus
Antoine Scheid Allen Tran Kevin Zielnicki
August 24, 2026
Abstract
We study an experiment with 8. 5 million users on Netflix’s recommender system to measure how
improvements in recommendation technology affect the set of products that get consumed.Where the paper says it
Pruning redundant reasoning tokens
The core finding is that ongoing improvements to recommendation technology increase overall engagement on the platform and users’ reliance on the recommendations for plays, primarily by shifting the recommendations – and subsequently plays – away from the most popular titles (“superstars”) and toward moderately popular titles (“middle-tail”), with minimal effects on the long-tail. This shift is consistent with a stylized model where algorithmic improvements allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data, attenuating the popularity bias inherent in existing algorithms. p. 1 As recommendation
technology matures, then, the plausible set of targetable titles continues to shift toward the middle-tail. We argue that these results are consistent with a stylized model in which algorithmic improvements
allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data. As recommendation
technology matures, then, the plausible set of targetable titles continues to shift toward the middle-tail. We argue that these results are consistent with a stylized model in which algorithmic improvements
allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data.Where the paper says it
Where the paper says it
Where the gains come from
The gains are largest for middle-tail titles: unlike superstars, they are not already reliably matched, and unlike long-tail titles, they generate enough data and command a large enough audience for better inference to matter. The primary justification for the assumption of middle-tail dominance is that the marginal precision gain is largest for the middle-tail: pM q′M(δ) > pL q′L(δ) ≥ 0. p. 3 Over the empirically relevant range of δ, the marginal precision gain is
largest for the middle-tail: pM q
′
M(δ) > pL q
′
L
(δ) ≥ 0. The primary justification for this assumption comes from the popularity bias widely documented in the
recommendation systems literature (Celma and Cano, 2008; Klimashevskaia et al. 1
Improvements in recommendation technology attenuate this bias by allowing the RecSys to identify relevant
titles with less data. The gains are largest for middle-tail titles: unlike superstars, they are not already
reliably matched, and unlike long-tail titles, they generate enough data and command a large enough audience
for better inference to matter.Where the paper says it
Where the paper says it
How the training signal is built
The Bayesian recommender system microfounds the two key components of Assumption 1: the popularity default and the larger marginal gains for middle-tail titles. The signal precision is given by αj = δλj, where δ indexes technology quality as in our primary model and λL < λM < λS captures the fact that more popular titles have more interaction data. p. 1 We argue that these results are consistent with a stylized model in which algorithmic improvements
allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data. Since less popular titles generate fewer interactions, the model captures this data constraint as a tendency
for the RecSys to default toward popular titles when preference signals for less popular titles are weak. 1 Setup
We consider a representative user i whose true utility from consuming title j is given by:
Uij = µj + θij ,
where µj ∈ R is baseline popularity and θij is drawn i. across (i, j) with bounded, strictly positive
density on R and finite mean.Where the paper says it
Where the paper says it
The analysis decomposes the effect of a marginal increase in δ into an inframarginal change in which title the user consumes and an extensive-margin change in whether the user consumes. The resulting average match quality is weakly increasing if and only if rinf(δ)E′(δ)ginf(δ)z true-utility gains from inframarginal switches per extensive-margin entry ginf(δ)z true utility gain per inframarginal switch ≥ U(δ) − uext(δ)z match-quality gap per extensive-margin entry. p. 4 Finally, it is important to understand whether, conditional on consumption, the improved recommendations lead users to better-matched titles. This is theoretically ambiguous in our model as there are
two counteracting forces: inframarginal switching may increase average match quality if the improved recommendations shift a user’s consumption from S to their preferred title, but extensive-margin entry may
decrease average match quality by inducing users who would otherwise consume the outside option to consume marginal titles.Where the paper says it
The results confirm Prediction 3: view hours increased by 0.37%, the total number of titles played by 0. p. 9 Bars denote 95%
confidence intervals computed via robust standard errors. Figure 4 presents the estimates of β from (2) across four different measures of engagement: total view
hours, number of plays, the number of days with at least one play, and the number of distinct titles played. Bars denote 95%
confidence intervals computed via robust standard errors. Figure 4 presents the estimates of β from (2) across four different measures of engagement: total view
hours, number of plays, the number of days with at least one play, and the number of distinct titles played. Bars denote 95%
confidence intervals computed via robust standard errors. Figure 4 presents the estimates of β from (2) across four different measures of engagement: total view
hours, number of plays, the number of days with at least one play, and the number of distinct titles played. Crucially, it implies that the shift away from superstars documented in Section 4 is
not diversification for diversification’s sake, but actually coincides with improved engagement. The intuition from our conceptual model for why the improvement leads to an increase in overall engagement is simple: the RecSys replaces less well-matched superstars with better matched middle-tail titles
which results in users being more likely to find content to play from the recommender system.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The analysis shows that d/dδ Pr(ci ∈ Ri) ≥ E′(δ) > 0, and we can conclude that Pr(ci ∈ Ri ci ≠ 0) is strictly increasing in δ. This strict increase is shown by comparing the derivatives of the two terms in (10), which weakly exceeds the bracketed term in (9).
The average match quality is strictly increasing when the inequality is strict, locally unchanged when it holds with equality, and decreasing when it is reversed. This final prediction establishes that at the current frontier, continued improvement lifts overall engagement and reallocates it toward the middle-tail. p. 2 (2026) show
using a structural model that previous RecSys improvements lift engagement by better identifying users with
high incremental consumption from recommendations, and that this value is largest for middle-tail titles. We provide experimental evidence that continued improvements to the RecSys further shift consumption
toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while
lifting engagement overall.Where the paper says it
The paper provides experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This demonstrates that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data. p. 0 Improvements
increase total consumption and users’ reliance on recommendations while diffusing recommendations and
consumption away from the most popular titles (“superstars”) toward a larger number of moderately
popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results
challenge the notion that recommender systems polarize consumption – raising the consumption shares
of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail
products grow as algorithms improve and platforms scale.Where the paper says it
The distribution shifts are measured by estimating specification (1) on recommendations users observe during the experiment for each of the title buckets. Figure 3c confirms that HHI falls by a statistically and economically significant 1. p. 7 We estimate the effects of the technology improvement on the Herfindahl–Hirschman Index
(HHI = PN
j=1 s
2
j
) – a standard measure for the degree of concentration that we compute at the title level –
to test this. Figure 2c confirms that HHI has a statistically and economically significant decrease of 5. 8
Finally, we measure whether this corresponds to a reduction in play concentration: Figure 3c shows that
HHI falls by a statistically and economically significant 1. 2%, consistent with viewership dispersing from
superstar titles to more personalized middle-tail titles. percentile in the control group. 8
Finally, we measure whether this corresponds to a reduction in play concentration: Figure 3c shows that
HHI falls by a statistically and economically significant 1.Where the paper says it
Where the paper says it
Where the paper says it
The analysis shows that the ratio of plays to visits increases by 0.47%, the share of plays originating from the recommendations increases by 0. p. 4 The main force behind this prediction is that some users who preferred M or L over S were previously
recommended S due to the popularity bias, which led them to consume nothing; as the RecSys shifts to their
preferred title, they consume it and the outside-option share falls. Because this newly realized consumption
comes from the recommended title, the share of consumption originating from recommendations should rise
as well:
Prediction 4 (Share of Plays from Recommendation). , my list). We then estimate (2) for the ratio of plays to visits,
the share of plays that originate from recommendations, and the mean reciprocal rank (MRR) of plays. As recommendation
technology matures, then, the plausible set of targetable titles continues to shift toward the middle-tail. We argue that these results are consistent with a stylized model in which algorithmic improvements
allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data. Improvements therefore shift recommendations toward these better-matched
titles, raising overall engagement. This paper contributes to a long-standing literature on how digitization has reshaped the types of goods
that get consumed (Goldfarb and Tucker, 2019), and in particular to conflicting findings on whether recommendation systems concentrate or disperse consumption.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The analysis shows that users complete 1.3% more titles at each of the X = 15, 30, 45-minute thresholds and provide 0. p. 2 (2026) show
using a structural model that previous RecSys improvements lift engagement by better identifying users with
high incremental consumption from recommendations, and that this value is largest for middle-tail titles. We provide experimental evidence that continued improvements to the RecSys further shift consumption
toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while
lifting engagement overall. 1 Setup
We consider a representative user i whose true utility from consuming title j is given by:
Uij = µj + θij ,
where µj ∈ R is baseline popularity and θij is drawn i. across (i, j) with bounded, strictly positive
density on R and finite mean. Finally, it is important to understand whether, conditional on consumption, the improved recommendations lead users to better-matched titles. This is theoretically ambiguous in our model as there are
two counteracting forces: inframarginal switching may increase average match quality if the improved recommendations shift a user’s consumption from S to their preferred title, but extensive-margin entry may
decrease average match quality by inducing users who would otherwise consume the outside option to consume marginal titles.Where the paper says it
Where the paper says it
Where the paper says it
The paper provides a simple Bayesian RecSys that microfounds the two key components of Assumption 1: the popularity default and the larger marginal gains for middle-tail titles. The structure motivates each component of Assumption 1: (i) Popularity default, (ii) qj monotonicity in δ, and (iii) Middle-tail dominance. p. 3 , 2024), by which deployed
algorithms over-recommend popular titles because less popular titles generate too little data to extract a
reliable signal. Appendix C motivates the assumption with a Bayesian recommender that generates this bias
endogenously and shows the popularity bias and, over the relevant range of δ, that the weighted identification
gain pj q
′
j
(δ) is largest for the middle-tail: the superstar’s q
′
S
(δ) is small, since it is already well targeted, and
while the long-tail’s q
′
L
(δ) may be sizeable, too few users rank it first for the identification gain to matter. An increase in δ increases total engagement: Pr(ci ̸= 0) is strictly
increasing in δ. The main force behind this prediction is that some users who preferred M or L over S were previously
recommended S due to the popularity bias, which led them to consume nothing; as the RecSys shifts to their
preferred title, they consume it and the outside-option share falls.Where the paper says it
Where the paper says it
The paper provides experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This suggests that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data. p. 0 Improvements
increase total consumption and users’ reliance on recommendations while diffusing recommendations and
consumption away from the most popular titles (“superstars”) toward a larger number of moderately
popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results
challenge the notion that recommender systems polarize consumption – raising the consumption shares
of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail
products grow as algorithms improve and platforms scale. Improvements
increase total consumption and users’ reliance on recommendations while diffusing recommendations and
consumption away from the most popular titles (“superstars”) toward a larger number of moderately
popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results
challenge the notion that recommender systems polarize consumption – raising the consumption shares
of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail
products grow as algorithms improve and platforms scale. Improvements
increase total consumption and users’ reliance on recommendations while diffusing recommendations and
consumption away from the most popular titles (“superstars”) toward a larger number of moderately
popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results
challenge the notion that recommender systems polarize consumption – raising the consumption shares
of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail
products grow as algorithms improve and platforms scale.Where the paper says it
Where the paper says it
Where the paper says it
The paper is a long-term holdback experiment comparing a frozen algorithm with an evolving production treatment incorporating all algorithmic innovations deployed during a 60-day experimental period. This experiment represents an ideal testing ground for the model for several reasons, including the inclusion of all innovations made to personalization algorithms during the experiment timeframe. p. 6 This experiment represents an ideal testing ground for our model for several reasons. First, the treatment
includes all innovations made to personalization algorithms during the experiment timeframe.Where the paper says it
The paper provides a stylized conceptual model where user utility is Uij = µj + θij, and choice utility is Vij = Uij + b · 1(j ∈ Ri), capturing the role of recommendations in reducing informational frictions. The platform’s recommendation technology is indexed by a single parameter δ, with Assumption 1 specifying the popularity default and middle-tail dominance. p. 3 The platform’s recommendation technology is indexed by a
single parameter δ and we make the following assumptions about it. Assumption 1 (Recommendation Technology).Where the paper says it
The paper contributes to a long-standing literature on how digitization has reshaped the types of goods that get consumed (Goldfarb and Tucker, 2019). The authors identify a second, dynamic dimension – algorithm maturity and the volume of title-level interaction data – that changes which titles benefit as recommendation quality improves. p. 2 Tucker and Zhang
(2011) show that popularity information reinforces superstar concentration under homogeneous preferences
but can benefit niche products when preferences are heterogeneous. Our contribution is to identify a second,
dynamic dimension – algorithm maturity and the volume of title-level interaction data – that changes which
titles benefit as recommendation quality improves. Improvements therefore shift recommendations toward these better-matched
titles, raising overall engagement. This paper contributes to a long-standing literature on how digitization has reshaped the types of goods
that get consumed (Goldfarb and Tucker, 2019), and in particular to conflicting findings on whether recommendation systems concentrate or disperse consumption.Where the paper says it
Where the paper says it
The authors provide experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This suggests that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data.
The paper is an empirical study on Netflix’s RecSys involving more than 8.5 million subscribers. p. 1 This paper studies how improvements to recommendations reshape viewership using a unique experiment
on Netflix’s RecSys involving more than 8. 5 million subscribers. To evaluate the empirical implications of our model, we analyze an experiment on
Netflix on 8,559,252 subscribers over a 60-day period from February to April 2025. This experiment was
designed as a “cumulative holdback” test to measure the collective impact of multiple innovations to personalization algorithms during this period. 21% increase in days with a play – represents about
5% of this magnitude. Crucially, it implies that the shift away from superstars documented in Section 4 is
not diversification for diversification’s sake, but actually coincides with improved engagement.Where the paper says it
Where the paper says it
Where the paper says it
The authors provide experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This suggests that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Recommendation system recalibration to target middle-tail products: The improved system can shift recommendations away from
superstars
towardmoderately popular titles,
as this is where recommendation quality improvements yield the largest gains for user engagement and consumption. -
Dynamic targeting based on algorithm maturity: The AI system should continuously monitor its own algorithmic maturity (indexed by delta) to determine which product distribution segment—long-tail, middle-tail, or superstar—is most effectively targetable for increased return on investment.
-
Enhanced estimation of consumer preference signals: By incorporating the mechanism where
improvements in recommendation technology attenuate this bias by allowing the RecSys to identify relevant titles with less data,
the AI can better infer consumer preferences from limited interaction data, especially for less popular titles. -
Increased engagement through precision gains: The system should prioritize recommendations that result in a strictly increasing probability of consumption originating from recommendations, as
an increase in delta results in a larger share of consumption originating from recommendations.
-
Adaptive match quality monitoring: The AI can assess the
average match quality
by tracking whether the shift toward middle-tail titles increasesthe likelihood that users find more 'high quality' titles to play,
which is measured by metrics like positive thumbs and watch duration shares.
Abstract
We study an experiment with 8.5 million users on Netflix's recommender system to measure how improvements in recommendation technology affect the set of products that get consumed. Improvements increase total consumption and users' reliance on recommendations while diffusing recommendations and consumption away from the most popular titles ("superstars") toward a larger number of moderately popular titles ("middle-tail"), with minimal effects on the most niche titles ("long-tail"). Our results challenge the notion that recommender systems polarize consumption -- raising the consumption shares of the head and tail at the expense of the middle -- and suggest that the returns to investing in middle-tail products grow as algorithms improve and platforms scale.
Related papers
- Huge If True: Learning from the Size of the Surprise
- The Rebundling of Freelance Work in the AI Age
- Dynamic investment and coordination in teams: Theory and experimental evidence
- Does Frontier AI Strengthen Cyber Defense? Evidence from Project Glasswing
- Generative AI at Work
- What is Work For? Labor, Politics, and Meaning in the Age of AI