Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix

arXiv:2608.21274 · econ.GN, cs.IR, q-fin.EC · Submitted 2026-08-21 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: We're looking at the paper: "Recommendation Quality and the Concentration of Consumption".

Paper summary: Ruth: So we’re going into a bit more detail on this study. The core thesis of "Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix" is that recommendation quality improvements don't necessarily lead to a concentration of consumption at the top, as previously thought.

Warren: They argue that instead, better algorithms diffuse consumption away from superstars and toward a larger number of moderately popular titles. This is what they call the middle-tail.

Iris: The paper claims that this shift happens because algorithmic improvements allow recommendation systems to better infer consumer preferences from the limited data they have.

Ruth: That’s the mechanism they are pointing to. They suggest that existing algorithms have a popularity bias, and as technology gets smarter, it starts overcoming that bias in a way that favors those in the middle of the distribution.

Warren: It challenges older literature that said recommendation systems inherently polarize consumption—meaning they just concentrate consumption among either the head or tail of products.

Iris: They are providing experimental evidence to show this distributional effect isn't fixed; it’s dynamic and depends on things like algorithm maturity and how much title-level interaction data is available.

Ruth: So why does that matter for the real world? It suggests that if we keep investing in these recommendation systems, the payoff isn't just making the top items slightly better; it’s unlocking a whole new segment of content consumption.

Warren: That means platforms can get bigger engagement and users are more likely to find something to play from their system, but they also have to structure their investment strategy around improving those middle-tail titles.

Conclusion: Ruth: To wrap up, this work by Aridor, Winston Chou, Kallus, Scheid, Tran, et al., is about how the technology we build for suggesting content affects consumer behavior in a very specific way.

Warren: The main implication is that as recommendation technology evolves, the system doesn't just get better at predicting what people will watch; it changes *what* they watch by shifting their focus to a more diverse set of titles.

Iris: Essentially, it suggests that the value of recommendation systems isn't solely in optimizing for extreme popularity; it’s in their ability to intelligently explore and serve the middle ground.

Ruth: It’s about recognizing that the returns on investing in middle-tail content are tied directly to how mature and data-rich your platform's recommender system is becoming.

Warren: So, if you’re running a service, you need to think about scaling not just toward more superstars, but how your algorithms handle the trade-off between getting better at matching the middle versus getting better at matching the extremes.

Netflix

econ.GN, cs.IR, q-fin.EC

Submitted: 2026-08-21

Updated: 2026-11-18

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: We study an experiment with 8.5 million users on Netflix’s recommender system to measure how improvements in recommendation technology affect the set of products that get consumed Improvements…

Key concepts

Superstars
These are the most popular titles on Netflix. The study found that as recommendation quality improves, users stop consuming these highly popular titles and start playing more moderately popular ones instead.
Middle-tail Titles
These are titles with moderate popularity. The research shows that better algorithms favor these middle-tail items over superstars because they offer a larger marginal gain in user satisfaction when the algorithm gets smarter.
Long-tail Titles
These are the least popular, most niche titles. The study found that recommendation quality improvements have minimal effect on these titles, suggesting their consumption patterns are less sensitive to algorithmic advancements.
Algorithmic Maturity ($\delta$)
This parameter represents the quality of the recommendation system's technology. As this maturity increases, the system becomes better at inferring user preferences from existing data, leading to more accurate and beneficial recommendations.

Terminology

Summary

We study an experiment with 8. p. 0

Where the paper says it

Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix∗ Guy Aridor Winston Chou Nathan Kallus Antoine Scheid Allen Tran Kevin Zielnicki August 24, 2026 Abstract We study an experiment with 8. 5 million users on Netflix’s recommender system to measure how improvements in recommendation technology affect the set of products that get consumed.

5 million users on Netflix’s recommender system to measure how improvements in recommendation technology affect the set of products that get consumed Improvements increase total consumption and users’ reliance on recommendations while diffusing recommendations and consumption away from the most popular titles (“superstars”) toward a larger number of moderately popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”) Our results challenge the notion that recommender systems polarize consumption – raising the consumption shares of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail products grow as algorithms improve and platforms scale

Pruning redundant reasoning tokens

The core finding is that ongoing improvements to recommendation technology increase overall engagement on the platform and users’ reliance on the recommendations for plays, primarily by shifting the recommendations – and subsequently plays – away from the most popular titles (“superstars”) and toward moderately popular titles (“middle-tail”), with minimal effects on the long-tail. This shift is consistent with a stylized model where algorithmic improvements allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data, attenuating the popularity bias inherent in existing algorithms. p. 1

Where the paper says it

As recommendation technology matures, then, the plausible set of targetable titles continues to shift toward the middle-tail. We argue that these results are consistent with a stylized model in which algorithmic improvements allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data.

The authors argue that these results are consistent with a stylized model in which algorithmic improvements allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data. p. 1
Where the paper says it

As recommendation technology matures, then, the plausible set of targetable titles continues to shift toward the middle-tail. We argue that these results are consistent with a stylized model in which algorithmic improvements allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data.

Where the gains come from

The gains are largest for middle-tail titles: unlike superstars, they are not already reliably matched, and unlike long-tail titles, they generate enough data and command a large enough audience for better inference to matter. The primary justification for the assumption of middle-tail dominance is that the marginal precision gain is largest for the middle-tail: pM q′M(δ) > pL q′L(δ) ≥ 0. p. 3

Where the paper says it

Over the empirically relevant range of δ, the marginal precision gain is largest for the middle-tail: pM q ′ M(δ) > pL q ′ L (δ) ≥ 0. The primary justification for this assumption comes from the popularity bias widely documented in the recommendation systems literature (Celma and Cano, 2008; Klimashevskaia et al.

This advantage stems from the fact that middle-tail titles are not already reliably matched, and unlike long-tail titles, they generate enough data and command a large enough audience for better inference to matter. p. 1
Where the paper says it

1 Improvements in recommendation technology attenuate this bias by allowing the RecSys to identify relevant titles with less data. The gains are largest for middle-tail titles: unlike superstars, they are not already reliably matched, and unlike long-tail titles, they generate enough data and command a large enough audience for better inference to matter.

How the training signal is built

The Bayesian recommender system microfounds the two key components of Assumption 1: the popularity default and the larger marginal gains for middle-tail titles. The signal precision is given by αj = δλj, where δ indexes technology quality as in our primary model and λL < λM < λS captures the fact that more popular titles have more interaction data. p. 1

Where the paper says it

We argue that these results are consistent with a stylized model in which algorithmic improvements allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data. Since less popular titles generate fewer interactions, the model captures this data constraint as a tendency for the RecSys to default toward popular titles when preference signals for less popular titles are weak.

The posterior mean of utility Uij is s ij = µj + w(αj) ηij, where w(αj) is the shrinkage weight as in s 2θ / (1 + s 2θ), j ∈ 5. p. 2
Where the paper says it

1 Setup We consider a representative user i whose true utility from consuming title j is given by: Uij = µj + θij , where µj ∈ R is baseline popularity and θij is drawn i. across (i, j) with bounded, strictly positive density on R and finite mean.

The analysis decomposes the effect of a marginal increase in δ into an inframarginal change in which title the user consumes and an extensive-margin change in whether the user consumes. The resulting average match quality is weakly increasing if and only if rinf(δ)E′(δ)ginf(δ)z true-utility gains from inframarginal switches per extensive-margin entry ginf(δ)z true utility gain per inframarginal switch ≥ U(δ) − uext(δ)z match-quality gap per extensive-margin entry. p. 4

Where the paper says it

Finally, it is important to understand whether, conditional on consumption, the improved recommendations lead users to better-matched titles. This is theoretically ambiguous in our model as there are two counteracting forces: inframarginal switching may increase average match quality if the improved recommendations shift a user’s consumption from S to their preferred title, but extensive-margin entry may decrease average match quality by inducing users who would otherwise consume the outside option to consume marginal titles.

The results confirm Prediction 3: view hours increased by 0.37%, the total number of titles played by 0. p. 9

Where the paper says it

Bars denote 95% confidence intervals computed via robust standard errors. Figure 4 presents the estimates of β from (2) across four different measures of engagement: total view hours, number of plays, the number of days with at least one play, and the number of distinct titles played.

62%, the distinct number of titles played by 1. p. 9
Where the paper says it

Bars denote 95% confidence intervals computed via robust standard errors. Figure 4 presents the estimates of β from (2) across four different measures of engagement: total view hours, number of plays, the number of days with at least one play, and the number of distinct titles played.

2%, and the number of days with at least one play by 0. p. 9
Where the paper says it

Bars denote 95% confidence intervals computed via robust standard errors. Figure 4 presents the estimates of β from (2) across four different measures of engagement: total view hours, number of plays, the number of days with at least one play, and the number of distinct titles played.

21%. The intuition for this increase in overall engagement is that the RecSys replaces less well-matched superstars with better matched middle-tail titles which results in users being more likely to find content to play from the recommender system. p. 9
Where the paper says it

Crucially, it implies that the shift away from superstars documented in Section 4 is not diversification for diversification’s sake, but actually coincides with improved engagement. The intuition from our conceptual model for why the improvement leads to an increase in overall engagement is simple: the RecSys replaces less well-matched superstars with better matched middle-tail titles which results in users being more likely to find content to play from the recommender system.

The analysis shows that d/dδ Pr(ci ∈ Ri) ≥ E′(δ) > 0, and we can conclude that Pr(ci ∈ Ri ci ≠ 0) is strictly increasing in δ. This strict increase is shown by comparing the derivatives of the two terms in (10), which weakly exceeds the bracketed term in (9).

The average match quality is strictly increasing when the inequality is strict, locally unchanged when it holds with equality, and decreasing when it is reversed. This final prediction establishes that at the current frontier, continued improvement lifts overall engagement and reallocates it toward the middle-tail. p. 2

Where the paper says it

(2026) show using a structural model that previous RecSys improvements lift engagement by better identifying users with high incremental consumption from recommendations, and that this value is largest for middle-tail titles. We provide experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall.

The paper provides experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This demonstrates that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data. p. 0

Where the paper says it

Improvements increase total consumption and users’ reliance on recommendations while diffusing recommendations and consumption away from the most popular titles (“superstars”) toward a larger number of moderately popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results challenge the notion that recommender systems polarize consumption – raising the consumption shares of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail products grow as algorithms improve and platforms scale.

The distribution shifts are measured by estimating specification (1) on recommendations users observe during the experiment for each of the title buckets. Figure 3c confirms that HHI falls by a statistically and economically significant 1. p. 7

Where the paper says it

We estimate the effects of the technology improvement on the Herfindahl–Hirschman Index (HHI = PN j=1 s 2 j ) – a standard measure for the degree of concentration that we compute at the title level – to test this. Figure 2c confirms that HHI has a statistically and economically significant decrease of 5.

2%, consistent with viewership dispersing from superstar titles to more personalized middle-tail titles. p. 8
Where the paper says it

8 Finally, we measure whether this corresponds to a reduction in play concentration: Figure 3c shows that HHI falls by a statistically and economically significant 1. 2%, consistent with viewership dispersing from superstar titles to more personalized middle-tail titles.

This dispersal of plays corresponds to a reduction in play concentration. p. 8
Where the paper says it

percentile in the control group. 8 Finally, we measure whether this corresponds to a reduction in play concentration: Figure 3c shows that HHI falls by a statistically and economically significant 1.

The analysis shows that the ratio of plays to visits increases by 0.47%, the share of plays originating from the recommendations increases by 0. p. 4

Where the paper says it

The main force behind this prediction is that some users who preferred M or L over S were previously recommended S due to the popularity bias, which led them to consume nothing; as the RecSys shifts to their preferred title, they consume it and the outside-option share falls. Because this newly realized consumption comes from the recommended title, the share of consumption originating from recommendations should rise as well: Prediction 4 (Share of Plays from Recommendation).

5%, and MRR increases by 0. p. 9
Where the paper says it

, my list). We then estimate (2) for the ratio of plays to visits, the share of plays that originate from recommendations, and the mean reciprocal rank (MRR) of plays.

1% – all consistent with Prediction 4. p. 1
Where the paper says it

As recommendation technology matures, then, the plausible set of targetable titles continues to shift toward the middle-tail. We argue that these results are consistent with a stylized model in which algorithmic improvements allow the RecSys to better infer consumer preferences from a fixed amount of title-level interaction data.

This confirms that improvements in δ raise the probability of recommendation-driven consumption faster than they raise overall engagement. p. 1
Where the paper says it

Improvements therefore shift recommendations toward these better-matched titles, raising overall engagement. This paper contributes to a long-standing literature on how digitization has reshaped the types of goods that get consumed (Goldfarb and Tucker, 2019), and in particular to conflicting findings on whether recommendation systems concentrate or disperse consumption.

The analysis shows that users complete 1.3% more titles at each of the X = 15, 30, 45-minute thresholds and provide 0. p. 2

Where the paper says it

(2026) show using a structural model that previous RecSys improvements lift engagement by better identifying users with high incremental consumption from recommendations, and that this value is largest for middle-tail titles. We provide experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall.

57% more positive thumbs. p. 2
Where the paper says it

1 Setup We consider a representative user i whose true utility from consuming title j is given by: Uij = µj + θij , where µj ∈ R is baseline popularity and θij is drawn i. across (i, j) with bounded, strictly positive density on R and finite mean.

The results indicate that users found more “high quality” titles to play and that the average title is of comparable quality. p. 4
Where the paper says it

Finally, it is important to understand whether, conditional on consumption, the improved recommendations lead users to better-matched titles. This is theoretically ambiguous in our model as there are two counteracting forces: inframarginal switching may increase average match quality if the improved recommendations shift a user’s consumption from S to their preferred title, but extensive-margin entry may decrease average match quality by inducing users who would otherwise consume the outside option to consume marginal titles.

The paper provides a simple Bayesian RecSys that microfounds the two key components of Assumption 1: the popularity default and the larger marginal gains for middle-tail titles. The structure motivates each component of Assumption 1: (i) Popularity default, (ii) qj monotonicity in δ, and (iii) Middle-tail dominance. p. 3

Where the paper says it

, 2024), by which deployed algorithms over-recommend popular titles because less popular titles generate too little data to extract a reliable signal. Appendix C motivates the assumption with a Bayesian recommender that generates this bias endogenously and shows the popularity bias and, over the relevant range of δ, that the weighted identification gain pj q ′ j (δ) is largest for the middle-tail: the superstar’s q ′ S (δ) is small, since it is already well targeted, and while the long-tail’s q ′ L (δ) may be sizeable, too few users rank it first for the identification gain to matter.

The popularity default persists at every finite δ – a user is still recommended S whenever the signal for their preferred title is insufficiently favorable – but this bias weakens as δ increases but always persists unless the technology is perfect. p. 4
Where the paper says it

An increase in δ increases total engagement: Pr(ci ̸= 0) is strictly increasing in δ. The main force behind this prediction is that some users who preferred M or L over S were previously recommended S due to the popularity bias, which led them to consume nothing; as the RecSys shifts to their preferred title, they consume it and the outside-option share falls.

The paper provides experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This suggests that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data. p. 0

Where the paper says it

Improvements increase total consumption and users’ reliance on recommendations while diffusing recommendations and consumption away from the most popular titles (“superstars”) toward a larger number of moderately popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results challenge the notion that recommender systems polarize consumption – raising the consumption shares of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail products grow as algorithms improve and platforms scale.

p. 0
Where the paper says it

Improvements increase total consumption and users’ reliance on recommendations while diffusing recommendations and consumption away from the most popular titles (“superstars”) toward a larger number of moderately popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results challenge the notion that recommender systems polarize consumption – raising the consumption shares of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail products grow as algorithms improve and platforms scale.

p. 0
Where the paper says it

Improvements increase total consumption and users’ reliance on recommendations while diffusing recommendations and consumption away from the most popular titles (“superstars”) toward a larger number of moderately popular titles (“middle-tail”), with minimal effects on the most niche titles (“long-tail”). Our results challenge the notion that recommender systems polarize consumption – raising the consumption shares of the head and tail at the expense of the middle – and suggest that the returns to investing in middle-tail products grow as algorithms improve and platforms scale.

The paper is a long-term holdback experiment comparing a frozen algorithm with an evolving production treatment incorporating all algorithmic innovations deployed during a 60-day experimental period. This experiment represents an ideal testing ground for the model for several reasons, including the inclusion of all innovations made to personalization algorithms during the experiment timeframe. p. 6

Where the paper says it

This experiment represents an ideal testing ground for our model for several reasons. First, the treatment includes all innovations made to personalization algorithms during the experiment timeframe.

The paper provides a stylized conceptual model where user utility is Uij = µj + θij, and choice utility is Vij = Uij + b · 1(j ∈ Ri), capturing the role of recommendations in reducing informational frictions. The platform’s recommendation technology is indexed by a single parameter δ, with Assumption 1 specifying the popularity default and middle-tail dominance. p. 3

Where the paper says it

The platform’s recommendation technology is indexed by a single parameter δ and we make the following assumptions about it. Assumption 1 (Recommendation Technology).

The paper contributes to a long-standing literature on how digitization has reshaped the types of goods that get consumed (Goldfarb and Tucker, 2019). The authors identify a second, dynamic dimension – algorithm maturity and the volume of title-level interaction data – that changes which titles benefit as recommendation quality improves. p. 2

Where the paper says it

Tucker and Zhang (2011) show that popularity information reinforces superstar concentration under homogeneous preferences but can benefit niche products when preferences are heterogeneous. Our contribution is to identify a second, dynamic dimension – algorithm maturity and the volume of title-level interaction data – that changes which titles benefit as recommendation quality improves.

This redefines the debate over whether recommendation systems concentrate or disperse consumption as neither fixed nor universal but evolves with algorithmic maturity. p. 1
Where the paper says it

Improvements therefore shift recommendations toward these better-matched titles, raising overall engagement. This paper contributes to a long-standing literature on how digitization has reshaped the types of goods that get consumed (Goldfarb and Tucker, 2019), and in particular to conflicting findings on whether recommendation systems concentrate or disperse consumption.

The authors provide experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This suggests that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data.

The paper is an empirical study on Netflix’s RecSys involving more than 8.5 million subscribers. p. 1

Where the paper says it

This paper studies how improvements to recommendations reshape viewership using a unique experiment on Netflix’s RecSys involving more than 8. 5 million subscribers.

The experiment was designed as a “cumulative holdback” test to measure the collective impact of multiple innovations to personalization algorithms during this period. p. 5
Where the paper says it

To evaluate the empirical implications of our model, we analyze an experiment on Netflix on 8,559,252 subscribers over a 60-day period from February to April 2025. This experiment was designed as a “cumulative holdback” test to measure the collective impact of multiple innovations to personalization algorithms during this period.

This provides robust evidence that the shift away from superstars documented in Section 4 is not diversification for diversification’s sake, but actually coincides with improved engagement. p. 9
Where the paper says it

21% increase in days with a play – represents about 5% of this magnitude. Crucially, it implies that the shift away from superstars documented in Section 4 is not diversification for diversification’s sake, but actually coincides with improved engagement.

The authors provide experimental evidence that continued improvements to the RecSys further shift consumption toward precisely this part of the distribution: away from superstars and toward middle-tail titles, while lifting engagement overall. This suggests that the returns of middle-tail products to platforms depend on algorithmic maturity and the ability of the RecSys to exploit limited data.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Recommendation system recalibration to target middle-tail products: The improved system can shift recommendations away from superstars toward moderately popular titles, as this is where recommendation quality improvements yield the largest gains for user engagement and consumption.

  2. Dynamic targeting based on algorithm maturity: The AI system should continuously monitor its own algorithmic maturity (indexed by delta) to determine which product distribution segment—long-tail, middle-tail, or superstar—is most effectively targetable for increased return on investment.

  3. Enhanced estimation of consumer preference signals: By incorporating the mechanism where improvements in recommendation technology attenuate this bias by allowing the RecSys to identify relevant titles with less data, the AI can better infer consumer preferences from limited interaction data, especially for less popular titles.

  4. Increased engagement through precision gains: The system should prioritize recommendations that result in a strictly increasing probability of consumption originating from recommendations, as an increase in delta results in a larger share of consumption originating from recommendations.

  5. Adaptive match quality monitoring: The AI can assess the average match quality by tracking whether the shift toward middle-tail titles increases the likelihood that users find more 'high quality' titles to play, which is measured by metrics like positive thumbs and watch duration shares.

Abstract

We study an experiment with 8.5 million users on Netflix's recommender system to measure how improvements in recommendation technology affect the set of products that get consumed. Improvements increase total consumption and users' reliance on recommendations while diffusing recommendations and consumption away from the most popular titles ("superstars") toward a larger number of moderately popular titles ("middle-tail"), with minimal effects on the most niche titles ("long-tail"). Our results challenge the notion that recommender systems polarize consumption -- raising the consumption shares of the head and tail at the expense of the middle -- and suggest that the returns to investing in middle-tail products grow as algorithms improve and platforms scale.

Related papers