When Does a Recommendation Spark Curiosity? Theory, Measurement, and Field Evidence
summary
The gist
Consumers on digital platforms are driven not only by what they like, but also by what they are curious about Abstract>.
In short
The paper formalizes consumer curiosity using an information-gap theory, creating a computational measure based on five axioms. This measure outperforms existing proxy metrics and significantly improves recommendation performance in offline experiments across multiple datasets and online tests, showing it boosts user engagement.
Key concepts
- Information Gap Theory of Curiosity
- This theory posits that curiosity arises from the difference between what a consumer knows about an item and what they currently know. The paper formalizes this idea into mathematical axioms to create a measurable quantity.
- Canonical Curiosity Measure
- This is the specific mathematical formula derived from the five axioms, taking the form F(g, f, τ) = f · exp − (g − τ(1 − f))² / 2σ². It quantifies how much curiosity a recommended item evokes based on three key components: information gap (g), surrounding context (f), and reference gap (τ).
- Information Gap
- This component measures the difference between the information a consumer has about an item and their current knowledge state. A larger, perceived gap between what is known and what is missing drives higher levels of curiosity.
- Surrounding Context
- This refers to the context surrounding an item, such as other items or information nearby. The theory suggests that the optimal information gap for sparking curiosity changes depending on this context; growing context can alter where the most curious gap lies.
Terminology used across episodes
This episode discusses
- When Does a Recommendation Spark Curiosity? Theory, Measurement, and Field Evidence · Paper Radio
- Session-based Recommendations with Recurrent Neural Networks
The paper
When Does a Recommendation Spark Curiosity? Theory, Measurement, and Field Evidence · Read on arXiv
Pan Li, Alexander Tuzhilin
Scheller College of Business Georgia Tech · Stern School of Business New York University
Curiosity, the drive to close the gap between what one knows and what one wants to know, pulls consumers beyond their familiar content, sustains engagement, and breaks them out of the filter bubbles of repetitive, relevance-only recommendations. Despite its importance, curiosity is hardly incorporated into modern recommender systems for lack of a formal measure. The information-gap theory of curiosity provides no formal quantitative model that a recommender system can operate on, and existing proxy metrics do not accurately measure the curiosity that a recommended item evokes. To fill this research gap, we formalize the information-gap theory into five axioms of curiosity and construct the canonical formulation of a computational measure of the curiosity that a recommended item evokes in a consumer. We validate our curiosity measure through a survey-based study, in which it matches consumers' self-reported curiosity and outperforms alternative proxy metrics. We subsequently build a curiosity-aware recommender system that incorporates this measure into the utility function design, and we show through offline experiments on three real-world datasets that it significantly outperforms the state-of-the-art methods. Furthermore, we conduct a large-scale online controlled experiment at a major video-streaming platform in Asia, where we show that our method significantly increases click-through rate, video views, and time spent over the latest production system, leading to tangible economic impact for the company. Our method has since been deployed on the platform and remains in operation. Our findings establish consumer curiosity as a measurable and actionable objective for recommender systems, with important managerial implications for digital platforms to recommend what consumers wonder about.
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "When Does a Recommendation Spark Curiosity? Theory, Measurement, and Field Evidence".
Warren: Here's the gist of the paper. Consumers on digital platforms are driven not only by what they like, but also by what they are curious about Abstract>.
Ruth: This paper formalizes the information-gap theory of curiosity into a computational measure and demonstrates that incorporating this measure into recommender systems significantly outperforms state-of-the-art methods in both offline and online experiments Key words>.
Warren: Formalizing the Curiosity Measure The authors formalize the information-gap theory of curiosity by proposing five axioms over three components of the consumer’s knowledge state: (a) the information gap between the consumer and the item, (b) surrounding context, and (c) reference gap Li and Tuzhilin: When Does a Recommendation Spark Curiosity?
Ruth: Working paper 3 The information-gap theory of curiosity provides no formal quantitative model that a recommender system can operate on, and existing proxy metrics (e.g., unexpectedness, variety-seeking, and Wundt-curve arousal) do not accurately measure the curiosity that a recommended item (e.g., a product, movie, or video) evokes Page 2>.
Paper summary: Warren: To wrap up this paper, Pan Li and Alexander Tuzhilin have shown that by formalizing the information-gap theory of curiosity into five axioms and shape conditions, they can construct a canonical measure for it >
Iris: And they validated this measure through surveys to confirm it matches consumer self-reported curiosity and then built a recommender system using that measurement >
Ruth: The main implication is that incorporating this measurable curiosity objective into recommendation systems significantly improves ranking performance across three different real-world datasets and outperforms existing methods like unexpectedness or variety-seeking >
Warren: And the online experiment showed tangible increases in click-through rate, video views, and time spent over the current production system when using this curiosity-aware model >
Iris: So to summarize, they've established that curiosity is a valid measure of consumer drive and that operationalizing it into recommendations raises consumer engagement across these different scenarios >
Conclusion: Ruth: So we've looked at how these authors took the information gap theory of curiosity and turned it into a real computational measure for recommender systems.
Warren: It’s interesting that they built this out of five specific axioms about what makes someone curious—like the gap between what they know and what you’re showing them.
Iris: And they showed that this isn't just some abstract idea; it has a concrete mathematical form, a specific function shape, which is important for making it usable in a computer.
Ruth: Exactly. The core idea is that curiosity isn't random; it follows these rules about how the item’s information relates to what the consumer already knows and what’s around them.
Warren: And they proved that this measure actually works by testing it against real consumer surveys, matching what people *say* they are curious about.
Iris: That validation is key because it means we aren't just guessing; we have a way to quantify this drive using those gap levels, the information level of the item, and even how recently the consumer interacted with things.
Ruth: So what does this mean for platforms? It suggests that instead of just recommending what’s popular or what’s similar, they can target exactly where a person's knowledge is missing in a way that sparks genuine interest.
Warren: If you can measure curiosity directly from consumption records, you can add it as an objective function to your recommendation system without needing to completely redesign the whole thing.
Iris: That could be powerful because it lets us tune the system specifically for what drives engagement, not just whatever metrics are easiest to optimize in a standard way.
Ruth: But there are still some caveats we need to keep in mind about how this measure behaves when people change their minds or when we scale it up across millions of users.
Warren: True. We saw some robustness in the offline tests, but how does this model hold up when you try to apply it to something completely different, like music recommendations instead of movies?
Iris: That’s exactly what we need to explore next. The framework isn't just for one domain; it might be useful wherever a system needs to decide what information to show someone who already knows something related.
More episodes
- Daily Summary for 2026-11-19
- 2608.21274-Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix
- 2408.02757-A nonparametric test for diurnal variation in spot correlation processes
- 2306.16393-High-Dimensional Canonical Correlation Analysis
- 2605.17299-Geometric Brownian motion with intermittent entries and exits
- 2607.04753-Fooling Yourself: how narratives shape beliefs
- 2408.14671-Inference on Treatment Effects with Repeatedly Mismeasured High-Dimensional Controls
- 2201.06898-Difference-in-Differences Estimators for Treatments Continuously Distributed at Every Period
- 2501.15307-Influence Functions: A Unified Differentiation-Based Framework
- The day in conversation for 2026-11-19