When Does a Recommendation Spark Curiosity? Theory, Measurement, and Field Evidence
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "When Does a Recommendation Spark Curiosity? Theory, Measurement, and Field Evidence".
Warren: Here's the gist of the paper. Consumers on digital platforms are driven not only by what they like, but also by what they are curious about Abstract>.
Ruth: This paper formalizes the information-gap theory of curiosity into a computational measure and demonstrates that incorporating this measure into recommender systems significantly outperforms state-of-the-art methods in both offline and online experiments Key words>.
Warren: Formalizing the Curiosity Measure The authors formalize the information-gap theory of curiosity by proposing five axioms over three components of the consumer’s knowledge state: (a) the information gap between the consumer and the item, (b) surrounding context, and (c) reference gap Li and Tuzhilin: When Does a Recommendation Spark Curiosity?
Ruth: Working paper 3 The information-gap theory of curiosity provides no formal quantitative model that a recommender system can operate on, and existing proxy metrics (e.g., unexpectedness, variety-seeking, and Wundt-curve arousal) do not accurately measure the curiosity that a recommended item (e.g., a product, movie, or video) evokes Page 2>.
Paper summary: Warren: To wrap up this paper, Pan Li and Alexander Tuzhilin have shown that by formalizing the information-gap theory of curiosity into five axioms and shape conditions, they can construct a canonical measure for it >
Iris: And they validated this measure through surveys to confirm it matches consumer self-reported curiosity and then built a recommender system using that measurement >
Ruth: The main implication is that incorporating this measurable curiosity objective into recommendation systems significantly improves ranking performance across three different real-world datasets and outperforms existing methods like unexpectedness or variety-seeking >
Warren: And the online experiment showed tangible increases in click-through rate, video views, and time spent over the current production system when using this curiosity-aware model >
Iris: So to summarize, they've established that curiosity is a valid measure of consumer drive and that operationalizing it into recommendations raises consumer engagement across these different scenarios >
Conclusion: Ruth: So we've looked at how these authors took the information gap theory of curiosity and turned it into a real computational measure for recommender systems.
Warren: It’s interesting that they built this out of five specific axioms about what makes someone curious—like the gap between what they know and what you’re showing them.
Iris: And they showed that this isn't just some abstract idea; it has a concrete mathematical form, a specific function shape, which is important for making it usable in a computer.
Ruth: Exactly. The core idea is that curiosity isn't random; it follows these rules about how the item’s information relates to what the consumer already knows and what’s around them.
Warren: And they proved that this measure actually works by testing it against real consumer surveys, matching what people *say* they are curious about.
Iris: That validation is key because it means we aren't just guessing; we have a way to quantify this drive using those gap levels, the information level of the item, and even how recently the consumer interacted with things.
Ruth: So what does this mean for platforms? It suggests that instead of just recommending what’s popular or what’s similar, they can target exactly where a person's knowledge is missing in a way that sparks genuine interest.
Warren: If you can measure curiosity directly from consumption records, you can add it as an objective function to your recommendation system without needing to completely redesign the whole thing.
Iris: That could be powerful because it lets us tune the system specifically for what drives engagement, not just whatever metrics are easiest to optimize in a standard way.
Ruth: But there are still some caveats we need to keep in mind about how this measure behaves when people change their minds or when we scale it up across millions of users.
Warren: True. We saw some robustness in the offline tests, but how does this model hold up when you try to apply it to something completely different, like music recommendations instead of movies?
Iris: That’s exactly what we need to explore next. The framework isn't just for one domain; it might be useful wherever a system needs to decide what information to show someone who already knows something related.
Pan Li, Alexander Tuzhilin
Scheller College of Business Georgia Tech · Stern School of Business New York University
econ.GN, cs.IR, cs.LG, q-fin.EC
Submitted: 2026-11-11
Updated: 2026-11-11
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 97/100
The gist: Consumers on digital platforms are driven not only by what they like, but also by what they are curious about Abstract>.
Key concepts
- Information Gap Theory of Curiosity
- This theory posits that curiosity arises from the difference between what a consumer knows about an item and what they currently know. The paper formalizes this idea into mathematical axioms to create a measurable quantity.
- Canonical Curiosity Measure
- This is the specific mathematical formula derived from the five axioms, taking the form F(g, f, τ) = f · exp − (g − τ(1 − f))² / 2σ². It quantifies how much curiosity a recommended item evokes based on three key components: information gap (g), surrounding context (f), and reference gap (τ).
- Information Gap
- This component measures the difference between the information a consumer has about an item and their current knowledge state. A larger, perceived gap between what is known and what is missing drives higher levels of curiosity.
- Surrounding Context
- This refers to the context surrounding an item, such as other items or information nearby. The theory suggests that the optimal information gap for sparking curiosity changes depending on this context; growing context can alter where the most curious gap lies.
Terminology
Summary
Consumers on digital platforms are driven not only by what they like, but also by what they are curious about Abstract>. p. 0 edu
Abstract. Consumers on digital platforms are driven not only by what they like, but also by what they are curious about. We subsequently build a curiosity-aware recommender system that incorporates this measure into the
utility function design, and we show through offline experiments on three real-world datasets that it significantly outperforms the
state-of-the-art methods. Furthermore, we conduct a large-scale online controlled experiment at a major video-streaming platform
in Asia, where we show that our method significantly increases click-through rate, video views, and time spent over the latest
production system, leading to tangible economic impact for the company.Where the paper says it
Where the paper says it
Formalizing the Curiosity Measure
The authors formalize the information-gap theory of curiosity by proposing five axioms over three components of the consumer’s knowledge state: (a) the information gap between the consumer and the item, (b) surrounding context, and (c) reference gap Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 3 The information-gap theory of curiosity provides no formal quantitative model that a recommender system can operate on, and existing proxy metrics (e. p. 0 Despite its importance,
curiosity is hardly incorporated into modern recommender systems for lack of a formal measure. The information-gap theory
of curiosity provides no formal quantitative model that a recommender system can operate on, and existing proxy metrics (e. The information-gap theory
of curiosity provides no formal quantitative model that a recommender system can operate on, and existing proxy metrics (e. ,
unexpectedness, variety-seeking, and Wundt-curve arousal) do not accurately measure the curiosity that a recommended item (e. ,
unexpectedness, variety-seeking, and Wundt-curve arousal) do not accurately measure the curiosity that a recommended item (e. , a
product, movie, or video) evokes. , a
product, movie, or video) evokes. To fill this research gap, we formalize the information-gap theory into five axioms of curiosity and
construct the canonical formulation of a computational measure of the curiosity that a recommended item evokes in a consumer.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The five axioms define the structure of this measure, including Axiom 1 (Gap Relativity), which states that curiosity depends only on the gap-relative quantities (g, f, τ) Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 9 > Axiom 2 (Perceivability) asserts that curiosity is zero with no surrounding context or at an extremely large gap Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 8 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 9
AXIOM 2 (Perceivability). For all 𝜏 > 0 and 0 < 𝑓 < 𝑓 ′ < 1, 𝑔
∗
( 𝑓
′
, 𝜏) <
𝑔
∗
( 𝑓 , 𝜏). This axiom states that, at a fixed reference gap 𝜏, the information gap that evokes the most
curiosity becomes smaller as the surrounding context grows. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al. Survey, Section 5. 2(f)
#5 Recency Adaptivity The reference gap is a recency-weighted average
of encountered gaps.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The canonical curiosity measure is characterized by Theorem 1, which states that if a function satisfies Axioms 1–5 and the shape conditions (S1)–(S4), then it must take the form F(g, f, τ) = f · exp − (g − τ(1 − f))2 / 2σ2, where σ2 determines how quickly curiosity falls as the gap gets farther from the peak Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 39 >
Validation and Empirical Testing
The authors validated their measure through a survey-based study where it matched consumers’ self-reported curiosity, and they showed that the measure outperformed alternative proxy metrics Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 5 > The canonical measure reaches a Pearson correlation of r = 0. p. 2 Working paper 3
between the consumer and the item, (b) the surrounding context that the consumer has accumulated
around the item, and (c) the reference gap that the consumer has adapted to over time. We prove
that these five axioms, together with additional shape conditions, determine a canonical functional
form of curiosity. AXIOM 5 (Recency Adaptivity). 𝜏𝑢𝑡 is a recency-weighted average of the encountered information gap levels:
𝜏𝑢𝑡 =
∑︁
( 𝑗,𝑡′
)∈𝐻𝑢𝑡
𝜔𝑗𝑡′ 𝑔𝑢 𝑗𝑡′,
where 𝜔𝑗𝑡′ ≥ 0 and Í
( 𝑗,𝑡′
) 𝜔𝑗𝑡′ = 1, with weights non-increasing and non-constant in elapsed time. However, the information-gap theory (Loewenstein 1994) provides no formal quantitative model
that a recommender system can operate on. It is worth noting that a subsequent model formalizes
preferences regarding the presence and absence of information gaps (Golman and Loewenstein
2018), while it does not provide a measure of the curiosity that an item evokes in a consumer. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 2 Working paper
Recommender systems, meanwhile, are a natural setting for using consumer curiosity since
many existing methods repeatedly recommend to consumers products similar to what they have
already consumed (Kaminskas and Bridge 2016), resulting in filter bubbles (Pariser 2011), reduced
sales diversity (Fleder and Hosanagar 2009), and consumer dissatisfaction (Kapoor et al. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 2 Working paper
Recommender systems, meanwhile, are a natural setting for using consumer curiosity since
many existing methods repeatedly recommend to consumers products similar to what they have
already consumed (Kaminskas and Bridge 2016), resulting in filter bubbles (Pariser 2011), reduced
sales diversity (Fleder and Hosanagar 2009), and consumer dissatisfaction (Kapoor et al. In our survey-based study in Section 5. 2, analysis (e), we
test the axiom by locating the curiosity peak separately in the low- and high-context scenarios, and
we confirm that the peak is located at a smaller gap under high context.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Performance Gains in Offline Experiments
Offline experiments on three real-world datasets (Yelp, MovieLens, and Alibaba-Youku) showed that the curiosity-aware recommender system significantly improves recommendation performance over state-of-the-art baseline methods across all evaluation metrics Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 6 > The curiosity objective also achieves the highest diversity and novelty on every dataset, and its advantage carries over to the non-sequential MF model in Panel (b) Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 5 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al.Where the paper says it
Impact of Online Experiment
A large-scale online controlled experiment at a major video-streaming platform in Asia showed that the curiosity-aware recommender system significantly increased click-through rate, video views, and time spent over the latest production system Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 27 > The curiosity-aware system raises all three outcomes over the production baseline, as click-through rises by 2. p. 2 Through offline experiments on three real-world
datasets from Yelp, MovieLens, and Alibaba-Youku, we demonstrate that our curiosity-aware recommender system significantly improves recommendation performance over state-of-the-art baseline methods across all the evaluation metrics, which capture the accuracy, diversity, and novelty of
the recommendations, and that the improvements hold across a wide range of experiment settings. Moreover, we conducted a large-scale online controlled experiment at a major video-streaming platform in Asia, where our curiosity-aware recommender system significantly increased click-through
rate, video views, and time spent over the latest production system. We subsequently build a curiosity-aware recommender system that incorporates this measure into the
utility function design, and we show through offline experiments on three real-world datasets that it significantly outperforms the
state-of-the-art methods. Furthermore, we conduct a large-scale online controlled experiment at a major video-streaming platform
in Asia, where we show that our method significantly increases click-through rate, video views, and time spent over the latest
production system, leading to tangible economic impact for the company. We subsequently build a curiosity-aware recommender system that incorporates this measure into the
utility function design, and we show through offline experiments on three real-world datasets that it significantly outperforms the
state-of-the-art methods. Furthermore, we conduct a large-scale online controlled experiment at a major video-streaming platform
in Asia, where we show that our method significantly increases click-through rate, video views, and time spent over the latest
production system, leading to tangible economic impact for the company. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Managerial Implications
The findings establish consumer curiosity as a measurable and actionable objective for recommender systems, leading to important managerial implications for digital platforms Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 8 > First, platforms can measure the curiosity that each candidate item evokes in each consumer directly from consumption records, so the curiosity objective can be added to a deployed system without redesigning it and with little additional serving cost Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 2 We subsequently build a curiosity-aware recommender system that incorporates our proposed
curiosity measure into the recommendation process, where we design a utility function that combines the curiosity objective with the relevance objective of the underlying recommendation model. Under our proposed framework, any standard relevance model can be augmented with the curiosity
objective without redesigning the deployed system. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 2 Working paper
Recommender systems, meanwhile, are a natural setting for using consumer curiosity since
many existing methods repeatedly recommend to consumers products similar to what they have
already consumed (Kaminskas and Bridge 2016), resulting in filter bubbles (Pariser 2011), reduced
sales diversity (Fleder and Hosanagar 2009), and consumer dissatisfaction (Kapoor et al.Where the paper says it
Where the paper says it
Where the paper says it
Robustness and Scalability
The offline gains of the curiosity objective are robust to alternative implementations of f, τ, and scalarization Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 24 > The training time per epoch scales linearly with the number of records, keeping the framework deployable at an industrial scale Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 5 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al.Where the paper says it
Future Research Directions
Future research can examine curiosity-aware recommendations on other platforms and in other applications, including music and e-commerce Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 31 > The framework can be applied beyond recommendations, wherever a system chooses what information to place before someone who already knows something adjacent to it, such as search, education, and news feeds Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 1 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 2 Working paper
Recommender systems, meanwhile, are a natural setting for using consumer curiosity since
many existing methods repeatedly recommend to consumers products similar to what they have
already consumed (Kaminskas and Bridge 2016), resulting in filter bubbles (Pariser 2011), reduced
sales diversity (Fleder and Hosanagar 2009), and consumer dissatisfaction (Kapoor et al.Where the paper says it
Conclusion
Curiosity is an important driver of consumer decisions Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 8 > In this paper, we formalized the information-gap theory of curiosity as a set of five axioms over the relation between an item’s information and a consumer’s knowledge state Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 5 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al. We prove
that these five axioms, together with additional shape conditions, determine a canonical functional
form of curiosity. Furthermore, we show how to operationalize each component of the proposed
framework within the context of recommender systems, and we validate the measure by comparing
it to consumers’ self-reported curiosity in a survey-based study. Under our proposed framework, any standard relevance model can be augmented with the curiosity
objective without redesigning the deployed system. Through offline experiments on three real-world
datasets from Yelp, MovieLens, and Alibaba-Youku, we demonstrate that our curiosity-aware recommender system significantly improves recommendation performance over state-of-the-art baseline methods across all the evaluation metrics, which capture the accuracy, diversity, and novelty of
the recommendations, and that the improvements hold across a wide range of experiment settings. We subsequently build a curiosity-aware recommender system that incorporates this measure into the
utility function design, and we show through offline experiments on three real-world datasets that it significantly outperforms the
state-of-the-art methods. Furthermore, we conduct a large-scale online controlled experiment at a major video-streaming platform
in Asia, where we show that our method significantly increases click-through rate, video views, and time spent over the latest
production system, leading to tangible economic impact for the company. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 5
unexpectedness, variety-seeking, and Wundt-curve arousal, which we present below together with
their differences from the concept of curiosity.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Appendix C Details
Appendix C presents the full questionnaire of the survey-based study, including background questions, movie recommendations, and questions about each recommendation Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 45 >
Appendix D Details
Appendix D describes the online implementation of the curiosity measure in the online controlled experiment, showing that it satisfies all five axioms Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 46 > The online surrounding context also keeps three of the four properties of Section 3. p. 6 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 7
Table 2 Notation
Symbol Meaning
𝑈, 𝐼, 𝑇 Sets of consumers, items, and discrete time periods
𝐻𝑢𝑡 Consumption history, the consumption events ( 𝑗, 𝑡′
) of consumer 𝑢 before time 𝑡
(𝐸, 𝑑) Representation space 𝐸 ⊆ R
𝑝 of dimension 𝑝 and its metric 𝑑
e𝑖
, s𝑢𝑡 Item vector and consumer state, with s𝑢𝑡 = Enc(𝐻𝑢𝑡 ) for a sequential encoder Enc(· )
𝑔𝑢𝑖𝑡 Information gap, 𝑑(e𝑖
,s𝑢𝑡 )
𝑓𝑢𝑖𝑡 Surrounding context, the acquired knowledge lying near item 𝑖
𝐾, Λ, 𝜌 Kernel, saturating map, and bandwidth that define 𝑓 (Section 3. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al.Where the paper says it
Where the paper says it
Appendix B Details
Appendix B provides proofs of the theoretical results in Section 3, including Lemma B.2 and Proposition B.3, which show that the curiosity-maximizing gap is well defined and that Axioms 1–4 lead to a location-family representation of the curiosity function Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 6 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 7
Table 2 Notation
Symbol Meaning
𝑈, 𝐼, 𝑇 Sets of consumers, items, and discrete time periods
𝐻𝑢𝑡 Consumption history, the consumption events ( 𝑗, 𝑡′
) of consumer 𝑢 before time 𝑡
(𝐸, 𝑑) Representation space 𝐸 ⊆ R
𝑝 of dimension 𝑝 and its metric 𝑑
e𝑖
, s𝑢𝑡 Item vector and consumer state, with s𝑢𝑡 = Enc(𝐻𝑢𝑡 ) for a sequential encoder Enc(· )
𝑔𝑢𝑖𝑡 Information gap, 𝑑(e𝑖
,s𝑢𝑡 )
𝑓𝑢𝑖𝑡 Surrounding context, the acquired knowledge lying near item 𝑖
𝐾, Λ, 𝜌 Kernel, saturating map, and bandwidth that define 𝑓 (Section 3. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al.Where the paper says it
Where the paper says it
Shape Conditions
The four shape conditions (S1)–(S4) are necessary to select the most parsimonious member of the family of curiosity functions that satisfy the five axioms Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 42 > These conditions describe how the three components combine, and each is a natural choice for a curiosity measure Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 1 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 2 Working paper
Recommender systems, meanwhile, are a natural setting for using consumer curiosity since
many existing methods repeatedly recommend to consumers products similar to what they have
already consumed (Kaminskas and Bridge 2016), resulting in filter bubbles (Pariser 2011), reduced
sales diversity (Fleder and Hosanagar 2009), and consumer dissatisfaction (Kapoor et al.Where the paper says it
Prediction Invariance
The qualitative predictions of Section 3.3 hold for every curiosity function that satisfies the five axioms, so they do not depend on the choice of the canonical form Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 3 It is worth noting that a subsequent model formalizes
preferences regarding the presence and absence of information gaps (Golman and Loewenstein
2018), while it does not provide a measure of the curiosity that an item evokes in a consumer. In
this paper, we formalize the structural claims of the theory in the form of a set of axioms and derive
the canonical form of the curiosity measure that satisfies them. Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 6 Working paper
Table 1 Systematic Comparison of Curiosity and Related Concepts
Concept What it measures Level Reference point Response shape
Novelty Whether the item is new or unfamiliar to the
consumer
Consumer–item Consumption history Monotone increasing in distance
Diversity Dissimilarity among the items within one
recommendation list
Item list The list itself Increasing in pairwise distance
Serendipity Pleasant surprise: unexpectedness combined with relevance
Consumer–item Expected set Monotone in deviation, gated by
relevance
Unexpectedness Distance of the item from the consumer’s
expected set
Consumer–item Expected set Inverted-U with a peak fixed per
consumer
Varietyseeking
Stable propensity to seek different content
that scales unexpectedness
Consumer (trait) Consumption history Monotone, scaled by the trait
Wundt-curve
arousal
Arousal response to surprise with one optimal stimulus level
Consumer–item Surprise stimulus axis Inverted-U with a fixed peak
Curiosity Curiosity evoked by an information gap that
the consumer can perceive
Consumer–item–time
(state)
Reference gap that adapts
to recent consumption
Inverted-U whose peak shifts
with the surrounding context and
the reference gap
cannot yet predict, formalized as the agent’s prediction error (Pathak et al.Where the paper says it
Where the paper says it
Final Conclusion
The three forms of validation establish that C is a valid measure of curiosity and that incorporating it into recommendations raises consumer engagement Li and Tuzhilin: When Does a Recommendation Spark Curiosity? Working paper 8 > The platform has since deployed the system in production Li and Tuzhilin: When Does a Recommendation Spark Curiosity? p. 3 Li and Tuzhilin: When Does a Recommendation Spark Curiosity? 4 Working paper
systems, which is validated against consumers’ self-reported curiosity and deployed in production.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Incorporate a curiosity objective into recommendation utility functions by designing a utility function Vui = (1 - alpha)rhat + alpha wutCi, allowing platforms to pursue both relevance and curiosity simultaneously. This enables systems to select items that match consumer preferences while also evoking consumer curiosity, balancing exploration and exploitation based on the platform policy parameter alpha.
-
Implement a formal computational measure of curiosity C(u, i, t) based on the information-gap theory axioms (Gap Relativity through Axiom 1), which quantifies what an item evokes by measuring the distance between the consumer's current knowledge state suti and item vector ei. This measure is derived from three components: information gap guti, surrounding context futi, and reference gap tuti.
-
Design item representation using a low-dimensional latent vector learned by an AutoEncoder or Item2vec model to define the item vector ei in a shared metric space E. The consumer state suti should be updated using a GRU to aggregate consumption history, allowing the information gap guti = d(ei, suti) to be computed directly between an item and a consumer.
-
Augment the curiosity measure with surrounding context futi, calculated as the kernel sum of consumed items near item i mapped to [0, 1]. This component ensures that a gap is only perceived when the consumer already knows something related to the item, satisfying Axiom 2 (Perceivability).
-
Incorporate a reference gap tuti into the measure by defining it as an exponentially weighted average of encountered information gaps, ensuring recently consumed items count more, which satisfies Axiom 5 (Recency Adaptivity). This allows the system to adapt its exploration focus based on the consumer's most recent knowledge.
-
Optimize recommendation selection using a utility function Vui = (1 - alpha)rhat + alpha wutCi over candidate set Cut, where wuti is estimated as the coefficient on Cuti in a regression of engagement before time t on Cuti that holds predicted relevance fixed. This allows the system to dynamically weight curiosity based on individual consumer responsiveness.
-
Deploy a curiosity-aware recommendation system in production by calculating Cuti using existing item vectors and consumer states, requiring only updates to tuti and futi, thus adding little serving cost while significantly improving click-through rate, video views, and time spent compared to the latest production system.
Abstract
Curiosity, the drive to close the gap between what one knows and what one wants to know, pulls consumers beyond their familiar content, sustains engagement, and breaks them out of the filter bubbles of repetitive, relevance-only recommendations. Despite its importance, curiosity is hardly incorporated into modern recommender systems for lack of a formal measure. The information-gap theory of curiosity provides no formal quantitative model that a recommender system can operate on, and existing proxy metrics do not accurately measure the curiosity that a recommended item evokes. To fill this research gap, we formalize the information-gap theory into five axioms of curiosity and construct the canonical formulation of a computational measure of the curiosity that a recommended item evokes in a consumer. We validate our curiosity measure through a survey-based study, in which it matches consumers' self-reported curiosity and outperforms alternative proxy metrics. We subsequently build a curiosity-aware recommender system that incorporates this measure into the utility function design, and we show through offline experiments on three real-world datasets that it significantly outperforms the state-of-the-art methods. Furthermore, we conduct a large-scale online controlled experiment at a major video-streaming platform in Asia, where we show that our method significantly increases click-through rate, video views, and time spent over the latest production system, leading to tangible economic impact for the company. Our method has since been deployed on the platform and remains in operation. Our findings establish consumer curiosity as a measurable and actionable objective for recommender systems, with important managerial implications for digital platforms to recommend what consumers wonder about.
Sources
Related papers
- Huge If True: Learning from the Size of the Surprise
- The Rebundling of Freelance Work in the AI Age
- Dynamic investment and coordination in teams: Theory and experimental evidence
- Does Frontier AI Strengthen Cyber Defense? Evidence from Project Glasswing
- Generative AI at Work
- What is Work For? Labor, Politics, and Meaning in the Age of AI