Calibrated Forecasting and Persuasion
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: We're looking at the paper: "Calibrated Forecasting and Persuasion".
Warren: Here's the gist of the paper. We study a dynamic game where an expert sends probabilistic forecasts to a decision-maker and we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem.
Ruth: The Dynamic Game Setup The paper introduces a dynamic game between a sender (expert) and a receiver (decision-maker) where the state of the world evolves over time.
Warren: In each period, the sender provides a probabilistic forecast about that period’s unknown state to the receiver, who then chooses an action.
Ruth: The sender is assumed to be informed and knows the distribution of states, while the receiver is uninformed.
Title and authors: Ruth: So we're looking at this paper by Atulya Jain and Vianney Perchet called "Calibrated Forecasting and Persuasion". The core idea is a dynamic game between an expert sending probabilistic forecasts and a decision-maker who tests those forecasts using a calibration test based on past data.
Warren: It sounds like we're trying to figure out how an expert should forecast over time to maximize her payoff while passing this credibility check, right? What does the paper actually do with this setup?
Iris: They reduce the dynamic game down to a static persuasion problem by showing that the distributions of forecasts that can pass calibration tests are exactly what we call mean-preserving contractions of the distribution of conditionals.
Ruth: Mean-preserving contractions, that's a heavy term. So, instead of trying to figure out the whole path over time, they simplify it to finding a good static policy based on those conditional distributions. But they also compare what an informed expert can achieve versus what an uninformed one can do.
Warren: And that comparison is key for us traders, because we're always dealing with imperfect information about future states. Does this mean the paper gives us a way to quantify the value of knowing the true underlying process?
Iris: It does, by characterizing the attainable payoff for both experts. For an informed expert, they find that calibrated strategies induce mean-preserving contractions of the distribution of conditionals and her maximum payoff is tied to Per(Cµ, uˆS) where Cµ is the distribution of conditionals and uˆS is the indirect utility.
Ruth: So we have a benchmark for how much value information provides. But what about when the expert doesn't know the data-generating process at all? What are her limits there?
Warren: The paper characterizes her attainable payoff function for an uninformed sender by looking at the closed convex hull of her indirect utility, CoˆuS(f*)(p) in an adversarial setting, and if it’s an ergodic Markov chain, she can attain uˆS(f*)(p) where p is the long-run empirical distribution of states.
Iris: That's a bit different because for an uninformed sender, they look at the set of possible strategies rather than just one optimal path. It suggests that even without knowing the true process, you can still find a strategy that performs well based on what you observe over time.
Ruth: So it’s like saying if you don't know the rules of the game perfectly, you can still construct a strategy that gives you a decent return by playing optimally against the worst possible version of those rules. But then they bring in regret minimization as an alternative approach to look at how this plays out in practice.
Warren: Regret minimization is interesting because it suggests that if the receiver just reacts myopically best to calibrated forecasts, he incurs no regret in the long run. But wait, the paper also says you can do better than that against a mean-based receiver. That's what I want to hear for running a strategy.
Title and authors: Iris: That means if you use those calibrated forecasts, even against someone who just tries to follow the average outcome every period, you can still get a strictly better payoff than the benchmark Per(Cµ, uˆS). It exploits how mean-based algorithms adapt gradually over time.
Ruth: That’s a big implication for us in markets. It suggests that being calibrated isn't just about hitting a minimum bar; it's an active way to beat simpler learning strategies if you can structure your forecasts correctly. So, what are the practical applications they show?
Warren: They apply this to financial platforms where the optimal calibrated strategy achieves Per(Cµ, uˆS) when the state space support is affinely independent. In that case, it shows that even though we don't always forecast the absolute truth of the conditional probability, announcing accurate but coarser forecasts can be better than trying to be perfectly truthful with every single piece of information.
Iris: That makes sense for real-world data where perfect knowledge isn't available, so coarse forecasting is a practical trade-off. It also points toward the fact that calibration and regret minimization are closely related ideas in online learning literature.
Ruth: So to wrap up, the main point of this paper by Atulya Jain and Vianney Perchet is that calibration provides a solid micro-foundation for commitment assumptions in Bayesian persuasion by linking it to mean-preserving contractions. They show what an informed expert can guarantee, what an uninformed one can attain, and how you can beat simple learning strategies using regret minimization heuristics.
Warren: It seems like the ultimate message is that even when the expert is not perfectly informed or when facing a less sophisticated decision-maker, there's a way to structure your forecasting strategy—through calibration—to guarantee at least this benchmark and sometimes do better. That means calibration isn't just academic; it’s a tool for maximizing expected payoff under uncertainty.
Iris: I just want to emphasize that the model assumes the expert is informed when they are trying to pass any complex statistical test like calibration, which is an important assumption we should keep in mind when applying this to real markets.
Ruth: Exactly, because if the expert isn't informed about the process, her ability to pass that calibration test becomes much more constrained. The paper sets a high bar for what's possible under those assumptions.
Warren: So, moving forward, it seems like the next step is figuring out exactly how to implement this in a trading system where we have continuous data streams and need to make these dynamic adjustments based on the state evolution.
Iris: And maybe exploring extensions, like allowing the receiver’s actions to actually affect the state transitions in a Markov Decision Process. That would add another layer of complexity that's worth investigating after we digest this paper.
The paper's summary: Ruth: So we're looking at "Calibrated Forecasting and Persuasion," and what they're doing is taking this dynamic game—the expert sending forecasts over time—and turning it into a static persuasion problem.
Warren: They’re saying that the way you can pass the credibility test, which they call calibration, is mathematically equivalent to finding mean-preserving contractions of the distribution of conditionals.
Iris: So instead of tracking every single state evolution period by period, they boil it down to this static distribution relationship between what we expect and what we actually see.
Ruth: That’s right. The main idea is that calibration isn't just some arbitrary rule for experts; it’s a precise mathematical structure—the mean-preserving contraction—that describes the set of forecasts that are credible over time.
Warren: And they use this to set up the value of the game. For an informed expert, her maximum payoff is determined by Per(Cµ, uˆS).
Iris: That means she’s not just aiming for any high probability; she’s aiming for a specific utility measure derived from that distribution of conditionals and her indirect utility.
Ruth: It connects the dynamic prediction problem directly to a static one, which is a big simplification. But they also look at what happens when the expert doesn't know the underlying process perfectly.
Warren: For an uninformed sender, they characterize her possible payoffs using the closed convex hull of her indirect utility, CoˆuS(f*)(p). That’s her attainable payoff boundary in a worst-case scenario.
Iris: If it’s an ergodic Markov chain, she can actually reach the utility uˆS(f* (p)), where p is just the long-run average empirical distribution of states.
Ruth: So the paper shows that even without full knowledge, there are specific achievable payoffs based on how you structure your strategy against whatever process you observe.
Warren: And they bring in regret minimization as a practical tool here, suggesting that if the receiver just plays myopically best against calibrated forecasts, he incurs no regret in the long run.
Iris: But they also show you can get a strictly better payoff than that benchmark against a mean-based receiver by exploiting how those mean-based algorithms adapt over time.
Ruth: So what this means for us is that calibration is not just about meeting some minimum standard; it’s an active way to structure forecasts that can actually beat simpler learning strategies if you're willing to play them correctly.
Warren: The application in finance shows that when the state space support has a certain property, like being affinely independent, the optimal calibrated strategy ends up announcing accurate but coarser forecasts instead of trying to be perfectly truthful every single time.
Iris: That’s a practical trade-off for real-world data where perfect knowledge isn't available, so coarse forecasting can be better than constant high-frequency adjustments.
Ruth: It also points out that calibration and regret minimization are fundamentally related ideas in online learning literature, which is an important connection to keep in mind when designing systems that learn.
Warren: The paper sets a benchmark for what an informed system can guarantee, and it gives us the tools to see how to improve upon those benchmarks when we introduce heuristics like regret minimization.
Iris: But they do flag that they’re working under assumptions about the expert being informed when she's trying to pass calibration, so we have to be careful applying this directly if our model assumptions are wrong.
Ruth: Exactly. The next step is seeing how this framework handles more complex scenarios, like when the receiver’s actions actually change the future state transitions in a Markov Decision Process.
The paper's improvements: Tom: So we're looking at how they improve on that initial framework for "Calibrated Forecasting and Persuasion," focusing on how you can actually get better results than just sticking to the baseline benchmark.
Ruth: They suggest using regret minimization as a heuristic, which is pretty practical. The idea is if the receiver just plays myopically best against calibrated forecasts, he incurs no regret over time.
Warren: That’s interesting because we know that if you use those calibrated forecasts, even against someone who just tries to follow the average outcome every period, you can get a strictly better payoff than that Per(Cµ, uˆS) benchmark.
Iris: They show this happens by exploiting how mean-based algorithms adapt gradually over time; they exploit the cumulative payoffs from action up to period t.
Ruth: So it’s not just about being calibrated right now; it’s about structuring your forecasts so that you capitalize on the receiver's slow learning process.
Warren: That means if we can model that adaptation, we can design a strategy that outperforms the static benchmark in a dynamic setting.
Iris: They also explore extensions, like allowing the receiver’s actions to actually affect how the state transitions in a Markov Decision Process. That adds another layer of complexity to what they modeled.
Ruth: That would mean moving from a simple game where you just report probabilities, to something where your own forecasts influence the environment itself.
Warren: If that were feasible, the whole payoff structure would change because the expert’s signal isn't just being tested; it’s actively shaping the distribution of future states.
Iris: That would require a completely new way to look at identification, because you’d be tracking how your forecast affects the evolution of that distribution.
Ruth: It gets complicated fast, but the implication is that this research opens up ways to model more complex interactions in strategic environments.
Warren: So while they give us a solid baseline for informed experts and show how heuristics can beat simple learning, the real challenge now is modeling these dynamic feedback loops where actions matter.
Conclusion: Tom: So we're wrapping up on "Calibrated Forecasting and Persuasion," which basically shows how you can turn this dynamic expert forecasting game into a static persuasion problem based on mean-preserving contractions.
Ruth: It really boils down to calibration being a test of credibility, and they provide a solid mathematical foundation for what that credibility means in terms of forecasts.
Warren: For the quantitative side, it gives us the payoff bound Per(Cµ, uˆS) for informed experts and even characterizes the attainable set for uninformed ones as CoˆuS(f* (p)).
Iris: The identification part is solid because they link these dynamic games to static persuasion problems, which helps us isolate exactly what variation in the process is doing the work.
Ruth: It means we have a clear benchmark for performance—that Per value—and we know how much better you can get if you introduce regret minimization heuristics.
Warren: The implication is that even against a simple learner, if you structure your forecasts correctly, you can beat that benchmark by exploiting adaptation over time.
Iris: And they do flag the limitation that the expert being informed when passing calibration is a necessary assumption we should keep in mind for real-world applications.
Ruth: It’s a solid piece of work because it connects market incentives with rigorous statistical concepts like Markov chains and concentration inequalities.
Warren: So, while this paper gives us tools to structure better forecasts, the next logical step is seeing how these ideas play out when the receiver's actions actually change the future state transitions in a Markov Decision Process.
Iris: That’s where we should look next to see if this framework can handle those more interactive scenarios.
ATULYA JAIN, VIANNEY PERCHET
University of Bonn · ENSAE & Criteo AI Lab
econ.TH, cs.GT
Submitted: 2024-06-21
Updated: 2026-10-29
Comments: The conference version of this work has been accepted to the Twenty-Fifth ACM Conference on Economics and Computation (EC'24)
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 83/100
The gist: We study a dynamic game where an expert sends probabilistic forecasts to a decision-maker and we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion…
Terminology
Summary
We study a dynamic game where an expert sends probabilistic forecasts to a decision-maker and we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. p. 0 How should the expert send forecasts to maximize her payoff while passing
the test? For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem.Where the paper says it
The Dynamic Game Setup
The paper introduces a dynamic game between a sender (expert) and a receiver (decision-maker) where the state of the world evolves over time. In each period, the sender provides a probabilistic forecast about that period’s unknown state to the receiver, who then chooses an action. p. 5 ATULYA JAIN AND VIANNEY PERCHET
that period’s unknown state. The receiver observes the forecast and then chooses an
action. , ωt−1) the history of states up to period t. We assume the sender is informed
and knows the distribution µ, whereas the receiver is uninformed. The state and the
receiver’s action in that period determine the sender’s and receiver’s payoffs, denoted by
uS(ωt
, at) and uR(ωt
, at). A forecasting strategy is truthful if, in every period t, the sender
reports the conditional, that is, ft = pt
.Where the paper says it
Where the paper says it
Where the paper says it
Calibration as a Credibility Test
The decision-maker assesses credibility using a calibration test based on past data. This test checks whether the forecast of an outcome is close to the actual proportion of times the outcome occurred when the forecast was made. p. 0 The idea is to check whether the forecast of an outcome is close to the actual proportion of
times the outcome occurred when the forecast was made. For example, the investor checks
whether an asset outperformed others on 70% of the days when the analyst predicted a 0. ATULYA JAIN AND VIANNEY PERCHET
A T-sequence of forecasts is ϵT -calibrated if the frequency-weighted average distance
between forecasts and the empirical distributions of states does not exceed the error
margin ϵT . 6 We illustrate the calibration test using a rain forecasting example. ATULYA JAIN AND VIANNEY PERCHET
Fail: If the sequence of past forecasts is not ϵt−1-calibrated, the sender fails the calibration test in period t. In this case, she incurs a punishment cost −c, where c > 0.Where the paper says it
Where the paper says it
Where the paper says it
Reduction to Static Persuasion
The main result establishes that the sender’s maximum payoff in the dynamic forecasting game equals the value of a static persuasion problem. This equivalence shows that the calibration test provides a micro-foundation for the commitment assumption in Bayesian persuasion. p. 2 Thus, the sender’s dynamic problem reduces to a static persuasion problem. This equivalence shows that the calibration test provides a micro-foundation for the
commitment assumption in Bayesian persuasion. Our main result (Theorem 1) establishes that the sender’s maximum payoff in the
dynamic forecasting game equals the value of a static persuasion problem. The state,
signal, and prior in the persuasion problem correspond to the conditional probability,
the forecast, and the distribution of conditionals in the dynamic game.Where the paper says it
Where the paper says it
Informed Sender Results
For a stationary ergodic process, calibrated strategies induce precisely the mean-preserving contractions of the distribution of conditionals. The sender’s maximum payoff equals Per(Cµ, uˆS) where Cµ is the distribution of conditionals and uˆS is the indirect utility. p. 9 In the dynamic game, uˆS(f) equals
the sender’s expected payoff when the receiver best responds to belief f. The sender chooses a distribution Q ∈ M(P) to maximize her expected indirect utility. For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. The distributions of
forecasts that can arise under calibration are precisely the mean-preserving contractions
of the distribution of conditionals.Where the paper says it
Where the paper says it
Uninformed Sender Results
For an uninformed sender, the attainable payoff function is characterized by the closed convex hull of her indirect utility CoˆuS(f∗(p)) in the adversarial case. When the unknown process is an ergodic Markov chain, she can attain uˆS(f∗(p)) where p is the long-run empirical distribution of states. p. 2 Second, when the unknown process is an
ergodic Markov chain, she can do better. In particular, she can approximately attain
the payoff corresponding to the no-disclosure policy in the persuasion problem with prior
given by the distribution of conditionals—the same problem as in the informed sender
case (Proposition 3).Where the paper says it
Regret Minimization and Heuristics
The paper examines regret minimization as an alternative heuristic for decision-making. It shows that if the receiver myopically best responds to calibrated forecasts, he incurs no regret in the long run. p. 2 Regret minimization is closely linked to calibration
(see Perchet, 2014), and we build on this connection to obtain new results. We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). A common theme in the literature is
that the sender can obtain higher payoff than the rational benchmark when facing such
algorithms. Similarly, we show the sender can obtain a higher utility than the calibration
benchmark against a mean-based learner.Where the paper says it
Where the paper says it
Application to Financial Platforms
The application to a financial platform shows that the optimal calibrated strategy achieves Per(Cµ, uˆS) when the state space support is affinely independent. The optimal strategy announces accurate but coarser forecasts rather than the truthful conditionals. p. 2 We characterize the platform’s optimal forecasting strategy
using the concavification approach and illustrate it graphically. We show that it may be
optimal for the platform to send coarse forecasts rather than truthful ones.Where the paper says it
Conclusion and Future Work
The paper concludes that calibration and regret minimization are closely related notions in the online learning literature. Many problems remain open, such as characterizing the optimal calibrated strategy for any stochastic process. p. 7 Unlike the finite test, the asymptotic notion does not depend on the
chosen sequence of error margins. The truthful forecasting strategy is calibrated for any stochastic process [Dawid, 1982]. Given that she is
tested by a calibration test, how should the expert send forecasts to maximize her payoff? To do this, we develop a dynamic sender-receiver model where the state of the world
evolves according to a stochastic process.Where the paper says it
Where the paper says it
Appendix B Details
Appendix B provides an equivalent formulation of the persuasion problem in terms of Blackwell experiments rather than mean-preserving contractions. It shows that a probability distribution Q is a simple mean-preserving contraction if and only if E(Q) is a garbling of E(P). p. 9 Definition 3. A probability distribution Q = (µ, r) is a simple mean-preserving contraction of P = (λ, q), if there exists a row-stochastic matrix G ∈ R
n×m satisfying
(3. Definition 3. A probability distribution Q = (µ, r) is a simple mean-preserving contraction of P = (λ, q), if there exists a row-stochastic matrix G ∈ R
n×m satisfying
(3. Thus, the sender’s dynamic problem reduces to a static persuasion problem. This equivalence shows that the calibration test provides a micro-foundation for the
commitment assumption in Bayesian persuasion. Thus, the sender’s dynamic problem reduces to a static persuasion problem. This equivalence shows that the calibration test provides a micro-foundation for the
commitment assumption in Bayesian persuasion. How should the expert send forecasts to maximize her payoff while passing
the test? For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. How should the expert send forecasts to maximize her payoff while passing
the test? For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. All omitted proofs are in Appendix A. Appendix B contains a
reformulation of the persuasion problem in terms of Blackwell experiments. All omitted proofs are in Appendix A. Appendix B contains a
reformulation of the persuasion problem in terms of Blackwell experiments. For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. The distributions of
forecasts that can arise under calibration are precisely the mean-preserving contractions
of the distribution of conditionals. For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. The distributions of
forecasts that can arise under calibration are precisely the mean-preserving contractions
of the distribution of conditionals. 3) Per(P, uˆS) := max
Q∈M(P)
EQ[ˆuS]. Let Q∗ denote an optimal distribution of forecasts solving (3. 3) Per(P, uˆS) := max
Q∈M(P)
EQ[ˆuS]. Let Q∗ denote an optimal distribution of forecasts solving (3. First, in
an adversarial setting where nature can adapt its strategy, there is no fixed prior, and the
attainable payoff is evaluated as a function of the realized long-run empirical distribution
of states. In this case, the sender can approximately guarantee the minimal payoff in the
canonical persuasion problem, where the mean of the prior equals the realized empirical
distribution (Theorem 2). First, in
an adversarial setting where nature can adapt its strategy, there is no fixed prior, and the
attainable payoff is evaluated as a function of the realized long-run empirical distribution
of states. In this case, the sender can approximately guarantee the minimal payoff in the
canonical persuasion problem, where the mean of the prior equals the realized empirical
distribution (Theorem 2). We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). Conversely, when the informed sender faces a regret-minimizing
receiver, she can guarantee at least the calibration benchmark—namely, the persuasion
value—and, for a natural class of regret-minimizing algorithms, sometimes strictly more
(Proposition 5 and Theorem 3). We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). Conversely, when the informed sender faces a regret-minimizing
receiver, she can guarantee at least the calibration benchmark—namely, the persuasion
value—and, for a natural class of regret-minimizing algorithms, sometimes strictly more
(Proposition 5 and Theorem 3). We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). Conversely, when the informed sender faces a regret-minimizing
receiver, she can guarantee at least the calibration benchmark—namely, the persuasion
value—and, for a natural class of regret-minimizing algorithms, sometimes strictly more
(Proposition 5 and Theorem 3). We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). Conversely, when the informed sender faces a regret-minimizing
receiver, she can guarantee at least the calibration benchmark—namely, the persuasion
value—and, for a natural class of regret-minimizing algorithms, sometimes strictly more
(Proposition 5 and Theorem 3). A forecasting strategy σ is calibrated if it is ϵ-calibrated, for every ϵ > 0. Intuitively, a forecasting strategy is calibrated if, for every forecast used with positive
long-run frequency, the empirical distribution of states given that forecast equals the
forecast itself. First, in
an adversarial setting where nature can adapt its strategy, there is no fixed prior, and the
attainable payoff is evaluated as a function of the realized long-run empirical distribution
of states. In this case, the sender can approximately guarantee the minimal payoff in the
canonical persuasion problem, where the mean of the prior equals the realized empirical
distribution (Theorem 2). Intuitively, this corresponds to merging low-value and high-value truthful forecasts with appropriate weights
so that the overall forecast matches the true frequencies. Thus, while the forecasts must
be correct on average, they can be less informative than truthful forecasts. Intuitively, this corresponds to merging low-value and high-value truthful forecasts with appropriate weights
so that the overall forecast matches the true frequencies. Thus, while the forecasts must
be correct on average, they can be less informative than truthful forecasts.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
The paper cites a wide range of literature, including works by Kamenica and Gentzkow on Bayesian persuasion, Foster and Vohra on asymptotic calibration, and studies on regret minimization in strategic environments. The references cover topics such as optimal persuasion via bi-pooling, and the relationship between calibration and online learning heuristics. p. 2 As shown by Foster and Vohra [1998], even an uninformed sender can pass the calibration
test for any stochastic process. As before, we characterize the sender’s attainable payoff
via a signaling policy in a persuasion problem. As shown by Foster and Vohra [1998], even an uninformed sender can pass the calibration
test for any stochastic process. As before, we characterize the sender’s attainable payoff
via a signaling policy in a persuasion problem. As shown by Foster and Vohra [1998], even an uninformed sender can pass the calibration
test for any stochastic process. As before, we characterize the sender’s attainable payoff
via a signaling policy in a persuasion problem. ATULYA JAIN AND VIANNEY PERCHET
1. Related Literature
Strategic Information Transmission: Our work contributes to the literature on communication between an informed sender and an uninformed receiver. ATULYA JAIN AND VIANNEY PERCHET
1. Related Literature
Strategic Information Transmission: Our work contributes to the literature on communication between an informed sender and an uninformed receiver. ATULYA JAIN AND VIANNEY PERCHET
1. Related Literature
Strategic Information Transmission: Our work contributes to the literature on communication between an informed sender and an uninformed receiver. 3 considers the case where the sender is uninformed. 4This holds in natural environments, such as finite Markov chains. 3 considers the case where the sender is uninformed. 4This holds in natural environments, such as finite Markov chains. 3 considers the case where the sender is uninformed. 4This holds in natural environments, such as finite Markov chains.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Appendix A Details
Appendix A contains omitted results and proofs, including Proposition 7 which states that if the punishment cost satisfies c > (1 + √2)L, then any optimal forecasting strategy passes the calibration test with long-run frequency one in expectation. The proof of Proposition 7 demonstrates that the sender’s average payoff is bounded by Per(Cµ, uˆS) under these conditions. p. 9 Given prior P ∈ ∆(∆Ω) and indirect utility uˆS, define the persuasion value by
(3. 3) Per(P, uˆS) := max
Q∈M(P)
EQ[ˆuS]. Regret minimization is closely linked to calibration
(see Perchet, 2014), and we build on this connection to obtain new results. We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). The truthful forecasts F1 pass the calibration test in this example. This reflects the fact
that, in the long run, empirical frequencies converge to the true conditional probabilities. For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. The distributions of
forecasts that can arise under calibration are precisely the mean-preserving contractions
of the distribution of conditionals. The truthful forecasting strategy is calibrated for any stochastic process [Dawid, 1982]. Moreover, one can choose the sequence ϵt → 0 slowly enough so that truthful forecasting
fails the finite calibration test only finitely often (see Proposition 6 in Appendix A). First, in
an adversarial setting where nature can adapt its strategy, there is no fixed prior, and the
attainable payoff is evaluated as a function of the realized long-run empirical distribution
of states. In this case, the sender can approximately guarantee the minimal payoff in the
canonical persuasion problem, where the mean of the prior equals the realized empirical
distribution (Theorem 2). Regret minimization is closely linked to calibration
(see Perchet, 2014), and we build on this connection to obtain new results. We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). We show
that if the receiver myopically best responds to the forecasts of any calibrated strategy,
then he incurs no regret, thereby justifying the use of calibration as a decision-making
heuristic (Proposition 4). Conversely, when the informed sender faces a regret-minimizing
receiver, she can guarantee at least the calibration benchmark—namely, the persuasion
value—and, for a natural class of regret-minimizing algorithms, sometimes strictly more
(Proposition 5 and Theorem 3). A common theme in the literature is
that the sender can obtain higher payoff than the rational benchmark when facing such
algorithms. Similarly, we show the sender can obtain a higher utility than the calibration
benchmark against a mean-based learner. A common theme in the literature is
that the sender can obtain higher payoff than the rational benchmark when facing such
algorithms. Similarly, we show the sender can obtain a higher utility than the calibration
benchmark against a mean-based learner. We first analyze the case of an informed sender who knows the data-generating process. She can pass the calibration test by forecasting truthfully.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The paper cites a wide range of literature, including works by Kamenica and Gentzkow on Bayesian persuasion, Foster and Vohra on asymptotic calibration, and studies on regret minimization in strategic environments. The references cover topics such as optimal persuasion via bi-pooling, and the relationship between calibration and online learning heuristics. The work also relates to strategic information transmission in dynamic environments. The authors reference specific results concerning concentration inequalities for Markov chains. The references cover topics such as optimal persuasion via bi-pooling, and the relationship between calibration and online learning heuristics. The work also relates to strategic information transmission in dynamic environments. The authors reference specific results concerning concentration inequalities for Markov chains.
Appendix B provides an equivalent formulation of the persuasion problem in terms of Blackwell experiments rather than mean-preserving contractions. It shows that a probability distribution Q is a simple mean-preserving contraction if and only if E(Q) is a garbling of E(P). This equivalence links the two ways of describing the persuasion problem. The optimization problem (B.1) is in terms of constrained Blackwell experiments while the equivalent problem (3.3) is in terms of mean-preserving contractions. The optimal garbling F≾E(P) maximizes EQ[ˆuS]. The special case for a perfectly informed sender reduces the problem to the canonical Bayesian persuasion setting. The proof of Proposition 3 uses opportunistic approachability to show that under certain restrictions on nature’s play, the sender can approach R+(Q). The final result shows that the sender can guarantee at least Per(Cµ, uˆS) against a regret-minimizing receiver. The construction of the strategy exploits the fact that mean-based algorithms adapt gradually over time to achieve a strictly higher payoff than the calibration benchmark. The overall average payoff in this example is approximately 28 > 2.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Dynamic forecasting strategy optimization for informed experts: The system can determine that
the distributions of forecasts that can arise under calibration are precisely the mean-preserving contractions of the distribution of conditionals.
This allows an AI to select forecasts that maximize payoff while satisfying a calibration test, effectively finding strategies wherethe distributions that can arise under calibration are precisely the mean-preserving contractions of the distribution of conditionals.
-
Financial platform strategic forecasting: The AI can optimize a financial platform's strategy by choosing forecasts based on the concave envelope of indirect utility:
The platform’s optimal calibrated strategy achieves Per(Cµ, uˆS).
This means the system will issue coarse forecasts rather than truthful ones when it maximizes its expected long-run average payoff. -
Uninformed sender performance benchmarking: The system can provide upper bounds for uninformed experts' payoffs by characterizing
the closed convex hull of her indirect utility,
which isCo ˆuS(f∗(p))
in the adversarial case anduˆS(f∗(p))
when the process is an ergodic Markov chain. -
Regret minimization heuristic integration: The AI can utilize regret minimization as a decision-making heuristic, knowing that
if the receiver myopically best responds to the forecasts of any calibrated strategy, then he incurs no regret,
thus justifying calibration as a practical tool. -
No-regret learner interaction modeling: When interacting with mean-based learning algorithms, the system can predict superior payoffs by exploiting adaptation:
the sender can attain a payoff strictly greater than Per(Cµ, uˆS)
by swapping forecasts to exploitcumulative payoffs from action up to period t.
Abstract
We study a dynamic game where an expert sends probabilistic forecasts to a decision-maker. The decision-maker verifies these forecasts using a calibration test based on past data. How should the expert send forecasts to maximize her payoff while passing the test? For a stationary ergodic process, we characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. Calibration restricts long-run forecast distributions to mean-preserving contractions of the distribution of truthful forecasts. We compare the payoffs attainable by an informed and uninformed expert, providing a benchmark for the value of information. Finally, we consider a decision-maker who uses a mean-based learning strategy and show that the expert can guarantee at least the calibration benchmark and sometimes strictly more.