Stochastic Optimal Control with Measurable Coefficients and Applications

arXiv:2502.02352 · math.OC, econ.GN, math.PR, q-fin.EC · Submitted 2025-02-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Mathematics. The day in mathematics: new theorems, new proofs, and what they open.

Ada: I'm Ada.

Emmy: And I'm Emmy, here with Kolya.

Ada: We're looking at the paper: "Stochastic Optimal Control with Measurable Coefficients and Applications".

Emmy: Here's the gist of the paper. Stochastic optimal control problems with merely measurable coefficients are not well understood <ref:2502.02352#pg2> In this manuscript, we consider fully non-linear stochastic optimal control problems in infinite horizon with measurable coefficients and (local) uniformly elliptic diffusion <ref:2502.02352#pg2>.

Ada: Using the theory of Lp-viscosity solutions, we show existence of an Lp-viscosity solution v ∈ W2,p loc of the Hamilton-Jacobi-Bellman (HJB) equation, which, in turn, is also a strong solution (i.e.

Emmy: it satisfies the HJB equation pointwise a.e.) <ref:2502.02352#pg2>.

Ada: These results allow us to construct optimal feedback controls and to characterize the value function as the unique Lp-viscosity solution of the HJB equation <ref:2502.02352#pg2>.

Paper summary: Ada: So to recap, this paper tackles fully non-linear stochastic optimal control problems with infinite horizon where the coefficients are only measurable, not necessarily smooth. The core thesis is that using Lp-viscosity solutions, they prove existence of a solution v in W 2,p locally for the Hamilton-Jacobi-Bellman equation <ref:2502.02352#pg1>.

Emmy: And they push that further by showing this Lp-viscosity solution is also a strong solution, which means it satisfies the Hamilton-Jacobi-Bellman equation pointwise almost everywhere. They then use these solutions to prove verification theorems which tell us exactly when a control strategy is optimal.

Kolya: I wonder what the implications are for someone looking at advertising models, since they mention that as an application area? Does this theory help solve problems where the underlying dynamics or costs aren't perfectly smooth?

Ada: The paper applies these ideas to a stochastic optimal control problem in economics, specifically in the context of optimal advertising. They set up a controlled one-dimensional stochastic differential equation for stock of advertising goodwill, and they minimize an expected cost functional.

Emmy: The results allow them to characterize the value function V as the unique Lp-viscosity solution of the Hamilton-Jacobi-Bellman equation. This means even with those tricky measurable coefficients, there's a definitive way to find that optimal value function.

Conclusion: Ada: Looking at the title, Stochastic Optimal Control with Measurable Coefficients and Applications, it really highlights the scope—it’s about solving control problems where we don't assume nice things like continuity or Lipschitz conditions on our functions. The authors are showing that this framework works for these rougher coefficient settings.

Emmy: What this means in plain terms is that you don't need perfectly smooth functions to find an optimal strategy in a complex stochastic system, as long as the diffusion part is locally uniformly elliptic. They establish that the value function can be characterized uniquely by this Lp-viscosity solution framework.

Kolya: So, for someone just listening to this show, what's the real-world impact? It means we can apply these techniques to economic models where things aren't perfectly smooth—like advertising spending decisions—and get a solid answer on what the best strategy is.

Ada: Right. It’s about extending control theory into areas where standard smoothness assumptions fail, giving us tools to solve problems that were previously out of reach. This paper provides a way to construct those optimal feedback controls directly from this unique solution structure.

Emmy: So we have the existence, the uniqueness of the value function as an Lp-viscosity solution, and a method to find the optimal control policy through measurable selections. It's a comprehensive toolkit for non-smooth settings in stochastic control.

FILIPPO DE FEO

Deutsche Forschungsgemeinschaft · INdAM · Gruppo Nazionale per l’ Analisi Matematica, la Probabilité e le loro Applicazioni · Italian Ministry of University and Research

math.OC, econ.GN, math.PR, q-fin.EC

Submitted: 2025-02-04

Updated: 2026-11-04

Comments: Accepted for publication on SIAM Journal on Control and Optimization

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 76/100

The gist: Stochastic optimal control problems with merely measurable coefficients are not well understood <ref:2502.02352#pg2> In this manuscript…

Key concepts

L p-viscosity solution
This is a mathematical tool used to define solutions for complex partial differential equations (PDEs) that have measurable coefficients. It extends the traditional notion of viscosity solutions by using a larger class of test functions, which helps in proving the existence and uniqueness of solutions even when standard methods fail.
Hamilton-Jacobi-Bellman (HJB) equation
This is a fundamental PDE in optimal control that describes the value function. Solving it is key to finding the optimal strategy for minimizing a cost functional. The paper shows that L p-viscosity solutions can be used to find strong solutions, meaning they satisfy the equation pointwise almost everywhere.
Verification Theorems
These theorems provide necessary and sufficient conditions for determining if a given control policy is optimal. By using these theorems, the authors can construct the actual optimal feedback controls and verify that the value function found is indeed unique among all possible solutions.
Strong Solution (W2,p loc)
A strong solution means that the solution to the HJB equation satisfies it in a stronger sense—pointwise almost everywhere. The paper proves that an L p-viscosity solution with $p ext{ ≥ } n$ is actually a strong solution, which is crucial for applying verification theorems and proving uniqueness.

Terminology

Summary

Stochastic optimal control problems with merely measurable coefficients are not well understood <ref:2502. p. 002352#pg2> In this manuscript, we consider fully non-linear stochastic optimal control problems in infinite horizon with measurable coefficients and (local) uniformly elliptic diffusion <ref:2502. p. 002352#pg2>. Using the theory of Lp-viscosity solutions, we show existence of an Lp-viscosity solution v ∈ W2,p loc of the Hamilton-Jacobi-Bellman (HJB) equation, which, in turn, is also a strong solution (i. p. 0e. it satisfies the HJB equation pointwise a. p. 0e.) <ref:2502. p. 002352#pg2>. These results allow us to construct optimal feedback controls and to characterize the value function as the unique Lp-viscosity solution of the HJB equation <ref:2502. p. 002352#pg2>.

Problem Formulation and Context

The paper addresses fully non-linear stochastic optimal control problems with infinite horizon where the goal is to minimize a cost functional J(x, u(·)) defined by an expectation involving the running cost l(y(t), u(t)) <ref:2502.02352#pg2>. The state dynamics are given by a controlled stochastic differential equation dy(t) = b (y(t), u(t)) dt + σ (y(t), u(t)) dW(t) <ref:2502. p. 302352#pg2>. Literature review indicates that standard approaches via viscosity solutions of the Hamilton-Jacobi-Bellman (HJB) equation are successful under standard continuity assumptions, but control problems with merely measurable coefficients have not been well studied <ref:2502. p. 102352#pg2>.

Theoretical Framework for Solutions

The paper introduces the notion of L p-viscosity solution to treat second-order fully non-linear uniformly elliptic and parabolic partial differential equations (PDEs) with measurable coefficients <ref:2502.02352#pg2>. This theory replaces C2 test functions of viscosity solutions with the larger class of W 2,p-test functions, leading to stronger notions of solution and allowing for uniqueness results on bounded domains <ref:2502. p. 102352#pg2>. Proposition 3. p. 13 shows that if v is an L p-viscosity solution with p ≥ n, then v ∈ W2,p loc, making it a strong solution of the HJB equation <ref:2502. p. 002352#pg2>.

Verification Theorems and Optimality

The authors use the theory of L p-viscosity solutions to prove verification theorems, which provide necessary and sufficient conditions for optimality <ref:2502.02352#pg2>. Theorem 4.1 establishes these conditions by considering four specific inequalities (4. p. 61) through (4.4) related to the value function v <ref:2502. p. 002352#pg2>. The verification theorems allow for the construction of optimal feedback controls and characterize V as the unique L p-viscosity solution of the HJB equation <ref:2502. p. 002352#pg2>.

Application to Economic Models

The developed theory is applied to solve a stochastic optimal control problem arising in economics within the context of optimal advertising <ref:2502.02352#pg2>. This involves considering a controlled 1-dimensional SDE for the stock of advertising goodwill y(t) and minimizing an expected cost functional related to investment u(t) <ref:2502. p. 202352#pg2>. The optimal policy is found by characterizing the value function V as the unique L p-viscosity solution of the HJB equation <ref:2502. p. 202352#pg2>.

Uniqueness and Synthesis

The paper establishes uniqueness results for strong solutions or L p-viscosity solutions of the HJB equation using control-verification techniques, which is different from analytic methods used for bounded domains <ref:2502.02352#pg2>. Corollary 5. p. 25 states that V is the unique L p-viscosity solution to (3. p. 01) in the class C(R n) with polynomial growth, and the unique strong solution in W 2,p loc (R n) with polynomial growth <ref:2502. p. 902352#pg2>. The optimal feedback controls are constructed via Borel measurable selections of the set-valued map Ψ <ref:2502. p. 802352#pg2>.

Summary of Key Findings

The paper demonstrates that for fully non-linear stochastic optimal control problems with measurable coefficients and local uniform elliptic diffusion, an Lp-viscosity solution v with p ≥ n is a strong solution of the HJB equation <ref:2502.02352#pg2>. This allows for the construction of optimal feedback controls and the characterization of the value function as the unique L p-viscosity solution <ref:2502. p. 102352#pg2>. The results are then applied to solve an optimal advertising problem, where V is characterized as this unique solution <ref:2502. p. 202352#pg2>.

Applications to Optimal Advertising Models

In the application section, the authors set b(x, u):= a(x) + cu, σ(x, u):= ν(x) + γu, l(x, u):= h(u) − g(x), leading to a Borel measurable Hamiltonian H <ref:2502.02352#pg2>. For p ≥ n = 1, Proposition 5. p. 14 and Corollary 5. p. 25 are used to completely solve the optimal advertising problem by characterizing V as the unique L p-viscosity solution in Cb(R) to the HJB equation <ref:2502. p. 202352#pg2>. The optimal policy is then found as u(t):= ψ(y(t)), where ψ is a Borel measurable selection of the set valued map Ψ <ref:2502. p. 802352#pg2>.

Characterization of the Optimal Policy

The optimal policy construction relies on Proposition 5.3, which states that if certain conditions hold, the pair (y(·), u(·)) is optimal and v(x) = V (x) <ref:2502. p. 502352#pg2>. The control process u(t) satisfies u(s) ∈ argminu∈UHcv(y∗(s), Dv(y∗)(s), D2v(y∗)(s), u), P-a. p. 2s., for a.e. s ≥ 0 <ref:2502. p. 0 p. 002352#pg2>.

Uniqueness of the Value Function

The uniqueness of the value function V is established by showing that any strong solution or L p-viscosity solution satisfying certain conditions must be equal to V <ref:2502.02352#pg2>. This is achieved through Theorem 4. p. 06, which shows that if (4.1) and (4.4) are satisfied by v = V, then the optimal pair (y∗, u∗(·)) must satisfy the necessary condition u∗(s) ∈ argminu∈UHcv(y∗(s), DV(y∗)(s), D2V(y∗)(s), u), P-a. p. 7s., for a.e. s ≥ 0 <ref:2502.02352#pg2>.

Mathematical Tools Utilized

The work heavily utilizes the theory of L p-viscosity solutions [2, 32] and Dynkin’s formula for W 2,n-functions [24] <ref:2502.02352#pg2>. Appendix D provides a measurable selection theorem in optimization to handle non-Borel measurable functions <ref:2502. p. 002352#pg2>. The analysis relies on properties of Sobolev spaces W 1,p(O) and W 2,p(O) <ref:2502. p. 002352#pg2>. The paper also uses comparison with existing literature regarding non-smooth solutions <ref:2502. p. 202352#pg2>.

Conclusion on Methodology

The overall methodology involves using the L p-viscosity solution theory to establish strong solutions and then employing verification theorems derived from Dynkin's formula to prove optimality conditions <ref:2502.02352#pg2>. This approach successfully tackles fully non-linear stochastic optimal control problems with measurable coefficients, providing a comprehensive solution for both existence and characterization of the value function <ref:2502. p. 002352#pg2>. The paper concludes by showing that V is the unique L p-viscosity solution in C(R n) with polynomial growth <ref:2502. p. 902352#pg2>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved AI system can solve fully non-linear stochastic optimal control problems with measurable coefficients in infinite horizon settings by constructing optimal feedback controls using a Borel measurable selection of an argmin function from the set-valued map Psi(x).

  2. Improved AI system can characterize the value function as the unique L p-viscosity solution or unique strong solution of the Hamilton-Jacobi-Bellman equation, providing necessary and sufficient conditions for optimality through verification theorems.

  3. Improved AI system can solve optimal advertising problems by characterizing the value function V as the unique L p-viscosity solution in Cb(R) to a specific HJB equation derived from the advertising model dynamics.

Sources

Related papers