Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker-Planck Transformation

arXiv:2604.12153 · math.OC · Submitted 2026-04-14 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Mathematics. The day in mathematics: new theorems, new proofs, and what they open.

Ada: We're looking at the paper: "Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker-Planck Transformation".

Emmy: Here's the gist of the paper. Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker–Planck Transformation develops a density-based theoretical framework for nonlinear stochastic optimal control problems with optimal stopping by establishing a density-based deterministic representation of the underlying diffusion.

Ada: Core Methodology The paper introduces a density-based approach, based on a Fokker–Planck transformation, to solve nonlinear stochastic optimal control problems with optimal stopping and terminal distributional constraints.

Emmy: For state-independent diffusion, the controlled Fokker–Planck equation is rewritten as a continuity equation driven by a score-corrected velocity field, yielding a deterministic characteristic dynamics that reproduces the marginal law of the stochastic system.

Ada: The authors leverage Stein-type identities to show that the associated distributional dynamic programming equation admits the same second-order differential operator as the distributional stochastic Hamilton–Jacobi–Bellman formulation, establishing an equivalence between formulations.

Paper summary: Ada: So, looking at "Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker–Planck Transformation", we see they tackle this hard problem by establishing this density based deterministic representation of the underlying diffusion >

Emmy: It’s about taking nonlinear stochastic control problems with optimal stopping and terminal distributional constraints and finding a way to unify how you treat control, stopping time assignment, and the evolution of the distribution itself >

Ada: The key contribution seems to be that they show an equivalence between the distributional dynamic programming equation and a standard Hamilton Jacobi Bellman formulation via Stein type identities >

Emmy: And they build on that by formulating an optimal control problem with state dependent terminal time assignment and terminal distributional constraints, leading to a distributional stochastic Hamilton Jacobi Bellman equation >

Ada: The final result is this system of first order necessary conditions OS1 through OS5 derived from the vanishing of the Gâteaux derivative of the augmented functional >

Kolya: For someone who only cares about the practical implications, this paper provides a concrete mathematical roadmap for finding optimal stopping strategies under these specific distribution constraints >

Ada: It means that instead of trying to solve an intractable stochastic problem directly, you can use this density based deterministic representation to derive necessary conditions that guide your search for the solution >

Emmy: It gives you a way to understand the underlying structure—the characteristic dynamics—which is what makes the problem tractable >

Kolya: And one thing they flag as a limitation in their discussion is that they focus on state independent diffusion, which means this framework might not directly apply to systems where the diffusion itself depends heavily on where you are >

Conclusion: Ada: So we're wrapping up this look at "Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker–Planck Transformation". This paper really shows how to take those tricky control problems with stopping times and terminal constraints and turn them into a deterministic description using that Fokker–Planck transformation >

Emmy: Kolya basically, they found a way to use density evolution instead of dealing with pure randomness >

Ada: Right, the authors are doing this for state independent diffusion problems, which is a big part of it >

Emmy: And they establish that this density approach is equivalent to the standard Hamilton Jacobi Bellman formulation through some Stein type identities >

Kolya: So what does that mean for us? It gives us a deterministic framework to find those optimal stopping strategies under these distributional constraints >

Ada: Exactly, it provides a systematic way to get those first order necessary conditions for the control and stopping time >

Emmy: It’s an important link between stochastic dynamics and deterministic characteristics >

Kolya: And they even give us that system of five necessary conditions derived from the Gâteaux derivative of the augmented functional for those constrained problems >

Daniel Guggenheim School of Aerospace, Georgia Institute of Technology · Department of Mechanical Engineering, University of Alabama

math.OC

Submitted: 2026-04-14

Updated: 2026-11-10

Comments: 40 pages, revised version of a journal submission. We acknowledge the insightful feedback from Tryphon T. Georgiou on an early version of this manuscript

Project page: https://pakniyat.github.io

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker–Planck Transformation develops a density-based theoretical framework for nonlinear stochastic optimal control problems…

Key concepts

Fokker–Planck Transformation
This technique rewrites the evolution of a probability density function (the stochastic system's distribution) into a deterministic continuity equation. This transformation provides a way to describe the underlying diffusion process using characteristic dynamics, which is easier to analyze deterministically.
Score-Corrected Velocity Field
The controlled Fokker–Planck equation is rewritten as a continuity equation where the velocity field is corrected by the score function. This correction ensures that the resulting deterministic characteristic dynamics accurately reproduce the marginal law of the original stochastic system, linking deterministic flow to stochastic behavior.
Distributional Value Function
Instead of solving for a standard value function based on state paths, this framework solves for a distributional value function. This function governs the evolution of entire probability distributions over time, which is necessary because the problem involves terminal distributional constraints.
Gâteaux Derivative Decomposition
To find optimal solutions under distribution constraints, the authors decompose variations of an augmented objective functional. This decomposition separates variations into those that fix the measure, those that flow along the measure, and those related to endpoint changes, providing a structured way to derive necessary optimality conditions.

Terminology

Summary

Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker–Planck Transformation develops a density-based theoretical framework for nonlinear stochastic optimal control problems with optimal stopping by establishing a density-based deterministic representation of the underlying diffusion. p. 0

Where the paper says it

Nonlinear Stochastic Optimal Control and Optimal Stopping using the Fokker–Planck Transformation Akan Selim , Siddhartha Ganguly , Ali Pakniyat , and Panagiotis Tsiotras Abstract. In this paper, we develop a theoretical framework for nonlinear stochastic optimal control problems with optimal stopping by establishing a density-based deterministic representation of the underlying diffusion.

Core Methodology

The paper introduces a density-based approach, based on a Fokker–Planck transformation, to solve nonlinear stochastic optimal control problems with optimal stopping and terminal distributional constraints. For state-independent diffusion, the controlled Fokker–Planck equation is rewritten as a continuity equation driven by a score-corrected velocity field, yielding a deterministic characteristic dynamics that reproduces the marginal law of the stochastic system. p. 0

Where the paper says it

In this paper, we develop a theoretical framework for nonlinear stochastic optimal control problems with optimal stopping by establishing a density-based deterministic representation of the underlying diffusion. For state-independent diffusion, we rewrite the controlled Fokker–Planck equation as a continuity equation driven by a score-corrected velocity field, yielding a deterministic characteristic dynamics that reproduces the marginal law of the stochastic system.

The authors leverage Stein-type identities to show that the associated distributional dynamic programming equation admits the same second-order differential operator as the distributional stochastic Hamilton–Jacobi–Bellman formulation, establishing an equivalence between formulations. p. 2
Where the paper says it

(C-c) Equivalence of the reformulated and original problems: Using Stein-type identities, we show that the transformed deterministic formulation retains the same distributional dynamic programming structure as the original stochastic problem. In particular, the associated distributional value function satisfies an equation involving the same second-order operator as the distributional stochastic Hamilton–Jacobi–Bellman equation, which establishes the equivalence of the two formulations at the level of value-function characterization.

Problem Formulation and Framework

The central problem (P0) involves minimizing the expected value of a Bolza-type cost functional consisting of a standard Lagrangian and a Mayer term subject to optimal stopping and terminal distributional constraints. The framework is built around evolving state distributions, where the problem is formulated at the level of evolving state distributions and value functions. p. 3

Where the paper says it

TSIOTRAS Overview: The paper develops a density-based route for stochastic control with stopping and terminal distributional goals: it begins with a distribution-level formulation, introduces a deterministic reformulation of density evolution, establishes consistency with the original stochastic problem, and derives a unified first-order optimality system. Distributional framework Formulate the problem at the level of evolving state distributions and value functions Deterministic reformulation Recast density evolution through a Fokker– Planck-based characteristic description Equivalence result Show that the reformulated problem preserves the dynamic programming structure Necessary conditions Derive unified first-order optimality conditions for common and free stopping with terminal distributional constraints Figure 1.

This leads to a distributional stochastic Hamilton–Jacobi–Bellman equation (sHJB) equation that governs the distributional value function. p. 2
Where the paper says it

(C-c) Equivalence of the reformulated and original problems: Using Stein-type identities, we show that the transformed deterministic formulation retains the same distributional dynamic programming structure as the original stochastic problem. In particular, the associated distributional value function satisfies an equation involving the same second-order operator as the distributional stochastic Hamilton–Jacobi–Bellman equation, which establishes the equivalence of the two formulations at the level of value-function characterization.

Deterministic Reformulation

A crucial aspect of the work is the deterministic reformulation via the Fokker–Planck transformation, which rewrites the stochastic density evolution as a continuity equation with a score-corrected velocity field. This yields a deterministic characteristic description whose evolving marginal law matches that of the original stochastic system, providing a density-based deterministic representation of the diffusion process. p. 2

Where the paper says it

(C-b) A deterministic reformulation via the Fokker–Planck transformation: For the case of state-independent diffusion, we introduce a Fokker–Planck transformation that rewrites the stochastic density evolution as a continuity equation with a scorecorrected velocity field. This yields a deterministic characteristic description whose evolving marginal law matches that of the original stochastic system, thereby providing a density-based deterministic representation of the diffusion process.

The reverse Fokker–Planck transformation recovers Itô’s formula at the level of the Fokker–Planck equation after applying it to a transformed system. p. 2
Where the paper says it

This is a crucial idea that enables further analysis. In addition, we characterize also the corresponding reverse Fokker-Planck transformation, showing how Itô’s formula [Øks03, Theorem 4.

Optimality Conditions and Results

The paper derives a unified first-order optimality system for both common terminal-time and general state-dependent stopping cases using variational analysis. Theorem 4.12 shows that the distributional value function for the transformed problem (P1) satisfies the same HJB equation as the original stochastic optimal control and stopping problem (P0). p. 2

Where the paper says it

(C-c) Equivalence of the reformulated and original problems: Using Stein-type identities, we show that the transformed deterministic formulation retains the same distributional dynamic programming structure as the original stochastic problem. In particular, the associated distributional value function satisfies an equation involving the same second-order operator as the distributional stochastic Hamilton–Jacobi–Bellman equation, which establishes the equivalence of the two formulations at the level of value-function characterization.

The final result is a system of first-order necessary conditions (OS1)–(OS5) derived from the vanishing of the Gâteaux derivative of the augmented functional. p. 3
Where the paper says it

TSIOTRAS Overview: The paper develops a density-based route for stochastic control with stopping and terminal distributional goals: it begins with a distribution-level formulation, introduces a deterministic reformulation of density evolution, establishes consistency with the original stochastic problem, and derives a unified first-order optimality system. Distributional framework Formulate the problem at the level of evolving state distributions and value functions Deterministic reformulation Recast density evolution through a Fokker– Planck-based characteristic description Equivalence result Show that the reformulated problem preserves the dynamic programming structure Necessary conditions Derive unified first-order optimality conditions for common and free stopping with terminal distributional constraints Figure 1.

Variational Analysis

The variational analysis for the distributionally-constrained problem (P2) is conducted by defining an augmented objective functional and computing its Gâteaux derivative using a decomposition into variations with fixed measure, measure flow, and endpoint variations. This process yields the first-order necessary conditions for the optimal control, co-state dynamics along the characteristic, optimal final-time (stopping) condition, and the terminal transversality condition. p. 3

Where the paper says it

(C-d) First-order necessary conditions: Building on this reformulation, we formulate an optimal control problem with state-dependent terminal-time assignment and terminal distributional constraints, and analyze it by variational methods in the space of probability measures. This yields a (Pontryagin-type) first-order optimality system, applicable to both the common terminal-time case and the more general free/statedependent stopping case, comprising control stationarity, adjoint dynamics with diffusion-induced correction terms, and appropriate transversality conditions.

The analysis relies on concepts like interior Otto perturbations to preserve the stopping surface and mass to first order. p. 0
Where the paper says it

Building on this representation, we formulate an optimal control problem with state-dependent terminal-time assignment and terminal distributional constraints and derive the first-order necessary conditions using variational analysis. We present the conditions both for a common terminal time and for the general case of state-dependent stopping.

Future Directions

Future work is planned to focus on scalable, sampling-based numerical methods for high-dimensional problems, including indirect physics-informed approaches that incorporate the derived optimality conditions together with score-matching algorithms, as well as direct sampling-based semi-Lagrangian methods with neural approximations of the adjoint and density fields. The paper concludes by establishing a novel approach for finding optimal stopping strategies in nonlinear stochastic optimal control problems subject to distributional constraints. p. 2

Where the paper says it

Despite the previous developments, general 𝑛-dimensional stochastic optimal control and stopping problems, including optimal common stopping, with distributional constraints remain largely unexplored. To address this gap, with the long-term goal of developing a scalable sampling-based method, we establish a novel approach for finding optimal stopping strategies in nonlinear stochastic optimal control problems subject to distributional constraints.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved AI systems can solve nonlinear stochastic optimal control problems where control decisions are coupled with an endogenous stopping mechanism by formulating them as a density-based deterministic representation using a Fokker–Planck transformation. This enables treating control, stopping, and distribution evolution in a unified manner.

  2. The system can derive first-order necessary conditions for optimal strategies, including those for common and state-dependent terminal times, using variational analysis on the transformed problem formulation (P1). These conditions include control stationarity and adjoint dynamics with diffusion-induced correction terms.

  3. AI systems can handle problems where the objective is to shape the entire distribution of system states rather than just a single trajectory. This capability is useful in applications like robotic swarms or aerospace guidance where distribution-level guarantees are more meaningful than nominal tracking alone.

  4. The framework allows for an equivalent deterministic representation of stochastic dynamics as a characteristic description driven by a score-corrected velocity field, which simplifies analysis and enables the application of calculus of variations over the space of probability measures.

  5. The system can solve distributionally constrained optimal control problems (P2) where the objective involves terminal distributional constraints, by transforming them into an unconstrained optimization problem using a measure flow operator. This allows for deriving optimality conditions with independent decision variables.

  6. AI systems can incorporate non-local drift terms in their dynamics, where the drift depends on the current probability measure, as seen in Problem (P1). The transformed velocity field is nonlocal, meaning it "depends explicitly on the current probability measure mu t at t >= 0."

  7. The system can determine optimal stopping strategies based on a time-dependent assignment function (Tad), which allows for state-dependent terminal constraints and provides a mechanism to derive the free-terminal-time (stopping) condition (OS5).

  8. AI systems can utilize a set of first-order necessary conditions derived from the variational analysis, such as Optimal control: d/dt u L(t, x(t), u(t)) + lambda(t) T 1/dx f(t, x(t), u(t)) = 0, to guide the optimal control policy.

Abstract

In this paper, we develop a theoretical framework for nonlinear stochastic optimal control problems with optimal stopping by establishing a density-based deterministic representation of the underlying diffusion. For state-independent diffusion, we rewrite the controlled Fokker-Planck equation as a continuity equation driven by a score-corrected velocity field, yielding a deterministic characteristic dynamics that reproduces the marginal law of the stochastic system. Leveraging Stein-type identities, we show that the associated distributional dynamic programming equation admits the same second-order differential operator as the distributional stochastic Hamilton-Jacobi-Bellman formulation. Building on this representation, we formulate an optimal control problem with state-dependent terminal-time assignment and terminal distributional constraints and derive the first-order necessary conditions using variational analysis. We present the conditions both for a common terminal time and for the general case of state-dependent stopping.

Sources

Related papers