Data-driven Linear Quadratic Integral Control: A Convex Formulation and Policy Gradient Approach

arXiv:2604.14905 · eess.SY, cs.SY · Submitted 2026-04-16 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Data-driven Linear Quadratic Integral Control".

Dev: Here's the gist of the paper. This paper studies the data-driven synthesis of linear quadratic integral (LQI) controllers for continuous-time systems to achieve optimal state-feedback control with integral action for reference tracking using only measured data.

Rosa: Core Methodology The objective is to derive a data-driven closedloop parameterization of the augmented dynamics that incorporates the integral state while relying solely on input-stateoutput measurements.

Dev: Based on this parameterization, a data-driven convex optimization problem is formulated whose solution yields the optimal linear quadratic regulator (LQR) feedback gain for the augmented system without explicit knowledge of the system matrices.

Rosa: In addition, a policy gradient flow is derived to compute the optimal controller within the space of stabilizing gains.

Title and authors: Rosa: So this paper is titled "Data-driven Linear Quadratic Integral Control: A Convex Formulation and Policy Gradient Approach." It's about making these linear quadratic integral controllers for continuous-time systems using only measured data.

Dev: That sounds like they're tackling the problem of getting optimal state feedback with integral action, which is usually hard without knowing the system matrices beforehand.

Rosa: Exactly. The core idea here is to get that optimal control using just input, state, and output measurements. It’s about synthesizing a controller without needing explicit knowledge of the underlying system dynamics for that part of the calculation.

Taro: So if I'm hearing this right, we're trying to build a feedback mechanism that has integral action for tracking but doesn't require us to model every single parameter of the plant upfront?

Dev: That’s right. They propose a way to do that by first creating a data-driven closed-loop parameterization of the augmented system, which includes that integral state.

The paper's summary: Rosa: So what they're actually doing is deriving this representation from measured input, state, and output trajectories. They set up a convex optimization problem based on that representation to find the optimal linear quadratic regulator feedback gain.

Dev: That’s the crucial part for me—using a convex optimization framework so we don't get stuck with something non-convex where finding the solution is tough. And they solve it without knowing A or B explicitly, which is a big deal.

Rosa: They also introduce this policy gradient flow to compute the optimal controller within the set of stabilizing gains. So it’s not just a static gain; it's something that can adapt based on what the data shows them during operation.

Taro: I wonder how robust this adaptive part is when things get messy, like if there's some noise or if the system starts behaving unexpectedly outside the tested conditions?

Dev: The paper shows it works by validating it on a distributed generation unit in a DC microgrid. They found that the feedback gain K converges to an optimal value of zero point four zero nine, which is pretty specific for that setup.

The paper's improvements: Rosa: One thing they highlight as an improvement is how they handle the parameterization itself. They use a data-driven closed-loop parameterization based on sampled covariance, and the resulting data matrices don't actually depend on the sample size T.

Dev: That’s significant because it means we don't have to keep collecting more and more samples just to make the representation accurate for a bigger system. The structure of those data matrices is independent of how long you measure, as long as you have enough measurements.

Rosa: And then they show how the optimal LQR feedback gain is recovered through this convex program. They find that the optimal linear quadratic regulator feedback gain K equals U times Z minus one, where Z and W are matrices derived from minimizing that program based on the data matrices.

Taro: So, it’s not just finding *a* controller; they show a way to find the *optimal* one by framing it as this constrained optimization problem over the data. Does that mean we can actually get a better result than just guessing?

Dev: It means mathematically, yes. The formulation allows them to compute that optimal LQR gain for the augmented system directly from those measured data matrices without having to solve the original state-space equations analytically first.

Conclusion: Rosa: So to wrap up, this paper introduces a data-driven approach for synthesizing linear quadratic integral controllers for continuous-time systems using only input, state, and output measurements. They achieve optimal state feedback control with integral action for reference tracking just by relying on measured data.

Dev: The main contribution is deriving the convex optimization problem that lets you compute the optimal LQR feedback gain without needing explicit knowledge of the system matrices, and then using a policy gradient flow to compute that optimal controller within the space of stabilizing gains.

Taro: For someone listening who just cares about real-world application, this means you can build a tracking controller for something like a microgrid or some other continuous system where you don't have perfect models—you just feed it the sensor readings and it figures out how to control it optimally.

Rosa: It’s moving toward systems that adapt their control law based on what they are actually measuring in real time. We’re looking at a data-driven approach for LQI controllers, and this paper shows how to frame that as a convex optimization problem solvable with policy gradient flows.

Dev: The implication is that we can move away from relying entirely on perfect system identification for control synthesis and instead use data collection to guide the optimization process itself.

Taro: I’m just thinking about the next steps, though, because they mention limitations. They say they didn't fully address scenarios with process and measurement noise or time-varying system dynamics yet. That’s where the real test will be for any practical deployment.

Rosa: Right now, it’s a proof of concept on a DGU in a DC microgrid, showing convergence to that optimal gain of zero point four zero nine for that example. It proves the concept works under certain conditions.

Dev: So the next paper we'll be looking at might be how this handles those noise issues you mentioned, because if it can’t handle noise gracefully, it doesn't really help us in a noisy power grid environment.

Karlsruhe Institute of Technology (KIT) · ABB Corporate Research Center

eess.SY, cs.SY

Submitted: 2026-04-16

Updated: 2026-11-10

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 82/100

The gist: This paper studies the data-driven synthesis of linear quadratic integral (LQI) controllers for continuous-time systems to achieve optimal state-feedback control with integral action for reference…

Key concepts

LQI Controller
An LQI controller is used for continuous-time systems that combines standard state feedback with an integral action. The integral action accumulates the error between the system's output and a desired reference signal, ensuring the system eventually tracks that reference perfectly. This combination provides both fast response and zero steady-state error.
Data-Driven Parameterization
This involves using measured input, state, and output data to create a mathematical model of the closed-loop system dynamics. Instead of needing exact system matrices beforehand, this method uses sampled data (like $y(0), y( riangle), ext{etc.}$) to define equivalent representations that describe how the system behaves based on what we have actually measured.
Convex Optimization
The paper sets up a mathematical problem that is convex, meaning it has a single best solution that can be found efficiently. This optimization problem is used to determine the optimal LQR feedback gain ($K^*$) for the augmented system. Because it's convex, we can solve it reliably using standard computational techniques.
Policy Gradient Flow
This is a technique used to compute the actual controller within a set of possible stabilizing gains. It starts with an initial guess and iteratively adjusts the gain in a direction that improves performance, guided by the optimization problem. This flow converges to the optimal control law, allowing for dynamic adaptation based on real-time system behavior.

Terminology

Summary

This paper studies the data-driven synthesis of linear quadratic integral (LQI) controllers for continuous-time systems to achieve optimal state-feedback control with integral action for reference tracking using only measured data. p. 0

Core Methodology

The objective is to derive a data-driven closedloop parameterization of the augmented dynamics that incorporates the integral state while relying solely on input-stateoutput measurements. Based on this parameterization, a data-driven convex optimization problem is formulated whose solution yields the optimal linear quadratic regulator (LQR) feedback gain for the augmented system without explicit knowledge of the system matrices. p. 0 In addition, a policy gradient flow is derived to compute the optimal controller within the space of stabilizing gains. p. 0

System Augmentation and Constraints

The LQI control law consists of a static state-feedback controller augmented with an integral action that accumulates the tracking error between the output and the reference signal. The augmented dynamics are defined by incorporating an integral state to enforce asymptotic tracking of a reference signal. p. 1 The closed-loop system under LQI control can be written in error coordinates as x˜˙a = Aax˜a + Bau˜a. p. 1 The optimal LQR feedback gain K∗ is synthesized by computing the optimal LQR feedback K∗ for the augmented system. p. 1

Data-Driven Parameterization

The paper introduces a data-driven closed-loop parameterization of the augmented system. This involves defining data matrices Y = y(0) y(∆). p. 0.. y((T − 1)∆). Theorem 2 states that the augmented closed-loop system (13) admits an equivalent data-driven representation x˜˙a = X′−YGx˜a. p. 3

Convex Optimization and Policy Gradient

The optimal LQR feedback gain K∗ is found by solving a convex optimization problem. This problem is formulated by introducing an epigraph variable S to bound a non-convex term. p. 4 The projected policy gradient flow G˙ = −αΠ∇fG converges to G∗, which yields the optimal LQR gain K(t) = −UG(t) 0 (53)>. p. 5

Validation and Results

The effectiveness of the method is demonstrated through a numerical example involving a distributed generation unit (DGU) in a DC microgrid. The simulation results show that the feedback gain K(t) converges to the optimal gain K⋆ ≈ 0. p. 5409 1.164 −9.997. The final controller in Fig. p. 6 4 implements the policy gradient flow in closed loop with (56), yielding a nonlinear dynamic state-feedback controller that adapts the gain toward the LQR optimal gain corresponding to the current load. p. 6

Conclusion

This paper introduced a data-driven approach for the synthesis of LQI controllers for continuous-time systems. Using a closed-loop data-driven parameterization, we derived a convex optimization problem that enables the computation of the optimal LQR feedback gain of the augmented system directly from measured data. p. 6 In addition, a policy gradient flow was introduced to compute the optimal controller within the set of stabilizing gains. p. 6 Future work will focus on extending the proposed approach to scenarios with process and measurement noise, as well as time-varying system dynamics. p. 7

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Data-driven LQI Controller Synthesis: The system can synthesize optimal linear quadratic integral (LQI) controllers for continuous-time systems using only measured input-stateoutput measurements, achieving optimal state-feedback control with integral action for reference tracking.

  2. Convex Optimization Formulation: The system allows the formulation of a data-driven convex optimization problem whose solution yields the optimal linear quadratic regulator (LQR) feedback gain for the augmented system without explicit knowledge of the system matrices.

  3. Policy Gradient Computation: A policy gradient flow can be derived to compute the optimal controller within the space of stabilizing gains, enabling data-driven optimal tracking control.

  4. Parameterization Adaptation: The closed-loop system dynamics can be represented using a data-driven closed-loop parameterization based on sampled covariance parameterization, resulting in data matrices independent of the sample size T.

  5. Optimal Gain Recovery: The optimal LQR feedback gain K star is recovered as K star equals U Z star W minus one, where Z star and W star minimize a convex program derived from the data matrices.

  6. Robust Tracking Under Load Variations: The system can adapt its feedback gain toward the LQR optimal gain corresponding to current load variations, enabling fast tracking without overshoot even when system parameters like load admittance change over time.

Sources

Related papers