Closing the Loop: Dynamic State Estimation and Feedback Optimization of Power Grids
summary
The gist
Closing the Loop: Dynamic State Estimation and Feedback Optimization of Power Grids considers online feedback optimization to solve the AC Optimal Power Flow in real-time in power grids This paper…
In short
The paper addresses controlling power grids in real-time by combining dynamic state estimation with online feedback optimization for AC Optimal Power Flow. It formally proves that linking these two components creates a stable, steady-state optimal system. This allows the grid to adapt quickly to changing conditions using only real-time measurements.
Key concepts
- Dynamic State Estimation (SE)
- This subsystem uses a Kalman filter to estimate the true grid state. It works by subtracting power flow equations at different times, which helps determine unmeasured states like loads, providing an unbiased estimate ($ ilde{V}(t)$) even when measurements are imperfect.
- Online Feedback Optimization
- Instead of solving the entire optimization problem statically, this method uses the real-time state estimate ($ ilde{V}(t)$) as feedback to continuously adjust control set-points. This allows the system to react immediately to fast changes in the power grid's conditions.
- Optimal Power Flow (OPF)
- This is a core problem in power systems where the goal is to determine the best settings for controllable energy sources. The paper shows how online feedback optimization helps solve this complex, time-varying problem efficiently in real-time.
- Stability and Convergence Proof
- The authors formally prove that the combination of state estimation and feedback optimization results in a closed-loop system that is stable. This mathematical proof guarantees that both the estimation errors and the control set-points will converge to their true, optimal values over time.
Terminology used across episodes
This episode discusses
The paper
Closing the Loop: Dynamic State Estimation and Feedback Optimization of Power Grids · Read on arXiv
Miguel Picallo, Saverio Bolognani, Florian Dorfler
Automatic Control Laboratory, ETH Zurich
DOI: 10.1016/j.epsr.2020.106753
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "Closing the Loop".
Dev: Here's the gist of the paper. Closing the Loop: Dynamic State Estimation and Feedback Optimization of Power Grids considers online feedback optimization to solve the AC Optimal Power Flow in real-time in power grids This paper formally proves that the interconnection of the grid dynamics, the SE, and the feedback optimization is stable and steady-state optimal The contribution of this work lies in formally proving that the interconnection of the grid dynamics, the SE, and the feedback optimization is stable and steady-state optimal Problem Context The operation of power grids is undergoing a paradigm shift due to increasing controllable elements and pervasive sensing State-of-the-art optimization methods typically solve a static Optimal Power Flow (OPF) to determine the set-points of these controllable energy sources However, these approaches may not succeed in efficiently controlling the system under fast time-varying conditions The online feedback optimization consists of collecting real-time grid state measurements at each time step and using them as feedback to a controller However, so far the analysis of all of these feedback optimization approaches have considered a stylized problem setup by assuming availability of noise-free measurements of all states that need to be controlled The proposed method combines a dynamic SE based on a Kalman filter with online feedback optimization The dynamic State Estimation builds a system by subtracting the power flow equations at subsequent times to circumvent the lack of precise knowledge of loads The Kalman filter is designed to output an estimate Vˆ(t) based on measurements y(t) The interconnection of these subsystems results in a closed-loop system represented in Figure 1 Theorem 1 establishes exponential convergence towards the unbiased and optimal solution for both the estimation error and the set-points Convergence Rates Theorem 1 provides specific convergence rates for the errors in expectation and mean square The expected value of the estimation error converges to 0 as kE e(t)k22 ≤ σˆσˇ1−ψσˇt kE e(0)k22 t→∞→ 0 The expected value of the optimization error converges to 0 as kSc,(t) − S E∗c k2 ≤ tCS1e−CS2t kSc,(0) − S E∗c,(0)k2 t→∞→ 0 The estimation error is shown to converge very quickly in a few seconds (iterations), while the optimization error needs about 20 min (1200 iterations) Validation and Results The approach was validated on a simulated test feeder using 1-second time intervals The results show that both the estimation error and the optimization errors decrease to a low value, less than 1% for the estimation case, and then remain stable The accuracy is maintained despite potential model mismatches due to the robustness of real-time feedback optimization The voltage magnitudes remain all the time within limits almost for every node during the simulation Future Directions Future work could include considering a nonlinear power flow and measurements equation This includes biased pseudomeasurements and process noise to have non-zero mean timevarying loads Using the uncertainty represented in the estimation error covariance matrix to increase voltage restrictions could further reduce violations The constants CV, CS are monotone increasing with respect to trace(Σl), trace(Σy) and can be lowered by using faster time scales The convergence of V(t) is a result of the convergence of Sc,(t) The feasibility set F that has been used is not constant due to the time-varying profiles in solar radiation and wind speed APPENDIX A PROOF OF THEOREM 1 The proof starts by defining the state x(t), output z(t), and disturbance ω(t) The operator φ(·) is the generalized gradient mapping adapted to the projected gradient descent in (9) The separation principle holds because there is a 0 multiplying the input u(t + 1) for the index corresponding to the estimation error e(t) The convergence of V(t) is a result of the convergence of Sc,(t) Lemma 1 shows that the objective as a function of Sc is η-strongly convex and has a Lipschitz continuous gradient Lemma 2 shows that there exists a Lipschitz constant LS,opt such that the operators mapping the subsequent values Sl,(t+1), to the respective optimal solutions S∗c,(t), S∗c,(t+1) of (4) satisfy Theorem 2 provides bounds on the convergence of a stochastic process z(t) The proof for the estimation error e(t) shows that e(t) is exponentially bounded in mean square and provides the corresponding bound (11) The proof for the optimality part shows that kSc,(t+1) − S∗c,(t+1)k2a) ≤ rkSc,(t) − S E∗c k2 + kBck2LgE e(t)k2b> The proof for the optimality part shows that kSc,(t+1) − S∗c,(t+1)k2d) ≤ p1−2η + 2L2kSc,(t) − S∗c,(t)k2 + kBck2Lgke(t)k2 The proof for the optimality part shows that kSc,(t+1) − S∗c,(t+1)k2e> ≤ rkSc,(0) − S E∗c k2 + kBck2LgqσˆσˇkE e(0)k2√1−ψσˇt The proof for the optimality part shows that kSc,(t+1) − S E∗c k2c) ≤ rtkSc,(0) − S E∗c k2 + kBck2LgqσˆσˇkE e(0)k2 t max r,√1−ψσˇt The proof for the optimality part shows that kSc,(t+1) − S E∗c k2e) ≤ rkSc,(0) − S E∗c k2 + kBck2LgqσˆσˇE e(0)k2 t max r,√1−ψσˇt t→∞→ 0 The proof for the estimation error e(t) shows that E
eT(t+1)P-1(t+1)e(t+1)e(t): ≤ (1−ψσˇ)eT(t)P-1(t)e The proof for the estimation error e(t) shows that kE e(k)k22 ≤ σˆσˇ1−ψσˇt kE e(0)k22 t→∞→ 0 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2a) ≤ rtE e(k)k2 + kBck2LgkE e(t)k2 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2b) ≤ rtE e(k)k2 + kBck2LgqσˆσˇkE e(0)k2√1−ψσˇt The proof for the optimality part shows that kSc,(t+1) − S E∗c k2c) ≤ rtkSc,(0) − S E∗c k2 + kBck2LgqσˆσˇE e(0)k2 t max r,√1−ψσˇt t→∞→ 1−rkBck2LgkBl(Sl,(t+1) − Sl,(t))k2 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2e) ≤ rkkSc,(0) − S E∗c k2 + kBck2LgqσˆσE e(0)k2 t max r,√1−ψσˇt t→∞→ 0 The proof for the estimation error e(t) shows that kE e(t)k22 ≤ σˆσˇ1−ψσˇt kE e(0)k22 t→∞→ 0 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2a) ≤ rtE e(k)k2 + kBck2LgkE e(t)k2 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2b) ≤ rtE e(k)k2 + kBck2LgqσˆσkE e(0)k2√1−ψσˇt The proof for the optimality part shows that kSc,(t+1) − S E∗c k2c) ≤ rtkSc,(0) − S E∗c k2 + kBck2LgqσˆσE e(0)k2 t max r,√1−ψσˇt t→∞→ 1−rkBck2LgkBl(Sl,(t+1) − Sl,(t))k2 The proof for the estimation error e(t) shows that kE e(t)k22 ≤ σˆσˇ1−ψσˇt kE e(0)k22 t→∞→ 0 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2a) ≤ rtE e(k)k2 + kBck2LgkE e(t)k2 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2b) ≤ rtE e(k)k2 + kBck2LgqσˆσkE e(0)k2√1−ψσˇt The proof for the optimality part shows that kSc,(t+1) − S E∗c k2c) ≤ rtkSc,(0) − S E∗c k2 + kBck2LgqσˆσE e(0)k2 t max r,√1−ψσˇt t→∞→ 1−rkBck2LgkBl(Sl,(t+1) − Sl,(t))k2 The proof for the estimation error e(t) shows that kE e(t)k22 ≤ σˆσˇ1−ψσˇt kE e(0)k22 t→∞→ 0 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2a) ≤ rtE e(k)k2 + kBck2LgkE e(t)k2 The proof for the optimality part shows that kSc,(t+1) − S E∗c k2b) ≤ rtE e(k)k2 + kBck2LgqσˆσkE e(0)k2√1−ψσˇt The proof for the optimality part shows that kSc,(t+1) − S E∗c k2c) ≤ rtkSc,(0) − S E∗c k2 + kBck2LgqσˆσE e(0)k2 t max r,√1−ψσˇt t→∞→ 1−rkBck2LgkBl(Sl,(t+1) − Sl,(t))k2 The proof for the estimation error e(t) shows that kE e(t)k22 ≤ σˆσˇ1−ψσˇt kE e(0)k22 t→∞→ 0 Where the gains come from
Title and authors: Rosa: So, we're looking at this paper called "Closing the Loop: Dynamic State Estimation and Feedback Optimization of Power Grids." It deals with figuring out how to control power grids in real time when things are changing fast.
Dev: It's about using online feedback optimization to solve the AC Optimal Power Flow in real time, which means continuously driving controllable power injections and loads towards the optimal set-points based on measurements.
Rosa: But what makes this paper different is that instead of assuming you have perfect, noise-free knowledge of every state, they connect a dynamic State Estimation system with the optimization scheme and prove that this whole interconnection stays stable and converges to the right solution.
Dev: That's a big deal because most feedback optimization schemes assume you already know all the states perfectly. This work shows how to handle that uncertainty by using available measurements to build an estimate, which is then fed back into the controller.
Taro: From my side, I’m interested in how robust this setup is when the world misbehaves and things get noisy, because that's where real-world deployment gets tricky.
Rosa: Exactly. The paper suggests they build a dynamic State Estimation using available measurements to get an unbiased estimate called V hat t, which then feeds into the optimization scheme.
Dev: And they prove that this interconnection is stable and converges exponentially towards both the true state values and the optimal set-points, which is pretty strong for a real-time control loop.
Taro: Exponential convergence sounds good on paper, but how fast does that actually happen when you're dealing with actual grid conditions? And what about when things are really unstable?
Rosa: The paper gives specific convergence rates for both the estimation error and the optimization error. For the estimation error, it says it falls with a factor of sigma hat sigma sub bish t to the power of one minus psi sub bish t times sigma hat e zero squared, as time goes to infinity.
Title and authors: Dev: That means for that part, we see convergence towards zero as time goes on, and they also give a bound on the resulting stochastic error for the optimization part. It converges with a factor involving t to the power of one minus psi sub bish t times sigma hat e zero squared plus some other terms.
Taro: So, if I'm just listening to this, what does that mean for me in terms of control? Does it mean my system will reliably get close to the right power levels even if the measurements aren't perfect?
Rosa: It means that even with imperfect measurements, as long as you have those real-time feedback measurements, the system is guaranteed to drive your controllable injections towards near-optimal set-points. The validation on a simulated test feeder showed that both errors decreased to low values, less than one percent for the estimation case.
Dev: And they maintained accuracy even when there were model mismatches because of this real-time feedback optimization correcting those inaccuracies. They also found that voltage magnitudes stayed within limits almost all the time during the simulation.
Taro: That’s interesting about robustness against model mismatch, but what about the practical application? The paper mentions their feasibility set F might be time-varying because of things like solar radiation and wind speed profiles, so how does that affect your system's performance?
Rosa: They acknowledge that the feasible set F isn't constant due to those changing profiles. They mention that the constants CV and CS are increasing with respect to things like trace of sigma sub l and trace of sigma sub y, but they can be lowered by using faster time scales.
Dev: That points toward a practical consideration: you might be able to speed up the convergence by running the estimation and optimization at a higher frequency, but that brings up latency concerns for real-time execution.
Taro: If we speed things up, does that make the system more sensitive to noise in those faster measurements? I mean, if you're dealing with fast time scales, you need to make sure the system doesn't just start oscillating wildly.
Title and authors: Rosa: They suggest that because the convergence of V t depends on the convergence of Sc t, and they showed that result using bounds involving r sub n times the integral of Sl t plus some terms involving sigma hat e zero squared, it suggests a controlled rate of change.
Dev: So, to wrap up this section, this paper on "Closing the Loop: Dynamic State Estimation and Feedback Optimization of Power Grids" formally proves that connecting dynamic state estimation with online feedback optimization is stable and converges in expectation towards the true state values and optimal set-points.
Taro: It sounds like a solid theoretical foundation for making these systems reliable, but I'm still waiting to see if it holds up when we throw actual noise and weird load changes at it outside of a simulation.
Rosa: Well, that’s the next step for them—considering nonlinear power flow and measurements with things like biased pseudomeasurements and process noise to include non-zero mean time-varying loads.
Dev: That would definitely test the stability guarantees they proved in Appendix A against those more realistic, messy conditions.
Taro: I’m curious if they can use that uncertainty in the estimation error covariance matrix to actively increase voltage restrictions, which could further reduce violations when the world gets unpredictable.
Rosa: They also noted that because CV and CS are monotone increasing with respect to things like trace of sigma sub l and trace of sigma sub y, you might be able to lower those constants by using faster time scales.
Dev: It’s a good path forward for making the control loop faster while maintaining stability, provided the underlying assumptions about the system dynamics still hold under those faster rates.
Taro: So, it looks like this paper lays out a solid mathematical proof for connecting estimation and optimization, giving us confidence in convergence rates even when dealing with imperfect information.
Rosa: That’s right; it gives us confidence that we can use real-time measurements to drive power injections towards optimal set-points reliably in changing conditions.
Dev: And the next paper we look at will be on how to handle those more complex nonlinear scenarios and non-ideal measurements that are more common in practice.
The paper's summary: Rosa: So, we just talked about how this paper is proving that linking state estimation and feedback optimization is stable even when things are messy, right?
Dev: Right. And what this paper really does is formally prove that if you connect your grid dynamics with a dynamic state estimation system and an online feedback optimization scheme, the whole thing stays stable and converges to the best possible solution in steady state.
Rosa: It’s about taking those real-time measurements you get at every time step and using them as feedback to constantly adjust what the optimal power set points should be, even when you don't have perfect information about everything.
Dev: Exactly. The core contribution here is that they formally prove that this interconnection—the grid dynamics, the state estimation, and the feedback optimization—is stable and reaches a steady state optimal solution.
Rosa: But what this means for us on the ground is that we don't have to assume we have perfect measurements of every single thing in the power grid to get good control. You can use whatever you measure right now to correct your plan for the next second, and they guarantee it will eventually settle down at a good spot.
Dev: The paper gives us specific numbers on how fast this happens. For the estimation error, it shows that its expected value falls with the square of the sampling interval times some factor related to sigma sub bish t to the power of one minus psi sub bish t times sigma hat e zero squared as time goes on.
Rosa: That means for just getting your state estimates right, you see convergence towards zero as time keeps going, and they even give a bound on the resulting stochastic error for your optimization part.
Dev: And for the optimization error, it converges with a factor involving t to the power of one minus psi sub bish t times sigma hat e zero squared plus some other terms. It shows that your control input converges to near-optimal set points over time, even when there's noise in your measurements.
Rosa: I mean, this is cool because it means you can keep driving those controllable power injections towards the optimal settings in real time, adapting to changes and even model mismatches because you're using the current grid state as feedback to fix those inaccuracies.
Dev: And they validated this on a simulated test feeder, showing that both the estimation error and optimization errors dropped to low values, under one percent for the estimation case, and stayed stable. The accuracy holds up even when there are model mismatches because of how robust this real-time feedback optimization is.
Rosa: It also showed that voltage magnitudes stayed within limits almost all the time during that simulation, which is a big win for reliability in practice. But they do flag some things for future work, like dealing with nonlinear power flow and measurements with noise or biased pseudomeasurements to include time-varying loads.
Dev: Right. They noted that the feasibility set F isn't constant because of profiles from things like solar radiation and wind speed, which means the constants CV and CS can change over time. But they also pointed out you can lower those constants by using faster time scales if you need to run it quicker, though that brings up latency concerns for real-time execution.
Rosa: So basically, this paper gives us a solid mathematical foundation proving that connecting dynamic state estimation with online feedback optimization is stable and converges in expectation towards the true state values and optimal set points. But the next step is testing those guarantees against more realistic conditions, like non-linear flow and noisy measurements, which I think will be really interesting for real deployment.
The paper's improvements: Taro: So, we just talked about how this paper proves stability in connecting state estimation and feedback optimization when things are changing fast, right?
Rosa: Right. And what they suggest moving forward is tackling the more messy realities of power grids where you have noisy measurements and non-linear power flow.
Dev: They actually look at incorporating those real-world issues into the framework by considering biased pseudomeasurements and process noise to include non-zero mean time-varying loads. That’s a big step because it moves away from the perfect, clean data they used in the initial proof.
Rosa: That makes sense. If you can model that mess—the noise and the changing load patterns—then you can test if their stability guarantees still hold when your sensors aren't perfect or when things are constantly shifting.
Dev: Exactly. They also have a suggestion about using the uncertainty represented in that estimation error covariance matrix to actively increase voltage restrictions, which could further reduce violations if you’re seeing high uncertainty there. That’s a proactive way to manage risk rather than just reacting to violations after they happen.
Taro: That sounds like a good safety feature. If the AI knows it's not sure about a state, it should probably slow down or tighten the constraints on that voltage immediately, not wait for the error to grow too large.
Rosa: It’s about using that mathematical uncertainty as an input to make smarter decisions in real time. And they also mentioned something interesting about their constants, CV and CS. They said those constants are increasing with respect to things like the trace of sigma sub l and the trace of sigma sub y, but you can lower those by using faster time scales.
Dev: That means if you can run your estimation and optimization at a higher frequency—faster time scales—you might be able to get a tighter control loop without sacrificing stability, provided those underlying assumptions about the system dynamics still hold up under that faster rate.
Taro: So, the implication is you have two ways to push performance: either you can make the estimation more robust by using uncertainty to tighten restrictions, or you can try speeding things up by using faster time scales. Which path is better depends on how much latency your hardware allows.
Rosa: That’s the trade-off there. It’s about finding that sweet spot where you get better accuracy with less computational lag, which is what matters when you're driving a physical system.
Dev: So, to summarize their improvements: they suggest using uncertainty for active restriction tightening and exploring faster time scales to lower convergence constants, all while pushing the model toward handling non-linear flow and noisy measurements.
Taro: It’s a solid plan for moving from the lab simulation to something that actually runs in a power grid environment where things are never clean or static.
Conclusion: Rosa: So we've covered how this paper proves that connecting dynamic state estimation with online feedback optimization is stable and converges in expectation towards the true state values and optimal set points, right?
Dev: Yeah. The main point is that you can use real-time measurements to drive power injections towards near-optimal set points reliably even when things are changing fast or you don't have perfect data.
Taro: I just want to emphasize that the robustness they proved is what matters most for me; it’s about how gracefully the system recovers when the world misbehaves, not just how fast it converges in a perfect scenario.
Rosa: Right. They showed this works on a simulated test feeder, with estimation errors dropping below one percent and voltage staying within limits almost all the time. That means we can trust this setup for real-time control under imperfect conditions.
Dev: The numbers show the estimation error falls with the square of the sampling interval times some factor related to sigma sub bish t to the power of one minus psi sub bish t times sigma hat e zero squared as time goes on. It converges, even if you’re running at a higher frequency, as long as that factor is less than one.
Taro: If we are pushing for faster time scales, we need to be careful about how much that convergence factor drops; that could be the point where your control loop becomes too sensitive to noise in the measurements.
Rosa: That’s a valid concern. They also mentioned you can lower those convergence constants by using faster time scales, but they also pointed out you need to consider things like biased pseudomeasurements and process noise for future work to see how this holds up in truly messy scenarios.
Dev: The safety case here is that the interconnection is formally proven stable, and the bounds they provided on the stochastic error give us a concrete idea of what level of control accuracy we can expect over time.
Taro: It changes things for autonomy because it gives us a mathematical way to guarantee that even if your perception or estimation gets fuzzy, the control action will still trend towards what is optimal and safe.
Rosa: So, to wrap up this paper on "Closing the Loop: Dynamic State Estimation and Feedback Optimization of Power Grids," it formally proves that linking dynamic state estimation with online feedback optimization is stable and converges in expectation towards the true state values and optimal set points.
Dev: It’s a strong piece of theory for making these real-time control loops more dependable under uncertainty.
Taro: I think the next step will be testing those stability guarantees against those non-linear power flow scenarios we talked about, because that's where the real world gets complicated.
More episodes
- 2608.13438-ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
- 2610.34593-Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies
- 2610.35385-EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
- 2610.36378-Data-Driven Hamiltonian for Neural Optimal Control
- 2610.37275-A Heuristic for Scheduling Autonomous Vehicles at Intersections
- 2610.37288-Dynamic Coordination of Autonomous Vehicles at Intersections
- 2610.37632-Real-Time Coordination of Autonomous Vehicles at Intersections
- 2611.13779-Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
- 2611.13411-Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
- 2611.14488-Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies