Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation

arXiv:2110.00954 · eess.SY, cs.SY · Submitted 2021-10-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation".

Dev: Here's the gist of the paper. In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e.g., at subsecond time scales.

Rosa: The proposed method is a model-free OFO approach that sequentially estimates a time-varying sensitivity while operating the grid, bypassing the need to know the whole grid model accurately and having full grid observability.

Dev: The core problem addressed The increasing amount of controllable yet sometimes unpredictable power resources in electrical grids leads to new challenges and opportunities in operation.

Rosa: Fast control decisions are required to avoid constraint violations such as overvoltages, especially in distribution grids.

Title and authors: Rosa: This paper is by Miguel Picallo, Lukas Ortmann, Saverio Bolognani, and Florian Dorfler from ETH Zurich. They’re tackling the challenge of real-time grid operation using this Online Feedback Optimization with Sensitivity Estimation technique.

Dev: And what they do is propose a method that lets the controller track the solution of a time-varying Optimal Power Flow problem—that's the AC-OPF—even when you don't have all those expensive, high-fidelity grid models ready to go.

Taro: So, if we think about what this means practically, it suggests we can get fast enough control decisions for distribution grids where things change constantly and constraints like voltage limits are really tight.

Rosa: Precisely. The main implication is that you don't need a perfect grid model to have safe and optimal control at subsecond time scales, which is crucial for handling unpredictable power resources in modern grids.

Dev: And they focus on the fact that because an inaccurate sensitivity estimate can cause performance issues, they build this learning mechanism—the recursive least squares estimation—to ensure the controller adapts as things change during operation.

The paper's summary: Rosa: The summary of this paper explains that instead of needing a full grid model, which is often unavailable or too expensive for distribution systems, the Online Feedback Optimization approach only requires knowing the steady-state sensitivity relating changes in controllable inputs to changes in measured outputs.

Dev: They then propose using a recursive least squares estimation based on measuring the change in outputs caused by changing controllable inputs as their feedback signal to learn this time-varying sensitivity during real-time operation.

Taro: So, the mechanism is that they are sequentially estimating this sensitivity while the grid is actually running, which bypasses the need to know the whole system model accurately and gives them full grid observability.

Rosa: That’s a big deal for deployment because it turns OFO into a model-free approach, meaning you don't have to spend all that time building an accurate nonlinear grid model just to run the control.

Dev: They combine this sensitivity estimation with a persistently exciting OFO controller, which gathers enough information about the sensitivity while driving the control inputs directly toward the AC-OPF solution.

The paper's improvements: Rosa: One of the main improvements they highlight is that by using this learned sensitivity estimate, the controller preserves stability even when you use a constant power flow sensitivity approximation instead of the actual one during operation.

Dev: That means they are robust against model mismatch because the system is actively learning what it needs to know about those sensitivities through real-time measurements from output variations and input changes.

Taro: From my side, I see that this is important for failure modes because if something unexpected happens, like a sudden change in topology or a line fault, the controller has learned enough to react gracefully instead of breaking down because it's not relying on a fixed guess.

Rosa: And they provide convergence guarantees—they certify that both the estimated sensitivity and the actual control input are converging toward their true values over time, even when there’s time variation or some noise in the measurements.

Conclusion: Dev: So, to wrap up, this paper on "Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation" suggests we can use this model-free plug-and-play controller for real-time power grid operation that allows for safe and optimal control.

Rosa: The big picture here is that because of the convergence guarantees they provide, this method can be used to maintain optimal power flow set points at subsecond time scales without needing a complete grid model or an accurate admittance matrix.

Taro: It’s interesting how they link persistent excitation design to specific aspects of the problem, like topology changes or communication issues. That suggests we can target where the learning is most needed when unforeseen operational conditions arise.

Dev: Right, so this work provides quantifiable tracking error bounds that depend on things like the persistency of excitation and measurement noise characteristics, which gives us a clearer safety case for how reliable this system is in practice.

Rosa: It’s a solid piece of work for making real-time grid operation more adaptable by letting the controller learn its environment instead of just relying on a fixed setup.

Miguel Picallo, Lukas Ortmann, Saverio Bolognani, Florian Dorfler

Automatic Control Laboratory, ETH Zurich

eess.SY, cs.SY

Submitted: 2021-10-03

Updated: 2022-05-16

Journal ref: Electric Power Systems Research, 2022 212, 108405

DOI: 10.1016/j.epsr.2022.108405

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 87/100

The gist: In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e.g…

Key concepts

Online Feedback Optimization (OFO)
A computationally efficient method used to track the solutions of a complex optimization problem, like AC-OPF, in real-time. It operates at fast control loop rates, allowing the system to adapt quickly to changes in the grid without needing a complete mathematical model of every component.
Sensitivity Estimation
The core innovation where the controller learns how much the output (like voltage) changes when inputs (like power flow commands) change. This is done recursively using measurements and a least-squares approach, allowing the system to estimate time-varying relationships without knowing the full grid model beforehand.
Model-Free Approach
A control strategy that does not require an accurate, complete mathematical model of the entire electrical grid. Instead, it learns the necessary system dynamics—the sensitivity—directly from real-time measurements during operation, making it adaptable to unpredictable resources and constraints.
Persistency of Excitation
A condition ensuring that the control inputs provide enough varied information over time so that all relevant aspects of the system's behavior can be observed. This mathematical requirement guarantees that the sensitivity estimates converge reliably to their true values.

Terminology

Summary

In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e. p. 0

Where the paper says it

Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation Miguel Picallo∗ , Lukas Ortmann∗ , Saverio Bolognani, Florian Dorfler ¨ Automatic Control Laboratory, ETH Zurich, 8092 Zurich, Switzerland {miguelp,ortmannl,bsaverio,dorfler}@ethz. ch Abstract—In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e.

g., at subsecond time scales. p. 0
Where the paper says it

ch Abstract—In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e. , at subsecond time scales.

The proposed method is a model-free OFO approach that sequentially estimates a time-varying sensitivity while operating the grid, bypassing the need to know the whole grid model accurately and having full grid observability. p. 1
Where the paper says it

However, [16] requires a sufficient time-scale separation between the sensitivity estimation procedure and the feedback optimization, which may lower the convergence rate of the entire approach if the measurement sample rate is restricted due to communication limits. Therefore, in this paper, with a similar spirit as in the extremum seeking approach [16], we propose a model-free OFO approach that sequentially estimates a time-varying sensitivity while operating the grid, bypassing the need to know the whole grid model accurately, and to have full grid observability.

The core problem addressed

The increasing amount of controllable yet sometimes unpredictable power resources in electrical grids leads to new challenges and opportunities in operation. Fast control decisions are required to avoid constraint violations such as overvoltages, especially in distribution grids. p. 0

Where the paper says it

On the other hand, given the volatility and unpredictability of these resources, fast control decisions are required to avoid constraint violations, e. , overvoltages.

Standard AC-OPF requires full grid observability and an accurate nonlinear grid model which are often unavailable or unaffordable at the distribution system level. p. 0
Where the paper says it

Unfortunately, standard AC-OPF requires a) full grid observability, e. , measurements of all active and reactive power injections and consumptions, and b) an accurate nonlinear grid model, e.

Online Feedback Optimization (OFO) is a computationally efficient approach that tracks the solutions of an AC-OPF problem under time-varying conditions using subsecond control-loop rates. p. 0
Where the paper says it

Yet, solving a computationally expensive AC-OPF may pose a limit on these rates. Online Feedback Optimization (OFO) [4]–[6] is a novel computationally efficient approach that allows to track the solutions of an AC-OPF problem under time-varying conditions using subsecond control-loop rates.

How the training signal is built

The authors propose a recursive least-squares estimation to learn the sensitivity from measurements during real-time operation, turning OFO into a model-free approach. They design a sensitivity learning approach via recursive least squares using the change in the outputs caused by a change of the controllable inputs. p. 1

Where the paper says it

Our contributions are as follows: First, we design a sensitivity learning approach via recursive least squares [17], [18]. We use as measurements the change in the outputs caused by a change of the controllable inputs.

The measurement equation for the sensitivity Ht is derived from a first-order Taylor approximation of yt. p. 2
Where the paper says it

We make the part Σp2 of the process noise proportional to k∆utk2, since a large ∆ut can trigger a larger change in the true sensitivity ∇uh(u, d) that depends on u, and the part Σp1 independent of ∆ut to account for a uncontrolled random change ∆dt = dt+1 − dt that can affect the sensitivity as well. Next, to derive a measurement equation for the sensitivity Ht, consider the first-order Taylor approximation of yt yt z }| { h(ut, dt) ≈ yt−1 z }| { h(ut−1, dt−1) + Ht−1 z }| { ∇uh(ut−1, dt−1) ∆ut−1 + ∇dh(ut−1, dt−1)∆dt−1.

The estimation is updated using a Bayesian update represented in a least-squares problem which can be expressed as a Kalman filter [24]. p. 2
Where the paper says it

We compute the new sensitivity estimate hˆ t through a Bayesian update represented in the following least-squares problem [17], [18]: hˆ t = arg min hˆ khˆ − hˆ t−1k 2 Σ −1 t−1 + k∆yt−1 − U∆,t−1hˆk 2 Σ −1 m,t−1 , where Σt is the covariance matrix representing the uncertainty of the sensitivity estimate hˆ t, and kxk 2 A = x T Ax is the norm of x with respect to a positive definite matrix A. The resulting recursive estimation can be expressed as a Kalman filter [24]: hˆ t =hˆ t−1 + Kt−1(∆yt−1 − U∆,t−1hˆ t−1) Σt = 1 − Kt−1U∆,t−1 Σt−1 + Σp,t−1, (9) where 1 is the identity matrix, and Kt = ΣtU T ∆,t(Σm,t + U∆,tΣtU T ∆,t) −1 is the Kalman gain, which is well defined for an invertible Σm,t, see later Assumption 1.

Where the gains come from

The approach combines this sensitivity estimation with a persistently exciting OFO that gathers enough information about the sensitivity while driving control inputs towards the AC-OPF solutions. The persistency of excitation condition ensures that there exists a time span T > 0 such that for all t > 0, the matrix formed by columns ∆ut for i ∈ (0,. p. 3

Where the paper says it

e, we need to use different ∆u to explore different reactions ∆y and infer different elements of Ht from them. This can be formalized via the persistency of excitation condition [25]: ∆ut is persistently exciting if there exists a time span T > 0, such that for all t > 0, the matrix formed by columns ∆ut+i for i ∈ {0, .

.., T) has full rank. p. 3
Where the paper says it

This can be formalized via the persistency of excitation condition [25]: ∆ut is persistently exciting if there exists a time span T > 0, such that for all t > 0, the matrix formed by columns ∆ut+i for i ∈ {0, . , T} has full rank, i.

This leads to the convergence result where the sensitivity estimates satisfy Unbiased mean and Bounded covariance bounds as t→∞. p. 3
Where the paper says it

In conclusion, Assumption 1 is reasonable. Then, with a persistently exciting ∆u as in (10), we have the following convergence result: Proposition 1: Under Assumption 1, and the persistently excited OFO updates (10), the sensitivity estimates (9) satisfy: Unbiased mean: kE[ht − hˆ t]k 2 2 ≤ Ch,1e −Ch,2t t→∞→ 0 Bounded covariance: E[kht − hˆ tk 2 2 ] = tr(Σt) ≤ Ch,3 + Ch,4e −Ch,5t→Ch,3, (11) where E[·] denotes the expectation, Ch,i > 0 are positive constants, and t→∞→ the limit as t goes to infinity.

Validation and performance

The proposed OFO controller with sensitivity estimation is validated on the IEEE 123-bus test feeder using real consumption data. The results show that the estimated sensitivity Hˆ t converges to the real time-varying sensitivity H t and the input u t converges to the AC-OPF solution u∗. p. 5

Where the paper says it

6 in Proposition 1: First, the estimated sensitivity Hˆ t converges to the real time-varying sensitivity Ht. Second, the input ut converges to the AC-OPF solution u ∗ t (2).

Furthermore, it performs better than a state-of-the-art OFO with a constant sensitivity approximation in terms of tracking the AC-OPF solution and voltage regulation. p. 0
Where the paper says it

We analytically certify the convergence of the proposed OFO with sensitivity estimation, and validate its performance on a simulation using the IEEE 123-bus test feeder, and comparing it against a state-of-the-art OFO with constant sensitivity. Index Terms—Online Feedback Optimization, Real-time AC Optimal Power Flow, Recursive Estimation, Voltage Regulation I.

Conclusion and outlook

The proposed OFO controller with sensitivity estimation can be used as a model-free plug-and-play controller for real-time power grid operation that enables safe and optimal control. The work provides convergence guarantees when approximating the time-varying sensitivity behavior by a random process with linear measurements. p. 5

Where the paper says it

In this work we have proposed a recursive estimation approach that provides Online Feedback Optimization (OFO) with a tool to learn the model sensitivity without extensive measurements, and thus improves its performance and turns OFO into a model-free approach. We have provided convergence guarantees when approximating the time-varying sensitivity behavior by a random process with linear measurements.

Future additions could investigate a more suitable way to design the persistency of excitation, possibly linked to the optimization problem. p. 6
Where the paper says it

Both Hˆ and H0 are initialized with an perturbed admittance matrix Y . suitable way to design the persistency of excitation, possibly linked to the optimization problem, so that it explores specific directions of interest.

The paper also suggests observing how the proposed sensitivity estimation approach performs under a sudden change of topology caused by, e. p. 6
Where the paper says it

suitable way to design the persistency of excitation, possibly linked to the optimization problem, so that it explores specific directions of interest. Additionally, it would be interesting to observe how the proposed sensitivity estimation approach performs under a sudden change of topology caused by, e.

g., a line fault. p. 6
Where the paper says it

Additionally, it would be interesting to observe how the proposed sensitivity estimation approach performs under a sudden change of topology caused by, e. , a line fault, network split, etc.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Model-free real-time grid operation via Online Feedback Optimization with sensitivity estimation allows AI systems to maintain optimal power flow set-points at subsecond time scales without requiring a full grid model or accurate admittance matrix. The improved system can track the solution of a time-varying AC Optimal Power Flow problem by recursively estimating the time-varying sensitivity, turning OFO into a model-free approach.

  2. Robustness against model mismatch is enhanced because the system sequentially updates its sensitivity estimate H hat t using a recursive least squares approach based on measured output variations delta y and input changes delta u. This allows the control system to learn the dependency of output on inputs during real-time operation, bypassing the need for extensive prior measurements or an accurate nonlinear model.

  3. The AI controller can perform superior voltage regulation compared to state-of-the-art methods by using the learned sensitivity H hat t instead of a constant sensitivity approximation H 0. This leads to lower optimization errors and fewer voltage violations when operating on real, time-varying grid data, as shown in simulation results.

  4. The system achieves convergence guarantees for both the estimated sensitivity and the control input towards their true counterparts, even under time-varying conditions and noise. This provides quantifiable tracking error bounds that depend on the persistency of excitation and the noise characteristics of the measurement process.

  5. The controller can be designed to explore specific directions of interest in the optimization landscape by linking persistent excitation design to specific aspects of the problem, such as topology changes or communication problems. This allows for targeted learning where it is most needed, improving adaptability under unforeseen operational conditions.

Abstract

In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e.g., at subsecond time scales. The OFO controller uses grid measurements as feedback to update the value of the controllable elements in the grid, and track the solution of a time-varying AC Optimal Power Flow (AC-OPF). Instead of relying on a full grid model, e.g., grid admittance matrix, OFO only requires the steady-state sensitivity relating a change in the controllable inputs, e.g., power injections set-points, to a change in the measured outputs, e.g., voltage magnitudes. Since an inaccurate sensitivity may lead to a model-mismatch and jeopardize the performance, we propose a recursive least-squares estimation that enables OFO to learn the sensitivity from measurements during real-time operation, turning OFO into a model-free approach. We analytically certify the convergence of the proposed OFO with sensitivity estimation, and validate its performance on a simulation using the IEEE 123-bus test feeder, and comparing it against a state-of-the-art OFO with constant sensitivity.

Sources

Related papers