Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation".
Dev: Here's the gist of the paper. In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e.g., at subsecond time scales.
Rosa: The proposed method is a model-free OFO approach that sequentially estimates a time-varying sensitivity while operating the grid, bypassing the need to know the whole grid model accurately and having full grid observability.
Dev: The core problem addressed The increasing amount of controllable yet sometimes unpredictable power resources in electrical grids leads to new challenges and opportunities in operation.
Rosa: Fast control decisions are required to avoid constraint violations such as overvoltages, especially in distribution grids.
Title and authors: Rosa: This paper is by Miguel Picallo, Lukas Ortmann, Saverio Bolognani, and Florian Dorfler from ETH Zurich. They’re tackling the challenge of real-time grid operation using this Online Feedback Optimization with Sensitivity Estimation technique.
Dev: And what they do is propose a method that lets the controller track the solution of a time-varying Optimal Power Flow problem—that's the AC-OPF—even when you don't have all those expensive, high-fidelity grid models ready to go.
Taro: So, if we think about what this means practically, it suggests we can get fast enough control decisions for distribution grids where things change constantly and constraints like voltage limits are really tight.
Rosa: Precisely. The main implication is that you don't need a perfect grid model to have safe and optimal control at subsecond time scales, which is crucial for handling unpredictable power resources in modern grids.
Dev: And they focus on the fact that because an inaccurate sensitivity estimate can cause performance issues, they build this learning mechanism—the recursive least squares estimation—to ensure the controller adapts as things change during operation.
The paper's summary: Rosa: The summary of this paper explains that instead of needing a full grid model, which is often unavailable or too expensive for distribution systems, the Online Feedback Optimization approach only requires knowing the steady-state sensitivity relating changes in controllable inputs to changes in measured outputs.
Dev: They then propose using a recursive least squares estimation based on measuring the change in outputs caused by changing controllable inputs as their feedback signal to learn this time-varying sensitivity during real-time operation.
Taro: So, the mechanism is that they are sequentially estimating this sensitivity while the grid is actually running, which bypasses the need to know the whole system model accurately and gives them full grid observability.
Rosa: That’s a big deal for deployment because it turns OFO into a model-free approach, meaning you don't have to spend all that time building an accurate nonlinear grid model just to run the control.
Dev: They combine this sensitivity estimation with a persistently exciting OFO controller, which gathers enough information about the sensitivity while driving the control inputs directly toward the AC-OPF solution.
The paper's improvements: Rosa: One of the main improvements they highlight is that by using this learned sensitivity estimate, the controller preserves stability even when you use a constant power flow sensitivity approximation instead of the actual one during operation.
Dev: That means they are robust against model mismatch because the system is actively learning what it needs to know about those sensitivities through real-time measurements from output variations and input changes.
Taro: From my side, I see that this is important for failure modes because if something unexpected happens, like a sudden change in topology or a line fault, the controller has learned enough to react gracefully instead of breaking down because it's not relying on a fixed guess.
Rosa: And they provide convergence guarantees—they certify that both the estimated sensitivity and the actual control input are converging toward their true values over time, even when there’s time variation or some noise in the measurements.
Conclusion: Dev: So, to wrap up, this paper on "Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation" suggests we can use this model-free plug-and-play controller for real-time power grid operation that allows for safe and optimal control.
Rosa: The big picture here is that because of the convergence guarantees they provide, this method can be used to maintain optimal power flow set points at subsecond time scales without needing a complete grid model or an accurate admittance matrix.
Taro: It’s interesting how they link persistent excitation design to specific aspects of the problem, like topology changes or communication issues. That suggests we can target where the learning is most needed when unforeseen operational conditions arise.
Dev: Right, so this work provides quantifiable tracking error bounds that depend on things like the persistency of excitation and measurement noise characteristics, which gives us a clearer safety case for how reliable this system is in practice.
Rosa: It’s a solid piece of work for making real-time grid operation more adaptable by letting the controller learn its environment instead of just relying on a fixed setup.
Miguel Picallo, Lukas Ortmann, Saverio Bolognani, Florian Dorfler
Automatic Control Laboratory, ETH Zurich
eess.SY, cs.SY
Submitted: 2021-10-03
Updated: 2022-05-16
Journal ref: Electric Power Systems Research, 2022 212, 108405
DOI: 10.1016/j.epsr.2022.108405
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 87/100
The gist: In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e.g…
Key concepts
- Online Feedback Optimization (OFO)
- A computationally efficient method used to track the solutions of a complex optimization problem, like AC-OPF, in real-time. It operates at fast control loop rates, allowing the system to adapt quickly to changes in the grid without needing a complete mathematical model of every component.
- Sensitivity Estimation
- The core innovation where the controller learns how much the output (like voltage) changes when inputs (like power flow commands) change. This is done recursively using measurements and a least-squares approach, allowing the system to estimate time-varying relationships without knowing the full grid model beforehand.
- Model-Free Approach
- A control strategy that does not require an accurate, complete mathematical model of the entire electrical grid. Instead, it learns the necessary system dynamics—the sensitivity—directly from real-time measurements during operation, making it adaptable to unpredictable resources and constraints.
- Persistency of Excitation
- A condition ensuring that the control inputs provide enough varied information over time so that all relevant aspects of the system's behavior can be observed. This mathematical requirement guarantees that the sensitivity estimates converge reliably to their true values.
Terminology
Summary
In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e. p. 0 Adaptive Real-Time Grid Operation via Online
Feedback Optimization with Sensitivity Estimation
Miguel Picallo∗
, Lukas Ortmann∗
, Saverio Bolognani, Florian Dorfler ¨
Automatic Control Laboratory, ETH Zurich, 8092 Zurich, Switzerland
{miguelp,ortmannl,bsaverio,dorfler}@ethz. ch
Abstract—In this paper we propose an approach based on
an Online Feedback Optimization (OFO) controller with grid
input-output sensitivity estimation for real-time grid operation,
e. ch
Abstract—In this paper we propose an approach based on
an Online Feedback Optimization (OFO) controller with grid
input-output sensitivity estimation for real-time grid operation,
e. , at subsecond time scales. However, [16] requires a sufficient
time-scale separation between the sensitivity estimation procedure and the feedback optimization, which may lower the
convergence rate of the entire approach if the measurement
sample rate is restricted due to communication limits. Therefore, in this paper, with a similar spirit as in the extremum seeking approach [16], we propose a model-free OFO
approach that sequentially estimates a time-varying sensitivity
while operating the grid, bypassing the need to know the whole
grid model accurately, and to have full grid observability.Where the paper says it
Where the paper says it
Where the paper says it
The core problem addressed
The increasing amount of controllable yet sometimes unpredictable power resources in electrical grids leads to new challenges and opportunities in operation. Fast control decisions are required to avoid constraint violations such as overvoltages, especially in distribution grids. p. 0 On the other hand,
given the volatility and unpredictability of these resources, fast
control decisions are required to avoid constraint violations,
e. , overvoltages. Unfortunately, standard
AC-OPF requires a) full grid observability, e. , measurements
of all active and reactive power injections and consumptions,
and b) an accurate nonlinear grid model, e. Yet, solving a computationally
expensive AC-OPF may pose a limit on these rates. Online Feedback Optimization (OFO) [4]–[6] is a novel
computationally efficient approach that allows to track the
solutions of an AC-OPF problem under time-varying conditions using subsecond control-loop rates.Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The authors propose a recursive least-squares estimation to learn the sensitivity from measurements during real-time operation, turning OFO into a model-free approach. They design a sensitivity learning approach via recursive least squares using the change in the outputs caused by a change of the controllable inputs. p. 1 Our
contributions are as follows: First, we design a sensitivity
learning approach via recursive least squares [17], [18]. We
use as measurements the change in the outputs caused by a
change of the controllable inputs. We make the
part Σp2
of the process noise proportional to k∆utk2, since
a large ∆ut can trigger a larger change in the true sensitivity
∇uh(u, d) that depends on u, and the part Σp1
independent
of ∆ut to account for a uncontrolled random change ∆dt =
dt+1 − dt that can affect the sensitivity as well. Next, to derive a measurement equation for the sensitivity
Ht, consider the first-order Taylor approximation of yt
yt
z }| {
h(ut, dt) ≈
yt−1
z }| {
h(ut−1, dt−1) +
Ht−1
z }| {
∇uh(ut−1, dt−1) ∆ut−1
+ ∇dh(ut−1, dt−1)∆dt−1. We compute
the new sensitivity estimate hˆ
t through a Bayesian update
represented in the following least-squares problem [17], [18]:
hˆ
t = arg min
hˆ
khˆ − hˆ
t−1k
2
Σ
−1
t−1
+ k∆yt−1 − U∆,t−1hˆk
2
Σ
−1
m,t−1
,
where Σt is the covariance matrix representing the uncertainty
of the sensitivity estimate hˆ
t, and kxk
2
A = x
T Ax is the norm
of x with respect to a positive definite matrix A. The resulting
recursive estimation can be expressed as a Kalman filter [24]:
hˆ
t =hˆ
t−1 + Kt−1(∆yt−1 − U∆,t−1hˆ
t−1)
Σt =
1 − Kt−1U∆,t−1
Σt−1 + Σp,t−1,
(9)
where 1 is the identity matrix, and Kt = ΣtU
T
∆,t(Σm,t +
U∆,tΣtU
T
∆,t)
−1
is the Kalman gain, which is well defined for
an invertible Σm,t, see later Assumption 1.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The approach combines this sensitivity estimation with a persistently exciting OFO that gathers enough information about the sensitivity while driving control inputs towards the AC-OPF solutions. The persistency of excitation condition ensures that there exists a time span T > 0 such that for all t > 0, the matrix formed by columns ∆ut for i ∈ (0,. p. 3 e, we
need to use different ∆u to explore different reactions ∆y and
infer different elements of Ht from them. This can be formalized via the persistency of excitation condition [25]: ∆ut is
persistently exciting if there exists a time span T > 0, such
that for all t > 0, the matrix formed by columns ∆ut+i for i ∈
{0, . This can be formalized via the persistency of excitation condition [25]: ∆ut is
persistently exciting if there exists a time span T > 0, such
that for all t > 0, the matrix formed by columns ∆ut+i for i ∈
{0, . , T} has full rank, i. In conclusion,
Assumption 1 is reasonable. Then, with a persistently exciting
∆u as in (10), we have the following convergence result:
Proposition 1: Under Assumption 1, and the persistently
excited OFO updates (10), the sensitivity estimates (9) satisfy:
Unbiased mean: kE[ht − hˆ
t]k
2
2 ≤ Ch,1e
−Ch,2t
t→∞→ 0
Bounded covariance: E[kht − hˆ
tk
2
2
] = tr(Σt)
≤ Ch,3 + Ch,4e
−Ch,5t→Ch,3,
(11)
where E[·] denotes the expectation, Ch,i > 0 are positive
constants, and t→∞→ the limit as t goes to infinity.Where the paper says it
Where the paper says it
Where the paper says it
Validation and performance
The proposed OFO controller with sensitivity estimation is validated on the IEEE 123-bus test feeder using real consumption data. The results show that the estimated sensitivity Hˆ t converges to the real time-varying sensitivity H t and the input u t converges to the AC-OPF solution u∗. p. 5 6
in Proposition 1: First, the estimated sensitivity Hˆ
t converges
to the real time-varying sensitivity Ht. Second, the input ut
converges to the AC-OPF solution u
∗
t
(2). We analytically certify the convergence of the proposed OFO
with sensitivity estimation, and validate its performance on a
simulation using the IEEE 123-bus test feeder, and comparing it
against a state-of-the-art OFO with constant sensitivity. Index Terms—Online Feedback Optimization, Real-time AC
Optimal Power Flow, Recursive Estimation, Voltage Regulation
I.Where the paper says it
Where the paper says it
Conclusion and outlook
The proposed OFO controller with sensitivity estimation can be used as a model-free plug-and-play controller for real-time power grid operation that enables safe and optimal control. The work provides convergence guarantees when approximating the time-varying sensitivity behavior by a random process with linear measurements. p. 5 In
this work we have proposed a recursive estimation approach
that provides Online Feedback Optimization (OFO) with a
tool to learn the model sensitivity without extensive measurements, and thus improves its performance and turns OFO
into a model-free approach. We have provided convergence
guarantees when approximating the time-varying sensitivity
behavior by a random process with linear measurements. Both Hˆ and H0 are initialized with an perturbed admittance
matrix Y . suitable way to design the persistency of excitation, possibly
linked to the optimization problem, so that it explores specific
directions of interest. suitable way to design the persistency of excitation, possibly
linked to the optimization problem, so that it explores specific
directions of interest. Additionally, it would be interesting
to observe how the proposed sensitivity estimation approach
performs under a sudden change of topology caused by, e. Additionally, it would be interesting
to observe how the proposed sensitivity estimation approach
performs under a sudden change of topology caused by, e. , a
line fault, network split, etc.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Model-free real-time grid operation via Online Feedback Optimization with sensitivity estimation allows AI systems to maintain optimal power flow set-points at subsecond time scales without requiring a full grid model or accurate admittance matrix. The improved system can track the solution of a time-varying AC Optimal Power Flow problem by recursively estimating the time-varying sensitivity, turning OFO into a model-free approach.
-
Robustness against model mismatch is enhanced because the system sequentially updates its sensitivity estimate H hat t using a recursive least squares approach based on measured output variations delta y and input changes delta u. This allows the control system to learn the dependency of output on inputs during real-time operation, bypassing the need for extensive prior measurements or an accurate nonlinear model.
-
The AI controller can perform superior voltage regulation compared to state-of-the-art methods by using the learned sensitivity H hat t instead of a constant sensitivity approximation H 0. This leads to lower optimization errors and fewer voltage violations when operating on real, time-varying grid data, as shown in simulation results.
-
The system achieves convergence guarantees for both the estimated sensitivity and the control input towards their true counterparts, even under time-varying conditions and noise. This provides quantifiable tracking error bounds that depend on the persistency of excitation and the noise characteristics of the measurement process.
-
The controller can be designed to explore specific directions of interest in the optimization landscape by linking persistent excitation design to specific aspects of the problem, such as topology changes or communication problems. This allows for targeted learning where it is most needed, improving adaptability under unforeseen operational conditions.
Abstract
In this paper we propose an approach based on an Online Feedback Optimization (OFO) controller with grid input-output sensitivity estimation for real-time grid operation, e.g., at subsecond time scales. The OFO controller uses grid measurements as feedback to update the value of the controllable elements in the grid, and track the solution of a time-varying AC Optimal Power Flow (AC-OPF). Instead of relying on a full grid model, e.g., grid admittance matrix, OFO only requires the steady-state sensitivity relating a change in the controllable inputs, e.g., power injections set-points, to a change in the measured outputs, e.g., voltage magnitudes. Since an inaccurate sensitivity may lead to a model-mismatch and jeopardize the performance, we propose a recursive least-squares estimation that enables OFO to learn the sensitivity from measurements during real-time operation, turning OFO into a model-free approach. We analytically certify the convergence of the proposed OFO with sensitivity estimation, and validate its performance on a simulation using the IEEE 123-bus test feeder, and comparing it against a state-of-the-art OFO with constant sensitivity.
Sources
- Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges
- Cross-layer Design for Real-Time Grid Operation: Estimation, Optimization and Power Flow
Related papers
- Industrial Load Modeling and Optimization for Market-Based Interaction with Power Systems
- Exact and Approximate Convex Reformulation of Linear Stochastic Optimal Control with Chance Constraints
- Certified Set Convergence for Piecewise Affine Systems
- Symbolic Control for Stochastic Systems via Finite Parity Games
- Multiobjective optimization-based design and dispatch of islanded, hybrid microgrids for remote, off-grid communities in sub-Saharan Africa
- Physiology-informed layered sensing for intelligent human-exoskeleton interaction