Guided Bayesian Optimization: Data-Efficient Controller Tuning with Digital Twin
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Guided Bayesian Optimization".
Dev: Here's the gist of the paper. Guided Bayesian Optimization (BO) presents an efficient data-driven method for iteratively tuning closed-loop controller parameters using a digital twin of the system The experimental results show that our method requires fewer experiments on the physical plant than Bayesian optimization to find the optimal controller parameters The proposed methodology is model-free, making it suitable for nonlinear and unmodelled plants with measurement noise The objective function consists of performance metrics modeled by Gaussian processes The method utilizes the available information in the closed-loop system to progressively maintain a digital twin that guides the optimizer, improving the data efficiency of our method Switching the digital twin on and off is triggered by our datadriven criteria related to the digital twin’s uncertainty estimations in the BO tuning framework Effectively, it replaces much of the exploration of the real system with exploration performed on the digital twin Digital Twin Construction The digital twin is built using closed-loop data acquired during standard BO iterations, and activated when the uncertainty in the Gaussian Process model of the optimization objective on the real system is high The DT plant model, denoted by G˜, is estimated based on D˜, which is a digital-twin data set constructed from informative data acquired in each experiment The fidelity of the DT plant model is estimated by Root Mean Square Error (RMSE) Guided BO Algorithm Algorithm 1 summarizes our guided BO method, which uses the information flow in the closed-loop system to define a DT The DT is activated when two conditions pvn(θ+) > η1, eRMSE < η2 are satisfied When the DT is activated, we use the DT instead of the actual system to estimate the cost value denoted by ˜Jˆ at ˜θ = θ+ This estimated performance J˜ is then added to the BO training data set in Line 11 of Algorithm 1 Performance Evaluation The guided BO with high-fidelity DT outperforms the nominal controller on average after approximately 2 experiments, whereas BO requires approximately 12 experiments The results on Table IV show that with the same number and frequency of DT integration, the guided BO with high-fidelity DT converges to a desired optimality ratio sooner than BO Guided BO requires only 10 experiments to perform the same, a 44% improvement The guided BO controller’s Kd is higher than the one obtained with BO, in line with the step responses shown in Fig.
Rosa: 15, causing smaller tracking error rate and a minor overshoot Controller Tuning Efficiency The results on Table VIII show that our guided BO is more time-efficient than the BO method because it requires fewer tedious experiments on the physical system and lower computational overdue by DT iterations The total tuning time for Guided BO on the rotary system is 503.54 s, compared to 785.19 s for BO The results of the total tuning time are shown in Table VIII Guided BO requires fewer experiments on the real DC rotary motor system and has a smaller variance between repetitions Indeed, the best among the 100 batches of guided BO accomplishes this in 3 experiments (compared to 7 experiments required for BO, a 57% improvement) We showed that our guided BO approach considerably improves the controller tuning data efficiency, generalizes across the system and industrial controller structures, and converges faster to optimum values We proved the guided BO’s superior performance on a noisy linear servo motor and DC rotary motor real-time hardware The digital twin carries out the exploration duty of the optimizer whereas the exploitation is performed on the real system The guided BO exploits more around the optimum cost, whereas BO explores further away from the optimum The paper is 2403.16619.
Paper summary: Rosa: So we're talking about this paper, "Guided Bayesian Optimization: Data-Efficient Controller Tuning with Digital Twin." Essentially, they're presenting a new way to tune closed-loop controller parameters using data. The big idea is to use a digital twin of the system to guide that tuning process.
Dev: Right, so the main claim is that this guided Bayesian optimization method requires fewer experiments on the physical plant than standard Bayesian optimization to find those optimal controller parameters. It's about improving data efficiency for tuning controllers, and it does this by using information from the closed-loop system to keep a digital twin updated.
Taro: I’m curious, how does that digital twin thing actually work when you're dealing with noisy systems? If the physical plant has measurement noise, how reliable is this virtual model guiding the optimizer?
Rosa: Well, they build this digital twin using closed-loop data gathered during the standard Bayesian optimization iterations. They activate it only when the uncertainty in that Gaussian process model of the objective function on the real system gets high.
Dev: That's a key part, isn't it? So when that condition is met, they switch to using this digital twin instead of actually testing the physical system to estimate the cost value. They call this estimated performance J-tilde at theta plus.
Taro: So instead of running another experiment on the physical hardware every time, they use what they have built up in that digital twin for the next step, which sounds like it cuts down on real-world testing a lot.
Rosa: Exactly. They're using the information available in the closed-loop system to progressively maintain this digital twin, and that digital twin then guides the optimizer, which improves how data efficient their method is. It replaces a lot of exploration in the real system with exploration on that digital twin instead.
Dev: And they define this framework independent of the actual controller or even the plant structure itself. The objective function they're trying to minimize, J-hat at theta plus, is defined as a weighted sum of performance metrics modeled by Gaussian processes.
Taro: So they model the performance using those Gaussian processes, but they can't just use an analytical formula for the cost value because getting real data from an industrial plant is often too expensive.
Rosa: That’s why they assume a black-box oracle for the closed-loop system that gives them that cost value J-hat given any parameter vector theta. The problem then becomes finding the best parameter vector theta star by minimizing J-hat over all possible parameters.
Paper summary: Dev: And they do this by solving for theta star as argmin theta epsilon Theta J-hat at theta epsilon, where epsilon is a noise realization from a normal distribution with zero mean and variance of sigma squared.
Taro: So the noise in the data collection process is something they have to account for explicitly in their cost function definition. That’s important because real data is never perfect.
Rosa: Right, so when you look at the results on Table IV, they show that their guided BO with a high-fidelity digital twin outperforms the nominal controller on average after only about two experiments, whereas standard BO requires about twelve experiments.
Dev: That’s a significant difference in sample efficiency right there. They also found that with the same number and frequency of integrating the digital twin, their guided BO converges to the desired optimality ratio sooner than standard BO.
Taro: I see they're saying that even when you do more work integrating the digital twin, guided BO still gets there faster in terms of reaching a good solution. What about how it actually performs once it’s done?
Rosa: They also noted that the guided BO controller’s kappa is higher than the one obtained with standard BO, and they saw this reflected in the step responses shown in Figure fifteen. This leads to a smaller tracking error rate and a minor overshoot.
Dev: So it seems like they're not just finding a better parameter set, but perhaps a controller that is inherently more stable or performs better in the real world because of how they found it.
Taro: That makes sense if the digital twin is giving them that extra information about how the system will behave under certain conditions. What about the practical side of this method? How time-efficient is it compared to just running standard BO?
Rosa: They looked at Table VIII, and they showed that their guided BO is more time-efficient because it requires fewer tedious experiments on the physical system, which means lower computational overhead from those digital twin iterations. The total tuning time for the rotary system was five hundred three point fifty-four seconds for guided BO compared to seven hundred eighty-five point nineteen seconds for standard BO.
Dev: And they also found that when comparing the best result among a hundred batches of guided BO, it only took three experiments to accomplish something similar, compared to seven experiments required for standard BO. That’s a fifty-seven point five percent improvement in experiment count.
Paper summary: Taro: So the practical implication is that for industrial systems where every experiment costs time and resources, this method really speeds up the whole tuning process while still getting good results.
Rosa: The paper concludes that their guided BO approach considerably improves data efficiency for controller tuning, it generalizes across different system types and even different industrial controller structures. They proved its superior performance on both a noisy linear servo motor and a DC rotary motor real-time hardware.
Dev: And they specifically highlighted that the digital twin carries out the exploration duty of the optimizer while exploitation is performed on the real system. This means guided BO explores more around where it thinks the optimum cost is, while standard BO explores further away from that optimum.
Taro: So it seems like this method smartly balances exploring new things with exploiting what we already know about how to get a good tuning result. What are the limitations they point out?
Rosa: One limitation they state is that there's a gap between the twinning property of digital twins in industrial systems, which limits how accurate the digital twin model can be. They tried to mitigate this by incorporating a generic learning algorithm into the digital twin to enable active braking control in one example.
Dev: That gap in accuracy is something we need to keep an eye on when we talk about applying this outside of controlled environments. The method's success heavily depends on how well that digital twin bridges the gap between its virtual model and the actual physical reality.
Taro: So, while it’s a powerful way to save experiments, you still have to deal with the uncertainty inherent in modeling a complex physical system virtually. It’s not magic; it’s guided by data and assumptions about how well your twin works.
Rosa: That's the gist of the paper on "Guided Bayesian Optimization: Data-Efficient Controller Tuning with Digital Twin." It shows a way to use a digital twin to make controller tuning much more data efficient, reducing experiments from twelve down to two in some cases while improving performance metrics like tracking error and overshoot.
Dev: It’s a method that relies on the system's own closed-loop data to build this twin, which then intelligently switches between exploring the real system and exploring the twin. The main point is that it’s faster than standard Bayesian optimization in terms of experiments needed, but they also acknowledge that the accuracy of this digital twin is a critical factor for its success.
Conclusion: Rosa: So, this paper is about Guided Bayesian Optimization using a digital twin to tune controllers more efficiently than just standard optimization.
Dev: Right, it's about using this virtual model to guide the tuning process by switching between testing the real system and testing that twin when it gets uncertain.
Rosa: The authors found that their guided method needs way fewer experiments on the physical plant to get good controller parameters compared to regular Bayesian optimization.
Dev: They showed they can reach a desired level of performance much faster, requiring about ten experiments for their guided method versus twelve for standard Bayesian optimization.
Rosa: And it's not just about finding the right numbers; they found that this method actually produces a better controller in terms of tracking error and overshoot on real hardware.
Dev: They did say the digital twin itself has its own limitations because building an accurate virtual model of a complex industrial system isn't always straightforward.
Rosa: So, it's a trade-off, right? You get faster tuning with fewer physical tests, but you have to trust how good that digital twin model is actually going to be in the real world.
Mahdi Nobar, Jurg Keller, Alisa Rupenyan, Mohammad Khosravi, John Lygeros
ETH Zurich · FHNW University of Applied Sciences Northwestern Switzerland · ZHAW Centre for AI Zurich University for Applied Sciences Zurich Winterthur Switzerland · Delft Center for Systems and Control Delft University of Technology Netherlands
eess.SY, cs.SY
Submitted: 2024-03-25
Updated: 2025-11-04
Comments: This work has been published in IEEE Transactions on Automation Science and Engineering
Journal ref: IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 11304-11317, 2025
DOI: 10.1109/TASE.2024.3454176
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 82/100
The gist: Guided Bayesian Optimization (BO) presents an efficient data-driven method for iteratively tuning closed-loop controller parameters using a digital twin of the system The experimental results show…
Key concepts
- Guided Bayesian Optimization (BO)
- A data-driven method for finding optimal controller parameters by iteratively running experiments. It is guided by a digital twin that helps direct the search, focusing exploration on areas most likely to yield better results based on system information.
- Digital Twin
- A virtual model of the physical system built using closed-loop data from standard BO iterations. It is activated when uncertainty in the performance model is high, allowing it to replace real-system exploration with simulation for guidance.
- Exploration vs. Exploitation
- In this method, the digital twin handles exploration—searching new areas of the parameter space. The real physical system then performs exploitation—refining the search around the best-found optimum cost value. This balances searching broadly and refining locally.
- Performance Metrics modeled by Gaussian Processes
- The performance metrics (like tracking error) are modeled using Gaussian Processes, which are statistical tools that provide a probabilistic view of how different controller parameters affect system performance, helping the optimization process understand uncertainty.
Terminology
Summary
Guided Bayesian Optimization (BO) presents an efficient data-driven method for iteratively tuning closed-loop controller parameters using a digital twin of the system The experimental results show that our method requires fewer experiments on the physical plant than Bayesian optimization to find the optimal controller parameters
Methodology Overview
The proposed methodology is model-free, making it suitable for nonlinear and unmodelled plants with measurement noise The objective function consists of performance metrics modeled by Gaussian processes The method utilizes the available information in the closed-loop system to progressively maintain a digital twin that guides the optimizer, improving the data efficiency of our method Switching the digital twin on and off is triggered by our datadriven criteria related to the digital twin’s uncertainty estimations in the BO tuning framework Effectively, it replaces much of the exploration of the real system with exploration performed on the digital twin
Digital Twin Construction
The digital twin is built using closed-loop data acquired during standard BO iterations, and activated when the uncertainty in the Gaussian Process model of the optimization objective on the real system is high The DT plant model, denoted by G˜, is estimated based on D˜, which is a digital-twin data set constructed from informative data acquired in each experiment The fidelity of the DT plant model is estimated by Root Mean Square Error (RMSE)
Guided BO Algorithm
Algorithm 1 summarizes our guided BO method, which uses the information flow in the closed-loop system to define a DT The DT is activated when two conditions pvn(θ+) > η1, eRMSE < η2 are satisfied When the DT is activated, we use the DT instead of the actual system to estimate the cost value denoted by ˜Jˆ at ˜θ = θ+ This estimated performance J˜ is then added to the BO training data set in Line 11 of Algorithm 1
Performance Evaluation
The guided BO with high-fidelity DT outperforms the nominal controller on average after approximately 2 experiments, whereas BO requires approximately 12 experiments The results on Table IV show that with the same number and frequency of DT integration, the guided BO with high-fidelity DT converges to a desired optimality ratio sooner than BO Guided BO requires only 10 experiments to perform the same, a 44% improvement The guided BO controller’s Kd is higher than the one obtained with BO, in line with the step responses shown in Fig. 15, causing smaller tracking error rate and a minor overshoot
Controller Tuning Efficiency
The results on Table VIII show that our guided BO is more time-efficient than the BO method because it requires fewer tedious experiments on the physical system and lower computational overdue by DT iterations The total tuning time for Guided BO on the rotary system is 503.54 s, compared to 785. p. 4 Notably, we stop BO on DT when for
n˜EI consecutive iterations on DT we meet one of the conditions
aEI,i(θ) ≤ η3 max
i−n˜EI≤j≤i−1
aEI,j(θ), (14a)
i > nmax, (14b)
where i is the number of BO iterations on DT after last time
DT is activated, aEI,i(θ) = aEI(θ
+) and η3 > 0 is a predefined
threshold to stop BO iteration on DT as in [50]. If the expected
improvement does not significantly improve compared with
the improvements in the last n˜EI iterations, then (14a) halts
the optimization with DT.Where the paper says it
Conclusion
We showed that our guided BO approach considerably improves the controller tuning data efficiency, generalizes across the system and industrial controller structures, and converges faster to optimum values We proved the guided BO’s superior performance on a noisy linear servo motor and DC rotary motor real-time hardware The digital twin carries out the exploration duty of the optimizer whereas the exploitation is performed on the real system The guided BO exploits more around the optimum cost, whereas BO explores further away from the optimum The paper is 2403.16619.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Guided Bayesian Optimization improves data efficiency by defining a digital twin that
guides the optimizer,
effectively replacing much ofthe exploration of the real system with exploration performed on the digital twin.
This allows for tuning controller parameters using fewer experiments on the physical plant, specifically requiring57% and 46% fewer experiments on the hardware than Bayesian optimization to tune the control parameters
for linear and rotary motor systems. -
The improved AI system can perform data-driven controller tuning for nonlinear and unmodelled plants with measurement noise by utilizing a model-free approach. This method is suitable for complex industrial applications where obtaining a precise plant model is
not feasible without adequate system excitation.
-
The guided BO algorithm can continuously update a digital twin based on closed-loop data, and it switches the digital twin on and off when
the uncertainty in the Gaussian Process model of the optimization objective on the real system is high,
leading to more targeted exploration. -
The AI system can adapt effectively to the real world by integrating measurements with simulation behaviors, combining process measurements with an optimization framework to provide
robustness in the presence of repetitive disturbances and plant-model mismatch.
-
The improved system can generate a performance cost function J(theta) by a weighted sum of four metrics—percentage overshoot (zeta), settling time (Ts), rise time (Tr), and integral of time-weighted absolute error (eITAE)—to optimize closed-loop performance.
Abstract
This article presents the guided Bayesian optimization algorithm as an efficient data-driven method for iteratively tuning closed-loop controller parameters using an event-triggered digital twin of the system based on available closed-loop data. We define a controller tuning framework independent of the controller or the plant structure. Our proposed methodology is model-free, making it suitable for nonlinear and unmodelled plants with measurement noise. The objective function consists of performance metrics modeled by Gaussian processes. We utilize the available information in the closed-loop system to identify and progressively maintain a digital twin that guides the optimizer, improving the data efficiency of our method. Switching the digital twin on and off is triggered by data-driven criteria related to the digital twin's uncertainty estimations in the BO tuning framework. Effectively, it replaces much of the exploration of the real system with exploration performed on the digital twin. We analyze the properties of our method in simulation and demonstrate its performance on two real closed-loop systems with different plant and controller structures. The experimental results show that our method requires fewer experiments on the physical plant than Bayesian optimization to find the optimal controller parameters.
Related papers
- Industrial Load Modeling and Optimization for Market-Based Interaction with Power Systems
- Exact and Approximate Convex Reformulation of Linear Stochastic Optimal Control with Chance Constraints
- Certified Set Convergence for Piecewise Affine Systems
- Symbolic Control for Stochastic Systems via Finite Parity Games
- Multiobjective optimization-based design and dispatch of islanded, hybrid microgrids for remote, off-grid communities in sub-Saharan Africa
- Physiology-informed layered sensing for intelligent human-exoskeleton interaction