Bayesian Optimisation under State-Preservation Constraints

arXiv:2610.12150 · cs.LG · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Bayesian Optimisation under State-Preservation Constraints".

Jane: The gist The central idea is to pre-compute the set of controls whose linearised constraint response stays within tolerance,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Let’s talk about the title and who wrote this paper. "Bayesian Optimisation under State-Preservation Constraints." The authors are Gabriel Diaz-Aylwin, Vignesh Gopakumar, Omkar Myatra, David Moulton, and Lorenzo Zanisi from Lancaster University in the UK.

Jane: They're tackling that state preservation problem directly. What that really means is they are focusing on design problems where the solution isn't just about minimizing an objective function; it’s about making sure certain state observables stay near trusted values throughout the design process.

Lu: The title points to a core challenge: how to run Bayesian optimization when your constraints are tied to the state itself, like in tokamak fusion reactor designs or complex physical simulations where you can't just treat the objective function in isolation.

Meng: So for someone who does engineering, it means they’re looking at scenarios where you have these crucial physical boundaries that must be respected, and they are trying to design around them efficiently instead of having to test every single possibility.

Lalam: It’s about structuring the optimization search space based on a known first-order behavior of the constraints, which is a foundational step for making AI systems more robust in complex physical modeling.

The paper's summary: Tom: The paper summarizes how they take this idea of pre-computing that linear approximation and use it to define the proposal geometry for Bayesian optimization. They start with a first-order expansion around a baseline design, x zero which gives you an ellipsoid defined by delta x M delta x tau squared where M is related to the Jacobian of the constraint function <ref:2610.12150#pg1>.

Jane: That ellipsoid is then intersected with the actual engineering domain, X, and that intersection becomes their proposal geometry for the Bayesian optimization loop. It’s this intersection that they use to generate candidates, which they found allows them to draw a large number of well-spread candidates from it at negligible cost compared to sampling blindly.

Lu: The summary emphasizes that this ellipsoidal region is available in closed form, and because it’s available in closed form, they can draw millions of well-spread candidates from it without needing expensive evaluations for each one. This is the core mechanism they are highlighting.

Meng: So the paper shows a method that moves away from struggling with the thin and anisotropic nature of those feasible sets by using this geometric object as a smart proposal strategy for Bayesian optimization. It’s about making sure your sampling efforts are directed where they matter most based on the first-order approximation.

Lalam: It suggests that instead of learning feasibility point by point, which is what standard constrained optimization often does, you can approximate the feasible set directly as a geometric shape, which is a classical technique from constrained optimization that they are applying here to guide the search.

The paper's improvements: Tom: What makes this work better than just using a simple linear approximation? The authors suggest several key improvements. First, they introduce an online refinement process for the response map J, which updates it by least squares regression over pairs observed so far, leading to an updated proposal geometry M c and region E b.

Jane: That means the initial guess isn't static; as the optimization runs and you get more data points, you continuously update your geometric shape to better fit the true constraint behavior. It’s an adaptive process that gets smarter with every evaluation.

Lu: Then there’s handling errors beyond the linearisation. They use a learnt feasibility model where they weight the acquisition function instead of modeling the scalar violation directly, approximating feasibility probability using moment matching, which is a generalized chi-squared tail. This allows them to safely operate even when the linear approximation has errors at the edges of their feasible region.

Meng: That's crucial for practical application. It means you don’t just trust the initial linear shape; you have a mechanism in place that handles those small deviations from linearity without immediately failing or wasting samples on infeasible points.

Lalam: And they also show how this framework integrates into multi-objective Bayesian optimization by suggesting you can swap the Expected Improvement function for the Expected Hyper-Volume Improvement, EHVI, which lets you jointly optimize multiple objectives like connection length and total flux expansion.

Conclusion: Tom: So to wrap up, the authors are showing that pre-computing the set of controls whose linearised constraint response stays within tolerance allows them to pull back the state-space constraint into design space by defining an ellipsoid as a proposal geometry. They also show online refinement and a field model for handling non-linear errors safely.

Jane: The main implication is that this method provides substantial gains in sample efficiency over standard constrained Bayesian optimization baselines, demonstrated by results where they reached a median detachment threshold five to seven times below the best competitor in the currents-only problem while keeping nearly ninety percent of evaluations within a very tight boundary tolerance.

Lu: This is really about making sure that when you’re optimizing complex physical systems, you can get much more useful information out of each evaluation, even when the constraints are super restrictive and weirdly shaped.

Meng: From an engineering side, if this works as well in tokamak divertor optimization—which they used as their primary application—it means designs that require extremely precise state preservation constraints become much faster to explore.

Lalam: It’s a general framework for the Bayesian optimization of trusted engineering designs under tightly constrained state variation, and the core contribution remains that continually updating candidate proposal geometry drawn from the linearised feasible region E is what drives all this improvement.

Gabriel Diaz-Aylwin, Vignesh Gopakumar

Lancaster University · UK Atomic Energy Authority

cs.LG

Submitted: 2026-10-08

Updated: 2026-10-08

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 79/100

The gist: The gist The central idea is to pre-compute the set of controls whose linearised constraint response stays within tolerance, thereby pulling back the state-space constraint into design space How it

Key concepts

Linearized Feasible Region (E)
This is a simplified, linear approximation of the actual constraint boundary around a baseline design. It's derived from a first-order Taylor expansion of the constraint function, allowing the optimization process to search within this manageable geometric shape instead of the complex nonlinear space.
Proposal Geometry
Instead of sampling randomly across all possible designs, this method uses the intersection between the linearized feasible region (E) and the engineering domain (X) as a focused proposal area. This directs Bayesian optimization to explore regions most likely to satisfy constraints, leading to faster convergence.
Feasibility Model
Since linear approximation introduces error, a learned model is used online to weight the acquisition function. This model estimates the probability that a new design satisfies the true constraint violation tolerance ($ au$), allowing the optimizer to balance exploration and feasibility effectively.

Terminology

Summary

The gist The central idea is to pre-compute the set of controls whose linearised constraint response stays within tolerance, thereby pulling back the state-space constraint into design space

How it works

The method introduces a classical construction from nonlinear programming into constrained BO by precomputing the first-order approximation of the preservation constraint and using this as a proposal geometry for Bayesian optimization Around the baseline design x0, a first-order expansion gives C(ψ(x0 + δx)) = C(ψ(x0)) + J δx + O(δx 2). This leads to the linearised feasible region E defined by δx⊤M δx ≤ τ2 where M = J⊤GJ. The intersection E ∩ X is used as the proposal geometry for the Bayesian optimization loop.

Candidate Generation and Refinement

At each stage of the optimization, candidates are drawn from E ∩ B, where E is the linearised feasible region and B is either the engineering domain X or a current objective-focused trust region. To generate a large candidate pool efficiently when E is highly anisotropic, ray-based sampling within E ∩ B is employed, followed by a maximin criterion to retain well-spread subsets. The initial response map J = D(C ◦ ψ)(x0) describes the infinitesimal change in the observable around the baseline. This map is refined online by least squares regression over pairs observed so far, leading to an updated proposal geometry Mc = Jb⊤GJ and E b = n x0 + δx: δx⊤Mδx c ≤ τ2 o.

Feasibility Beyond Linearization

To handle the linearisation error, a learnt feasibility model is used to weight the acquisition function instead of modeling the scalar violation directly. This field model approximates the change in observable as δC(x) = J δx ˆ + X K j=1 aj (x) ϕj, where Jˆ is the online-refit linear response map and aj are zeromean GPs on mode coefficients. The resulting feasibility probability P[δC(x)G ≤ τ] is approximated by moment matching, which is a generalized χ2 tail.

Application in Divertor Optimization

The primary application is divertor design for tokamak fusion reactors, where the objective is to minimise the detachment-threshold impurity fraction cz while preserving the plasma boundary of a baseline equilibrium known as the last closed flux surface (LCFS). The constraint C is defined by C(ψ(x)) − C(ψ(x0))G ≤ τ, where v(x):= C(ψ(x)) − C(ψ(x0))G is the constraint violation. The method demonstrates substantial gains in sample efficiency over standard constrained-BO baselines, with ablations showing that the proposal geometry is the principal source of improvement.

Experimental Results

Experiments show that our proposal geometry is the principal source of improvement, reaching a median detachment threshold 5–7× below the best competitor in the currents-only problem while holding ∼ 90% of evaluations within a 0.5 cm boundary tolerance. The method reaches a final cZ of 0.012, which is 5.2× better than the best competitor setting, while keeping 89.5% of its evaluations feasible. The proposed BO module is self-contained and can be easily plugged into orthogonal BO advances, including batch (Gonz´alez et al. 2016), multi-fidelity (Snoek, Larochelle, and R. Adams 2012), multi-task (Moss, Leslie, and Rayson 2020) and high-dimensional (Doumont et al. 2025) BO.

The method provides a general framework for the Bayesian optimization of trusted engineering designs under tightly constrained state variation. The paper suggests that the proposed BO module is indifferent to the choice of acquisition function when applied to multi-objective Bayesian optimization by swapping EI with EHVI. The core contribution is the continually updating candidate proposal geometry, which is drawn from the linearised feasible region E and refined online.

The gist The central idea is to pre-compute the set of controls whose linearised constraint response stays within tolerance

How it works

The method introduces a classical construction from nonlinear programming into constrained BO by precomputing the first-order approximation of the preservation constraint and using this as a proposal geometry for Bayesian optimization. Around the baseline design x0, a first-order expansion gives C(ψ(x0 + δx)) = C(ψ(x0)) + J δx + O(δx 2). This leads to the linearised feasible region E defined by δx⊤M δx ≤ τ2 where M = J⊤GJ. The intersection E ∩ X is used as the proposal geometry for the Bayesian optimization loop.

Candidate Generation and Refinement

At each stage of the optimization, candidates are drawn from E ∩ B, where E is the linearised feasible region and B is either the engineering domain X or a current objective-focused trust region.

Improvements for AI systems

  1. Bold Header: Pre-computation of Linearised Feasible Geometry

The system can precompute the first-order approximation of the arXiv:2610.12150v1 [cs.LG] 8 Oct 2026 to define an ellipsoid, which is then used as the proposal geometry for the Bayesian optimisation loop. This allows for drawing a large number of well-spread candidates from it at negligible cost.

  1. Bold Header: Online Refinement of Response Map

The system can continuously update the constraint model by refitting the linear response map Jb by least squares over the pairs observed so far, including the initial finite-difference probes, ensuring that the proposal geometry is updated through Mc = Jb⊤GJ, E b = n x0 + δx: δx⊤Mδx c ≤ τ 2 o.

  1. Bold Header: Field Model for Feasibility Beyond Linearisation

The system can handle violations of the linear approximation by using a field model where it models the change in the observable, δC(x) and take its norm, approximating feasibility via a generalized chi-squared tail. This allows the method to safely operate when linearisation error causes edges of the feasible region to contain infeasible points.

  1. Bold Header: Enhanced Sample Efficiency in High-Cost Design

The improved AI system can achieve substantial gains in sample efficiency by incorporating the constraint directly into the proposal geometry, as demonstrated by reaching a median detachment threshold 5–7× below the best competitor in the currents-only problem while holding ∼ 90% of evaluations within a 0.5 cm boundary tolerance.

  1. Bold Header: Multi-Objective Optimization Integration

The system can be adapted for multi-objective Bayesian optimisation by swapping EI with EHVI, allowing it to jointly optimise objectives like connection length Lc (field-line distance from midplane to target) and total flux expansion.

Abstract

In many engineering design problems, the objective and constraints depend on the state: the solution of a PDE determined by the design parameters. We consider improving a design while holding selected state observables near trusted values, which we call state preservation constraints. Constrained Bayesian optimisation handles these with a learnt feasibility model, but struggles with this problem's highly anisotropic feasible set. Our central idea is to pre-compute the set of controls whose linearised constraint response stays within tolerance, thereby pulling back the state-space constraint into design space. This linearisation defines an ellipsoid from which we can efficiently draw a large number of well-spread candidates. The underlying linear response map is refined online, and the ellipsoid is rebuilt accordingly. We demonstrate the method end-to-end on our key application - Tokamak divertor optimisation under plasma-boundary preservation.

Sources

Related papers