A Unified Information-Theoretic Approach to Constrained Multi-Fidelity Multi-Objective Bayesian Optimization

The gist The authors propose CMF-PFEV, an information-theoretic approach to constrained multi-fidelity multiobjective Bayesian optimization that measures query utility by the information gain about the target-fidelity feasible Pareto frontier, providing a cost-aware acquisition function without separate heuristics for fidelity selection or constraint handling

Problem Setting and Goal

The C-MF-MO problem involves optimizing L objective functions subject to C constraint functions across M fidelity levels, where the highest fidelity M is the target fidelity The goal is to approximate the highest-fidelity feasible Pareto frontier F∗ under a finite evaluation budget The target Pareto frontier F∗ is defined as (f(M)x x in Xfeas, there n' exists x' in Xfeas: f(M)x' ≻ f(M)x)

Unified Information Criterion

The basic idea is that a query is useful if its observation reduces uncertainty about the feasible target-fidelity Pareto frontier This utility is measured by the mutual information MI between an observation r(m)x and the Pareto-frontier F∗ The authors derive a tractable variational lower bound for this intractable mutual information using a mixture of under- and over-truncated approximations to the Pareto-consistent region This construction expresses information about the frontier through Pareto-consistency events defined at the target fidelity, which reduces acquisition computation to Gaussian probabilities of these regions before and after a prospective observation

Variational Approximation and Sampling

The variational conditional density q(r(m)x F∗) is approximated by imposing the Pareto-consistency condition h(M)x ∈ AF∗ as q r(m)x F∗:= p r(m)x h(M)x ∈ AF∗ The finite frontier Fb∗ is represented by a set of NPF points, and the variational distribution is constructed using a mixture of over-truncated (qU) and under-truncated (qO) components: qλ r(m)x Fb∗ = λqU r(m)x Fb∗ + (1 − λ)qO r(m)x Fb∗ The lower bound is approximated by LCMF, which involves the expectation over the random frontier and prospective noisy observation, calculated via nested Monte Carlo sampling

Query Selection and Optimization

The acquisition value is defined as am = CalcAcq(xm, λm; S), which is then maximized jointly over input x and mixture weight λ for each fidelity m The final query selection involves choosing the fidelity that maximizes the cost-normalized acquisition value α(x, m, λ) = 1/cm LbCMF(x, m, λ) The CMF-PFEV algorithm iteratively updates the multi-fidelity GP posterior and selects the next query based on this criterion

Experimental Results

Experiments on synthetic, benchmark, and real-world problems demonstrated effectiveness across diverse objective, constraint, and fidelity settings CMF-PFEV performs favorably across all three settings The method shows a pronounced advantage on GP sample path problems where the data-generating process matches the GP modeling assumptions, while consistent improvements are also observed on benchmark and hyperparameter optimization problems In comparison to other methods, MF-JESMOC showed significantly longer computation times due to its use of deep GPs and variational approximations The results are reported as inference relative hypervolume (IRHV) against cumulative evaluation cost

Conclusion

CMF-PFEV provides a unified information-theoretic criterion by expressing information about the target-fidelity feasible Pareto frontier through Pareto-consistency events, yielding a tractable variational acquisition function The method is effective across diverse problem settings, although the authors note that unknown approximation error on the information gain still requires investigation The complete Bayesian optimization loop is summarized in Algorithm 1 in Appendix B

How it works

The authors develop a unified information-theoretic approach to constrained multi-fidelity multi-objective BO centered on the feasible Pareto frontier at the target fidelity The key idea is to represent frontier information through Pareto-consistency events defined at the target fidelity, allowing objectives, constraints, and fidelity selection to be handled within the same information criterion This approach avoids separate mechanisms for design, fidelity, and constraint handling

The method derives a tractable variational lower bound based on over- and under-truncated approximations of these Pareto-consistency events by extending PFEV This construction reduces the acquisition computation to Gaussian region probabilities before and after a prospective observation, where arbitrary-fidelity queries are connected to the target fidelity through standard cross-fidelity Gaussian conditioning The final acquisition function is cost-aware, as it evaluates query utility by the information gain about this frontier, normalized by its evaluation cost

Key Components of the Acquisition Function

The acquisition function is derived from the lower bound MI ≡ LCMF(x, m, λ) which incorporates multiple objectives, constraints, and fidelity selection into a single information criterion The calculation of ZT and ZT (· r(m)x) involves evaluating Gaussian CDFs based on the joint posterior distribution of the target-fidelity outputs conditioned on the prospective observation r(m)x The objective-space probabilities are evaluated by partitioning the region dominated by the finite frontier into mutually disjoint hyperrectangles The optimization involves jointly optimizing candidate input x and mixture weight λ for each fidelity m

Cross-Fidelity Propagation

The role of fidelity is explicit in the cross-fidelity covariance, where a query is informative about the target frontier only insofar as the cross-fidelity covariance allows its observation to change posterior beliefs about the target-fidelity outputs The posterior mean vector and variance matrix for the target fidelity h(M)x are updated using a formula that explicitly includes terms related to lower fidelity observations and noise This makes the role of fidelity explicit in how a query is informative about the target frontier

Evaluation Metrics

Performance is measured by inference relative hypervolume (IRHV), which is the hypervolume of posterior-mean recommendations normalized by an approximation of the true Pareto hypervolume The IRHV is reported against cumulative evaluation cost The method's performance is compared against MF-OSEMO, MF-HVKG, MFMOSEGO, and MF-CMOBO The comparison shows that CMF-PFEV performs favorably across all three settings The computational time for methods like MFJESMOC is significantly higher than for others, due to the estimation of complicated deep GP and variational approximations

Limitations

A limitation noted is unknown approximation error on the information gain, as MI is an essential criterion but its estimation accuracy still requires investigation The method's effectiveness is demonstrated across synthetic problems, benchmark functions, and machine-learning hyperparameter optimization problems

How it works

The authors develop a unified information-theoretic approach to constrained multi-fidelity multi-objective Bayesian optimization centered on the feasible Pareto frontier at the target fidelity The key idea is to represent frontier information through Pareto-consistency events defined at the target fidelity, allowing objectives, constraints, and fidelity selection to be handled within the same information criterion This approach avoids separate mechanisms for design, fidelity, and constraint handling<ref:2610.

1. Improved query selection via CMF-PFEV allows for joint continuous optimization over (x, λ) with m held fixed to select the most cost-effective fidelity in a sense of the approximate MI, leading to more efficient budget utilization compared to methods like MF-JESMOC, which has a significantly higher next-query selection time.

2. Enables identification of the highest-fidelity feasible Pareto frontier by measuring query utility by the information gain about this frontier, provided by an observation, ensuring that every evaluation contributes maximally to identifying the target fidelity set.

3. Facilitates decision-making for complex problems where multiple objectives, constraints, and fidelity selection are incorporated into a single information criterion rather than handled through separate acquisition mechanisms, leading to a unified and potentially more robust optimization strategy.

Bayesian optimization often involves multiple objectives, constraints, and fidelity levels. We address the challenge of jointly selecting where and at which fidelity to evaluate to identify the highest-fidelity feasible Pareto frontier in this combined setting. From a unified information-theoretic perspective, we measure query utility by the information gain about this frontier, provided by an observation. Since this mutual information is intractable, we derive a variational lower bound using a mixture of under- and over-truncated approximations to the Pareto-consistent region. Multi-fidelity surrogate models propagate the information to arbitrary fidelities, yielding a cost-aware acquisition function without separate heuristics for fidelity selection or constraint handling. Experiments on synthetic, benchmark, and real-world problems demonstrate effectiveness across diverse objective, constraint, and fidelity settings.