LLM-Augmented Bayesian Optimization with Adaptive Policies for Chiplet Design Space Exploration

arXiv:2610.14612 · cs.LG · Submitted 2026-10-10 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "LLM-Augmented Bayesian Optimization with Adaptive Policies for Chiplet Design Space Exploration".

Jane: The gist The proposed framework integrates a Large Language Model (LLM) with Bayesian Optimization (BO) to enable architectural-level simulation and automated optimization for 2.5D/3D CPU-GPU heterogeneous chiplets,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we’re looking at this paper called "LLM-Augmented Bayesian Optimization with Adaptive Policies for Chiplet Design Space Exploration". It’s about using a Large Language Model with Bayesian Optimization to figure out the best designs for these complex CPU-GPU chiplets.

Jane: It sounds like they are tackling the difficulty of designing these heterogeneous systems when you have so many different parts interacting. They are looking at how to make that design space exploration much more efficient than traditional methods.

Lu: The core idea here is combining the LLM's ability to understand complex architecture with Bayesian Optimization’s strength in efficiently searching a design space, which is usually very sparse data-wise.

Meng: So, what does this mean for us on the ground? Are we talking about faster design iterations or just a clever way to look at the problem?

Lalam: It focuses on using the LLM to generate search policies that guide how Bayesian Optimization explores those chiplet designs, which is a big step because it tries to move beyond just treating the LLM as some kind of oracle.

The paper's summary: Tom: Okay, so the paper explains that traditional methods struggle with these chiplet systems because they have very few early-stage platforms to test them on, and the relationships between design parameters and performance metrics are super complex and nonlinear.

Jane: They propose a framework that first extends an existing simulator to handle three dee architectures by adding graph topologies, which lets them simulate things like 2 point 5D or even three dee chiplets more accurately <ref:2610.14612#pg1>.

Lu: Then they introduce the LLM component, which is supposed to generate high-quality search policies to direct the Bayesian Optimization process in a knowledge-driven way, instead of relying only on static rules.

Meng: But they also admit that just treating the LLM as a perfect predictor doesn't work because it can be unreliable for precise numerical optimization and might even introduce bias when it sees designs it hasn't seen much of.

Lalam: Exactly, so the paper is focused on how to make that LLM guide the search without letting its unreliability derail the actual optimization process.

The paper's improvements: Tom: They introduce three main improvements to tackle this problem: first, they have Adaptive Directional Policy Reasoning, or ADPR, which lets the LLM use current simulation feedback to generate policies that actually change as the search progresses.

Jane: And then they have this exploration gain-aware fine-tuning method that tries to improve the quality of those LLM policies by rewarding them based on how much they are expected to help find better designs later on.

Lu: The third improvement is policy-guided Bayesian optimization, which uses a new acquisition function called P-EHVI, which incorporates those LLM policies as soft search priors into the Bayesian Optimization process.

Meng: So, it’s like the LLM tells BO where to look next, and the fine-tuning makes sure the LLM is actually telling it useful things about those promising areas.

Lalam: The P-EHVI function specifically softens this guidance by boosting candidates that are within a certain subspace defined by those policies, which helps guide BO without completely removing its ability to explore other promising regions.

Conclusion: Tom: So, to wrap up the paper "LLM-Augmented Bayesian Optimization with Adaptive Policies for Chiplet Design Space Exploration", they show that this combination of ADPR, fine-tuning, and policy-guided BO can actually improve design space exploration efficiency on both three dee and 2 point 5D chiplet datasets <ref:2610.14612#pg1,LLM-Augmented Bayesian Optimization with Adaptive Policies for Chiplet Design Space Exploration>.

Jane: They reported that this method increased the hypervolume by thirteen point three percent on their primary three dee dataset compared to previous methods, which is a solid improvement in how they find good Pareto designs.

Lu: The results show they achieved a nine point three percent improvement in hypervolume on the 2 point 5D dataset without needing to retrain anything, which suggests the framework has good generalization capabilities across different chiplet architectures.

Meng: From my side, what’s interesting is that even though this relies on an LLM for guidance, they managed to keep the online runtime shorter compared to some baselines while getting these results.

Lalam: It really shows how you can use the LLM's interpretation of design context to direct the numerical search of Bayesian Optimization, which is a powerful way to make complex optimization problems tractable.

Tom: That's what they set out to do: integrate the LLM for qualitative guidance and BO for quantitative selection, and this framework seems to achieve that synergy effectively.

Jane: It’s an interesting direction for how we can approach chiplet design challenges moving forward, so keep an eye on how these adaptive policies evolve in the community.

Hang Liu, Zhengfei Chen, Zhengrui Chen, Hao Geng, Yu Li, Qi Sun

Zhejiang University · ChipFlux Company · ShanghaiTech University

cs.LG

Submitted: 2026-10-10

Updated: 2026-10-10

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 85/100

The gist: The gist The proposed framework integrates a Large Language Model (LLM) with Bayesian Optimization (BO) to enable architectural-level simulation and automated optimization for 2.5D/3D CPU-GPU

Key concepts

Chiplet Technology Challenges: Chiplet technology allows modular chip integration but faces hurdles like scarce early evaluation platforms and complex, nonlinear relationships between design parameters and performance metrics. Traditional methods struggle to explore this high-dimensional complexity efficiently.
Adaptive Directional Policy Reasoning (ADPR): ADPR enables the LLM to generate search policies by analyzing the current optimization state, identifying bottlenecks in benchmark information, predicting parameter effects, and reflecting on past results. This allows the LLM to intelligently suggest which design parameters to adjust next.
Policy-Guided Bayesian Optimization (P-EHVI): This method modifies Expected Hypervolume Improvement (EHVI) by applying a policy boost. It uses the LLM's generated directional policy to softly increase the acquisition values of promising candidates, focusing the Bayesian optimization search on regions suggested by the AI.
Exploration Gain-Aware Fine-Tuning: This technique aligns LLM reasoning with Pareto front improvement goals using Group Relative Policy Optimization (GRPO). It uses a reward function that balances thinking, answering, and gain to train the LLM policy generation to prioritize designs that lead to better overall performance metrics.

Terminology

Bayesian Optimization — 1× in this paper · explained in 3 episodes

Summary

The gist The proposed framework integrates a Large Language Model (LLM) with Bayesian Optimization (BO) to enable architectural-level simulation and automated optimization for 2.5D/3D CPU-GPU heterogeneous chiplets, improving design space exploration efficiency.

Chiplet Technology and Challenges

Chiplet technology is a key approach for extending Moore’s Law in the post-Moore era by allowing flexible integration of modular dies from diverse process nodes and functionalities <ref:2610.14612#pg6> However, its design process is constrained by two primary challenges: the scarcity of early-stage platforms for rapidly evaluating chiplet-based heterogeneous systems, and the complex relationships between design parameters and optimization metrics <ref:2610.14612#pg6>. A critical bottleneck in chiplet development is the lack of adequate early-stage evaluation tools for CPU-GPU heterogeneous chiplets <ref:2610.14612#pg6>. Furthermore, the inherent heterogeneity of chiplet systems creates a significant optimization challenge where integrating various IPs from different process nodes establishes highly complex and nonlinear relationships between architectural parameters and performance metrics <ref:2610.14612#pg6>. Traditional Design Space Exploration (DSE) methods struggle to handle this surging complexity efficiently <ref:2610.14612#pg6>.

Framework Components

The proposed framework synergizes an extended simulator with LLM-guided Bayesian Optimization <ref:2610.14612#pg6>. The framework consists of three main components:

** We first extend the 2.5D simulator LegoSim [31] to support 3D architectures by introducing a graph-based topology representation and routing <ref:2610.14612#pg6>. The optimization engine is decoupled from the architectural evaluator, allowing the framework to be extended to other target systems, such as DRAM, NPU, and CIM-based architectures <ref:2610.14612#pg6>. **

** We introduce an adaptive policy reasoning mechanism and an exploration gain-aware fine-tuning method to improve the quality of LLM-generated search policies <ref:2610.14612#pg6>. The LLM generates high-quality policies to guide the optimization process, enabled by our proposed adaptive directional policy reasoning (ADPR) and an exploration gain-aware fine-tuning method <ref:2610.14612#pg6>. **

** We propose a policy-guided Bayesian optimization using a novel P-EHVI acquisition function, which leverages LLM policies to focus the exploration on more promising regions <ref:2610.14612#pg6>. The framework combines Adaptive Directional Policy Reasoning (ADPR), exploration gainaware fine-tuning, and policy-guided Bayesian optimization <ref:2610.14612#pg6>. **

Adaptive Policy Reasoning and Fine-Tuning

Adaptive Directional Policy Reasoning (ADPR) enables the LLM to use the current exploration state and simulation feedback to generate adaptive directional policies <ref:2610.14612#pg6>. This procedure consists of three stages:

Bottleneck Analysis: The LLM analyzes the benchmark information and the current Pareto set Ωt to identify the primary bottleneck of the optimization anchor x∗t <ref:2610.14612#pg6>. The anchor is selected as the design with the largest hypervolume contribution <ref:2610.14612#pg6>.
Adjustment Effect Prediction: Based on the identified bottleneck, the LLM uses parameter descriptions, domain knowledge, and current observations to predict the effects of increasing, decreasing, or maintaining related parameters and determine promising adjustment directions <ref:2610.14612#pg6>.
Feedback Reflection: The LLM revises its reasoning using the previous state st−1, directional policy at−1, newly evaluated designs, and the resulting hypervolume change <ref:2610.14612#pg6>.

The exploration gain-aware fine-tuning method aligns the LLM’s reasoning with ADPR by adopting group relative policy optimization (GRPO) [11] with an exploration gain-aware reward that aligns model training with the objective of improving the final Pareto front <ref:2610.14612#pg6>. The resulting reward is defined as Rtotal = αRthink + βRanswer + γRgain, where Rgain directly quantifies a policy’s effectiveness by evaluating its expected contribution to the Pareto front <ref:2610.14612#pg6>.

Policy-Guided Bayesian Optimization

To translate the LLM-generated directional policy into candidate selection, the framework proposes a policy-guided Bayesian optimization method <ref:2610.14612#pg6>. This involves extending Expected Hypervolume Improvement (EHVI) [8] to Policy-Boosted EHVI (P-EHVI), which softly increases the acquisition values of candidates within Xa <ref:2610.14612#pg6>. The formula is P-EHVI(xD) = EHVI(xD) · (1 + λ · I(x ∈ Xa)) <ref:2610.14612#pg6>. This approach introduces soft policy guidance without sacrificing BO’s exploration capability, as the uniform boost preserves their relative EHVI ranking while prioritizing the suggested region <ref:2610.14612#pg6>.

Experimental Results and Comparison

Experiments on both 3D and 2.5D chiplet datasets reveal that our framework improves design space exploration efficiency <ref:2610.14612#pg6>. Results demonstrate that our framework increases hypervolume by 13.3% compared to the previous method on the primary 3D dataset, and achieves a 9.3% improvement in hypervolume on the 2.5D dataset without retraining <ref:2610.14612#pg6>. Table 2 shows that our method achieves the best average HV and ADRS on both chiplet design spaces <ref:2610.14612#pg6>. The framework obtains a higher hypervolume with a shorter online runtime compared to baselines <ref:2610.14612#pg6>.

Conclusion

In this paper, we propose an LLM-augmented Bayesian optimization framework for chiplet DSE <ref:2610.14612#pg6>. The framework uses adaptive directional policy reasoning to enable the LLM to generate search policies from the current exploration state and simulation feedback, while BO remains responsible for uncertainty-aware candidate selection <ref:2610.14612#pg6>. Exploration gain-aware fine-tuning further aligns policy generation with the Pareto optimization objective <ref:2610.14612#pg6>. Experiments on 2.5D and 3D chiplet design spaces demonstrate improved exploration efficiency over representative baselines and promising generalization across related architectures <ref:2610.14612#pg6>.

--- Page 7 ---

References [3] Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven K Reinhardt, Ali Saidi, Arkaprava Basu, Joel Hestness, Derek R Hower, Tushar Krishna, Somayeh Sardashti, et al. 2011. The gem5 simulator <ref:2610.14612#pg6> [4] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell et al. 2020. Language models are few-shot learners <ref:2610.14612#pg6> [8] Samuel Daulton, Maximilian Balandat and Eytan Bakshy. 2020. Differentiable expected hypervolume improvement for parallel multi-objective Bayesian optimization <ref:2610.14612#pg6> [9] Yuanhang Gao, Donger Luo, Chen Bai, Bei Yu et al. 2024. Is Vanilla Bayesian Optimization Enough for High-Dimensional Architecture Design Optimization? <ref:2610.14612#pg6> [30] Runjie Zhang, Mircea R Stan and Kevin Skadron. 2015. Hotspot 6.0: Validation, acceleration and extension <ref:2610.14612#pg6> [33] Haocong Zhi, Xianuo Xu et al. 2021. A methodology for simulating multi-chiplet systems using open-source simulators <ref:2610.14612#pg6> [37] Zhuoping Yang, Shixin Ji et al. 2024. Challenges and opportunities to enable large-scale computing via heterogeneous chiplets <ref:2610.14612#pg6> [38] Xufeng Yao, Wenqian Zhao et al. 2025. High-level Synthesis Directives Design Optimization via Large Language Model <ref:2610.14612#pg6> [47] Qwen Team et al. 2024. Qwen2 technical report <ref:2610.14612#pg6> [39] David Allen Van Veldhuizen. 1999.

Improvements for AI systems

textbfPolicy-Guided Bayesian Optimization (P-EHVI) Integration for Focused Search Focus: The framework proposes policy-guided Bayesian optimization using a novel P-EHVI acquisition function, which leverages LLM policies to focus the exploration on more promising regions. This mechanism ensures that BO's numerical search is directed by the LLM's reasoning, enhancing DSE efficiency by guiding candidates toward high potential improvement regions defined by the policy.</item
textbfAdaptive Policy Reasoning (ADPR) for Dynamic Bottleneck Identification: ADPR enables the LLM to use the current exploration state and simulation feedback to generate adaptive directional policies. This involves three stages: Bottleneck Analysis, Adjustment Effect Prediction, and Feedback Reflection, allowing the policy to adapt iteratively based on observed outcomes, moving beyond static heuristics.</item
textbf Exploration Gain-Aware Fine-Tuning for Policy Quality: The method introduces a group relative policy optimization (GRPO) with an exploration gain-aware reward to improve policy generation. The reward function, where Rgain = TMHI(aLLM) / max a' ∈ A TMHI(a') (8), directly aligns the LLM training with improving the final Pareto front, rewarding policies based on their expected contribution to hypervolume.</item
textbf Surrogate Model Augmentation for Policy Consistency: The framework utilizes policy-consistent subspace Xa and extends Expected Hypervolume Improvement (EHVI) to Policy-Boosted EHVI (P-EHVI), defined as P-EHVI(xD) = EHVI(xD) · (1 + λ · I(x ∈ Xa)) (9). This soft policy guidance ensures that BO's candidate selection respects the LLM's directional prior, allowing the system to escape misleading LLM guidance while prioritizing promising subspaces.</item
textbf Domain-Specific Architecture Modeling: The system extends existing simulators to support graph topologies and introduces a TSV power model. This allows for the simulation of complex 2.5D/3D heterogeneous chiplet systems, enabling the framework to evaluate novel architectures beyond monolithic SoC limitations.