BOReFT: Manifold Steering of Language Models for Black-box Optimization
cs.LG, cs.AI, cs.CL
Submitted: 2026-09-27
Updated: 2026-09-27
Code: https://github.com/dhdhagar/boreft
License: http://creativecommons.org/licenses/by/4.0/
The gist: Language models are increasingly used as proposal models for black-box search, from program optimization to molecular design.
Terminology
Abstract
Language models are increasingly used as proposal models for black-box search, from program optimization to molecular design. Existing approaches typically improve proposals through iterative prompting or parameter updates, offering limited control over how completely and efficiently the model's search space is explored. Continuous optimization methods, such as Bayesian optimization, provide a principled way to search but require a suitable domain to operate over. To address this, we introduce BOReFT, which learns a compact, low-dimensional space of hidden-state interventions in a frozen language model, and uses this space as the search domain for Bayesian optimization with an external scoring function. Empirically, we find that the learned domain spans semantic regions and exhibits smoothness properties that support search. Theoretically, we show that semantic coverage and interpolation control the best score available in the learned space, and that decoding from this space yields a standard stochastic-bandit observation model for adaptive search. We evaluate BOReFT on the interpretable word search task "Semantle" and on three more real-world discovery tasks in de novo molecule property optimization. Compared to strong LLM baselines, BOReFT finds in Semantle a higher number of hidden targets and, on two out of three molecular objectives, achieves higher property scores. Consequently, our method provides a principled new bridge between discrete proposal spaces of LLM-based search and continuous black-box optimization.
Sources
- MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Models
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- Active Flow Expansion for Out-of-Distribution Discovery: from Theory to Molecules
- A Tutorial on Bayesian Optimization
- Verbalizable Representations Form a Global Workspace in Language Models
- Distilling the Knowledge in a Neural Network
- MiGrATe: Mixed-Policy GRPO for Adaptation at Test-Time
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
- Representation Engineering: A Top-Down Approach to AI Transparency
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks