WHALE: A Simple Recipe for Joint Harness-Weight Optimization
cs.LG, cs.AI
Submitted: 2026-08-31
Updated: 2026-08-31
Code: https://github.com/krafton-ai/WHALE
License: http://creativecommons.org/licenses/by/4.0/
The gist: Agent performance depends jointly on the model parameters and the executable harness code that manages context and control flow.
Terminology
Abstract
Agent performance depends jointly on the model parameters and the executable harness code that manages context and control flow. Optimizing either component in isolation can leave the system bottlenecked by its frozen counterpart: weight updates can change which harness is effective, while harness updates can change which model capabilities are exposed. Existing joint-adaptation methods optimize weights and textual prompts but leave the broader harness fixed. We propose Weight-Harness Alternating LEarning (WHALE), a simple recipe that alternates two phases: updating the model under the current harness, then searching for a better harness under the updated model. We instantiate these two phases with online rejection-sampling fine-tuning and Meta-Harness, respectively. When to switch is a key design choice: to separate real improvements from noise without over-optimizing against a changing counterpart, WHALE uses either fixed phase durations or an adaptive patience rule over training signals. Using Qwen3.5-2B/4B agents across three domains (search question answering, mathematical reasoning, and chess puzzles), WHALE outperforms weight-only, harness-only, and Fast-Slow Training by 4.15-24.38 percentage points in best mean@8 accuracy. Either component can be the bottleneck: harness search matches peak weight-only accuracy with far fewer rollouts in SearchQA, but improves math accuracy only after a weight update. Small interleaved updates also outperform stagewise weight-then-harness optimization in accuracy and rollout cost. The code is available at https://github.com/krafton-ai/WHALE.
Sources
- FireAct: Toward Language Agent Fine-tuning
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMs
- Reinforced Self-Training (ReST) for Language Modeling
- Automated Design of Agentic Systems
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- Billion-scale similarity search with GPUs
- Meta-Harness: End-to-End Optimization of Model Harnesses
- Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
- P 2O: Joint Policy and Prompt Optimization
- Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference
- HARBOR: Automated Harness Optimization
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
- Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
- Learning, Fast and Slow: Towards LLMs That Adapt Continually
- Large Language Models as Optimizers
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- STaR: Bootstrapping Reasoning With Reasoning
- AFlow: Automating Agentic Workflow Generation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks