HarnessTTT: Learn to Optimize Harness at Test-Time for Scientific Discovery

arXiv:2610.17370 · cs.LG · Submitted 2026-10-12 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "HarnessTTT: Learn to Optimize Harness at Test-Time for Scientific Discovery".

Jane: The gist HarnessTTT introduces a test-time learning framework that trains a harness proposer while keeping the executor frozen to improve scientific discovery through learned search procedures.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we’ve talked about what HarnessTTT is, but let’s look at the paper's title and who wrote it. It’s titled HarnessTTT: Learn to Optimize Harness at Test-Time for Scientific Discovery >

Jane: The authors are Yingzi Ma, Yulong Cao, Nanxi Li, Fang Wu, Yizhe Zhang, James Zou, Yejin Choi and Chaowei Xiao from institutions like the University of Wisconsin-Madison and Stanford >

Lu: It’s important to note that they are bringing together expertise from a few different areas—language models at Meta and NVIDIA, plus researchers from Johns Hopkins and Stanford >

Meng: It’s interesting that they have people from multiple labs collaborating on this kind of system. Are they just combining different ideas, or is there a specific synergy between the machine learning aspect of the proposer and the executor?

Lalam: The paper says this work is about finding the best valid solution within a finite computational budget for scientific tasks >

Tom: That’s because scientific discovery involves exploring many possibilities, and they are focusing on making that search more efficient when time is limited >

Jane: So, what does this actually mean in practice? It suggests we can design AI systems that don't just solve a single problem but learn a strategy for solving a whole class of problems efficiently.

Lu: They show that the system evolves task-specific harnesses, which means the search procedure itself becomes adaptable to different kinds of scientific challenges >

Meng: Adaptable search procedures sound powerful, but how do you ensure this learned harness isn't just brittle? What happens if we move it to a completely new scientific domain?

Lalam: They claim that transferring these learned harnesses to a stronger executor, like GPT-five point five, can match or exceed the strongest reported system on nine out of fourteen tasks > <ref:2610.17370#pg1>

Tom: That transferability is what gets me excited. If you can train a search procedure on one model and move it to a much bigger, more powerful one without retraining the whole thing, that’s a big deal for scaling models >

The paper's summary: Jane: Now let’s look at the core summary of HarnessTTT. Basically, they introduce this test-time learning framework where you train a harness proposer while keeping the executor frozen >

Tom: That means the proposer is responsible for creating executable harnesses—system prompts, skills, tools, middleware—that guide the fixed executor through a discovery rollout >

Lu: The key is that each harness guides a budgeted discovery rollout whose outcome provides feedback to update that proposer >

Meng: So the loop is: Proposer generates harness, runs it against the frozen solver, gets feedback on the result, and then updates the proposer’s weights based on that experience >

Lalam: They describe this using two evolution loops: TTT-Discover updates executor weights during rollout one to N rollouts, and HarnessTTT updates proposer weights while keeping the executor M0 frozen >

Tom: So TTT-Discover is focused on adapting the solver itself, but HarnessTTT focuses entirely on improving the search strategy that directs that solver >

Jane: It’s about decoupling the learning of *how to search* from the learning of *how to solve* which is a really clean separation >

Lu: They emphasize that they are not just looking for any good solution, but using a specific objective function—gap-normalized rewards, RLOO, and max-sharpening—to favor high performing proposals >

Meng: That objective function sounds complex to manage. It’s trying to balance rewarding good results while keeping the search guided by some sort of relative improvement measure >

Lalam: The reward is calculated as rk = three tanh(gk/three) if gk is positive, else max(-one gk), where gk is defined based on the incumbent score and a task reference score u tau >

Tom: That’s a specific mechanism for handling the feedback. They aren't just using a simple reward signal; they are using this RLOO-based update to constrain how much the proposer changes based on that indirect feedback >

The paper's improvements: Jane: So, what are the actual improvements they claim? They show significant results across fourteen discovery tasks, including math construction, systems benchmarks, GPU kernels and even biology >

Tom: They achieved improvement on OpenEvolve on all eleven benchmark tasks when using a frozen Qwen3 point 5-9B executor > <ref:2610.17370#pg2>

Lu: And with the larger model, scaling the proposer and the frozen executor to Qwen3 point 8-27B, they improved on eleven of fourteen tasks >

Meng: Eleven out of fourteen is substantial, especially considering the complexity of those tasks like astrodynamics or heuristic programming. But what about when you move from a smaller model to a larger one? Does it just get better linearly?

Jane: They also showed that at matched rollouts, updating the proposer’s weights reaches better results earlier than updating the executor or even just using context updates >

Lalam: The ablation study confirmed that the RLOO objective with max-sharpening actually outperforms standard GRPO and a frozen proposer on matched rollouts >

Tom: And they validated every single generated tool before execution, passing static checks and sandboxed self-tests, which adds a layer of reliability to this whole process >

Conclusion: Jane: So to wrap up, HarnessTTT suggests that we can learn to optimize harnesses at test time for scientific discovery by training a proposer while keeping the executor frozen >

Tom: The main implication is that we’re not just looking for a single good answer; we’re learning how to build a search strategy that finds the best valid solution within tight computational budgets >

Lu: And they show that this harness search procedure can itself be learned at test time with a small frozen model and carried over to much stronger executors >

Meng: For deployment, it suggests we can create specialized AI systems for scientific work where the initial effort is spent learning a good search harness rather than just fine-tuning the massive solver model repeatedly >

Lalam: The paper HarnessTTT: Learn to Optimize Harness at Test-Time for Scientific Discovery is a framework that learns task specific search strategies and allows those strategies to be transferred across different model sizes >

Tom: It’s a lot of moving parts, but the focus on learning the procedure rather than just the final answer is what makes this work interesting for me. That’s where we are headed next >

Yingzi Ma, Yulong Cao, Nanxi Li, Fang Wu, Yizhe Zhang, James Zou, Yejin Choi

University of Wisconsin-Madison · NVIDIA Corporation · Johns Hopkins University

cs.LG

Submitted: 2026-10-12

Updated: 2026-10-12

Comments: 31 pages, 9 figures, 10 tables. Project page: https://gray311.github.io/harnessrl/

Code: https://github.com/skydiscover-ai/skydiscover

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist HarnessTTT introduces a test-time learning framework that trains a harness proposer while keeping the executor frozen to improve scientific discovery through learned search procedures.

Key concepts

Harness Proposer: This is the AI component that learns how to design an executable harness. It generates the complete plan, including system prompts, reusable skills, tools (Python functions), middleware hooks for control (like budget reminders), and execution parameters. Its goal is to create effective search strategies.
Frozen Executor: The actual model or system that performs the task execution remains unchanged throughout the learning process. This keeps the core execution logic stable while allowing the proposer to learn how to best guide that fixed executor through a discovery rollout.
Discovery-Oriented Policy-Gradient Objective: This is a specific reward mechanism used to train the proposer. It combines gap-normalized rewards, leave-one-out advantages (RLOO), and max-sharpening. This objective focuses on favoring high-performing proposals rather than just following a standard gradient, guiding the learning towards better search strategies.
Harness Representation: The harness is represented as an executable program built on NexAU. It consists of five editable parts: a system prompt for strategy, skills for procedures, tools (like Python functions), middleware hooks for step control, and control parameters for sampling. The proposer edits this representation.

Summary

The gist HarnessTTT introduces a test-time learning framework that trains a harness proposer while keeping the executor frozen to improve scientific discovery through learned search procedures.

How it works

HarnessTTT operates with two evolution loops, where the only difference is what gets updated: TTT-Discover updates executor weights, whereas HarnessTTT updates proposer weights while keeping the executor frozen The proposer generates executable harnesses comprising system prompts, reusable skills, tools, middleware, and execution controls Each harness guides a budgeted discovery rollout whose outcome provides feedback for updating the proposer A discovery-oriented policy-gradient objective combines gap-normalized rewards, leave-one-out advantages (RLOO), and max-sharpening to favor high-performing proposals Generated components are validated before execution, and the search retains the best valid solution and selected harnesses across rounds.

Harness Representation and Proposer Learning

The proposer learns to generate complete harnesses comprising system prompts, skills, tools, middleware, and execution-control parameters The harness is represented as an executable program built on NexAU with five editable components: a system prompt for strategy and workflow, skills for reusable procedures, tools including proposer-written Python functions, middleware hooks that run between executor steps (e.g., budget reminders, restarts after stalls), and control parameters for sampling and iteration limits The proposer edits a private copy of H⋆ and cannot touch the task program, evaluator, executor weights, or shared runtime The proposer's objective is to minimize a clipped surrogate loss with a KL penalty toward the frozen base proposer.

Reward Mechanism and Update Strategy

The reward mechanism is designed to handle indirect and noisy feedback by using an RLOO-based update with gap-normalized rewards, leave-one-out advantages without variance normalization, and max-sharpening The reward is calculated as rk = 3 tanh(gk/3) if gk > 0, else max(-1, gk), where gk is defined based on the incumbent score and a task reference score uτ. This objective favors high-scoring proposals, not an unbiased gradient of JK The proposer update minimizes the clipped surrogate loss with a KL penalty toward the frozen base proposer to constrain the change made from a small batch of indirect feedback.

Key Results and Transferability

HarnessTTT shows significant improvements across fourteen discovery tasks, including mathematical construction, systems benchmarks, GPU kernels, heuristic programming, astrodynamics, and biology With a frozen Qwen3.5-9B executor, HarnessTTT improves on OpenEvolve on all eleven benchmark tasks and holds the best open-weight sub-10B result on nine. Scaling the proposer and the frozen executor to Qwen3.8-27B improves on eleven of fourteen tasks, and transferring the learned harnesses to GPT-5.5 matches or exceeds the strongest reported system of any backbone on nine of fourteen tasks.

Ablation and Validation

The ablation study shows that at matched rollouts, updating the proposer’s weights reaches better results earlier than updating the executor or the proposer’s context The RLOO objective with max-sharpening outperforms standard GRPO and a frozen proposer. Every generated tool is validated before rollouts, passing static checks and sandboxed self-tests The final incumbent harness is retained as the parent for the next round, ensuring solution quality accumulates.

The paper demonstrates that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. The results show that the search procedure itself can be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors. It is shown that harness search can itself be learned at test time with a small frozen model and carried over to stronger executors<ref:

Improvements for AI systems

Harness TT-T learns to optimize harnesses at test-time for scientific discovery by training a proposer to generate executable harnesses that guide a frozen executor, allowing harness search can itself be learned at test time. This enables the system to develop task-specific strategies, such as generating tools like packing feasibility for circle packing or specialized playbooks like block family sweeper for Erdős minimum overlap.
The system gains explicit, reusable search procedures through learned harnesses that are explicit and separate from the executor’s weights, facilitating a weak-to-strong model transfer. This means a harness learned with a small frozen executor can be moved to GPT-5.5, carrying over effective search strategies without needing to retrain the executor.
The proposer is updated using a discovery-oriented RLOO-based update combining leave-one-out advantages (RLOO), and max-sharpening, which is designed to limit noise amplification while favoring high-performing proposals, as noted by the objective: The RLOO term thus retains the task-relative magnitude established by gap normalization.
The system can perform complex, multi-stage scientific discovery tasks with optimized search procedures, such as circle packing where the final harness incorporates a verifier and a transform in front of every evaluation and specific skills like circle-packing-construction.