DemoEvolve: Demonstration-Guided Harness Evolution under Sparse Feedback

arXiv:2605.24539 · cs.AI · Submitted 2026-05-23 · Read on arXiv

cs.AI

Submitted: 2026-05-23

Updated: 2026-09-29

Code: https://github.com/CharTyr/STS2-Agent

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Adaptation — 1× in this paper · explained in 1 episode
• Agent Harness — 1× in this paper · explained in 2 episodes

Sources

• GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
• CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments
• TextArena
• EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems
• GameArena: Evaluating LLM Reasoning through Live Computer Games
• Continual Harness: Online Adaptation for Self-Improving Foundation Agents
• Meta-Harness: End-to-End Optimization of Model Harnesses
• TextAtari: 100K Frames Game Playing with Language Agents
• Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
• AutoHarness: improving LLM agents by automatically synthesizing a code harness
• SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
• AlphaEvolve: A coding agent for scientific and algorithmic discovery
• BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
• Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games
• A Self-Improving Coding Agent
• Voyager: An Open-Ended Embodied Agent with Large Language Models
• Huxley-G\"odel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine
• Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
• SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
• Learning to Continually Learn via Meta-learning Agentic Memory Designs

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems