Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback

arXiv:2608.15591 · cs.AI · Submitted 2026-08-16 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback".

Jane: The gist The framework addresses the static agent dilemma by introducing Agent Gym, a modular framework for continuous evaluation and evolution of LLM agents through human-in-the-loop feedback.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Moving on to the architecture of Agent Gym... they structure it into three main zones. Zone one is the Constitution Architecture where you put your domain configuration layer.

Jane: So that means all the specific business rules and data for your domain live in declarative artifacts, like a Master Data YAML and a Rules Book Markdown. That way, you can describe things in a way that both humans can read and machines can parse.

Lu: Exactly. That separation is important because it means you only need to change the configuration files if you want to onboard a new domain, not rewrite the agent’s actual code or prompt structure.

Meng: That sounds clean on paper, but how easy is it for an engineer to manage all those YAML and Markdown files at scale when you have dozens of different domains?

Lalam: The Constitution Architecture captures domain knowledge in these declarative artifacts which serve as both human-readable documentation and machine-parseable specifications.

The paper's summary: Tom: So, the core idea is to solve the tension between a static agent and evolving business rules by creating this continuous loop of Act, Evaluate, Investigate, Correct, Learn and Observe.

Jane: They point out that traditional evaluation methods only check performance against labeled ground truth sets when they exist. But in production there's no ground truth for novel cases or the nuanced rules might be missing from the reference data.

Lu: The paper introduces a dual evaluation model here: one part validates against available reference data, and another part validates decisions directly against the business rules themselves. This is crucial because it covers both known cases and unknown edge cases.

Meng: So you have two ways to check if an agent is doing its job, which sounds like it adds complexity to the deployment pipeline. What about the speed?

Lalam: The framework addresses these challenges by introducing a hybrid deterministic-LLM correction engine called ALF with twenty-one condition operators and three tiers of actions, allowing for surgical patches or pipeline continuation based on error severity.

The paper's improvements: Tom: A big focus is their correction mechanism, the hybrid deterministic-LLM correction engine, or ALF. They use twenty-one condition operators to detect violations without needing the LLM to do the heavy lifting of detection.

Jane: And for correcting those violations, they have a three-tier action model. Tier one might be a small adjustment, Tier two is a surgical patch, and Tier three allows the agent to continue its pipeline from where it stopped if it terminated early.

Lu: They also have this three-layer investigation architecture for ground-truth-free compliance validation, which means you can check decisions against the rules book without any pre-labeled data. Layer three is Conservative Cross-Validation, which runs the evaluation twice more when a potential violation is flagged.

Meng: That sounds like it adds significant computation time to every single decision path. How do they manage that overhead in a real production environment?

Lalam: To handle the investigation, they use this three-layer architecture for ground-truth-free compliance validation, where Layer three is Conservative Cross-Validation which uses a triple-check mechanism to re-run the evaluation twice more when a potential violation is detected.

Conclusion: Tom: So to wrap up, Agent Gym gives you this continuous post-deployment behavioral correction loop by wrapping any existing agent in those six capabilities. It separates the acting agent from the correction layer, which keeps things independently versioned and testable.

Jane: The real value here seems to be that it lets domain experts contribute knowledge through a programmatic safety loop for rule discovery, ensuring human approval before new rules get added to production.

Lu: This approach preserves auditability because the agent itself stays untouched; you just have this separate layer applying the corrections based on the human-approved rules. It moves away from self-modifying agents like Self-Refine or Reflexion which can be hard to track.

Meng: For deployment, having those components independently versioned and tested sounds better than having an engineer constantly poking at the agent's underlying prompt or code because of a new edge case.

Lalam: Agent Gym provides the environment where agents can be continuously measured, understood, and improved by domain experts contributing knowledge that only they possess.

Tom: So we've covered how this framework uses its six capabilities and its ALF engine to keep agents aligned with evolving rules without needing a full system rebuild.

Jane: It’s about moving beyond just checking performance at one point in time to actively managing the agent's behavior post-deployment through human feedback.

Lu: The separation between the acting agent and the correction layer is what makes this framework useful for maintaining long-term systems where rules are dynamic, not static.

Meng: It seems like a solid approach because it keeps the core AI model stable while allowing a dedicated process to handle the drift in business logic.

Lalam: Agent Gym is this modular, domain agnostic framework for continuous evaluation and evolution of LLM agents through human-in-the-loop feedback.

Google Cloud

cs.AI

Submitted: 2026-08-16

Updated: 2026-10-14

Comments: 15 pages, 4 figures

Code: https://github.com/google/adk-samples

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The gist The framework addresses the static agent dilemma by introducing Agent Gym, a modular framework for continuous evaluation and evolution of LLM agents through human-in-the-loop feedback.

Key concepts

Agent Gym: A modular framework that wraps any existing LLM agent to enable continuous evaluation and evolution. It provides six capabilities—Act, Evaluate, Investigate, Correct, Learn, and Observe—organized across three architectural zones to facilitate human-in-the-loop feedback.
Constitution Architecture: The layer where domain knowledge is codified in configuration artifacts like Master Data YAML and Rules Book Markdown. This separates domain knowledge from the agent's code, allowing new domains to be added simply by updating these declarative files.
Hybrid Deterministic-LLM Correction Engine (ALF): A system that detects issues using 21 condition operators without relying on the LLM itself for detection. It then applies a three-tier action model to correct errors, ensuring rule correctness is verified before human approval.

Terminology

Human-in-the-Loop — 1× in this paper · explained in 3 episodes

Summary

The gist The framework addresses the static agent dilemma by introducing Agent Gym, a modular framework for continuous evaluation and evolution of LLM agents through human-in-the-loop feedback.

Agent Gym Framework Overview

Agent Gym is a modular, domainagnostic framework that wraps any existing LLM-based agent in a continuous evaluation-and-evolution loop The framework provides six composable capabilities — Act, Evaluate, Investigate, Correct, Learn, and Observe — organized across three architectural zones Zone 1 is the Constitution Architecture which consists of a domain configuration layer Zone 2 is the Runtime Inference Pipeline which chains acting, investigation, and adaptive correction Zone 3 enables human-guided evolution through SME feedback and the Rule Learning Agent.

Key Technical Contributions

The key technical contributions include a hybrid deterministic-LLM correction engine (ALF) with 21 condition operators and three-tier actions A three-layer investigation architecture for ground-truth-free compliance validation and a programmatic safety loop that guarantees rule correctness before human approval. The framework introduces the Spec-to-Note Gap, an autoencoder-inspired view of agentic system transparency.

Architecture and Design Principles

Agent Gym is built on four foundational principles that guide every architectural decision Principle 1 states that the agent is a black box, making no assumptions about the agent’s internal architecture Principle 3 dictates that domain knowledge lives in configuration, not code, meaning only configuration changes are required for new domains. Principle 4 emphasizes that humans govern the loop, requiring every correction rule to be reviewed and approved by a subject matter expert before it is added to the production rule base.

Runtime Pipeline Components

The runtime inference pipeline consists of three sequential stages for each case 3.4.1 The Acting Agent processes input documents and produces a folder of output artifacts per case, including intermediate results and a final structured output file containing the agent’s decision. 3.4.2 The Investigation Agent validates agent decisions against the rules book without requiring any ground truth data This validation employs a three-layer architecture summarized in Table 2. Layer 3 is Conservative Cross-Validation, which uses a triple-check mechanism to re-run the evaluation twice more when a potential violation is detected.

Correction and Learning Mechanisms

The ALF engine sits downstream of the acting agent, separating detection from correction Detection uses 21 condition operators with no LLM involvement, while correction supports a three-tier action model. Tier 3 is used when the agent terminated early, allowing the LLM to continue the pipeline from the resumption point. The Learning Agent enables SMEs to discover new correction rules conversationally through a programmatic safety loop This loop includes Schema validation, Target match verification, and Collateral assessment before SME approval.

Governance and Transparency

The Constitution Architecture captures domain knowledge in declarative artifacts like the Master Data YAML and the Rules Book Markdown These artifacts serve as both human-readable documentation and machine-parseable specifications. The Spec-to-Note Gap proposes an autoencoder view of system transparency, where an LLM auditor generates a reconstruction note comparing the specification against the implemented system. This pattern maps naturally to the Observe capability, allowing auditors to regenerate this note as the constitution and rule base evolve.

Evaluation and Adoption

The Evaluation Engine provides quantitative accuracy measurement when ground truth is available It performs deterministic comparison across configured comparison groups and optionally uses LLM-as-judge for holistic alignment verdicts. An open-source reference implementation for invoice processing demonstrates that the framework is fully operational and ready for adoption. The framework's strength lies in its separation of concerns, ensuring that the acting agent, correction layer, and learning loop are independently versioned and tested. The framework positions itself alongside traditional Business Rule Management systems while offering continuous post-deployment behavioral correction. The paper concludes that Agent Gym provides the environment in which agents can be continuously measured, understood, and improved by domain experts contributing knowledge that only they possess.

Improvements for AI systems

Bold header: Hybrid deterministic-LLM correction engine (ALF)

The ALF engine provides a three-tier action model to correct errors, allowing for Surgical patch (Tier 2) or Pipeline cont. (Tier 3) actions based on the severity of the error, ensuring targeted and appropriate fixes.

Bold header: Three-layer investigation architecture

This architecture validates agent decisions against the rules book without requiring any ground truth data, using a process that includes Deterministic Checks followed by LLM Rule Discovery and finally Conservative Cross-Validation.

Bold header: Programmatic safety loop for rule discovery

This loop ensures rule correctness before human approval by iteratively validating candidate rules through three steps: Schema validation, Target match verification, and Collateral assessment to eliminate unintended matches.

Bold header: Constitution-based governance model

Domain knowledge is captured in declarative artifacts (Master Data is a structured YAML document and Rules Book is a human-readable Markdown document) which serve as both documentation and machine specifications, allowing new agents to be created from these artifacts alone.

Bold header: Spec-to-Note Gap for system transparency

This autoencoder-inspired view generates transparency notes in natural language by contrasting the spec against the transparency note, surfacing missing capabilities and behaviors that evaluation suites never measured.

Bold header: Continuous post-deployment behavioral correction

The framework addresses the static agent dilemma by wrapping any existing LLM agent in a loop of Act, Evaluate, Investigate, Correct, Learn, and Observe to handle evolving business rules without modifying the agent’s source code.