HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents

summary

The gist

AI web agents can perform complex, multi-step tasks such as searching for products, comparing options, and making purchases on behalf of users HANSEL helps you verify without reading full trajectory.

In short

HANSEL extracts minimal, interactive evidence from web agent trajectories to replace passive log reading for verification. It transforms raw execution data into a focused set of relevant pages, allowing users to directly inspect and correct errors in a live interface. Empirical results show HANSEL significantly reduces user effort and completion time while improving verification accuracy.

Key concepts

Passive Verification Methods
Current methods treat agent logs, screenshots, or summaries as passive reading tasks. Users must sift through overwhelming data to verify an agent's work, which often leads to low trust because the evidence is not interactive or contextually rich.
Evidence Extraction Pipeline
HANSEL uses a standardized trajectory representation ($ ext{observation}, ext{reasoning}, ext{action}$) and an LLM to intelligently select only the pages and snippets directly supporting the final answer, discarding irrelevant data like errors or blocked pages.
Interactive Verification Interface
This design goal renders extracted evidence as live, interactive webpages within HANSEL. Users can inspect these pages in a state similar to when the agent reached them, allowing them to directly check and correct errors without manually replaying complex actions.

Terminology used across episodes

This episode discusses

The paper

HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents · Read on arXiv

Yujin Zhang, Daye Nam

University of California, Irvine

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents".

Jane: Here's the gist of the paper. AI web agents can perform complex, multi-step tasks such as searching for products, comparing options, and making purchases on behalf of users HANSEL helps you verify without reading full trajectory.

Tom: The Problem with Current Verification Methods Existing transparency mechanisms treat verification as a passive reading task, leaving users to sift through overwhelming logs or trust potentially unfaithful explanations.

Jane: Execution logs record every page visited, click, search query, and reasoning step, which in principle allows users to trace how the agent worked but in practice produces an overwhelming log that few users will read.

Tom: Source links point users to the pages the agent visited but lose important context such as the page state when clicked.

Paper summary: Tom: So HANSEL's core claim is that you don't need to read the whole trajectory to verify an agent’s work; instead, they extract a minimal sufficient set of visited pages and snippets. They use a standardized representation of observations, reasoning, and actions to do this abstraction while keeping what you need for verification.

Jane: It claims that by presenting these extracted evidence pages as live, interactive webpages embedded in the system, users can inspect and even correct errors if they find something wrong. This moves verification from being a passive reading task to an active one where you can actually check things on your screen.

Lu: They focus on extracting all evidence pages and snippets that directly support the final answer, discarding anything unsuccessful or blocked like connection errors or login gating, which keeps the evidence focused on what actually mattered for the result.

Meng: That sounds like it solves a real problem for production where you need fast validation without drowning in data. The paper mentions reducing information volume from two hundred seventy-one trajectory pages down to just one hundred four extracted evidence pages, which is a sixty-one point six percent reduction in volume.

Lalam: And they achieved that through an evidence extraction pipeline that uses an LLM to decide which visited pages are important and what specific content on those pages should be included as snippets, while also discarding abandoned plans or blocked pages.

Conclusion: Tom: So looking at HANSEL, the authors are Yujin Zhang, Daye Nam, and they’re showing how to make post-hoc verification much more concrete for web agents. They've moved away from just summaries and logs toward providing interactive evidence.

Jane: The implication is that instead of users blindly trusting a fluent summary of an agent's reasoning, we can give them a tool where they can directly inspect the exact pages the agent used and see exactly what was there when it made its decisions.

Lu: It suggests that for complex tasks like making a purchase or sending an email, giving users this level of access to the actual execution trajectory makes them much more likely to actually verify if the process is sound.

Meng: I think from a deployment view, if you can give end-users a way to click into and fix an error in the evidence directly, that lowers the barrier for trust significantly when deploying these agents.

Lalam: It's about surfacing interactive evidence so users can verify agent behavior after execution, which is much more faithful than relying on potentially inaccurate reasoning summaries.

More episodes

← Home