HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents
summary
The gist
AI web agents can perform complex, multi-step tasks such as searching for products, comparing options, and making purchases on behalf of users HANSEL helps you verify without reading full trajectory.
In short
HANSEL extracts minimal, interactive evidence from web agent trajectories to replace passive log reading for verification. It transforms raw execution data into a focused set of relevant pages, allowing users to directly inspect and correct errors in a live interface. Empirical results show HANSEL significantly reduces user effort and completion time while improving verification accuracy.
Key concepts
- Passive Verification Methods
- Current methods treat agent logs, screenshots, or summaries as passive reading tasks. Users must sift through overwhelming data to verify an agent's work, which often leads to low trust because the evidence is not interactive or contextually rich.
- Evidence Extraction Pipeline
- HANSEL uses a standardized trajectory representation ($ ext{observation}, ext{reasoning}, ext{action}$) and an LLM to intelligently select only the pages and snippets directly supporting the final answer, discarding irrelevant data like errors or blocked pages.
- Interactive Verification Interface
- This design goal renders extracted evidence as live, interactive webpages within HANSEL. Users can inspect these pages in a state similar to when the agent reached them, allowing them to directly check and correct errors without manually replaying complex actions.
Terminology used across episodes
This episode discusses
- HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents · Paper Radio
- Challenges in Human-Agent Communication
- Measuring Progress on Scalable Oversight for Large Language Models
- Visibility into AI Agents
- Evaluating Long-Context Reasoning in LLM-Based WebAgents
- Mind2Web: Towards a Generalist Agent for the Web
- Cocoa: Co-Planning and Co-Execution with AI Agents
- Learning to Navigate the Web
- WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- Measuring Faithfulness in Chain-of-Thought Reasoning
- AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
- Ask Not What AI Can Do, But What AI Should Do: Towards a Framework of Task Delegability
- GAIA: a benchmark for General AI Assistants
- Magentic-UI: Towards Human-in-the-loop Agentic Systems
- NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild
- WebGPT: Browser-assisted question-answering with human feedback
- A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models
- On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
- On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)
- Explanations Can Reduce Overreliance on AI Systems During Decision-Making
The paper
HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents · Read on arXiv
Yujin Zhang, Daye Nam
University of California, Irvine
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents".
Jane: Here's the gist of the paper. AI web agents can perform complex, multi-step tasks such as searching for products, comparing options, and making purchases on behalf of users HANSEL helps you verify without reading full trajectory.
Tom: The Problem with Current Verification Methods Existing transparency mechanisms treat verification as a passive reading task, leaving users to sift through overwhelming logs or trust potentially unfaithful explanations.
Jane: Execution logs record every page visited, click, search query, and reasoning step, which in principle allows users to trace how the agent worked but in practice produces an overwhelming log that few users will read.
Tom: Source links point users to the pages the agent visited but lose important context such as the page state when clicked.
Paper summary: Tom: So HANSEL's core claim is that you don't need to read the whole trajectory to verify an agent’s work; instead, they extract a minimal sufficient set of visited pages and snippets. They use a standardized representation of observations, reasoning, and actions to do this abstraction while keeping what you need for verification.
Jane: It claims that by presenting these extracted evidence pages as live, interactive webpages embedded in the system, users can inspect and even correct errors if they find something wrong. This moves verification from being a passive reading task to an active one where you can actually check things on your screen.
Lu: They focus on extracting all evidence pages and snippets that directly support the final answer, discarding anything unsuccessful or blocked like connection errors or login gating, which keeps the evidence focused on what actually mattered for the result.
Meng: That sounds like it solves a real problem for production where you need fast validation without drowning in data. The paper mentions reducing information volume from two hundred seventy-one trajectory pages down to just one hundred four extracted evidence pages, which is a sixty-one point six percent reduction in volume.
Lalam: And they achieved that through an evidence extraction pipeline that uses an LLM to decide which visited pages are important and what specific content on those pages should be included as snippets, while also discarding abandoned plans or blocked pages.
Conclusion: Tom: So looking at HANSEL, the authors are Yujin Zhang, Daye Nam, and they’re showing how to make post-hoc verification much more concrete for web agents. They've moved away from just summaries and logs toward providing interactive evidence.
Jane: The implication is that instead of users blindly trusting a fluent summary of an agent's reasoning, we can give them a tool where they can directly inspect the exact pages the agent used and see exactly what was there when it made its decisions.
Lu: It suggests that for complex tasks like making a purchase or sending an email, giving users this level of access to the actual execution trajectory makes them much more likely to actually verify if the process is sound.
Meng: I think from a deployment view, if you can give end-users a way to click into and fix an error in the evidence directly, that lowers the barrier for trust significantly when deploying these agents.
Lalam: It's about surfacing interactive evidence so users can verify agent behavior after execution, which is much more faithful than relying on potentially inaccurate reasoning summaries.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game