HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "HANSEL: Interactive Evidence for Post-Hoc Verification of Web Agents".
Jane: Here's the gist of the paper. AI web agents can perform complex, multi-step tasks such as searching for products, comparing options, and making purchases on behalf of users HANSEL helps you verify without reading full trajectory.
Tom: The Problem with Current Verification Methods Existing transparency mechanisms treat verification as a passive reading task, leaving users to sift through overwhelming logs or trust potentially unfaithful explanations.
Jane: Execution logs record every page visited, click, search query, and reasoning step, which in principle allows users to trace how the agent worked but in practice produces an overwhelming log that few users will read.
Tom: Source links point users to the pages the agent visited but lose important context such as the page state when clicked.
Paper summary: Tom: So HANSEL's core claim is that you don't need to read the whole trajectory to verify an agent’s work; instead, they extract a minimal sufficient set of visited pages and snippets. They use a standardized representation of observations, reasoning, and actions to do this abstraction while keeping what you need for verification.
Jane: It claims that by presenting these extracted evidence pages as live, interactive webpages embedded in the system, users can inspect and even correct errors if they find something wrong. This moves verification from being a passive reading task to an active one where you can actually check things on your screen.
Lu: They focus on extracting all evidence pages and snippets that directly support the final answer, discarding anything unsuccessful or blocked like connection errors or login gating, which keeps the evidence focused on what actually mattered for the result.
Meng: That sounds like it solves a real problem for production where you need fast validation without drowning in data. The paper mentions reducing information volume from two hundred seventy-one trajectory pages down to just one hundred four extracted evidence pages, which is a sixty-one point six percent reduction in volume.
Lalam: And they achieved that through an evidence extraction pipeline that uses an LLM to decide which visited pages are important and what specific content on those pages should be included as snippets, while also discarding abandoned plans or blocked pages.
Conclusion: Tom: So looking at HANSEL, the authors are Yujin Zhang, Daye Nam, and they’re showing how to make post-hoc verification much more concrete for web agents. They've moved away from just summaries and logs toward providing interactive evidence.
Jane: The implication is that instead of users blindly trusting a fluent summary of an agent's reasoning, we can give them a tool where they can directly inspect the exact pages the agent used and see exactly what was there when it made its decisions.
Lu: It suggests that for complex tasks like making a purchase or sending an email, giving users this level of access to the actual execution trajectory makes them much more likely to actually verify if the process is sound.
Meng: I think from a deployment view, if you can give end-users a way to click into and fix an error in the evidence directly, that lowers the barrier for trust significantly when deploying these agents.
Lalam: It's about surfacing interactive evidence so users can verify agent behavior after execution, which is much more faithful than relying on potentially inaccurate reasoning summaries.
Yujin Zhang, Daye Nam
University of California, Irvine
cs.HC
Submitted: 2026-06-17
Updated: 2026-11-09
Comments: 26 pages, 7 figures
Code: https://github.com/cloudsreal/hansel_study
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 85/100
The gist: AI web agents can perform complex, multi-step tasks such as searching for products, comparing options, and making purchases on behalf of users HANSEL helps you verify without reading full trajectory.
Key concepts
- Passive Verification Methods
- Current methods treat agent logs, screenshots, or summaries as passive reading tasks. Users must sift through overwhelming data to verify an agent's work, which often leads to low trust because the evidence is not interactive or contextually rich.
- Evidence Extraction Pipeline
- HANSEL uses a standardized trajectory representation ($ ext{observation}, ext{reasoning}, ext{action}$) and an LLM to intelligently select only the pages and snippets directly supporting the final answer, discarding irrelevant data like errors or blocked pages.
- Interactive Verification Interface
- This design goal renders extracted evidence as live, interactive webpages within HANSEL. Users can inspect these pages in a state similar to when the agent reached them, allowing them to directly check and correct errors without manually replaying complex actions.
Terminology
Summary
AI web agents can perform complex, multi-step tasks such as searching for products, comparing options, and making purchases on behalf of users HANSEL helps you verify without reading full trajectory. p. 0
The Problem with Current Verification Methods
Existing transparency mechanisms treat verification as a passive reading task, leaving users to sift through overwhelming logs or trust potentially unfaithful explanations. Execution logs record every page visited, click, search query, and reasoning step, which in principle allows users to trace how the agent worked but in practice produces an overwhelming log that few users will read. p. 1 Source links point users to the pages the agent visited but lose important context such as the page state when clicked. p. 1 Screenshots are static and users cannot interact with them to check whether a filter was correctly applied or if content is off-screen. p. 3 Reasoning summaries can rationalize incorrect answers as fluently as correct ones, increasing over-reliance by creating an illusion that verification has already been performed. p. 3 The common gap across all four current mechanisms is that verification is still treated as a passive reading task; the user is a consumer of logs, images, links, or text. p. 3
HANSEL's Evidence Extraction Pipeline
HANSEL extracts interactive evidence pages and snippets from the agent’s trajectory log to surface a minimal sufficient set of visited pages that directly support the final answer. The system uses a standardized trajectory representation ⟨observation, reasoning, action⟩ to abstract low-level execution details while preserving information necessary for evidence extraction and answer verification. p. 5 The first goal of HANSEL is to extract all evidence pages and identify all evidence snippets on each page that directly support the final answer. p. 5 This involves using an LLM to identify which visited pages are evidence pages and which specific content on those pages should be included as evidence snippets. p. 5 To ensure conciseness, prompt rules instruct the LLM to discard abandoned or unsuccessful plans and remove blocked pages like connection errors or login gating. p. 5
Interactive Verification Interface Design
HANSEL's design is guided by three goals: Surface a Minimal Sufficient Evidence Set (DG.1), Enable Direct Evidence Access (DG. p. 32), and Improve Evidence Gap Awareness (DG. p. 33). DG.1 focuses on reducing evidence to what matters by extracting only the subset of visited pages that directly support the agent’s answer. p. 1 DG.2 enables direct evidence access by rendering each page as a live, interactive webpage embedded within HANSEL, where users can inspect and correct errors when necessary. p. 5 This is achieved by reconstructing each page into the state the agent reached through replaying scripted actions so users do not need to reapply these actions manually. p. 5 DG.3 improves evidence gap awareness by presenting evidence pages in two complementary layouts: grid view and carousel view, allowing users to spot errors in high-level reasoning or inspect them step-by-step. p. 5
Empirical Results of HANSEL
The technical evaluation revealed that HANSEL accurately identifies core evidence pages from agent trajectories, achieving 83.7% precision and 88. p. 08% recall against ground-truth annotations across 45 tasks from AssistantBench [52] and OnlineMind2Web [49]. p. 1 HANSEL reduced the information volume from 271 trajectory pages to 104 extracted evidence pages, representing a 61. p. 66% reduction in volume. p. 6 In a controlled user study with 14 participants, HANSEL significantly reduced task completion time and perceived effort compared to a standard agent interface. p. 0 Participants also reported significantly lower effort in their judgments when using HANSEL compared to the baseline. p. 1
User Perception and Implications
Participants consistently rated HANSEL higher than the baseline across all five dimensions, including Understanding, Verification, Error Identification, Correction, and Usability. Participants found HANSEL significantly easier to use (mu = 2. p. 893 → 4.71) and to verify the agent’s response (mu = 2. p. 143 → 4.43). Qualitative analysis showed that participants used fewer web pages when using HANSEL than when using the baseline, consistently browsing evidence pages in order directly within the interface. p. 9 However, some participants still confidently submitted incorrect answers when using HANSEL, indicating a risk of miscalibrated trust. p. 8 The paper suggests that confidence alone is not a reliable indicator of successful verification and that future design should incorporate cognitive forcing functions to prompt critical engagement with evidence before accepting an answer.
Limitations and Future Directions
Limitations include the potential for learning effects, as participants could improve at verification over the course of the session. The study setting also differs from everyday use, potentially introducing latency and reduced visual quality through remote control features. p. 7 HANSEL’s effectiveness depends on the quality of the agent’s trajectory and how well it aligns with users’ mental models of the task. p. 0 Furthermore, a single-session lab study does not capture long-term adaptation, and its current BrowserView-based page reconstruction may not scale well to large-scale deployment. The paper also notes that some agent answers cannot be verified through page evidence, such as those relying on domain knowledge where no corresponding evidence pages exist. p. 9
Conclusion
We designed HANSEL, a system that presents evidence from web agent trajectories as interactive, verifiable artifacts. Our technical evaluation showed accurate evidence extraction (F1 = 0. p. 6861) with a 61. p. 66% reduction in trajectory volume. p. 6 A user study with 14 participants demonstrated that HANSEL significantly reduces completion time and effort while improving usability. p. 0 We hope that HANSEL’s approach of surfacing interactive evidence for agent verification can inspire further research and adoption in real-world agent products where robust verification support remains an open and pressing need. p. 2
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Evidence Extraction Pipeline for Minimal Sufficient Set: Implement an automated pipeline that extracts only those pages and snippets that
directly support
the final answer, as defined by a step contributing to reasoning toward the final answer. This reduces evidence volume by 61.6% while maintaining high precision in identifying supporting content. -
Interactive Evidence Reconstruction: Develop a system that reconstructs each evidence page into the exact state the agent reached during execution, allowing users to
inspect and correct errors when necessary
without needing to manually reapply actions like filters or search queries. This ensures that users canresume from the last successful step rather than restarting the task from scratch.
-
Multi-View Reasoning Visualization: Provide dual layouts for evidence pages, including a grid view for spotting high-level reasoning errors and a carousel view with a timeline and embedded browser window for sequential inspection. This allows users to
spot errors in the high-level reasoning
or inspect pagesin order,
aiding in identifying where an agent's claims areunverifiable or potentially incorrect.
-
State-Aware Verification Interface: Design an interface that allows users to
directly interact with the embedded page, including clicking, scrolling, inputting, or navigating,
while highlighting evidence snippets directly on the live page. This facilitates verification by allowing users tomodify them directly on the webpage within the interface.
-
Adaptive Evidence Modeling: Extend the extraction pipeline to adapt its definition of evidence based on task structure rather than solely relying on webpages. For subjective tasks, this would involve identifying
decision steps as evidence
where users can assess if agent choicesreflect their actual preferences and revise them if needed.
Sources
- Challenges in Human-Agent Communication
- Measuring Progress on Scalable Oversight for Large Language Models
- Visibility into AI Agents
- Evaluating Long-Context Reasoning in LLM-Based WebAgents
- Mind2Web: Towards a Generalist Agent for the Web
- Cocoa: Co-Planning and Co-Execution with AI Agents
- Learning to Navigate the Web
- WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- Measuring Faithfulness in Chain-of-Thought Reasoning
- AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories
- Ask Not What AI Can Do, But What AI Should Do: Towards a Framework of Task Delegability
- GAIA: a benchmark for General AI Assistants
- Magentic-UI: Towards Human-in-the-loop Agentic Systems
- NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild
- WebGPT: Browser-assisted question-answering with human feedback
- A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models
- On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
- On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)
- Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Related papers
- EmoPhone: A Multi-Wave Dataset for In-the-Wild Mobile and Wearable Affect Sensing
- Tasteply: Externalizing Stable and Situational Preference as Manipulable Spatial Objects
- Shielded Trajectory Sampling for Personalized Relational Safety in Conversational AI
- Reconstruction and Reflection of Positive Experiences through Resurfacing Laughter-indexed Everyday Moments
- LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems
- Explainable Iterative Data Visualization Refinement via an LLM Agent