VizCopilot: Fostering Appropriate Reliance on Enterprise Chatbots with Context Visualization

arXiv:2510.11954 · cs.HC · Submitted 2025-10-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "VizCopilot: Fostering Appropriate Reliance on Enterprise Chatbots with Context Visualization".

Jane: Here's the gist of the paper. Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers However, when the retrieved context misaligns with user intentions, these chatbots often produce “irrelevantly right” responses that raise the risk of overreliance This work explores the value of giving users control over context engineering through data visualization to foster appropriate reliance on AI The Problem with Current Context Engineering Enterprise AI chatbots retrieve context from large databases, but autonomous retrieval approaches are susceptible to context misalignment This misalignment manifests in two key issues: first, the retrieved context may be irrelevant to the user’s prompt, and second, the chatbot may synthesize the retrieved context inappropriately due to a lack of background knowledge or incorrect assumptions Current retrieval methods struggle with messy formats and out-of-context terminologies in enterprise databases Design Requirements for Context Control The paper identifies three design requirements to address the challenges of large-scale context.

Tom: These requirements are: DR1 Sensemaking: The need is that users need to develop enough knowledge of context to be able to steer the chatbot.

Jane: Corpus visualizations can scaffold this process by organizing data with topics or keywords, allowing users to navigate information incrementally through progressive disclosure DR2 Verification Support: The system should support users in verification by both raising awareness of the need to verify responses and by reducing the cognitive load of verifying LLM outputs against context data.

Tom: Corpus visualizations can highlight areas requiring verification to minimize the amount of information users must process during verification DR3 Control at Group-Level: Users need the ability to examine and modify context data at a group level rather than file-by-file, ensuring changes are substantial enough to steer chatbots effectively VizCopilot Prototype Design VizCopilot was developed as a research prototype extending an AI chatbot with treemap-based context visualization.

Title and authors: Tom: So the title itself tells us the main goal is fostering appropriate reliance, which is a big deal because we want people to use these tools without blindly trusting them when they might be wrong. It's not about making the AI smarter, it's about making sure humans stay in charge of the process.

Jane: Exactly. They’re looking at how visualization can give users control over context engineering so they can build a better relationship with the AI rather than just being passively reliant on its potentially flawed synthesis.

Lu: The authors explore a specific approach that supports human oversight and intervention through this data visualization, which is their main contribution to the field. They evaluate if this method actually helps foster that sense of control and confidence in users.

Meng: So, it’s not just a nice UI addition; they are testing if this visualization actually changes how people interact with these systems when they're dealing with enterprise data retrieval. That sounds like a necessary test.

Lalam: They report findings from a research-through-design study using the VizCopilot prototype, which showed that incorporating this visualization did lead to improved user confidence and a stronger sense of control over the context provided to the chatbot.

The paper's summary: Tom: What they found is that users who used VizCopilot interacted with it in a more structured way, actually consulting the data context panel or visualization when cross-referencing what the AI gave them with what was actually in the data.

Jane: They also found that users could identify two main types of errors when they looked at the visualization—missing or redundant context, and misinterpretation of what was being presented. These were things they could spot by just glancing at the file view within that visualization.

Lu: This suggests that visualizing the data allows users to develop a better sense of what's relevant and helps them detect when the AI is pulling in garbage context. It scaffolds their knowledge incrementally, which is key for understanding complex enterprise information.

Meng: So, it’s not just about seeing the data; it’s about using that structure to actively check the AI’s assumptions against reality before they make a big decision based on that synthesized answer. That seems like a pragmatic step for deployment.

Lalam: The system organizes information across three levels: first you get a topical overview, then there are AI summaries for those subtopics, and finally you have a file view for raw data inspection if you need to see the details. This progressive disclosure helps manage that cognitive load we talked about earlier.

The paper's improvements: Tom: They suggest three big design requirements to make this work better: first, sensemaking—users need enough knowledge of the context to steer the chatbot, and visualizations can help scaffold that by organizing data by topic or keyword.

Jane: Then there’s verification support. The system should help users know when they actually need to check something and reduce how much mental energy it takes to verify those LLM outputs against the source data.

Lu: And third, they emphasize control at a group level, meaning users shouldn't have to go through every single file one by one; they should be able to examine and modify context data in bulk rather than doing it file-by-file.

Meng: The idea of modifying context at a group level is interesting because if you’re working on a big project, you don't want to manually edit hundreds of files just to subtly change the retrieval behavior for one AI interaction. That sounds efficient at scale.

Lalam: They propose that the visualization can surface misalignment between what the AI retrieved and what the user intended by highlighting expected context areas against the user’s query, which helps users detect and correct that misaligned context before it leads to a bad answer.

Conclusion: Tom: So, to wrap up, they argue that visualization fosters appropriate reliance because it suggests to the user rather than just telling them the answer is right or wrong, forcing that active sensemaking we talked about. It makes the retrieval process visible without creating a big concern about how faithful the AI is being.

Jane: They suggest that incorporating these visualizations into AI systems can actually enhance human cognition by taking those critical but demanding tasks off our plates, leading to a better way for us to collaborate with AI.

Lu: The limitation they point out is that they used a synthetic dataset for their study, and there’s also the need to improve how computationally scalable this visualization method is when dealing with much larger real-world enterprise datasets.

Meng: From my side, I worry about the scaling issue. If this visualization has to process petabytes of data, we're going to run into serious computational bottlenecks unless they solve that efficiency problem first.

Lalam: The paper on VizCopilot is about giving users control over context engineering through data visualization, and it shows that when you give people tools to see the structure and verify the context, their trust in the AI actually improves because they are actively involved.

University of California, Davis University of California, Davis Department of Computer Science Microsoft Microsoft

cs.HC

Submitted: 2025-10-13

Updated: 2026-11-12

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 76/100

The gist: Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers However…

Key concepts

Context Misalignment
This occurs when the data retrieved by an AI chatbot does not align with what the user actually intended. It can mean the context is entirely irrelevant or that the chatbot misinterprets it due to lacking background knowledge, resulting in an 'irrelevantly right' answer.
Sensemaking
This is a crucial skill where users develop enough understanding of the data context to guide the AI correctly. The paper suggests using visualizations to scaffold this process by organizing complex information into topics or keywords, helping users navigate and understand the data incrementally.
Progressive Disclosure
This design technique organizes information in layers, starting with a high-level overview and allowing users to drill down into more detailed views only when necessary. This reduces cognitive load by preventing users from being overwhelmed by too much raw data at once.
Group-Level Control
This feature enables users to modify context data for the entire group simultaneously, rather than having to edit files individually. This allows for substantial changes that are effective in steering the chatbot's behavior across multiple documents efficiently.

Terminology

Summary

Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers However, when the retrieved context misaligns with user intentions, these chatbots often produce “irrelevantly right” responses that raise the risk of overreliance This work explores the value of giving users control over context engineering through data visualization to foster appropriate reliance on AI

The Problem with Current Context Engineering

Enterprise AI chatbots retrieve context from large databases, but autonomous retrieval approaches are susceptible to context misalignment This misalignment manifests in two key issues: first, the retrieved context may be irrelevant to the user’s prompt, and second, the chatbot may synthesize the retrieved context inappropriately due to a lack of background knowledge or incorrect assumptions Current retrieval methods struggle with messy formats and out-of-context terminologies in enterprise databases

Design Requirements for Context Control

The paper identifies three design requirements to address the challenges of large-scale context. These requirements are:

** DR1 Sensemaking:**

The need is that users need to develop enough knowledge of context to be able to steer the chatbot. Corpus visualizations can scaffold this process by organizing data with topics or keywords, allowing users to navigate information incrementally through progressive disclosure

** DR2 Verification Support:**

The system should support users in verification by both raising awareness of the need to verify responses and by reducing the cognitive load of verifying LLM outputs against context data. Corpus visualizations can highlight areas requiring verification to minimize the amount of information users must process during verification

** DR3 Control at Group-Level:**

Users need the ability to examine and modify context data at a group level rather than file-by-file, ensuring changes are substantial enough to steer chatbots effectively

VizCopilot Prototype Design

VizCopilot was developed as a research prototype extending an AI chatbot with treemap-based context visualization. The interface consists of a visualization panel and a chat panel. p. 5

Where the paper says it

6 Sam Yu-Te lee, Jingya Chen, Albert Calzaretto, Richard Lee, Samir Passi, Alice Ferng, and Mihaela Vorvoreanu 4 VizCopilot: Prototype Design Based on these design requirements, we developed our research prototype: VizCopilot, an interactive prototype that extends the typical chat interface with a context visualization panel. VizCopilot is not intended as a mature system; rather, it serves as a design probe that operationalizes the design requirements to examine the viability of giving users control over context engineering through data visualization and to surface opportunities for future design.

The extended treemap visualizes context, where each cell represents a topic, and data items are represented as circles to support volume estimation Coordination between the visualization and chat panel allows for highlighting retrieved context and supports group-level control through subtopic thumbnails Progressive disclosure organizes information across three levels: topical overview, AI summaries for subtopics, and a file view for raw data inspection

User Study Findings

A qualitative user study with 14 participants compared VizCopilot with a chat-only interface. Key findings include:

** Theme 3.3:**

Participants demonstrated a more structured interaction flow with VizCopilot, consulting the data context panel or visualization to cross-reference context with the response.

** Theme 2.4:**

Participants were able to identify two major types of errors: missing or redundant context and misinterpretation of context, which they could detect by inspecting the file view in the visualization

Discussion and Future Directions

Visualization fosters appropriate reliance by suggesting rather than telling users about potentially incorrect answers, requiring active sensemaking. Transparency through visualization makes the retrieval process visible without the concern of faithfulness, which can significantly reduce cognitive load Design trade-offs involve a choice between immediate usability and long-term goals, with default visualization chosen to support sensemaking even before a prompt. p. 2

Where the paper says it

Based on these insights, we discuss the potential for visualization to help foster appropriate reliance through increased transparency and user control, design trade-offs regarding usability, as well as a vision to facilitate a sustainable human-AI collaboration paradigm. This work makes the following contributions: • We explore a context engineering approach that supports human oversight and intervention through data visualization, and evaluate its potential for fostering appropriate reliance.

The work suggests that incorporating visualizations into AI systems can enhance human cognition by offloading critical yet cognitively demanding tasks, leading to a sustainable paradigm of human–AI collaboration Limitations include the use of a synthetic dataset and the need for improved computational scalability for larger datasets.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Context Visualization to Detect Misalignment: The system can use topic structures in a treemap to surface misalignment between retrieval and user intent by highlighting expected context areas against the user's query, helping users detect and correct misaligned context.

  2. Progressive Disclosure for Cognitive Load Reduction: Implement a three-level information organization—topical overview, AI summaries for subtopics, and file view—to allow users to navigate data incrementally, reducing the cognitive load of context sensemaking by allowing deeper inspection only when needed.

  3. Group-Level Context Modification: Allow users to modify context at a group level rather than file-by-file by supporting direct manipulation of visual elements in the corpus visualization, enabling efficient and precise context modification to steer chatbot behavior.

  4. Verification Support through Highlighted Areas: The system should support verification by highlighting areas of context that require verification within the visualization, allowing users to focus their cross-referencing efforts and reduce the cognitive load of verifying LLM outputs against context data.

  5. Transparency in Generative Steps: Make the intermediate steps of this mechanism visible, which helps participants reason about why and how responses are generated, leading to increased trust by showing where VizCopilot gets the data from.

  6. Error Identification through Visual Cues: The system can use visualization to help users identify errors, such as missing context or misinterpretation, where participants were able to identify issues by glancing at the file view in the visualization and spotting sparse highlighting for irrelevant topics.

Abstract

Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers. However, when the retrieved context misaligns with user intentions, these chatbots often produce ``irrelevantly right'' responses that raise the risk of overreliance. To foster appropriate reliance, we need a different approach than asking users to verify chatbot outputs. We posit that human oversight and modification of the context retrieval process can improve the quality of chatbots' outputs and users' awareness, resulting in more appropriate reliance and higher accountability. In this work, we explore this approach through a Research-through-Design study using VizCopilot, a chatbot extended with corpus visualization techniques. Through a within-subject study comparing VizCopilot and a simplified pure-text chatbot, we evaluate the role of visualization in enterprise chatbots and surface future design opportunities for fostering appropriate reliance through context engineering. Our findings show that visualization not only helps users detect and correct misaligned context, but also enhances user agency and system transparency. At the same time, the study reveals limitations regarding verification support and trust in AI summaries. Based on these findings, we discuss how visualization can foster appropriate reliance, the associated design trade-offs, and our vision towards transparent and sustainable human-AI interaction.

Sources

Related papers