VizCopilot: Fostering Appropriate Reliance on Enterprise Chatbots with Context Visualization
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "VizCopilot: Fostering Appropriate Reliance on Enterprise Chatbots with Context Visualization".
Jane: Here's the gist of the paper. Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers However, when the retrieved context misaligns with user intentions, these chatbots often produce “irrelevantly right” responses that raise the risk of overreliance This work explores the value of giving users control over context engineering through data visualization to foster appropriate reliance on AI The Problem with Current Context Engineering Enterprise AI chatbots retrieve context from large databases, but autonomous retrieval approaches are susceptible to context misalignment This misalignment manifests in two key issues: first, the retrieved context may be irrelevant to the user’s prompt, and second, the chatbot may synthesize the retrieved context inappropriately due to a lack of background knowledge or incorrect assumptions Current retrieval methods struggle with messy formats and out-of-context terminologies in enterprise databases Design Requirements for Context Control The paper identifies three design requirements to address the challenges of large-scale context.
Tom: These requirements are: DR1 Sensemaking: The need is that users need to develop enough knowledge of context to be able to steer the chatbot.
Jane: Corpus visualizations can scaffold this process by organizing data with topics or keywords, allowing users to navigate information incrementally through progressive disclosure DR2 Verification Support: The system should support users in verification by both raising awareness of the need to verify responses and by reducing the cognitive load of verifying LLM outputs against context data.
Tom: Corpus visualizations can highlight areas requiring verification to minimize the amount of information users must process during verification DR3 Control at Group-Level: Users need the ability to examine and modify context data at a group level rather than file-by-file, ensuring changes are substantial enough to steer chatbots effectively VizCopilot Prototype Design VizCopilot was developed as a research prototype extending an AI chatbot with treemap-based context visualization.
Title and authors: Tom: So the title itself tells us the main goal is fostering appropriate reliance, which is a big deal because we want people to use these tools without blindly trusting them when they might be wrong. It's not about making the AI smarter, it's about making sure humans stay in charge of the process.
Jane: Exactly. They’re looking at how visualization can give users control over context engineering so they can build a better relationship with the AI rather than just being passively reliant on its potentially flawed synthesis.
Lu: The authors explore a specific approach that supports human oversight and intervention through this data visualization, which is their main contribution to the field. They evaluate if this method actually helps foster that sense of control and confidence in users.
Meng: So, it’s not just a nice UI addition; they are testing if this visualization actually changes how people interact with these systems when they're dealing with enterprise data retrieval. That sounds like a necessary test.
Lalam: They report findings from a research-through-design study using the VizCopilot prototype, which showed that incorporating this visualization did lead to improved user confidence and a stronger sense of control over the context provided to the chatbot.
The paper's summary: Tom: What they found is that users who used VizCopilot interacted with it in a more structured way, actually consulting the data context panel or visualization when cross-referencing what the AI gave them with what was actually in the data.
Jane: They also found that users could identify two main types of errors when they looked at the visualization—missing or redundant context, and misinterpretation of what was being presented. These were things they could spot by just glancing at the file view within that visualization.
Lu: This suggests that visualizing the data allows users to develop a better sense of what's relevant and helps them detect when the AI is pulling in garbage context. It scaffolds their knowledge incrementally, which is key for understanding complex enterprise information.
Meng: So, it’s not just about seeing the data; it’s about using that structure to actively check the AI’s assumptions against reality before they make a big decision based on that synthesized answer. That seems like a pragmatic step for deployment.
Lalam: The system organizes information across three levels: first you get a topical overview, then there are AI summaries for those subtopics, and finally you have a file view for raw data inspection if you need to see the details. This progressive disclosure helps manage that cognitive load we talked about earlier.
The paper's improvements: Tom: They suggest three big design requirements to make this work better: first, sensemaking—users need enough knowledge of the context to steer the chatbot, and visualizations can help scaffold that by organizing data by topic or keyword.
Jane: Then there’s verification support. The system should help users know when they actually need to check something and reduce how much mental energy it takes to verify those LLM outputs against the source data.
Lu: And third, they emphasize control at a group level, meaning users shouldn't have to go through every single file one by one; they should be able to examine and modify context data in bulk rather than doing it file-by-file.
Meng: The idea of modifying context at a group level is interesting because if you’re working on a big project, you don't want to manually edit hundreds of files just to subtly change the retrieval behavior for one AI interaction. That sounds efficient at scale.
Lalam: They propose that the visualization can surface misalignment between what the AI retrieved and what the user intended by highlighting expected context areas against the user’s query, which helps users detect and correct that misaligned context before it leads to a bad answer.
Conclusion: Tom: So, to wrap up, they argue that visualization fosters appropriate reliance because it suggests to the user rather than just telling them the answer is right or wrong, forcing that active sensemaking we talked about. It makes the retrieval process visible without creating a big concern about how faithful the AI is being.
Jane: They suggest that incorporating these visualizations into AI systems can actually enhance human cognition by taking those critical but demanding tasks off our plates, leading to a better way for us to collaborate with AI.
Lu: The limitation they point out is that they used a synthetic dataset for their study, and there’s also the need to improve how computationally scalable this visualization method is when dealing with much larger real-world enterprise datasets.
Meng: From my side, I worry about the scaling issue. If this visualization has to process petabytes of data, we're going to run into serious computational bottlenecks unless they solve that efficiency problem first.
Lalam: The paper on VizCopilot is about giving users control over context engineering through data visualization, and it shows that when you give people tools to see the structure and verify the context, their trust in the AI actually improves because they are actively involved.
University of California, Davis University of California, Davis Department of Computer Science Microsoft Microsoft
cs.HC
Submitted: 2025-10-13
Updated: 2026-11-12
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 76/100
The gist: Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers However…
Key concepts
- Context Misalignment
- This occurs when the data retrieved by an AI chatbot does not align with what the user actually intended. It can mean the context is entirely irrelevant or that the chatbot misinterprets it due to lacking background knowledge, resulting in an 'irrelevantly right' answer.
- Sensemaking
- This is a crucial skill where users develop enough understanding of the data context to guide the AI correctly. The paper suggests using visualizations to scaffold this process by organizing complex information into topics or keywords, helping users navigate and understand the data incrementally.
- Progressive Disclosure
- This design technique organizes information in layers, starting with a high-level overview and allowing users to drill down into more detailed views only when necessary. This reduces cognitive load by preventing users from being overwhelmed by too much raw data at once.
- Group-Level Control
- This feature enables users to modify context data for the entire group simultaneously, rather than having to edit files individually. This allows for substantial changes that are effective in steering the chatbot's behavior across multiple documents efficiently.
Terminology
Summary
Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers However, when the retrieved context misaligns with user intentions, these chatbots often produce “irrelevantly right” responses that raise the risk of overreliance This work explores the value of giving users control over context engineering through data visualization to foster appropriate reliance on AI
The Problem with Current Context Engineering
Enterprise AI chatbots retrieve context from large databases, but autonomous retrieval approaches are susceptible to context misalignment This misalignment manifests in two key issues: first, the retrieved context may be irrelevant to the user’s prompt, and second, the chatbot may synthesize the retrieved context inappropriately due to a lack of background knowledge or incorrect assumptions Current retrieval methods struggle with messy formats and out-of-context terminologies in enterprise databases
Design Requirements for Context Control
The paper identifies three design requirements to address the challenges of large-scale context. These requirements are:
** DR1 Sensemaking:**
The need is that users need to develop enough knowledge of context to be able to steer the chatbot. Corpus visualizations can scaffold this process by organizing data with topics or keywords, allowing users to navigate information incrementally through progressive disclosure
** DR2 Verification Support:**
The system should support users in verification by both raising awareness of the need to verify responses and by reducing the cognitive load of verifying LLM outputs against context data. Corpus visualizations can highlight areas requiring verification to minimize the amount of information users must process during verification
** DR3 Control at Group-Level:**
Users need the ability to examine and modify context data at a group level rather than file-by-file, ensuring changes are substantial enough to steer chatbots effectively
VizCopilot Prototype Design
VizCopilot was developed as a research prototype extending an AI chatbot with treemap-based context visualization. The interface consists of a visualization panel and a chat panel. p. 5 6 Sam Yu-Te lee, Jingya Chen, Albert Calzaretto, Richard Lee, Samir Passi, Alice Ferng, and Mihaela Vorvoreanu
4 VizCopilot: Prototype Design
Based on these design requirements, we developed our research prototype: VizCopilot, an interactive prototype that
extends the typical chat interface with a context visualization panel. VizCopilot is not intended as a mature system;
rather, it serves as a design probe that operationalizes the design requirements to examine the viability of giving users
control over context engineering through data visualization and to surface opportunities for future design.Where the paper says it
User Study Findings
A qualitative user study with 14 participants compared VizCopilot with a chat-only interface. Key findings include:
** Theme 3.3:**
Participants demonstrated a more structured interaction flow with VizCopilot, consulting the data context panel or visualization to cross-reference context with the response.
** Theme 2.4:**
Participants were able to identify two major types of errors: missing or redundant context and misinterpretation of context, which they could detect by inspecting the file view in the visualization
Discussion and Future Directions
Visualization fosters appropriate reliance by suggesting rather than telling users about potentially incorrect answers, requiring active sensemaking. Transparency through visualization makes the retrieval process visible without the concern of faithfulness, which can significantly reduce cognitive load Design trade-offs involve a choice between immediate usability and long-term goals, with default visualization chosen to support sensemaking even before a prompt. p. 2 Based on these insights, we discuss the potential
for visualization to help foster appropriate reliance through increased transparency and user control, design trade-offs
regarding usability, as well as a vision to facilitate a sustainable human-AI collaboration paradigm. This work makes the following contributions:
• We explore a context engineering approach that supports human oversight and intervention through data
visualization, and evaluate its potential for fostering appropriate reliance.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Context Visualization to Detect Misalignment: The system can use topic structures in a treemap to
surface misalignment between retrieval and user intent
by highlighting expected context areas against the user's query, helping usersdetect and correct misaligned context.
-
Progressive Disclosure for Cognitive Load Reduction: Implement a three-level information organization—topical overview, AI summaries for subtopics, and file view—to allow users to navigate data incrementally, reducing the
cognitive load of context sensemaking
by allowing deeper inspection only when needed. -
Group-Level Context Modification: Allow users to modify context at a group level rather than file-by-file by supporting
direct manipulation of visual elements in the corpus visualization,
enabling efficient and precise context modification to steer chatbot behavior. -
Verification Support through Highlighted Areas: The system should support verification by highlighting
areas of context that require verification
within the visualization, allowing users to focus their cross-referencing efforts andreduce the cognitive load of verifying LLM outputs against context data.
-
Transparency in Generative Steps: Make the
intermediate steps of this mechanism visible,
which helps participants reason about why and how responses are generated, leading to increased trust by showingwhere VizCopilot gets the data from.
-
Error Identification through Visual Cues: The system can use visualization to help users identify errors, such as missing context or misinterpretation, where participants were able to identify issues by
glancing at the file view in the visualization
and spotting sparse highlighting for irrelevant topics.
Abstract
Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers. However, when the retrieved context misaligns with user intentions, these chatbots often produce ``irrelevantly right'' responses that raise the risk of overreliance. To foster appropriate reliance, we need a different approach than asking users to verify chatbot outputs. We posit that human oversight and modification of the context retrieval process can improve the quality of chatbots' outputs and users' awareness, resulting in more appropriate reliance and higher accountability. In this work, we explore this approach through a Research-through-Design study using VizCopilot, a chatbot extended with corpus visualization techniques. Through a within-subject study comparing VizCopilot and a simplified pure-text chatbot, we evaluate the role of visualization in enterprise chatbots and surface future design opportunities for fostering appropriate reliance through context engineering. Our findings show that visualization not only helps users detect and correct misaligned context, but also enhances user agency and system transparency. At the same time, the study reveals limitations regarding verification support and trust in AI summaries. Based on these findings, we discuss how visualization can foster appropriate reliance, the associated design trade-offs, and our vision towards transparent and sustainable human-AI interaction.
Sources
- A Qualitative Study of User Perception of M365 AI Copilot
- "It makes you think": Provocations Help Restore Critical Thinking to AI-Assisted Knowledge Work
- BERTopic: Neural topic modeling with a class-based TF-IDF procedure
- Dense Passage Retrieval for Open-Domain Question Answering
- Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback
- The Role of Visualization in LLM-Assisted Knowledge Graph Systems: Effects on User Trust, Exploration, and Workflows
- A Survey of Context Engineering for Large Language Models
- Evaluating Retrieval Quality in Retrieval-Augmented Generation
- Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong
Related papers
- EmoPhone: A Multi-Wave Dataset for In-the-Wild Mobile and Wearable Affect Sensing
- Tasteply: Externalizing Stable and Situational Preference as Manipulable Spatial Objects
- Shielded Trajectory Sampling for Personalized Relational Safety in Conversational AI
- Reconstruction and Reflection of Positive Experiences through Resurfacing Laughter-indexed Everyday Moments
- LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems
- Explainable Iterative Data Visualization Refinement via an LLM Agent