One Is Not Enough: How People Use Multiple AI Models in Everyday Life

arXiv:2603.26107 · cs.HC, cs.AI · Submitted 2026-03-27 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "One Is Not Enough".

Jane: The gist: People increasingly use multiple Multimodal Large Language Models (MLLMs) concurrently, selecting each based on its perceived strengths.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper called "One Is Not Enough: How People Use Multiple AI Models in Everyday Life". It looks at how people actually use several Multimodal Large Language Models concurrently, choosing each one based on what they think each one is best at.

Jane: That seems like a really practical problem because when you're juggling different AIs, there are these coordination challenges. Things like figuring out how to adjust your prompts for different interfaces and trying to keep trust consistent when the models behave differently depending on what you're asking them to do.

Lu: Prior research has really focused on single-agent interactions, so this paper is looking at the orchestration of multiple AI systems, which hasn't been explored much before. They did a diary study with ten participants and some interviews to see how people organize their work across these competing systems.

Meng: I'm curious about what they found about how users structure these hierarchies. Are we talking about a fixed setup or something more flexible?

Tom: It turns out users build primary and secondary hierarchies among the models that they switch between depending on the situation. The paper shows this structure actually shifts based on whether you're using the AI for personal stuff versus professional work.

Jane: That shift is interesting because it suggests that what’s most important to a user changes depending on their context, like personal time versus work time.

Lu: They found three recurring hierarchy patterns across the participants in this study: a cross-context primary structure, a work-focused structure, and a split-primary structure.

Meng: That sounds complex to manage. Can you tell me more about what those different structures actually look like in practice?

Tom: For the cross-context primary structure, four participants centered their workflows around one favorite model they used for both personal and professional life. They saw this favorite model as a general-purpose assistant and kept their interactions pretty consolidated.

Jane: So it sounds like some people have one go-to AI that handles everything from casual questions to work tasks because they just feel familiar with it.

Lu: Then there was the work-focused structure, where three participants kept one primary model dedicated specifically to professional tasks, treating it like a productivity tool. They deliberately didn't feel a strong need to use that primary model in personal contexts.

Meng: That makes sense for someone who really needs reliability for their job; they wouldn't want their work AI getting distracted by personal errands. But what about the split-primary structure?

Tom: Three participants maintained distinct primary models for personal and professional contexts. They usually used a lighter or faster option for low-stakes personal tasks, while reserving a stronger model for work tasks that require higher reliability.

Paper summary: Jane: It’s like having two different specialists: one quick and casual, and one robust and serious. So what else was influencing these choices, beyond just the task type?

Lu: The preference wasn't always purely about perceived performance. Sometimes it came down to things like subscription fees or usage costs for the different models. For instance, one participant offloaded trivial daily queries to a fast mode version of a model just to keep operational costs low.

Meng: That makes sense from an engineering standpoint; if you're running things at scale, cost is always a factor in deployment decisions. Are there any initial factors that set these preferences?

Tom: Early experiences with a model often create what they call first-impression lock-in, which continues to shape their later choices. One participant established Claude as their primary writing agent based on a very strong initial impression they had of it.

Jane: So the very first time you try an AI, that initial feeling can stick around and guide your use for a long time, even if you switch later.

Lu: They also mentioned expert consensus and social signals, where participants would pay attention to which models were being perceived as improving or leading in community discussions. One participant used expert consensus as a strategic filter when deciding when to re-evaluate their tool choices.

Meng: So it’s not just about the raw numbers; the community chatter matters for their strategy too. But what about how they actually work together across these systems? How do they handle switching between them?

Tom: They developed coordination strategies to switch intentionally by breaking a task into different stages and assigning each stage to the model they think is strongest for that specific part of the job. For example, ideation with one model, drafting with another, and then polishing or refining with a third.

Jane: That’s smart; you don't try to use one AI for everything from start to finish; you let each tool do the thing it does best in that sequence.

Lu: They also managed effort by task difficulty. Some participants switched models based on how hard or small the immediate task was, trying to maximize efficiency by using a different model for a very simple thing versus a complex one.

Meng: That sounds like dynamic routing based on the workload, which is what we try to build into production systems, but I wonder if that’s actually scalable when you have ten different models competing.

Tom: Then there was cross-checking for credibility when information needed to be reliable. When outputs were critical, they would switch across platforms to verify the results and reduce the risk of hallucinations. One participant used a three-stage sequence with ChatGPT, Gemini, and then back to ChatGPT to find evidence before assessing validity.

Jane: It shows that users aren't just blindly trusting one model; they build in a verification step when accuracy matters most.

Paper summary: Lu: They also noted that sometimes they chose to iterate with a single model instead of migrating because the model already knew their context, making switching too costly, or because they found the problem was actually in their own prompt specification rather than the model's capability.

Meng: So it sounds like when things aren't working perfectly, users sometimes focus on fixing their instructions instead of immediately jumping to a new tool. That’s a practical way to handle it at scale.

Tom: And some people simply decided not to migrate across platforms because reestablishing the background and personal context elsewhere felt too much effort compared to any potential performance gain they might see. One participant avoided switching away from Gemini for personal hobbies because retraining a new model on their specific persona would take excessive effort.

Jane: It seems like maintaining a relationship with an AI, even if it’s not the absolute best for every single moment, can be valuable if the context is already established. So we're seeing more about these complex systems, but what does this mean for the actual design of these tools?

Lu: The findings point toward two specific opportunities for tool design. First, they suggest a workflow-aware interface that could allow users to transfer a structured task representation between models without losing continuity.

Tom: They proposed a portable task card that captures the current task state—including goals, active constraints, and prior model outputs—which the user can then present to a new model to resume work without having to re-ground everything.

Jane: That’s interesting; it means the tool doesn't have to force you into one system; it needs to help you manage the handoff between systems.

Lu: The second idea is model-level memory partitioning within a single interface that could segregate interaction histories by domain or task type. This would help users protect distinct model contexts, addressing mismatches between what the user thinks the system remembers and how the system actually stores its history.

Meng: I like the memory partitioning idea because if you can separate contexts, it might help manage what breaks at scale by isolating failures to a specific domain or task type.

Tom: So, in short, users are treating AI models as an evolving ecosystem where they construct their own priorities and negotiate those boundaries between them. The paper "One Is Not Enough: How People Use Multiple AI Models in Everyday Life" shows us that this coordination is driven by context and task demand rather than just picking the single best model for everything.

Jane: It really highlights that the challenge isn't just building a better single model; it’s about designing systems that help people effectively manage these diverse tools when they are all working at once.

Conclusion: Tom: So we're wrapping up on "One Is Not Enough: How People Use Multiple AI Models in Everyday Life." It really boils down to how people aren't just picking one perfect AI, but actually building a whole system of them.

Jane: Exactly. We talked about those hierarchies they build—the primary and secondary models, shifting depending on whether you’re doing personal stuff or work stuff. It shows that the use case dictates the tool choice more than just raw performance metrics.

Lu: What I find really interesting is how they map out these switching patterns based on task aggregation. When a user has a big project, they don't switch randomly; they have these deliberate strategies for assigning different stages of work to different models.

Meng: From my side, what I want to know is how hard it is for an actual system to handle this kind of coordination at scale. If every user is constantly switching roles across ten models, the overhead just explodes.

Tom: That’s a fair point, Meng. But the paper suggests that users are actually iterating with a single model sometimes instead of jumping between platforms because they see their own prompts as the main failure point if the output isn't right.

Jane: So it's not always about finding a better tool; sometimes it's about fixing how you talk to the tool you already have. That’s a big mental shift for how we think about AI interaction.

Lu: It also touches on trust calibration, which is new territory for me in this context. They look at how users cross-check information by switching models just to make sure the output is actually reliable before they commit to it.

Meng: That verification step adds latency and complexity to the workflow, which is something I have to worry about when deploying these things. But if that verification reduces errors enough for a critical task, then maybe it’s worth the extra computation.

Tom: The authors suggest future tool design needs to help users keep their context alive even when they switch platforms. They are talking about portable task cards that carry the entire state of a conversation so you don't have to re-explain everything.

Jane: That means we stop thinking of AI tools as isolated silos and start thinking of them as parts of a larger, shared workspace where the work travels with it.

Lu: It’s a shift from treating AI like individual software packages to treating it like an evolving ecosystem where priorities are constantly negotiated by the user.

Meng: If that's what they're saying, then for us in production, we have to design interfaces that respect those distinct contexts so that users don't lose their established relationships with the models.

KAIST

cs.HC, cs.AI

Submitted: 2026-03-27

Updated: 2026-03-27

Comments: Accepted as a poster at CHI 2026

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 81/100

The gist: The gist: People increasingly use multiple Multimodal Large Language Models (MLLMs) concurrently, selecting each based on its perceived strengths.

Key concepts

Individual MLLM Hierarchy Structures: Users create internal rankings of AI models, designating one as the primary tool for core tasks and others as secondary tools for verification or refinement. These hierarchies change depending on whether the user is in a personal or professional setting, showing that model roles are context-dependent.
MLLM Coordination Strategies: To manage multiple AIs effectively, users intentionally switch models by breaking down complex tasks into stages and assigning each stage to the best-suited model. They also switch based on task difficulty to optimize speed and use cross-checking across platforms to ensure output reliability.
Contextual Switching Drivers: The decision to switch between MLLMs is often driven by needing to re-establish a specific conversational context or personal background. Users may stick with a familiar model if retraining it for a new context seems too costly, prioritizing familiarity over potential performance gains.

Terminology

MLLMs — 12× in this paper · explained in 2 episodes
Verification — 9× in this paper · explained in 2 episodes
Multimodal Large Language Models — 6× in this paper
Agent Interactions — 4× in this paper · explained in 1 episode
Benchmark — 4× in this paper · explained in 2 episodes
Calibration — 4× in this paper · explained in 7 episodes
Generative AI — 4× in this paper · explained in 2 episodes
Hallucination — 4× in this paper · explained in 4 episodes
Aggregation — 3× in this paper · explained in 1 episode
Accuracy — 2× in this paper · explained in 1 episode
Adam — 2× in this paper · explained in 2 episodes
Core — 2× in this paper · explained in 2 episodes
Explainability — 2× in this paper · explained in 3 episodes
Grounding — 2× in this paper · explained in 3 episodes
Perplexity — 2× in this paper · explained in 3 episodes
Consistency — 1× in this paper · explained in 3 episodes
National Research Foundation of Korea — 1× in this paper
Prompt Engineering — 1× in this paper · explained in 5 episodes
Thinking mode — 1× in this paper · explained in 2 episodes

Summary

The gist: People increasingly use multiple Multimodal Large Language Models (MLLMs) concurrently, selecting each based on its perceived strengths.

Individual MLLM Hierarchy Structures

Participants constructed primary and secondary hierarchies among models that shifted over usage context <ref:2603.26107#pg2> The primary model was used for frequent, core tasks, while secondary models were used for verification (P2, P3, P5, P7), refinement (P3, P6, P9, P10), or more specialized needs (P2, P4, P5, P6, P7 <ref:2603.26107#pg5>). However its configuration shifted with context—particularly between personal and professional use <ref:2603.26107#pg6>. Across participants, three recurring hierarchy patterns described how model roles were arranged across personal versus work contexts <ref:2603.26107#pg6>.

(a) Cross-Context Primary Structure Four participants centered their workflows around a single favorite model used across both personal and professional contexts <ref:2603.26107#pg6>. They perceived the favorite model as a general-purpose assistant and kept interactions consolidated <ref:2603.26107#pg6>.

(b) Work-Focused Structure However, three participants kept one primary model dedicated to professional tasks, treating it as a productivity tool <ref:2603.26107#pg6>. They deliberately did not feel a strong need to use this primary model in personal contexts <ref:2603.26107#pg6>.

(c) Split-Primary Structure Furthermore, three participants maintained distinct primary models for personal versus professional contexts <ref:2603.26107#pg6>. They typically used a lighter or faster option for low-stakes personal tasks, while reserving a stronger model for work tasks that demanded higher reliability <ref:2603.26107#pg6>.

Factors Shaping Model Hierarchies

Model preferences were sometimes shaped less by perceived performance, as they balanced task suitability against recurring subscription fees or usage costs <ref:2603.26107#pg5>. For example, P1 offloads trivial daily queries to Gemini Fast Mode to minimize operational costs <ref:2603.26107#pg5>. First-Impression Lock-in Early experiences with a model often set a reference point that continued to shape later preferences <ref:2603.26107#pg6>. P3 established Claude as their primary writing agent based on a strong initial impression <ref:2603.26107#pg6>. Expert Consensus and Social Signals Some participants stayed attentive to which models were perceived as improving or leading, using expert opinions and community discussions as cues for when to re-evaluate their tool choices <ref:2603.26107#pg6>. P9 treated expert consensus as a strategic filter, monitoring the market without the burden of constant hands-on testing <ref:2603.26107#pg6>.

MLLM Coordination Strategies

Participants developed strategies to coordinate their interactions across multiple MLLM platforms, switching intentionally by assigning different task stages to different models, adjusting effort and latency to task demands, and cross-checking credibility-critical information <ref:2603.26107#pg6>.

Assigning Models to Specialized Roles A common coordination strategy was to break a task into stages and assign each stage to the model perceived as strongest for that subtask (P2, P5, P6, P7, P9)—for example, ideation with one model, drafting with another, and polishing or refinement with a third <ref:2603.26107#pg6>.
Managing Effort by Task Difficulty Three participants switched models based on the scale and difficulty of the immediate task to maximize efficiency <ref:2603.26107#pg6>. P9 switched between modes within a single service to manage latency, treating them as agents with different speeds and ranks <ref:2603.26107#pg6>.
Cross-Checking for Credibility When information needed to be reliable, participants adopted a cross-checking strategy by switching across platforms to verify outputs and reduce hallucination risk <ref:2603.26107#pg6>. This strategy was often implemented as a simple one-step verification: participants took the initial output to another model to confirm key claims <ref:2603.26107#pg6>. P7 used a three-stage sequence (ChatGPT → Gemini → ChatGPT) to locate evidence and then return to the original model to assess validity <ref:2603.26107#pg6>.

Don’t Migrate, Iterate Instead

Rather than migrating across platforms, some participants sought to improve output quality by iterating with a single model <ref:2603.26107#pg6>. They described two reasons for this choice: (1) the model already knew their context, making switching costly; and (2) when outputs fell short, they sometimes located the source of failure in their own input specification rather than model capability, leading them to revise and clarify their requests instead of migrating <ref:2603.26107#pg6>.

No Resetting the Relationship

Four participants sometimes chose to stick with a familiar model because reestablishing background and personal context elsewhere felt more costly than any likely performance gain <ref:2603.26107#pg6>. P2 avoided switching away from Gemini for personal hobbies, noting that retraining a new model on his specific persona would require excessive effort <ref:2603.26107#pg6>. Blame the Prompt, Not the Model When results were unsatisfying, participants sometimes focused on improving how they prompted rather than blaming the model, iterating until the output met their needs <ref:2603.26107#pg6>. P9 described this mindset explicitly, comparing poor prompting to user error in driving <ref:2603.26107#pg6>.

Design Implications

The findings suggest two design opportunities for tools that support users navigating an increasingly heterogeneous AI landscape <ref:2603.26107#pg6>. First, a workflow-aware interface could allow users to transfer a structured task representation between models without losing continuity <ref:2603.26107#pg6>. A portable task card could capture the current task state—including goals, active constraints, and prior model outputs—which the user presents to a new model to resume work without re-grounding <ref:2603.26107#pg6>. Second, model-level memory partitioning to segregate interaction histories by domain or task type within a single interface could help users to protect distinct model contexts <ref:2603.26107#pg6>. This would address the mismatches between user mental models and system memory behavior <ref:2603.26107#pg6>.

The paper concludes that users treat AI models as an evolving ecosystem, constructing priority hierarchies and negotiating context boundaries <ref:2603.26107#pg6>. These findings inform design opportunities for supporting effective model selection, cross-model coordination, and reliability-oriented use in multi-MLLM environments <ref:2603.26107#pg6>. The study reveals that switching is often driven by re-establishing conversational context rather than purely physical access <ref:2603.26107#pg6>. The work extends trust-calibration research to a networked, multi-agent verification practice <ref:2603.26107#pg6>.

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2021-NR059056) <ref:2603.26107#pg5>.

References

[1] Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan <ref:2603.26107#pg6>.

[2] Virginia Braun and Victoria Clarke <ref:2603.26107#pg6>.

[3] Jenna Butler, Jina Suh, Sankeerti Haniyur, and Constance Hadley <ref:2603.26107#pg5>.

[4] Butler, Jenna, Jaffe, Sonia, Janßen, Rebecca, Baym, Nancy <ref:2603.26107#pg5>.

[5] H. Dang, J. Lehman, and D. Buschek <ref:2603.26107#pg5>.

[6] Katy Ilonka Gero, Tao Long, and Lydia B Chilton <ref:2603.26107#pg6>.

[7] Xinyi Hou, Yanjie Zhao, and Haoyu Wang <ref:2603.26107#pg6>.

[8] Imagining the Digital Future Center <ref:2603.26107#pg5>.

[9] Tero Jokela, Jarno Ojala, and Thomas Olsson <ref:2603.26107#pg6>.

Improvements for AI systems

This improved system incorporates a workflow-aware interface that allows users to transfer a structured task representation between models without losing continuity. This means capturing the current task state—including goals, active constraints, and prior model outputs into a portable task card that can be presented to a new model to resume work without re-grounding.
The system implements model-level memory partitioning within a single interface to segregate interaction histories by domain or task type. This addresses the user's observed need to protect distinct model contexts, enabling users to create topic containers (e.g., 'Coding,' 'Health,' 'Project 1') that isolate histories within or across models.
The system introduces an automated cross-checking mechanism tailored for multi-MLLM verification, allowing users to implement complex validation sequences like the one P7 used: I paste ChatGPT’s output into Gemini and ask it to find supporting evidence. Then, I feed that evidence back into ChatGPT and ask: ‘Is this evidence actually correct?’

Abstract

People increasingly use multiple Multimodal Large Language Models (MLLMs) concurrently, selecting each based on its perceived strengths. This cross-platform practice creates coordination challenges: adapting prompts to different interfaces, calibrating trust against inconsistent behaviors, and navigating separate conversation histories. Prior HCI research focused on single-agent interactions, leaving multi-MLLM orchestration underexplored. Through a diary study and semi-structured interviews (N=10), we examine how individuals organize work across competing AI systems. Our findings reveal that users construct primary and secondary hierarchies among models that shift over usage context. They also develop personalized switching patterns triggered by task aggregation to adjust effort and latency, and output credibility. These insights inform future tool design opportunities, supporting users to coordinate multi-MLLM workflows.

Sources

LLM Applications: Current Paradigms and the Next Frontier