Unified Agent: Managing Interactions across Devices
Xinshuang Liu, Runfa Blark Li, Shaoxiu Wei, Xin Lin, Truong Nguyen
cs.AI, cs.CL, cs.CV, cs.HC
Submitted: 2026-08-06
License: http://creativecommons.org/licenses/by/4.0/
The gist: As capabilities rapidly increase, AI agents can move from running inside one app to acting across a user's devices over time.
Terminology
Abstract
As capabilities rapidly increase, AI agents can move from running inside one app to acting across a user's devices over time. Yet existing agent systems still fall short in this scenario. This is because observations are scattered across devices and moments, but mainstream systems are not designed around this fact: a single agent that treats devices as tools lacks effective state management for all devices across time, and multi-agent systems coordinate across agents but do not maintain the compact carried state a cross-device, cross-time request needs. We argue that the agent should maintain an effectively designed state that organizes engagement evidence, stated facts, and the standing request in a compact, action-ready form for deciding its action given the current observation. To compare state designs, we construct a benchmark of user-agent interaction across devices and time. We instantiate this principle in Unified Agent, a stateful agent that carries interaction evidence across devices and moments and uses it with the current observation to act. In the default setting, it significantly outperforms our adaptations of four published designs. Across changes in multimodal large language model (MLLM) family, capability, and reasoning effort, it remains ahead of all compared systems, demonstrating that the state-design advantage is robust across MLLM settings. Our code and data will be publicly available on GitHub.
Sources
- CLUE: Crossmodal disambiguation via Language-vision Understanding with attEntion
- Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation
- M2HRI: An LLM-Driven Multimodal Multi-Agent Framework for Personalized Human-Robot Interaction
- Inner Monologue: Embodied Reasoning through Planning with Language Models
- Explicit World Models for Reliable Human-Robot Collaboration
- ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents
- REMem: Reasoning with Episodic Memory in Language Agent
- RoboOS-NeXT: A Unified Memory-based Framework for Lifelong, Scalable, and Robust Multi-Robot Collaboration
- MemGPT: Towards LLMs as Operating Systems
- UI-TARS: Pioneering Automated GUI Interaction with Native Agents
- Zep: A Temporal Knowledge Graph Architecture for Agent Memory
- Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control
- SAGE: Smart home Agent with Grounded Execution
- PSI: Shared State as the Missing Layer for Coherent AI-Generated Instruments in Personal AI Agents
- A-MEM: Agentic Memory for LLM Agents
- FindingDory: A Benchmark to Evaluate Memory in Embodied Agents
- EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models
- Mixed-Initiative Dialog for Human-Robot Collaborative Manipulation
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems