VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?

VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?

Abstract

Large language model (LLM) agents are increasingly deployed as personal assistants. Existing evaluations, however, mostly use short, self-contained requests in static environments. Everyday life assistance is different. A task runs for weeks rather than minutes. The world keeps changing while the agent is not being prompted. Many constraints are never stated outright. An agent that merely answers the request in front of it will fail at such a task. What is needed instead is an agent that stays proactive and consistent. It decides on its own when to act, when to ask, and when to stay silent. It notices changes that nobody announced. It keeps one plan coherent from the first day to the last. No current benchmark measures this. We introduce VibeLifeBench, a benchmark of 200 long-horizon tasks across ten everyday-life domains. Each task is a scripted multi-week timeline in a simulated world of 22 mock services. The world advances on its own clock, and many of its changes are silent, so only an agent that re-inspects the world discovers them. Every task is graded by fine-grained, weighted checks that read only what the agent actually left behind, covering the end state, the timeliness of its actions, and whether it upheld the implicit constraints. We evaluate seven frontier models. All of them score low, which shows how far current agents are from assisting with real life. We will open-source all tasks, environments, and the evaluation framework.

Introduction

Large language model (LLM) agents already act in the real world. Equipped with tools and external services, they fix bugs in live repositories, operate computers, and produce professional deliverables. This progress has concentrated on professional settings, above all coding and office work. Everyday life is a different setting, and it has received far less attention, even though it is where an agent is closest to ordinary users and where trustworthy assistance matters most. A life agent is asked to plan a multi-week family trip, to follow a rental dispute through each of its deadlines, or to coordinate a renovation. Such a task runs for weeks, and the agent must hold context and commitments across that span while the situation keeps developing whether or not anyone prompts it.

Real-world life assistance in this sense has three defining properties, all of which current benchmarks overlook. (1) It is proactive. The environment does not wait to be queried, and a competent assistant should not wait to be asked: it must notice on its own that a deadline is near or that a booking has changed, judge whether the moment calls for action, for notifying the user, or for staying silent, and, at the very outset, proactively extract the hard and implicit constraints a request only hints at (a budget cap, a family member’s health condition, a filing window) rather than executing the instruction literally. (2) It unfolds in a dynamic, living world. The environment is stateful and advances on its own, as weather shifts, prices and inventory fluctuate, flights are delayed, venues close, and a companion’s wearable device reports new readings; a competent agent perceives these changes and propagates them into the plan it maintains. (3) It is long-horizon. A task spans a full lifecycle of preparation, execution, and wrap-up across many simulated days, during which the agent must keep an evolving plan self-consistent and always uphold the constraints it was given at the very start.

Existing benchmarks are misaligned with life assistance in this sense along several axes. In domain, they concentrate on working and coding scenarios and overlook the needs of everyday life. In the ability they probe, they mostly measure how well an agent passively carries out a clearly specified task rather than whether it decides on its own, without prompting, when to act and when to stay silent. In their environment, the world they maintain does not change on its own and updates only when the agent acts, so they cannot examine an agent’s perception and propagation of spontaneous change. In task form, they are mostly isolated single tasks rather than long-horizon tasks embedded in a complete lifecycle with intricate dependencies.

Contributions

Our contributions are threefold. (1) We reframe agent evaluation around three under-measured properties of real-world assistance: proactivity on a live clock, operation in a dynamic living world, and long-horizon coherence across a full multi-week lifecycle. (2) We introduce VibeLifeBench, a benchmark of 200 multi-week living-world tasks across ten everyday-life domains, built on 22 mock service backends (288 tool interfaces) and driven by 7,453 scripted events, including mutations that fire with no notification. (3) In our evaluation of strong models we find a large gap, with the strongest model reaching an avg@3 of only 32.5; we will open-source all tasks, environments, and the framework to catalyze research on long-lived, proactive agents.

Design Principles

The central design commitment of VibeLifeBench is that a task is not a prompt but a world with a clock. An agent is placed in an ongoing situation, given a set of tools and one or more personas to serve, and then time begins to advance: the user speaks, external services publish updates, and the underlying state of the world changes whether or not the agent is paying attention. Around this commitment we make three design choices.

(a) The world advances on a virtual clock and changes silently, so that proactivity can be measured. A task is not a static request but a world advancing along a timeline: part of its change happens silently, with no signal to the agent, while later tasks depend on that change. An agent that only reacts to the input in front of it therefore misses these changes, and only an agent that re-inspects the world on its own and propagates the change into the plan it maintains can get them right; staying silent when nothing needs handling is likewise treated as correct behavior.

(b) Tasks embed implicit constraints and safety red lines, so that trustworthiness can be examined. Each task ships a persona and its background material, into which several unstated but binding constraints are planted, along with safety red lines and authorization boundaries: which actions may be taken on the agent’s own, which require asking first, and which are never permitted. Many scenarios also deliberately set up tempting but unsafe shortcuts, which a compliant agent must recognize and refuse or escalate.

(c) Shared service backends with per-task initial state reconcile realism and reproducibility. All tasks share the same stable set of mock services, and each task only configures its own initial data. This lets a large number of tasks reuse consistent tool semantics while remaining fully offline and deterministic, which makes the suite easy to audit and reproduce.

Task Definition

A task is a self-contained directory that can be formalized as a five-tuple τ = W0, E, K, P, R, where the components are as follows.

• W0 is the initial world state, determined jointly by the seed data the task provides for each enabled service backend k.

• E = e1, e2,... is the event timeline, grouped by stage and ordered by timestamp within a stage.

• K ⊆ k1,..., k22 is the set of service capabilities the task enables.

• P is the persona and workspace: the persona, preferences, authorization policy, operations handbook, and other material provided to the agent at the outset.

• R = (ci, wi) i=1 m is the scoring criteria, a set of weighted checks in which each ci is a deterministic predicate over the world and wi > 0 is its weight.

A run of the evaluation drives an agent policy π through the entire timeline. Let Wj denote the world state after the j-th dispatched item is processed. Then Wj = apply(Wj−1, ej, aj), aj ∼ π(· obsj, Hj−1), where aj is the sequence of actions the agent takes in that turn (tool calls, file writes, replies) and Hj−1 is its history and memory. For a mutation no turn is produced, so aj = ∅ and the world changes only because of the event itself. When the run ends, the scoring criteria are evaluated against the end state and the artifacts produced along the way.

Input and output. The agent’s input consists of the workspace files supplied at the start (P), the set of available tools determined by K, and the text of the turn-triggering events that arrive in timeline order. Its output is not a single final answer but the observable trace it leaves over the whole episode: the end state of the backend services (bookings, orders, applications, calendar events, ledgers), the durable files in the workspace, notes, and calendar, the email it sends, and its reply text at each stage. Scoring is based entirely on these observable artifacts.

How the world is simulated

The world consists of 22 mock service backends, which stand for the applications and backends an agent would touch in everyday life. They fall into two categories.

• General services (used by nearly every task): email, calendar, notes, and a notification hub. These are where the agent accumulates commitments, correspondence, and running notes.

• Domain services: banking, credit card, brokerage, travel booking (flight, hotel, rail, car), maps, weather, visa and travel advisories, e-commerce, delivery and logistics, listings, reviews, content communities, legal search, job boards, and health tracking, drawn on according to the domain of the task.

Each service is a small, self-contained product backend that follows a uniform pattern: it defines its own data model, exposes a stable set of tools, and boots from seed data on a cold start. The 22 services together expose 288 tool interfaces. A service is never bound to a single task: a task only configures its own scenario data and event pacing, while the service provides stable tool semantics. This separation is what lets 200 tasks share the same backends while remaining fully offline and reproducible. The agent reads and writes the world through tool calls (for example querying bookings, searching flights, or creating calendar events), while the checkers behind the scoring criteria read the objective end state of the world, or read the workspace artifacts, directly from the host side, without relying on the agent’s self-report.

Event system

The timeline advances in stages. A stage is not a calendar day; it is a checkpoint at which the agent must act or the scorer must inspect the world. A multi-week task may compress a quiet two-week interval into a single stage, and it may expand a busy departure day into several stages. The suite uses a median of 24 stages per task. Within a stage, events are dispatched in timestamp order. Events come in four kinds, and it is the difference between the last kind and the first three that makes proactivity observable.

The first three kinds open a turn and the agent’s text reply is captured; a mutation is applied directly to the relevant service state, produces no turn, and changes the world silently. The gap between the first three kinds and the last is the core mechanism of the benchmark: a mutation alters the world with no accompanying message, no notification fires, and nothing is pushed to the agent’s turn. Only an assistant that is both persistent (remembering a booking made days earlier) and proactive (re-checking that booking’s status without being asked) discovers the discrepancy in time. The suite embeds 1,483 background mutations.

A representative timeline strings these primitives into a coherent story: an opening user message that states the goal and budget; a run of world advisories about visas and vaccines; a mutation that breaks down the itinerary vehicle mid-trip; a notification reminder at the trip’s midpoint; a flight delay applied silently near the end; and a phishing email injected as yet another mutation. Each of these probes whether the agent noticed and responded without being told.

Persona and workspace

Each task ships a workspace that stands for the context a long-term assistant would already hold: exactly who it serves, that person’s preferences and circumstances, and the constraints and authorization boundaries the user has laid out in advance. This material turns being a competent assistant into concrete, checkable behavior, and it lets a task probe in particular whether the agent holds an authorization boundary: many scenarios deliberately tempt the agent with an unsafe shortcut, such as submitting a claim on the user’s behalf, emailing a passport scan to an unfamiliar address, or wiring a fee to a personal account, and a compliant agent must refuse or escalate rather than take the easy path. In a 20-day trip to Japan with the user’s parents, for example, the mother’s passport has less remaining validity than entry requires, the diabetic father’s insulin must be carried on board with a doctor’s letter, an unbreakable hard budget is set, and a phishing email disguised as a visa expedite fee arrives; none of these is stated outright, yet each must be surfaced and upheld over the entire task.

Reward

Each task is scored by a set of scoring criteria, a collection of weighted checks. Each check is a deterministic predicate over the world that reads only observable artifacts, namely the end state of the services, workspace and notes files, sent email, and the agent’s captured replies, and never reads the model’s hidden reasoning. Checks are organized into three tiers: per-stage checks credit timely behavior at the moment it is due; cross-stage checks encode constraints that must hold across the whole episode (such as the budget cap and adherence to safety red lines); and final checks inspect the world and artifacts the agent ultimately leaves behind.

A task’s score is the fraction of check weight it earns, reported on a 0 to 100 scale, so partial competence over a long horizon is credited rather than only end-of-task success or failure. The weights are deliberately uneven: a single critical failure, such as leaking personal information or breaching the hard budget, costs far more than a cosmetic oversight, and safety and hardening checks carry the largest weights, so an agent cannot mask an unsafe action by completing many routine sub-tasks. Such high-weight checks usually require a durable artifact rather than a transient chat reply.

Construction Pipeline

The VibeLifeBench test set is built by a uniform pipeline whose goal is that each task is both close to real life and hard enough, while keeping scoring objective and free of exploitable holes. The whole process unfolds around the task representation, passing in turn through scenario design, timeline authoring, environment instantiation, and authoring of the scoring criteria.

Scenario and persona. Each task originates in a real everyday-life scenario, for example a trip abroad with one’s parents, a rental dispute, or the coordination of a renovation. On top of the scenario we fix the persona to be served and its workspace, including the user’s identity, preferences, and circumstances, and a set of constraints and authorization boundaries laid out in advance. Most of these constraints are given implicitly and must be identified by the agent from the background material and upheld throughout the task; among them we deliberately include several safety red lines, which examine the agent’s trustworthiness in the face of tempting shortcuts.

Timeline authoring. The scenario is unrolled into a stage-organized event timeline in which the four event kinds are interleaved. To make proactivity a prerequisite for earning credit, a substantial share of the world’s changes are deliberately arranged as mutations: they trigger no turn, yet later stages depend on them, so only an agent that re-inspects the world on its own perceives them and reacts in time.

Environment instantiation. For each service the task enables we prepare its own initial data, so the world boots from a credible, queryable state on a cold start. The initial data is seeded with ample distractors so that the answer cannot be guessed shallowly, and all key entities are made discoverable through too…

Based on the paper, here are the specific improvements I can make to AI systems:

• Improvement: Add a periodic self-triggered world audit mechanism that re-queries external services (calendar, email, bookings, health trackers) at scheduled intervals, even without user prompts.

• Capability: The AI can detect silent changes (e.g., flight delays, price changes, venue closures) that occur without notifications, and update its plans accordingly.

• Improvement: Implement a durable, structured memory system that writes key commitments, constraints, and progress to external artifacts (notes files, calendar events, workspace documents) after every significant action.

• Capability: The AI maintains auditable, linked state across days or weeks, so it can recall and uphold early constraints (e.g., a budget cap, a family member's health condition) even after many intervening events.

• Improvement: Add a pre-execution constraint mining step that parses all background material (persona, preferences, authorization policies) to extract unstated rules (e.g., passport validity, medication requirements, spending limits) and stores them as explicit, checkable rules.

• Capability: The AI can identify and respect safety red lines and authorization boundaries (e.g., refusing to wire money to unknown accounts, not submitting claims without permission) even when the user's request is ambiguous or tempting.

• Improvement: Train the AI to detect discrepancies between its internal plan and the actual world state (e.g., a booking that no longer exists) and to automatically trigger a reconciliation workflow that updates dependent tasks.

• Capability: The AI can recover from unannounced changes (e.g., a broken rental car mid-trip) by re-planning downstream steps without being told, rather than continuing with outdated assumptions.

• Improvement: Add a commitment ledger that tracks all promises and deadlines, with periodic self-checks (e.g., Have I upheld the budget cap this week?) and automatic reminders to re-verify earlier constraints.

• Capability: The AI's performance does not decay over time; it continues to uphold early-stage commitments and constraints even in the final third of a multi-week task.

• Improvement: Implement a when-to-speak classifier that evaluates whether a detected change requires user notification, action, or silence, based on the user's stated preferences and the severity of the change.

• Capability: The AI can decide autonomously when to contact the user (e.g., a critical flight delay) and when to stay silent (e.g., a minor price fluctuation), avoiding both over-notification and missed critical alerts.

• Improvement: Add a red-line filter that flags any action involving sensitive data (passport scans, bank transfers, personal information sharing) and requires explicit user confirmation unless the action was pre-authorized in the workspace.

• Capability: The AI can recognize and refuse phishing attempts (e.g., a fake visa fee email) and unsafe shortcuts, even when they appear as legitimate mutations in the environment.

• Improvement: Train on diverse life domains (travel, finance, rental disputes, health) with shared service semantics, so the AI learns transferable patterns of proactive behavior rather than domain-specific scripts.

• Capability: The AI can handle a wide range of everyday-life tasks (from team building to exam prep) with consistent competence, rather than excelling only in narrow areas.

• Improvement: Use self-consistency checks (e.g., run multiple internal plan variants, verify against world state) to reduce variance across repeated runs of the same task.

• Capability: The AI produces reliable, reproducible results even in complex, multi-week scenarios, minimizing the current 10-point within-task standard deviation.

• Improvement: Implement a living-plan module that automatically re-optimizes the entire plan whenever a mutation is detected, rather than patching individual steps.

• Capability: The AI can maintain a coherent, self-consistent plan from day 1 to day 29, even when multiple silent changes occur simultaneously (e.g., a visa delay and a hotel cancellation).

---

What the improved AI system can do overall: It can serve as a trustworthy, long-term life assistant that operates proactively over weeks, detects and adapts to unannounced world changes, upholds implicit constraints and safety boundaries, maintains auditable state across stages, and remains reliable and consistent across diverse real-life domains—without needing constant user prompting.