CoPlan: A Trustworthy Co-Intelligence Interface for Care Planning through Role-Based Contestable Argument Graphs

This paper presents CoPlan, a Co-Intelligent and Contestable Interface for Human-AI Care Planning, designed to address challenges in AI-supported care planning for aging-in-place contexts.

The paper begins by noting that care planning is a collaborative and ongoing process in which clinicians, patients, caregivers, and care teams define goals, coordinate interventions, and revise decisions as conditions change. The authors argue that many AI systems present recommendations as fixed outputs, limiting stakeholders’ ability to inspect, challenge, and revise plans when they conflict with clinical judgment, patient values, or real-world feasibility. They emphasize that trustworthiness "cannot rely only on accuracy, model performance, or explanation, because users also need meaningful ways to respond when AI recommendations are incomplete, unsafe, biased, or misaligned with clinical judgment and patient goals. While explainable AI (XAI) helps users understand recommendations, explanation alone does not ensure that users can correct the system, challenge its assumptions, or reshape the proposed plan."

CoPlan is designed around two principles:

• Co-intelligence: "a mode of human-AI collaboration in which clinicians, patients, caregivers, and AI agents contribute different forms of knowledge to the care planning process. Rather than treating AI as an autonomous planner, CoPlan positions AI as a collaborative participant that helps synthesize clinical evidence, surface risks, generate alternatives, and support reflection across stakeholders."

• Contestability: the ability of stakeholders to inspect an AI-supported care recommendation, question its evidence and assumptions, provide counterevidence, express disagreement, and trigger revision or justified confirmation.

The authors position this against the shift from explainability to contestability, noting that Instead of presenting AI-generated care plans as final recommendations, CoPlan treats them as proposals that can be inspected, challenged, revised, justified, or escalated.

The environment is formally modeled as a tuple: M = ⟨P, D, O, A, Γ, H, V, Π⟩, where:

• P describes patient information (health conditions, functional status, contextual factors)

• D is the set of retrieved evidence documents

• O is the set of candidate care options proposed by a generative model

• A is the recruited subset of healthcare roles serving as expert agents

• Γ is the argumentative pool of supporting and challenging arguments

• ΓH is the human-revised argument set

• V is a validation operator grounded in quantitative bipolar argumentation semantics

• Π is a care plan operator synthesizing the final recommended care plan

The argumentation framework uses a Quantitative Bipolar Argumentation Framework (QBAF) defined as Q = ⟨X, R+, R−, τ⟩, where R+ captures support relations, R− captures challenge/conflict relations, and τ assigns each argument an intrinsic strength score in [0,1] based on clinical relevance, its factual consistency with the retrieved documents D, and the transparency of its reasoning. The system computes final confidence degrees f(x) through an iterative process combining intrinsic strength with influences from supporters and attackers, then aggregates option-level confidence scores F(oi).

The system operates across four connected stages:

Stage 1 – Complexity Assessment and Care Team Recruitment: The system constructs the initial state from the client's interRAI Home Care assessment, a standardized, comprehensive clinical instrument widely used in community and long-term care settings. A patient analyzer agent computes case complexity c = C(P) ∈ low, moderate, high, very high, and a team recruitment agent recruits provider agents A = S(P, c), where each agent corresponds to a clinical role (registered nurse, pharmacist, general practitioner, nutritionist, physical therapist, occupational therapist, psychiatrist, social worker, home health aide, care coordinator).

Stage 2 – Argumentative Agent Framework and Collaborative Discussion: For each candidate intervention oi, every provider agent generates supporting and challenging arguments. The full argumentative pool is defined as Γ = ⋃(Sⱼ,i ∪ Cⱼ,i), where each argument is represented as [content(x), type(x), role(x)]. Agentic RAG allows agents to repeatedly query the historical medical vector database for similar case evidence to enhance the factual grounding of their arguments. Role-based argument generation uses role-specific prompts with professional background, expertise areas, and care priorities, ensuring the pool contains not only reasons in favor of a recommendation, but also possible risks, limitations, and conflicts that can later be reviewed by humans.

Stage 3 – Role-Based Human-in-the-loop Contestation: Human care team members review each argument and may: "(1) accept an argument, (2) reject an argument judged irrelevant, unsafe, or inconsistent with clinical practice, (3) modify an argument’s content to better reflect the client’s situation, or (4) add new arguments based on professional insight that was not captured by the agent team. This is supported through an interactive graph review interface that allows users to switch between candidate care options, inspect supporting and challenging arguments grouped by professional role, and directly interact with individual argument nodes. The authors note this operationalizes contestability at the interaction level by making the argumentative structure visible and editable, thereby enabling human reviewers to actively shape the reasoning process rather than merely observe it." After human edits, the system re-applies quantitative bipolar semantics to the revised set ΓH, producing updated degrees ΓV.

Stage 4 – Final Care Plan Generation and Human Approval: The care plan operator Π synthesizes the final plan using validated arguments, option set, and retrieved evidence, determining priority, recommendation strength, and required risk-mitigation steps for each intervention. The generated plan is presented through a final care plan viewer with structured sections, and the interface includes a chatbot assistant for follow-up questions. Importantly, the generated care plan is not treated as an automatically finalized decision—the human care planner retains responsibility for reviewing the plan and may reopen the relevant arguments, add feasibility constraints, revise the recommendation, or reject the plan component. After final plan generation, a Model Context Protocol (MCP) agent translates recommendations into concrete scheduling tasks, arranging appointments, requesting assessments, and scheduling provider visits.

In the aging-in-place scenario, the system demonstrated an end-to-end pipeline from patient information to a finalized care plan through four connected stages. In the illustrated case, the system prioritized home safety assessment and modification, social engagement and support groups, and regular health monitoring and care coordination as the highest-validity recommendations, with the final output including implementation steps, argument validity summaries, and evidence-based justification.

The authors summarize three main contributions: (1) CoPlan, a co-intelligent and contestable interface that makes AI-supported care recommendations inspectable, challengeable, revisable, and auditable (with implementation available at the provided GitHub link); (2) a contestability-centered framing that connects trustworthy AI, human-centered design, and clinical accountability in human-AI care planning; and (3) design implications for future AI-supported care planning systems that aim to preserve human agency while benefiting from AI reasoning and multi-agent coordination.

The paper acknowledges a current limitation: validity and confidence scores support review rather than autonomous decision-making. Future work should evaluate alternative aggregation methods and examine their effects on transparency, fairness, legitimacy, and overreliance and assess whether clinicians, patients, and caregivers can effectively express value-based concerns through contestation. The paper concludes that "trustworthy AI-supported care planning requires not only accurate recommendations or explanations, but also accountable negotiation. CoPlan provides a design direction that preserves human agency, clinical responsibility, and meaningful oversight."

Improvement 1: Argument-Graph Output Instead of Single Recommendations

Replace flat recommendation outputs with a structured, inspectable argument graph. The system constructs a Quantitative Bipolar Argumentation Framework (QBAF) where every candidate action is attached to supporting and attacking arguments, each assigned an intrinsic strength score derived from clinical relevance, factual consistency with retrieved evidence, and reasoning transparency.

• What the improved AI can do: For any recommendation it makes, the system visually renders the full reasoning structure—each argument, its source role, its evidence links, and its support/attack relationships—so users can traverse the logic rather than accept a black-box output. Confidence scores are computed via iterative bipolar semantics, allowing users to see exactly why one option scores higher than another and which arguments tipped the balance.

Improvement 2: Role-Diverse Adversarial Multi-Agent Argument Generation

Instead of a single model generating one rationale, spawn multiple specialized agents corresponding to distinct professional roles (e.g., pharmacist, physical therapist, social worker, general practitioner). Each agent generates both supporting and challenging arguments for each candidate option, using role-specific prompts that encode professional background, expertise areas, and care priorities, and each can query a historical medical vector database for similar-case evidence.

• What the improved AI can do: The system surfaces risks and limitations a single-perspective model would miss—e.g., the pharmacist flags a drug interaction the care coordinator's rationale ignores, and the physical therapist challenges a mobility plan as infeasible for the patient's home layout. The argument pool becomes genuinely multi-perspective, capturing professional disagreements before human review, so the system can reason over conflicts rather than smoothing them away.

Improvement 3: Direct Human Contestation with Semantic Re-Computation

Let human reviewers interactively modify the argument graph itself. Users can accept, reject, edit, or add argument nodes through a graph interface. After each human edit, the system re-applies the QBAF semantics to the revised argument set and recomputes the confidence scores and resulting plan priorities.

• What the improved AI can do: When a clinician deems an argument clinically unsafe and removes it, or adds a new argument about the patient's stated preferences, the system immediately re-derives the confidence scores for all affected care options and shows the new ranked plan—demonstrating in real time that human judgment measurably changes the AI's reasoning, not merely the final display. This converts contestation from a post-hoc veto into a first-class component of the computation.

Improvement 4: Justified-Confirmation and Escalation Protocol

Build an explicit disagreement-handling protocol into the system workflow. When a human challenges a recommendation, the system must either: (a) revise the plan in response to the new/changed arguments, (b) produce a strengthened justification (new evidence, stronger argument) for the original recommendation, or (c) escalate the contested component to a higher-authority human decision-maker with a structured summary of the disagreement.

• What the improved AI can do: The system never silently overrides a human challenge, nor does it reflexively capitulate. It maintains an auditable record of every challenge, the system's response, and the final resolution (revision, justified confirmation, or escalation). This yields a transparent chain of accountability where every contested decision has a documented rationale that meets clinical and ethical scrutiny.

Improvement 5: Evidence-Grounded Argument Strength with Auditable Scoring

Ensure every argument's intrinsic strength score is itself decomposable and inspectable. The score combines (a) clinical relevance of the argument to the patient's specific conditions, (b) factual consistency against the retrieved evidence documents (via agentic RAG verification), and (c) transparency of the reasoning (how clearly the argument links evidence to conclusion).

• What the improved AI can do: When asked why is this argument weak?, the system replies with a breakdown—e.g., this argument is factually inconsistent with two retrieved documents and uses indirect reasoning, enabling users to distinguish a well-evidenced challenge from a misinformed one. The system can also flag low-quality arguments automatically before human review, reducing the burden on clinicians.

Improvement 6: From Plan to Actionable Task Execution via Protocol Agents

Add a final execution stage where a Model Context Protocol (MCP) agent translates the approved care plan into concrete scheduling tasks: arranging assessments, booking provider visits, and requesting specific interventions.

• What the improved AI can do: After the human planner approves the final plan, the system autonomously converts each recommendation into time-bound tasks (e.g., home safety assessment within 7 days, follow-up call with the pharmacist on medication review), coordinates them with the care team calendar, and tracks completion—so the AI's value extends beyond reasoning into operational care coordination, while still leaving final approval with the responsible human.

Improvement 7: Adaptable Aggregation Transparency

Replace opaque score aggregation with multiple, visible aggregation alternatives that can be compared. The system exposes how option-level confidence is derived from argument-level strengths and lets users examine sensitivity—e.g., what happens if attack arguments are weighted differently than support arguments, or if consensus across roles is prioritized over raw scores.

• What the improved AI can do: The system reveals when a recommendation is robust across aggregation methods versus fragile (highly sensitive to weighting choices), enabling users to identify borderline plans that warrant extra scrutiny. This mitigates overreliance by showing that confidence scores are modeling choices, not objective truths, and gives stakeholders a basis for debating which aggregation policy aligns with their values.

AI-supported care planning can help clinicians, patients, caregivers, and care teams coordinate complex decisions across clinical, functional, psychosocial, and environmental needs. However, many AI systems present recommendations as fixed outputs, limiting stakeholders' ability to inspect, challenge, and revise plans when they conflict with clinical judgment, patient values, or real-world feasibility. We present CoPlan - a Co-Intelligent and Contestable Interface for Human-AI Care Planning. CoPlan uses a multi-agent workflow in which specialized AI agents generate candidate interventions and supporting or challenging arguments, while human care planners can accept, reject, modify, or add arguments before final plan generation. Through this design, CoPlan combines co-intelligence, in which humans and AI agents contribute complementary expertise, with contestability, where recommendations remain open to inspection, revision, and justification. We demonstrate CoPlan in an aging-in-place care planning scenario. The system supports adaptive care team recruitment, role-based argument review, final care plan generation, and practical follow-up through scheduling agents. This work contributes a contestable care planning interface and a design framing for trustworthy human-AI care planning that preserves human agency and clinical accountability.