Reading Cognition as Decisions Unfold in Words: A Factorized Inverse Decision Model

The paper introduces a factorized inverse decision model (FIDM) that decomposes each individual's task-execution likelihood into an action factor and an effort factor, governed by separate individual-specific parameters. The model is applied to verbalized cognitive task execution, specifically the Hong Kong Grocery Shopping Dialog Task (HK-GSDT), where participants verbally direct navigation through a supermarket while collecting target items.

The authors note that "existing formulations base inference primarily on action trajectories. In cognitive assessment, however, task execution performance reflects both the actions participants select and the response dynamics that accompany each choice. They observe that two participants may select similar actions yet differ markedly in their fluency, hesitation, and need for external guidance at each step, and that action-only formulations therefore capture only part of task performance, leaving response dynamics unmodeled."

The model represents individual j's task execution as a trace:

ζj = (si, ai, ei, si+1) for i = 0 to Lj−1,

where si is the current task state, ai is the selected action, ei is the observable execution signal aligned with the step, and si+1 is the resulting state.

The individual is characterized by descriptive parameters ϕj = (ηj, θj), where ηj governs action choices and θj governs the conditional distribution of execution signals. The inverse problem is (M, ζj) → ϕj.

The action factor uses a softmax policy: πηj(as) = exp(ηj Q(s,a)) / Σ exp(ηj Q(s,a')), where ηj controls sensitivity to task-value differences. The effort factor models execution signals as ei ∼ Fθj(·si, ai).

The factorized likelihood is:

P(ζj ϕj, M) = Π πηj(aisi) · Fθj(eisi,ai) · T(si+1si,ai)

1. FIDM introduction: separates how an individual selects actions from how those actions are executed, yielding an interpretable action–effort profile from verbalized task-execution traces.

2. Validation of factorization: The authors validate that the action–effort decomposition is selectively recoverable and not reducible to aggregate behavioral statistics.

3. Cognitive-screening application: Using data from over 400 older adults performing a grocery-shopping dialog task for cognitive screening, FIDM provides interpretable, task-localized evidence and complementary information beyond conventional clinical and data-driven representations.

Semi-synthetic traces with independently varied factors showed a strongly diagonal response matrix under complete observations, with matched responses ranging from 0.70 to 0.97 and cross-factor responses remaining near zero. This structure was preserved under 30% missing effort observations and increased observation noise.

When paired semi-synthetic traces had identical movement exposure but different action and effort parameters, Raw totals and length-normalized totals remain at chance (AUC = 0.502 and 0.503), whereas the FIDM action factor, effort factors, and full representation achieve AUCs of 0.929, 0.989, and 0.996, respectively.

For route-divergence onsets, action unexpectedness achieved a macro AUC of 0.902 and a 5.95× top-decile enrichment. Target-opportunity bypasses show a consistent pattern, with a macro AUC of 0.983 and 3.47× enrichment.

In binary classification, FIDM achieves the strongest standalone cross-validation performance, with an AUC of 70.39% and balanced accuracy of 65.12%. Adding FIDM improves every baseline across all four metrics. The best combinations included BERT+FIDM achieving the highest held-out AUC (75.63%) and MacBERT+FIDM achieving the highest held-out balanced accuracy (72.23%).

In three-class classification, FIDM alone achieves a held-out macro AUC of 67.85% and balanced accuracy of 56.34%. The strongest held-out results are obtained by GSDT score+FIDM, reaching 78.13% macro AUC and 71.72% balanced accuracy.

Under binary labels, the action factor alone outperforms the effort factor alone, while their combination yields the highest AUC and balanced accuracy. Under three-class labels, the effort factors alone outperform the action factor alone, though combined factors achieve the highest macro AUC.

The authors identify three main limitations: (1) the empirical evaluation is centered on a single cognitive-assessment task and generality across other sequential behaviors remains to be established; (2) the study includes 400 participants, comparable in scale to established clinically collected datasets in this area, but modest by conventional machine-learning standards; (3) task-trajectory reconstruction relies on automatic grounding, and although 19 of 20 sampled trajectories were semantically consistent in a manual audit, grounding errors may still propagate to downstream inference.

The authors conclude that extending inverse decision models beyond action trajectories to structured execution behavior may provide a useful principle for characterizing individual differences in sequential tasks.

Improvements to AI systems:

1. Dual-channel inverse decision models: Extend standard inverse reinforcement learning (IRL) to jointly infer separate parameters for action selection (sensitivity to task value) and execution effort (response dynamics like hesitation, fluency, or guidance needs). This allows AI to model how a user acts, not just what they choose, capturing individual differences that action-only models miss.

2. Factorized user profiling for sequential tasks: Build AI assistants that decompose a user's behavior into an action factor (goal-directedness) and an effort factor (cognitive load or confidence) from observed traces. This enables the system to adapt its interventions—e.g., providing more guidance to users with high effort signals, or more autonomy to those with high action sensitivity—without requiring explicit self-reports.

3. Robust behavior classification under partial observability: Improve AI systems that classify user states (e.g., cognitive decline, skill level, or engagement) by using the factorized likelihood that handles missing execution signals (e.g., 30% missing effort data) without degrading performance, as shown by the recovery experiments. This makes real-world deployment more reliable where sensor or log data is incomplete.

4. Interpretable anomaly localization in user trajectories: Enhance AI monitoring systems (e.g., for safety-critical tasks or user assistance) to pinpoint when and where a user deviates from expected behavior—using action unexpectedness and effort-based signals—achieving high localization accuracy (macro AUC > 0.90) and top-decile enrichment. This allows targeted, timely interventions rather than post-hoc aggregate analysis.

5. Complementary fusion with existing AI classifiers: Integrate FIDM-derived features into existing deep learning or clinical scoring systems (e.g., BERT, MacBERT, or standard test scores) to boost classification performance. The paper shows that adding FIDM improves every baseline across all metrics, with the best combinations raising held-out AUC to 75.63% and balanced accuracy to 72.23%—enabling more accurate screening in healthcare or user-adaptive systems.

6. Task-localized cognitive screening: Build AI systems that provide interpretable, step-by-step evidence of cognitive or functional decline from verbal task execution, rather than a single global score. This allows clinicians or automated systems to identify specific task segments (e.g., route divergences, target-opportunity bypasses) that are most indicative of impairment, improving diagnostic explainability and targeted follow-up.

What the improved AI system can do:

• Personalized adaptive tutoring: An AI tutor that observes a student solving multi-step problems can separately infer whether the student knows which action to take (action factor) versus how fluently they execute it (effort factor). It then adjusts hints or pacing—e.g., offering more scaffolding when effort signals are high, or more challenging problems when action sensitivity is high—leading to more effective learning.

• Clinical decision support for cognitive assessment: A system that analyzes a patient's verbal navigation in a shopping task can produce a dual profile (goal-directedness vs. cognitive effort) and flag specific problematic steps (e.g., hesitations or wrong turns) with high precision. It can combine this with standard test scores or language models to achieve state-of-the-art accuracy in detecting mild cognitive impairment, while providing interpretable evidence for clinicians.

• Human-robot collaboration: A robot assistant that infers a human partner's action preferences and execution confidence from their task traces can dynamically decide when to take initiative or offer help. For example, if the human's effort factor indicates high uncertainty at a step, the robot proactively suggests a next action; if the action factor is high, the robot stays passive to preserve autonomy.

• Fraud or error detection in sequential workflows: An AI system monitoring a user performing a multi-step process (e.g., data entry, assembly, or navigation) can distinguish between wrong choices (low action factor) and hesitant execution (high effort factor). It can localize the exact step where the anomaly occurs, enabling real-time alerts or post-hoc audits with far greater precision than aggregate statistics.

• Robust user modeling in noisy environments: Because the factorized model remains accurate with missing or noisy execution data, the AI system can be deployed in settings where sensors fail or logs are incomplete (e.g., mobile health apps, wearable devices) without losing predictive power—ensuring consistent performance in real-world conditions.