Seeing Is Not Deciding: Can Multimodal LLMs Act as Effective CEOs?

The paper investigates whether multimodal large language models (MLLMs) can act as effective Chief Executive Officers (CEOs) by integrating visual business evidence into executive decision-making. The authors note that while LLMs are increasingly applied as autonomous decision-making agents, in executive business decisions, existing benchmarks are limited to text-only settings, leaving it unclear whether models can perceive visual business evidence and effectively integrate it to improve decision quality.

The work is motivated by the observation that "Executive decisions (e.g., whether to cut investment, expand production, or reassure investors) require integrating complex dimensions, such as financial dashboards, operational KPIs, growth trajectories, competitive dynamics, and long-term strategic objectives, making the CEO-decision naturally multimodal. The authors identify a gap: Existing evaluations treat reasoning and visual perception as separate capabilities—general-purpose agent benchmarks emphasize reasoning in text-based scenarios, while multimodal benchmarks focus on chart reading and visual QA rather than strategic judgment, and financial benchmarks remain overwhelmingly text-centric, with nearly little visual business evidence."

The authors introduce C-SuiteBench, a controlled benchmark for evaluating multimodal executive decision making that places MLLMs in the role of a chief executive officer reasoning over both textual reports and business dashboards, including financial performance charts, operational KPIs, and growth trend visualizations. The benchmark instantiates 50 formally structured scenarios spanning four organizational groups by difficulty, with each scenario evaluated across five executive tasks, yielding 250 scenario-task instances per input setting (500 in total for text-only and multimodal).

Each scenario depicts a mid-sized enterprise at a strategic inflection point requiring executive judgment, with examples including revenue decline despite healthy operations, stalled growth with hidden unit-level risks, and conflicting departmental signals that require explicit resource trade-offs. Scenarios are organized into four difficulty families: Easy (consistent and unambiguous signals), Fragile (favorable surface indicators concealing underlying vulnerabilities), Tension (competing legitimate priorities requiring explicit trade-offs), and Adversarial (intentionally misleading cues requiring models to distinguish relevant evidence from distractors).

Critically, the design is controlled: "text-only and multimodal conditions are constructed from the same underlying organizational state, ensuring that the two conditions differ only in the availability of visual business evidence rather than in the decision problem itself. Visuals are rendered directly from the scenario's underlying data and expose quantitative patterns, trend inflections, rates of change, and cross-metric divergences, which are not explicitly stated in the textual report."

C-SuiteBench decomposes executive reasoning into five complementary tasks, allowing multimodal gains and failures to be identified to specific stages of the decision-making pipeline:

• T1: Organizational Diagnosis — identifies the company's primary organizational challenge.

• T2: Evidence Prioritization — ranks candidate evidence items by decision relevance.

• T3: Resource Reallocation — produces a concrete budget reallocation plan across organizational units under numerical constraints.

• T4: Risk Forecasting — identifies the most significant organizational risks over a long horizon and supports each prediction with an appropriate leading indicator.

• T5: Board Justification — produces a structured justification of its decisions for a board audience by integrating the evidence into a coherent executive narrative.

Evaluation uses a task-specific automatic scorer using rule-, overlap-, and constraint-based dimensions, aggregated into a single score in [0, 1]. The key metric is multimodal uplift, Δ = score multimodal − score text-only, which captures the change in decision quality when visual business evidence is added to an otherwise identical decision problem.

Nine frontier models were evaluated: GPT-5.4 mini, GPT-5.4 nano, Claude Opus 4.8, Qwen2.5-VL 72B, Mistral Small, Gemini 3 Flash, MiniMax-M3, MiniMax-M2.7, and MiniMax-M2.7-HighSpeed. All models are queried with identical prompts across text-only and multimodal conditions, with visual information passed as inline images, and No model-specific prompt tuning is performed. This yielded 2,250 paired text-only vs. multimodal task comparisons.

The central finding is that visual business information improves executive decision making, but the magnitude, and even the direction of this benefit depends strongly on the stage of the decision-making pipeline. The average uplifts across all nine models were: T1 Diagnose +0.07, T2 Prioritize +0.10, T3 Allocate −0.08, T4 Forecast +0.24, T5 Justify +0.17.

Evidence-centric stages consistently benefit from multimodal inputs. "The largest gains occur in T4 Risk Forecasting (mean Δ = +0.27, maximum +0.37 for MiniMax-M2.7) and T5 Board Justification (mean Δ = +0.21, positive for all nine models), while T1 and T2 exhibit smaller but reliable improvements (mean Δ = +0.07 and +0.10, respectively). The paper states that visual business information is particularly valuable for tasks centered on identifying, prioritizing, and synthesizing evidence. Specifically, in T4, gains are driven primarily by improvements in leading indicator recall, suggesting that temporal trends and rate-of-change signals are more effectively extracted from visual representations than from textual summaries, and in T5, models show consistent gains in evidence citation score and tradeoff coverage."

Statistically, T2, T4, T5 are statistically reliable (p = 0.004); T1 shows a positive directional trend (p = 0.180, not significant at model-level sign test).

Resource Reallocation (T3) is the lone systematic exception: All nine models exhibit negative uplift, with the largest degradations observed for MiniMax-M3 (Δ = −0.13) and MiniMax-M2.7-HighSpeed (Δ = −0.12), while even the smallest degradations remain below zero. The authors term this the multimodal integration paradox.

Sub-dimension decomposition reveals a striking dissociation: "Under multimodal conditions, visual grounding proxy... increases substantially across all models (mean Δ ≈ +0.42), confirming that models do perceive and attend to the additional visual information. However, this improvement is accompanied by systematic declines in both constraint validity... and strategic fit. In other words, models become better at incorporating visual evidence while simultaneously becoming worse at producing valid executive decisions. The paradox is that improved visual grounding does not translate into improved constrained action. Modern MLLMs successfully extract and reason over visual business signals, yet struggle to reconcile those signals with hard decision constraints when converting reasoning into executable actions."

The authors propose a mechanism: "multimodal inputs introduce a richer numerical signal environment... that competes with the explicit allocation constraints during decoding. Under text-only conditions, the model's attention is concentrated on satisfying the constraint specification. Once multiple visual evidence sources are introduced, however, the additional quantitative signals may interfere with constraint satisfaction, particularly when the correct decision requires discounting visually salient but strategically misleading evidence."

Stratifying by difficulty family, the paradox shows a context-dependent pattern. For evidence-centric tasks, multimodal gains remain largely robust across difficulty levels. For T3, Multimodal uplift is negative across all four difficulty families, but the magnitude of the degradation grows under higher-conflict decision regimes: the largest negative shifts appear in tension and adversarial scenarios, whereas easy scenarios show a smaller but still negative effect. Pooled results confirm this: Easy −0.028, Fragile −0.061, Tension −0.118, Adversarial −0.102. The authors conclude the paradox is not an inherent limitation of multimodal reasoning. Instead, it is a context-dependent failure that emerges when visually salient but competing evidence must be reconciled with hard decision constraints.

To test the crowding hypothesis, the authors conducted single-channel ablations (finance-only, growth-only, ops-only) on T3 and T4, using MiniMax-M3 as the representative case. On T4, All three visual channels produce substantial improvements over the text-only baseline (finance: +0.13, growth: +0.10, ops: +0.12), and no single channel consistently dominates. On T3, however, a fundamentally different pattern emerges: "Every single-channel condition substantially outperforms the full multimodal setting (full multimodal score: 0.542), with ops-only achieving the highest task score (0.711) and constraint validity (0.933), followed by growth-only (0.708) and finance-only (0.693). The degradation is therefore not caused by any individual channel; it emerges only when multiple evidence sources must be integrated simultaneously. This pattern holds across all nine models: the best single channel improves T3 by +0.052 over text-only, whereas combining all three degrades it by −0.08."

A text-plus-numbers control (supplying the same panel values as structured text, with no charts) decomposes the effect into numerical (Δ num) and visual (Δ vis) components, revealing an asymmetry: "on T3, numbers alone reproduce 60% of the degradation (Δ num = −0.046), with charts contributing the rest (Δ vis = −0.031, negative for 9/9 models, p = 0.004); on T4, charts instead carry the majority of the gain (Δ vis = +0.152, positive for 8/8 non-trivial models, p = 0.008). The authors conclude: Visual presentation thus does not help or hurt in itself: it amplifies whatever effect concurrent evidence load already produces. We term this evidence-load crowding, with visual signal crowding as its chart-specific instantiation. This leads to a practical implication: Because 60% of the T3 degradation survives removal of the visual modality, the operative intervention is reducing concurrent quantitative signals at the action stage, not merely withholding charts."

A human study with 30 evaluators (18 PhD-level researchers, 8 master's-level researchers, and 4 industry practitioners) using a blinded pairwise protocol over 250 paired scenario-task comparisons confirmed the automatic scoring: human majority-vote judgments agreed with the automatic scorer on 76% of cases, with inter-rater reliability of Fleiss' κ = 0.78, and reproduced the task asymmetry—multimodal outputs were strongly preferred on T2, T4, and T5... and T3 showed a consistent human preference against multimodal outputs... consistent with the multimodal integration paradox.

The paper argues that current multimodal evaluation practices conflate two fundamentally different capabilities: perceiving visual information and acting upon it, and proposes a three-stage capability hierarchy: (1) perceiving visual evidence, (2) integrating it into coherent reasoning, and (3) transforming reasoning into valid constrained actions.

The conclusion summarizes: "Our experiments reveal that multimodal inputs produce consistent gains on evidence-centric tasks, but uncover a systematic multimodal integration paradox in constrained resource allocation: visual grounding improves while decision quality degrades. Ablation experiments identify visual signal crowding, the collective interference of individually beneficial visual channels, as the underlying mechanism. In general, these findings show that the bottleneck of multimodal decision making is no longer visual perception, but the integration of heterogeneous evidence into constraint-satisfying action. The authors suggest that progress will likely require advances not only in model architecture and training, but in how we design evaluation environments that make the distinction between perception and action measurable, and offer design principles for future multimodal agents including two-stage reasoning architectures: separate evidence acquisition from decision execution," selective evidence exposure, and constraint-aware decision execution: preserve task constraints independently of the surrounding context.

1. Two-stage perception–action architecture

Split the pipeline into an Evidence Integration Module (ingests text + all charts and outputs a compact structured summary: key numbers, trends, divergences, leading indicators) and a Decision Execution Module (receives only that summary + hard constraints, and produces the final plan). Because the paper shows that full multimodal context degrades resource allocation (score 0.542) while single-channel visual input outperforms it (ops-only 0.711), this architecture lets the system exploit all visual evidence for understanding while preventing raw charts from interfering with constrained action generation.

What it can do: A CEO agent can ingest financial dashboards, KPI charts, and growth trend visuals to build a rich understanding of the firm, yet still produce budget reallocation plans that respect all numerical constraints even when many evidence sources are present.

2. Constraint-isolated decoding with a verification/repair layer

Add a hard constraint checker (rule-based or a small solver) that validates any generated allocation against the stated constraints (e.g., total budget, per-unit bounds, non-negativity). If the model's output violates a constraint, a separate repair module — which does not see the visual evidence — minimally adjusts the plan to satisfy all constraints before returning it.

3. Task-conditional evidence routing (selective multimodal exposure)

Use a lightweight classifier to identify the task type, then regulate evidence input accordingly:

• T1 Diagnosis, T2 Prioritization, T4 Risk Forecasting, T5 Justification → full multimodal input (where gains are strong, e.g., T4 Δ = +0.27).

• T3 Resource Allocation → feed either a distilled evidence summary, a single best visual channel chosen on the fly, or text-only input (since full multimodal input was negative for all nine models).

4. Salience-discounting mechanism for conflicting evidence

In Tension and Adversarial scenarios (where allocation degradation is worst: −0.118 and −0.102 versus −0.028 for Easy), the system should weight evidence by strategic relevance to the stated constraints, not by visual salience. Implement this via attention gating that down-weights volatile or distractor channels (e.g., visually prominent but decision-irrelevant metrics) when the task involves hard constraints.

5. Perception–action disentangled training objective

Train models with a multi-task objective containing two terms: (a) reward improvement on evidence-centric tasks when multimodal input is present, and (b) penalize any degradation in constrained-action validity when multimodal input is present — e.g., a constraint-validity term that is invariant to input modality. This makes the perception and action capabilities explicitly separable in the loss, rather than conflated as in current MLLM training.

6. Adaptive single-channel selection for planning tasks

At inference time, for allocation-style tasks, run quick ablation probes (as in the paper: finance-only, growth-only, ops-only) or a predictor to determine which single evidence channel is most decision-relevant, then route only that channel to the planner. The paper shows the best single channel improves T3 by +0.052 over text-only, while combining all three degrades it by −0.08.

7. Evidence-load capacity monitoring

Add a runtime monitor that estimates the system's current evidence load (number of concurrent quantitative channels, degree of cross-metric conflict, scenario difficulty family). When the load exceeds a calibrated threshold — or when the system detects adversarial/tension conditions — it automatically switches to a constraint-isolated mode (Improvements 1 and 2) for the action stage.

Large language models are increasingly applied as autonomous decision-making agents. However, in executive business decisions, existing benchmarks are limited to textonly settings. This makes it unclear whether models can perceive visual business evidence and effectively integrate it to improve decision quality. We introduce C-SUITEBENCH, a controlled multimodal benchmark that includes five decision tasks under paired text-only and multimodal conditions across 50 scenarios. We place nine frontier models in the role of a chief executive officer and evaluate their decision-making ability. Multimodal inputs consistently improve evidence-centric reasoning, with the largest and most reliable gains appearing in risk forecasting and board-facing justification. However, we uncover a multimodal integration paradox: adding visual business information degrades constrained resource allocation for all nine models, even as visual grounding itself improves. Ablation experiments reveal that this failure emerges from signal crowding, although each visual channel helps individually, their combination disrupts constraint satisfaction during decoding. These findings demonstrate that visual perception and constrained action are separable bottlenecks in multimodal agents, and that indiscriminate visual augmentation can harm high-stakes decision making, motivating selective grounding strategies for future executive AI systems.