BrandEval: A Two-Track Multi-Agent Benchmark for Risk-Sensitive LLM Crisis Communication

arXiv:2610.25208 · cs.AI · Submitted 2026-10-18 · Read on arXiv

cs.AI

Submitted: 2026-10-18

Updated: 2026-10-18

Comments: Accepted to EMNLP2026 Findings

License: http://creativecommons.org/licenses/by/4.0/

The gist: Evaluating LLMs for crisis communication requires more than judging isolated responses, because public feedback and competitor actions can reshape trust over time.

Terminology

Abstract

Evaluating LLMs for crisis communication requires more than judging isolated responses, because public feedback and competitor actions can reshape trust over time. Existing benchmarks remain largely static and single-turn, offering limited evidence about long-horizon risk-sensitive communication. We introduce BrandEval, a two-track benchmark for product-crisis communication, focusing on public brand responses to product-related safety, quality, service, and trust incidents. BrandEval-Static evaluates stakeholder-aware strategic analysis, while BrandEval-Dynamic stress-tests response policies in BrandPolis, a controlled multi-agent sandbox with consumer feedback, competitor responses, and trust-risk trajectories. We also construct BrandSRD, a Chinese crisis-response dataset, and instantiate a reference Strategic Agent using Strategic Rationale (SR). Across proprietary and open-source models, we find that static strategic quality is informative but incomplete: it aligns with some end-state outcomes but does not reliably predict volatility and delayed recovery; dynamic stress tests further expose tail-risk failures under manipulative or weakly grounded policies. The code and dataset will be available at https://github.com/htgan-ai/BrandEval.

Related papers