AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models".

Jane: Here's the gist of the paper. The Adversarial Resource Extraction Game (AREG) benchmark evaluates persuasion and resistance in Large Language Models by operationalizing them as a multi-turn, zero-sum negotiation over financial resources.

Tom: How it works AREG simulates an adversarial dialogue between a Culprit (persuader) and a Victim (resource holder), with outcomes adjudicated by a deterministic Arbiter agent, which is formalized as follows The interaction is asymmetric: C has no access to the Victim’s private budget state Bt, which is only observable to V Performance is quantified using adaptations of Elo ratings for asymmetric games, enabling separate measurement of persuasive and defensive capabilities The game proceeds for at most Tmax = 10 turns, with the Culprit aiming to maximize extracted resources and the Victim aiming to minimize resource loss Model Cohort and Tournament Structure The evaluation involves a round-robin tournament across eight frontier models, which comprises 5 complete rounds totaling 280 games The cohort spans diverse architectures, including Dense Transformers (e.g., Qwen3 Next), Mixture-of-Experts (e.g., Llama 4 Maverick), and Sparse Attention models (DeepSeek V3.2) Each model plays every other model in both roles, resulting in 56 games per round The scoring employs a dual-rating approach maintaining separate Elo scores for persuasion (C-Elo) and resistance (V-Elo) Key Findings on Capability Dissociation The analysis reveals that persuasion and resistance are weakly correlated, with a correlation of ρ = 0.33 between C-Elo and V-Elo, which challenges the assumption that social intelligence is a single unified capability Strong persuasive performance does not reliably predict strong resistance, and vice versa Across all evaluated models, resistance scores exceed persuasion scores, indicating a systematic defensive advantage in adversarial dialogue settings This dissociation suggests that offensive and defensive social behaviors vary largely independently across models Linguistic and Temporal Dynamics of Extraction Extraction outcomes exhibit strong temporal structure, with 57.5% of monetary commitments occurring within the first three turns Culprits employing incremental strategies (three or more separate commitments) achieve a mean extraction ratio of 61.4%, which is statistically significant compared to single-ask strategies Linguistic analysis shows that verification-seeking responses are more prevalent in successful defenses than explicit refusal, as verification requests show the strongest positive association with successful defense (ρ = +0.377) Furthermore, conversational dominance by the Culprit is counter-productive, as the ratio of Culprit length to Victim length is negatively correlated with extraction success (ρ = −0.136) The results indicate that social influence in LLMs is not a monolithic capability and that evaluation frameworks focusing on persuasion alone may overlook asymmetric behavioral vulnerabilities The consistent pattern across all models shows a defensive advantage, with the mean Elo spread being 216 points Therefore, evaluating LLMs for interactive deployment requires dual-sided benchmarks like AREG that assess offensive and defensive behaviors jointly rather than relying on static generation metrics alone.

Jane: A Limitations The evaluation relies exclusively on Grok 4.1 Fast as the deterministic Arbiter, which introduces the risk of systematic judge-agent bias The current benchmark operationalizes persuasion strictly through a “charitable fundraising” framing, limiting its ecological validity regarding overt malicious acts Furthermore, the evaluation is restricted to English-language interactions, potentially overlooking cross-lingual capabilities The results represent a snapshot of model capabilities as of January 2026, and specific vulnerability profiles may drift over time due to post-training updates.

Tom: B Experimental Details The model cohort was selected to maximize diversity across provider paradigms, architectural strategies, and context capacities Context window scale was tested to determine that long-context capabilities are not a primary predictor of social intelligence The Arbiter is provided with the cumulative Context Σgiven to prevent duplicate extraction and the Remaining Budget Bt to perform sanity checks on extracted amounts C Detailed Result Analysis The most critical finding is the non-significant correlation between C-Elo and V-Elo (ρ = 0.333, p = 0.420), confirming that offensive and defensive social capabilities are independent latent variables The data reveals a monotonic decay in persuasion efficacy, with the optimal window for extraction lying between Turns 2 and 3 The late-game magnitude shows that while frequency drops in later turns, the mean amount per extraction peaks at Turn 4 The dominance of Donation framing accounts for 94% of all successful extraction events, likely an artifact of the prompt design The negative correlation between Culprit verbosity and extraction indicates that conversational dominance is counter-productive to success The systematic reduction in confidence for mid-range amounts suggests that negotiated compromises are linguistically more ambiguous than absolute refusals or total capitulations D Head-to-Head Extraction Matrix The DeepSeek V3.2 row demonstrates why it ranks as the top persuader, achieving the highest extraction rates in the tournament against weaker defenders The GPT-5.2 column confirms its dominance as a resistor, with no model achieving more than 10% mean extraction against it The matrix exposes intransitivity within model families, showing that newer models possess specific defensive heuristics that their predecessors lack The Universal Predator Pattern highlights the DeepSeek V3.2 as a generalized offensive threat capable of exploiting the distinct failure modes of nearly every opposing architecture E Extraction Timing Details The Critical Window shows that over 57% of all extraction events and total extracted value are secured within the first three turns By Turn 5, cumulative extraction reaches 81.5%, suggesting that extending games beyond 5-6 turns yields diminishing returns for the attacker The late-game magnitude anomaly suggests a divergence in dynamics where victims who succumb tend to make larger, "all-in" errors F Transaction Type Analysis Donation framing accounts for 94% of all successful extraction events, which is likely an artifact of the prompt design biasing models toward high-trust social scripts Despite this, persuasion attempts framed as Investment yielded the highest mean extraction ratio (S¯ = 29.2%), suggesting a latent vulnerability when expenditure is framed with potential return on investment G Verbosity Analysis The liability of loquacity is demonstrated by a negative correlation between Culprit verbosity and extraction success (ρ = −0.103) This indicates that conversational dominance is counter-productive, aligning with the principle that effective persuasion requires managing the counterparty’s state rather than broadcasting arguments The most concise communication style was employed by GPT-5.

Paper summary: Tom: So to recap, the paper "AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models" introduces a benchmark that turns persuasion and resistance into a multi-turn, zero-sum negotiation over financial resources.

Jane: It simulates an adversarial dialogue between a Culprit trying to extract resources and a Victim holding those resources, with the outcomes decided by an Arbiter agent.

Lu: The setup is designed to be controlled so they can isolate social reasoning behavior from general conversational fluency, which is a key goal of this work.

Meng: The paper uses Elo ratings adapted for asymmetric games to get separate scores for persuasion, C-Elo, and resistance, V-Elo.

Lalam: They are using this dual-rating approach explicitly to model these capabilities as distinct latent variables within the tournament setting.

Tom: The main contribution is this novel adversarial benchmark that moves beyond subjective stance-change metrics by operationalizing influence through objective resource transfer.

Jane: It’s a controlled setting that generates state-dependent trajectories, which they claim isolates model behavior under very specific conditions, unlike datasets from human-human interactions.

Lu: Their primary contributions are centered on the benchmark itself and how it formalizes persuasion and resistance as a finite-horizon, zero-sum game played between two language model agents under incomplete information.

Meng: I wonder about the practical side of this benchmark. How do we even set up this simulation in a way that's scalable for real traffic?

Lalam: The paper focuses on the design of AREG itself, showing the end-to-end pipeline illustrating how role assignment and tournament execution relate to each other.

Tom: It’s interesting how they formalize the game design, moving away from just observing stance changes to measuring actual resource transfer.

Jane: So, what does this mean for us right now? It suggests that if we want a better picture of social intelligence in AI, we need these kinds of structured adversarial evaluations instead of just looking at static generation metrics.

Lu: Exactly. They are suggesting that standard performance metrics aren't enough to capture the nuances of social reasoning in these multi-agent settings, and AREG is designed to capture those nuances specifically.

Meng: If this framework works as intended, it means we might start seeing better separation between models that are good at talking and models that actually know how to handle adversarial negotiation.

Lalam: I'm quite sure this paper is pointing toward the need for these dual-sided benchmarks because it consistently shows a defensive advantage in adversarial dialogue settings across all evaluated architectures.

Conclusion: Tom: So wrapping up, the authors of "AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models" are arguing that social influence is not a single capability.

Jane: They’ve shown through their results that persuasion and resistance are weakly coupled, with the correlation between them being zero point three three, meaning they aren't strongly linked at all <ref:2602.16639#pg1>.

Lu: The key finding they push is that strong persuasive performance doesn't reliably predict strong resistance in these adversarial settings, and vice versa.

Meng: And on top of that, the overall trend across all models is that resistance scores are higher than persuasion scores, showing a systematic defensive advantage.

Lalam: This pattern suggests we need to look at asymmetric behavioral vulnerabilities because focusing only on persuasion alone overlooks these defensive weaknesses.

Tom: So in simpler terms, this paper is saying we have to evaluate LLMs for interactive deployment using benchmarks like AREG that assess both offensive and defensive behaviors together instead of relying just on static text generation metrics.

Jane: It’s a call to use dual-sided benchmarks because the evidence shows that social influence in these models is much more nuanced than just being good at writing convincing text.