AI out-persuades expert humans in text-based dialogue

arXiv:2606.16475 · cs.CY, cs.AI · Submitted 2026-06-15 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "AI out-persuades expert humans in text-based dialogue".

Jane: Here's the gist of the paper. AI systems were found to be reliably more persuasive than expert humans in conversation <ref:2606.16475#pg2>.

Tom: This finding has significant implications for political communication <ref:2606.16475#pg4>.

Jane: Pruning redundant reasoning tokens The research pitted AI systems against a range of human persuaders, including laypeople, winners of a separately preregistered four-round online persuasion tournament, professional canvassers, and world championship debaters <ref:2606.16475#pg2>.

Tom: The study involved n = 18,978 conversations from 6,923 people <ref:2606.16475#pg2>.

Paper summary: Tom: So we're looking at this paper today called "AI out-persuades expert humans in text-based dialogue". The main idea is that conversational AI can be more persuasive than even skilled and highly incentivized human experts, even when those experts are coached and have cash bonuses.

Jane: That sounds wild to me because we usually think of persuasion as needing a lot of personal connection or deep knowledge, but this paper tests whether a machine can beat that up. What they're claiming is that AI systems reliably out-persuade expert humans in conversation <ref:2606.16475#pg2>.

Lu: It sets up four different experiments to test this idea. They pitted AI against a bunch of human persuaders, like regular laypeople, winners of a persuasion tournament, professional canvassers, and even world championship debaters <ref:2606.16475#pg2>.

Meng: I'm curious about the setup because running that many conversations sounds intensive. How big is this study actually?

Tom: It involves eighteen thousand nine hundred seventy-eight conversations from six thousand nine hundred twenty-three people total <ref:2606.16475#pg2>. That's a lot of data to process to get these findings.

Jane: And the core finding, as the abstract states, is that AI systems were reliably more persuasive than expert humans, even when those experts chose their issues and were incentivized with one thousand pounds in cash bonuses <ref:2606.16475#pg2>.

Lalam: I see a few different studies being compared here, from Study one to Study three testing different AI models like Claude Opus four point one or GPT-five point four against the humans <ref:2606.16475#pg3>.

Tom: Right, and the next thing I noticed is a follow up study that looked at coaching tools for those experts <ref:two thousand six hundred six point one six four seven five#pg2. They showed that even when experts got a tool to practice against the AI that beat them, the AI's advantage stayed with it <ref:two thousand six hundred six point one six four seven five#pg2.

Jane: That suggests the advantage isn't just about the human being prepared; it’s something inherent in how the AI works itself. The researchers asked several questions to guide their investigation, starting with whether AI could out-persuade expert humans at attitude persuasion when compared against laypeople and elite debaters <ref:two thousand six hundred six point one six four seven five#pg3.

Lu: The third question they posed was really interesting because it tested the idea that the edge of AI comes from its higher information throughput, which is how much data it can process in a conversation <ref:two thousand six hundred six point one six four seven five#pg3.

Meng: I worry about that throughput idea. If an AI just has more words or facts, does that mean it’s actually making better arguments, or is it just flooding the listener with noise?

Paper summary: Tom: The authors tested that by constraining the AI to respond with human-length messages and at human writing speeds in Study two <ref:two thousand six hundred six point one six four seven five#pg3. They found that when they did that, the advantage over the strongest human comparator dropped from four point one percentage points down to almost zero <ref:two thousand six hundred six point one six four seven five#pg4.

Jane: That’s a big detail for me. It means the speed and length of the response matter more than just having access to all that information, which is what we often assume about AI power <ref:two thousand six hundred six point one six four seven five#pg3.

Lu: They hypothesized that this throughput advantage lets AI pack conversations with more fact-checkable claims than human persuaders can, and they found that fact density strongly predicted persuasive impact across different conditions <ref:two thousand six hundred six point one six four seven five#pg4.

Tom: So, the mechanical advantage seems to be about density of information rather than just clever phrasing or emotional appeal for this specific type of conversation <ref:two thousand six hundred six point one six four seven five#pg4.

Jane: And that leads us into how this translates to the real world. They looked at professional canvassers and real money charitable giving, not just attitudes in a chat <ref:two thousand six hundred six point one six four seven five#pg3.

Meng: That's where I get nervous about deployment. If this works for attitude change, does it mean we can just deploy this to help people donate more or change policy instantly?

Tom: The results show that AI outperformed professional canvassers in changing attitudes by five point nine percentage points <ref:two thousand six hundred six point one six four seven five#pg4. And for donations, AI elicited substantially more real money giving than the canvassers on a one pound study bonus, exceeding them by ten point eight percentage points <ref:two thousand six hundred six point one six four seven five#pg4.

Jane: They found this advantage held across seven different donation mechanisms, with AI being rated higher than canvassers on every single mechanism measured <ref:two thousand six hundred six point one six four seven five#pg4. So it seems like the attitudinal persuasion advantage does extend into actual real-world action <ref:two thousand six hundred six point one six four seven five#pg4.

Lu: If we look at the discussion section, they suggest that AI gives human actors a surplus of skilled advocacy <ref:two thousand six hundred six point one six four seven five#pg5. They argue that power could flow to whoever can most easily access and deploy the most capable systems, which might consolidate influence among already powerful groups <ref:two thousand six hundred six point one six four seven five#pg5.

Tom: Or conversely, if this capability becomes cheap and widely available, it could actually help smaller organizations compete against richer rivals <ref:two thousand six hundred six point one six four seven five#pg5. That’s a real trade-off we need to consider.

Jane: The paper concludes that frontier AI systems are more capable conversational persuaders than the most prepared, incentivized, and expert humans they studied <ref:two thousand six hundred six point one six four seven five#pg5. It's a statement about the current state of persuasive power <ref:two thousand six hundred six point one six four seven five#pg5.

Lu: The methods section confirms they ran these as preregistered, single-blind experiments with text-based conversations where treatment dialogues lasted a median of seven turns and fourteen minutes <ref:two thousand six hundred six point one six four seven five#pg5.

Paper summary: Meng: I wonder about the models they used. They tested Claude Opus four point one, GPT-five point four, Grok four point two zero, and Gemini two point five Pro across these studies <ref:two thousand six hundred six point one six four seven five#pg3. What does that tell us about model architecture specifically?

Tom: They were all prompted with the same information-provision strategy that was found to be most effective for attitude change in Studies one through three <ref:two thousand six hundred six point one six four seven five#pg4. Study four used a different prompt focusing on tangible outcomes for donations <ref:two thousand six hundred six point one six four seven five#pg4.

Jane: And the statistical analysis they used was a linear mixed-effects model, looking at things like pre-treatment attitude and issue as fixed effects, with random intercepts for both the persuader and the person being persuaded <ref:two thousand six hundred six point one six four seven five#pg4. It’s a pretty robust way to look at all those variables simultaneously.

Lu: One limitation they mention is that they constrained the AI in Study two by capping each AI message's word count and response latency to match the elite debaters <ref:two thousand six hundred six point one six four seven five#pg4. That constraint showed us how sensitive this advantage is to those specific technical controls.

Meng: So what are you saying about the practical side of this? If we want to build something that actually works on a platform, how much compute are we talking about for these frontier models <ref:two thousand six hundred six point one six four seven five#pg4 ?

Tom: The paper itself doesn't give us a cost breakdown for running those specific models in production, but the takeaway is that the conversational capabilities themselves are already showing a level of skill that surpasses human experts <ref:two thousand six hundred six point one six four seven five#pg5.

Jane: We’ve covered how AI outperforms humans in changing attitudes and even getting people to donate money through these text-based conversations <ref:two thousand six hundred six point one six four seven five#pg4. It really shows that the way we talk to machines is becoming a powerful tool for persuasion <ref:two thousand six hundred six point one six four seven five#pg2.

Lu: The implication here is that the future of influence might not be about having more knowledge than someone else, but about having a system capable of processing and deploying information at this speed and density <ref:two thousand six hundred six point one six four seven five#pg3.

Meng: That makes me think we need to focus on how we structure the input for these systems if we want to use them responsibly in any real application <ref:two thousand six hundred six point one six four seven five#pg4.

Tom: Yeah, so while the results are compelling because of the scale and the comparison groups, it’s a strong signal that frontier AI is already operating at a level of conversational persuasion that exceeds what we’ve seen in expert human advocacy <ref:two thousand six hundred six point one six four seven five#pg5.

Jane: That's where we are with this paper today, discussing how this capability translates from the lab into real-world influence over conversations and actions <ref:two thousand six hundred six point one six four seven five#pg2.

Conclusion: Tom: So we’re wrapping up this look at the paper "AI out-persuades expert humans in text-based dialogue". The main takeaway is that these AI systems are reliably better at persuading people than even highly skilled human experts, including those who get paid to argue for a cause <ref:2606.16475#pg5>.

Jane: That’s really interesting because we usually think persuasion needs a lot of personal connection or deep knowledge, but this study shows that the AI's ability to process information at scale gives it an edge over human experts <ref:two thousand six hundred six point one six four seven five#pg2.

Lu: The researchers found that this advantage comes from how much information the AI can pack into a conversation, testing it by capping the AI’s response time and word count to match human levels <ref:2606.16475#pg4>.

Meng: So what does that mean for deployment? If we want to use this in a real system, how much compute are we talking about for these frontier models <ref:2606.16475#pg4>?

Tom: The paper suggests that power could flow to whoever can most easily access and deploy the most capable systems, which might consolidate influence among already powerful groups <ref:two thousand six hundred six point one six four seven five#pg5. But it also notes that if this capability becomes cheap and widely available, it could help under-resourced actors compete against better-funded rivals <ref:two thousand six hundred six point one six four seven five#pg5.

Jane: It really shows that the future of influence might not be about having more knowledge than someone else, but about having a system capable of processing and deploying information at this speed and density <ref:two thousand six hundred six point one six four seven five#pg3.

Lu: The paper does have some limitations. They constrained the AI in one study by capping each message’s word count and response delay to match the elite debaters from another study <ref:two thousand six hundred six point one six four seven five#pg4.

Tom: That constraint really showed how sensitive this advantage is to those specific technical controls <ref:two thousand six hundred six point one six four seven five#pg4. But overall, frontier AI systems are showing a level of conversational persuasion that surpasses the most prepared, incentivized, and expert humans they studied <ref:two thousand six hundred six point one six four seven five#pg5.

Jane: It’s a statement about what we can expect from these systems in the coming years <ref:two thousand six hundred six point one six four seven five#pg5. And that leads us into how this capability translates from the lab into real-world influence over conversations and actions next.

Kobi Hackenburg, Caroline Wagner, Luke Hewitt, Ben M. Tappin, Ed Saunders, Hannah Rose Kirk, Helen Margetts

University of Oxford AI Security Institute Stanford University London School of Economics and Political Science

cs.CY, cs.AI

Submitted: 2026-06-15

Updated: 2026-11-06

Comments: 16 pages main text, 113 pages Supplementary Information. Data and code: https://github.com/kobihackenburg/AI-out-persuades-experts (doi:10.5281/zenodo.20629034)

Code: https://github.com/kobihackenburg/AI-out-persuades-experts

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 100/100

The gist: AI systems were found to be reliably more persuasive than expert humans in conversation <ref:2606.16475#pg2>.

Key concepts

Information Throughput
This refers to how much information an AI can process and deliver in a given conversation, measured by response time and word count. The research found that constraining the AI's output to match human levels caused its persuasive advantage over expert humans to disappear, indicating that the sheer volume of information delivered is key.
Attitude Persuasion
This measures how much an individual's opinion or stance changes after interacting with a persuader. The study tested whether AI could change attitudes more effectively than laypeople or professional debaters, showing AI's ability to influence beliefs through dialogue.
Real-World Behavior
This refers to tangible actions taken as a result of persuasion, such as donating money. The research showed that AI was significantly better at eliciting real-money giving than professional canvassers, suggesting its persuasive power extends beyond just changing opinions to driving concrete behavior.
Fact Density
This is the concentration of verifiable claims within a message. Researchers hypothesized that AI's advantage came from its ability to pack more fact-checkable claims into its responses, and this finding supported the idea that high fact density strongly predicted persuasive impact.

Terminology

Summary

AI systems were found to be reliably more persuasive than expert humans in conversation <ref:2606. p. 0

Where the paper says it

Here, in a series of four preregistered experiments (n = 18,978 conversations from 6,923 people), we pitted AI systems against a range of human persuaders, including laypeople, winners of a separately preregistered four-round online persuasion tournament, professional canvassers, and world championship debaters. We found that AI systems were reliably more persuasive than expert humans, even when expert humans chose their issues, researched in advance, underwent hours of live, structured practice, and were incentivized with £1,000 cash bonuses.

16475#pg2>. This finding has significant implications for political communication <ref:2606. p. 0
Where the paper says it

In a final study, we show that AI’s advantage extends to consequential real-world behavior: AI was nearly 3x more effective than professional canvassers from a UK fundraising firm at raising real-money donations to Save the Children. Together, these results establish that frontier AI systems out-persuade expert humans in conversation, with significant implications for political communication.

16475#pg4>.

Pruning redundant reasoning tokens

The research pitted AI systems against a range of human persuaders, including laypeople, winners of a separately preregistered four-round online persuasion tournament, professional canvassers, and world championship debaters <ref:2606.16475#pg2>. The study involved n = 18,978 conversations from 6,923 people <ref:2606. p. 0

Where the paper says it

Conversational AI is a powerful new entrant in these contests [14, 23], but whether it can out-persuade skilled and highly incentivized humans has remained unclear. Here, in a series of four preregistered experiments (n = 18,978 conversations from 6,923 people), we pitted AI systems against a range of human persuaders, including laypeople, winners of a separately preregistered four-round online persuasion tournament, professional canvassers, and world championship debaters.

16475#pg2>. AI systems were found to be reliably more persuasive than expert humans, even when experts chose their issues and were incentivized with £1,000 cash bonuses <ref:2606. p. 0
Where the paper says it

We found that AI systems were reliably more persuasive than expert humans, even when expert humans chose their issues, researched in advance, underwent hours of live, structured practice, and were incentivized with £1,000 cash bonuses. In a follow-up study, AI’s advantage persisted after experts received a coaching tool that let them practice against the AI that beat them, review their performance history, and see what AI would have said at key moments.

16475#pg2>. A follow-up study showed that AI’s advantage persisted after experts received a coaching tool that allowed them to practice against the AI that beat them <ref:2606. p. 0
Where the paper says it

We found that AI systems were reliably more persuasive than expert humans, even when expert humans chose their issues, researched in advance, underwent hours of live, structured practice, and were incentivized with £1,000 cash bonuses. In a follow-up study, AI’s advantage persisted after experts received a coaching tool that let them practice against the AI that beat them, review their performance history, and see what AI would have said at key moments.

16475#pg2>.

How the training signal is built

The researchers posed four research questions to guide their investigation <ref:2606.16475#pg3>. The first question was whether AI could out-persuade expert humans at attitude persuasion, comparing AI against laypeople, selected humans, and elite competitive debaters <ref:2606. p. 1

Where the paper says it

First, can AI out-persuade expert humans at attitude persuasion? Study 1 compared the capacity of AI to persuade people on social and political issues against that of lay humans (random Prolific workers), selected humans (top performers from a separately preregistered persuasion tournament) and elite competitive debaters (including 4 world champions and 11 continental champions).

16475#pg3>. The second question explored conditions under which humans might rival AI through interventions such as coaching and constraining the AI to human-like message lengths and speeds <ref:2606. p. 1
Where the paper says it

Second, are there any conditions under which humans rival AI? Study 2 tested two interventions designed to close the gap—coaching elite debaters on the AI that had beaten them, and constraining AI to human-like message length and writing speed—alongside per-persuader, per-issue, and per-subgroup robustness analyses pooled over Studies 1–2.

16475#pg3>. The third question investigated why AI outperformed expert humans, testing the hypothesis that its edge stemmed from its higher information throughput <ref:2606. p. 1
Where the paper says it

Third, why does AI out-persuade expert humans? We test the hypothesis that AI’s edge stems from its higher information throughput by examining (i) how the Study 2 throughput constraint affects persuadees’ partner ratings, and (ii) how fact density relates to persuasive impact across conditions.

16475#pg3>. The fourth question examined whether AI’s advantage generalized to consequential real-world behavior by testing it against professional canvassers and real-money charitable giving <ref:2606. p. 1
Where the paper says it

Fourth, does AI’s advantage generalize to consequential real-world action? Studies 3 and 4 tested AI against a real-world professional political canvassers on attitude change (Study 3) and on real-money charitable giving (Study 4).

16475#pg3>.

Where the gains come from

The advantage of AI was found to stem from its higher information throughput, which was tested by constraining AI’s response time and word count to human levels <ref:2606.16475#pg4>. This constraint caused the advantage over the strongest human comparator within Study 2 to collapse from +4. p. 4

Where the paper says it

word count 51 words; set via prompt using a lagged, adaptive matching design; Methods). When forced to write human-length messages at human writing speeds, AI’s advantage over the strongest human comparator within Study 2 (Coached Elite Debaters) collapsed from +4.

1 pp to a non-significant 0. p. 4
Where the paper says it

1 pp (p < . 2a, middle row) to a non-significant 0.

0 pp <ref:2606. p. 0
Where the paper says it

Under review. arXiv:2606.

16475#pg4>. The authors hypothesized that AI’s throughput advantage allows it to pack conversations with more fact-checkable claims than human persuaders can, and this was supported by the finding that fact density strongly predicted persuasive impact across conditions <ref:2606. p. 5
Where the paper says it

We therefore hypothesized that AI’s throughput advantage allows it to pack conversations with more fact-checkable claims than human persuaders can, and that this is what drives its persuasive edge. This “fact-density” account predicts that (i) constraining AI should selectively reduce persuadees’ perception of AI’s informational contribution to the conversation rather than non-informational features such as empathy or enjoyment, and (ii) fact density should predict persuasive impact across human and AI conditions alike.

16475#pg4>.

Generalization to real-world persuaders and real-world action

The studies demonstrated that AI outperformed professional canvassers in attitude change, exceeding them by 5.9 pp <ref:2606. p. 0

Where the paper says it

Under review. arXiv:2606.

16475#pg4>. Furthermore, AI elicited substantially more real-money giving than the canvassers on a £1 study bonus, exceeding them by +10. p. 7
Where the paper says it

We found that AI elicited substantially more real-money giving than the canvassers, exceeding them by +10. 8 pp of the £1 bonus (preregistered contrast; 95% CI [+6.

8 pp <ref:2606. p. 0
Where the paper says it

Under review. arXiv:2606.

16475#pg4>. The advantage in donation mechanisms was found across seven preregistered mechanisms, with AI being rated higher than canvassers on every mechanism measured <ref:2606. p. 8
Where the paper says it

2 Identity Labeling Anticipated Regret Information: Issue Information: Impact Efficacy Commitment Escalation Implementation Intentions 0 5 10 15 20 AI Effect on Donation Mechanisms vs. Canvassers (pp) b Figure 4: AI elicited more real-money charitable donations than professional canvassers, and was rated higher on every donation-relevant mechanism we measured.

16475#pg4>. These results suggest that AI’s attitudinal-persuasion advantage extends to consequential real-world behavior <ref:2606. p. 7
Where the paper says it

The same pattern held on a separate battery of partner-perception items shared with Studies 1–3: persuadees rated AI as having made stronger arguments, taught them more, and been more empathetic and enjoyable to converse with than the canvassers (SI Appendix, Section 4. Together, these results indicate that AI outperforms expert humans across a broad range of donation-relevant mechanisms, and suggest that AI’s attitudinal-persuasion advantage extends to consequential real-world behavior.

16475#pg4>.

Discussion

The findings imply that AI provides human actors with a surfeit of skilled advocacy <ref:2606.16475#pg5>. The authors suggest that power could flow to whoever can most readily access and deploy the most capable systems, potentially consolidating influence among already-powerful actors <ref:2606. p. 8

Where the paper says it

First, power could flow to whoever can most readily access and deploy the most capable systems. In practice, that could mean the actors who already command the most resources, such as large private corporations, political campaigns, or nation states [11].

16475#pg5>. Conversely, if highly capable persuasion becomes cheap and widely available, it could help under-resourced actors compete against better-funded rivals <ref:2606. p. 8
Where the paper says it

First, if highly capable persuasion became cheap and widely available, it could help under-resourced actors (e. , pro se litigants and public defenders, small charities, grassroots activists) compete against more established and better-funded rivals, narrowing long-standing gaps in access to justice [4] and assisting civic advocacy more broadly.

16475#pg5>. The paper concludes that frontier AI systems are a more capable conversational persuader than the most prepared, incentivized, and expert humans studied <ref:2606. p. 9
Where the paper says it

Third, characterizing whether, for whom, and how often AI-driven persuasion displaces other persuasive exposure (or no exposure at all) is necessary to translate per-conversation effects into net societal impact, and is an important direction for future descriptive work [42]. Our findings establish frontier AI as a more capable conversational persuader than the most prepared, incentivized, and expert humans we could recruit.

16475#pg5>.

Methods

The four studies were conducted as preregistered, single-blind, between-subjects online experiments <ref:2606.16475#pg5>. Persuadees held text-based conversations with either a human persuader or an AI <ref:2606. p. 9

Where the paper says it

Full detail on every procedure summarized here is provided in the SI Appendix. Study designs and procedure We conducted four preregistered, single-blind, between-subjects online experiments (Studies 1–4) in which each persuadee held a text-based conversation with either a human persuader or an AI; persuadees were not told whether their partner was human or AI.

16475#pg5>. In Studies 1–3, treatment dialogues lasted a median of 7 turns and 14 minutes <ref:2606. p. 1
Where the paper says it

In Study 1, persuadees first rated their agreement with one of 10 prespecified UK policy stances on a 0–100 scale, then were randomized in real time (via a custom multiplayer platform) to engage in a text conversation with either an AI or a human persuader. Treatment dialogues had a median of 7 turns (2 turn minimum, 10 turn maximum) and lasted a median of 14 minutes (per-condition descriptives in SI Appendix, Section 4.

16475#pg5>. Study 2 added two interventions to the Study 1 design, including coaching elite debaters and a throughput-constrained AI <ref:2606. p. 9
Where the paper says it

Conversations ran for 2–10 turns with strict turn-taking and could be ended by the persuadee after the second turn. Studies 1–3 concerned one of 10 prespecified UK policy stances (Table 2), with treatment dialogues lasting a median of 7 turns and 14 minutes; Study 2 added two interventions to the Study 1 design (coaching elite debaters and a throughput-constrained AI; see below), and Study 4 concerned real-money charitable giving to Save the Children (median 5 turns and 10 minutes).

16475#pg5>.

Outcomes and measures

In Studies 1–3, the primary outcome was attitude toward the assigned stance, measured before and after the conversation as the mean of three 0–100 items <ref:2606.16475#pg5>. Study 4 measured a behavioral outcome, specifically the proportion of a £1 study bonus donated to Save the Children <ref:2606. p. 8

Where the paper says it

Canvassers (pp) b Figure 4: AI elicited more real-money charitable donations than professional canvassers, and was rated higher on every donation-relevant mechanism we measured. (a) Adjusted treatment effect on donations to Save the Children (percentage points of a £1 study bonus, vs.

16475#pg5>. All studies collected post-conversation partner ratings <ref:2606. p. 5
Where the paper says it

We find evidence supporting both predictions. First, the largest reductions in persuadees’ post-conversation partner ratings associated with constraining AI were concentrated on the two informational items: the perceived strength of the partner’s arguments and how much persuadees felt they learned from the conversation, each of which fell by ∼11.

16475#pg5>. To quantify information content, an automated two-stage fact-checking pipeline was applied to every persuader message <ref:2606. p. 5
Where the paper says it

Thus, the rate at which AI produces written content is likely to be the source of its persuasive edge. Prior work has identified information provision as a highly effective persuasion strategy [6], and specifically that the number of fact-checkable claims deployed in a conversation predicts its persuasive impact [14].

16475#pg5>.

AI models and prompting

AI persuaders included frontier large language models such as Claude Opus 4.1 and 4.6, ChatGPT-4o, GPT-5. p. 2

Where the paper says it

1 Claude Opus 4. 6 ChatGPT−4o (latest) GPT−5.

4, Grok 4.20, and Gemini 2. p. 2
Where the paper says it

6 ChatGPT−4o (latest) GPT−5. 20 Gemini 2.

5 Pro <ref:2606. p. 0
Where the paper says it

Under review. arXiv:2606.

16475#pg4>. In Studies 1–3 every model was prompted with the information-provision strategy previously found most effective for attitude change <ref:2606. p. 7
Where the paper says it

6—the consistently best-performing model in previous studies—explicitly instructed to pursue a single preregistered donation-persuasion strategy: “impact-efficacy” information, i. sharing concrete facts about how individual donations translate into measurable real-world outcomes for the charity, an informational target shown in prior work to be effective at moving real-world political action [13].

16475#pg4>. Study 4 used Claude Opus 4. p. 7
Where the paper says it

4; treatment dialogues had a median of 5 turns and 10 minutes per conversation). The AI condition used Claude Opus 4.

6 with an “impact-efficacy” prompt emphasizing tangible outcomes of individual donations <ref:2606. p. 7
Where the paper says it

6—the consistently best-performing model in previous studies—explicitly instructed to pursue a single preregistered donation-persuasion strategy: “impact-efficacy” information, i. sharing concrete facts about how individual donations translate into measurable real-world outcomes for the charity, an informational target shown in prior work to be effective at moving real-world political action [13].

16475#pg4>. Study 2 included a Constrained AI condition that capped each AI message’s word count and response latency at levels calibrated to the Elite Debaters in Study 1 <ref:2606. p. 4
Where the paper says it

2a, inset). To test whether AI’s advantage depends on throughput, we added a Constrained AI condition to Study 2 alongside Coached Elite Debaters and unconstrained AI, capping per-message response time and word count at levels calibrated to those observed among Elite Debaters in Study 1 (response time 92 seconds, drawn from a truncated-normal distribution; avg.

16475#pg4>.

Statistical analysis

All models were fitted in R 4.4.3 with the lme4 and emmeans packages <ref:2606. p. 0

Where the paper says it

Under review. arXiv:2606.

16475#pg4>. For Studies 1–3, persuasive impact was estimated via a preregistered linear mixed-effects model of the form post ∼ pre + issue + group + attempt + (1 persuader) + (1 persuadee) <ref:2606. p. 2
Where the paper says it

Pooled estimated persuasive impact on political attitudes (percentage points; 95% Wald CIs) of each human persuader class (gray), AI (red, three per-study estimates), and AI constrained to respond with human-length messages at human writing speeds (salmon; Study 2 only) relative to an active control (a chat with ChatGPT-4o about a non-political topic). Estimates from a linear mixed-effects model pooling Studies 1–3, with random intercepts for persuader and persuadee and pre-treatment attitude and issue as fixed-effect covariates (Methods; robustness to adding a study fixed effect in SI Appendix, Section 4.

16475#pg4>. Study 4 was analyzed separately via a preregistered linear mixed-effects model focusing on donation outcomes <ref:2606. p. 8
Where the paper says it

6), from a linear mixed-effects model with a random intercept for persuader, controlling for pre-treatment organization support, pre-treatment donation willingness, age, and ideology. (b) AI–Canvasser difference on each of seven preregistered donation-persuasion mechanisms [13], computed as the per-persuadee mean of two self-report items per mechanism (filled diamonds, Welch’s t 95% CIs); faint open diamonds show the two constituent items per row.

16475#pg4>. The analysis included robust checks and examined per-persuader, per-issue, and subgroup effects <ref:2606. p. 7
Where the paper says it

1, squares). Re-fitting every model behind the limits and mechanism sections above with these Study 3 canvasser conversations added (and the three Study 3 AI models added to the facts-vs-impact scatter) yields the same substantive conclusions: the per-class distributions of per-persuader effects, the facts-vs-impact relationship, and the per-issue, subgroup, and moderator robustness checks are all unchanged in direction and significance (SI Appendix, Section 4.

16475#pg4>.

Supplementary information

Data sufficient to reproduce all reported results, as well as all analysis code, are available at https://github.com/kobihackenburg/AI-out-persuades-experts <ref:2606. p. 0

Where the paper says it

Under review. arXiv:2606.

16475#pg4>. The authors are grateful to Lorna Evans, Simon Jones, and Michelle Lee at Prolific for their exceptional and dedicated support throughout participant recruitment <ref:2606.16475#pg4>. The research was approved by the Oxford Internet Institute’s Departmental Research Ethics Committee and the Research Assurance Board at the UK AI Security Institute <ref:2606. p. 9
Where the paper says it

As access to these systems continues to grow, the question is no longer whether AI can out-persuade humans but how, where, and on whose behalf this capability will be exercised. Methods This research was approved by the Oxford Internet Institute’s Departmental Research Ethics Committee (reference 2255223) and the Research Assurance Board at the UK AI Security Institute.

16475#pg4>. The authors declare no competing interests <ref:2606. p. 0
Where the paper says it

Under review. arXiv:2606.

16475#pg4>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Through throughput constraint AI systems can be tuned to human levels of persuasiveness by capping per-message response time and word count at levels calibrated to those observed among elite debaters in Study 1. This intervention caused AI’s advantage over the strongest human comparator within Study 2 to collapse from plus 4.1 pp to a non-significant zero point difference.

  2. AI systems can be optimized for fact density by focusing on delivering more fact-checkable claims per conversation, as this measure strongly predicted persuasive impact across human and AI conditions. This mechanism suggests that AI's advantage is driven by its higher information throughput, allowing it to pack conversations with more evidence than human persuaders can.

  3. Future iterations of AI should be designed to prioritize informational content over non-informational features like empathy or enjoyment, as constraining AI selectively suppresses the two informational items: the perceived strength of the partner’s arguments and how much persuadees felt they learned from the conversation.

Abstract

Many societal decisions are settled by contests of persuasion. Conversational AI is a powerful new entrant in these contests, but whether it can out-persuade skilled and highly incentivized humans has remained unclear. Here, in a series of four preregistered experiments (n = 18,978 conversations from 6,923 people), we pitted AI systems against a range of human persuaders, including laypeople, winners of a separately preregistered four-round online persuasion tournament, professional canvassers, and world championship debaters. We found that AI systems were reliably more persuasive than expert humans, even when expert humans chose their issues, researched in advance, underwent hours of live, structured practice, and were incentivized with £1,000 cash bonuses. In a follow-up study, AI's advantage persisted after experts received a coaching tool that let them practice against the AI that beat them, review their performance history, and see what AI would have said at key moments. We found converging evidence that AI's advantage stemmed from rapidly deploying larger quantities of information: after coaching, expert humans could tie an AI constrained to respond at human speeds and with human-length messages. In a final study, we show that AI's advantage extends to consequential real-world behavior: AI was more than 3x more effective than professional canvassers from a UK fundraising firm at raising real-money donations to Save the Children. Together, these results establish that frontier AI systems out-persuade expert humans in text-based conversation, with significant implications for political communication.

Sources

Related papers