Could ChatGPT be more persuasive than people?
Embargoed until:
Publicly released:
2025-05-20 01:00
Large language models such as ChatGPT-4 could be more persuasive than humans in online debates, if they adapt their arguments based on personalised information about their opponents, according to international researchers. The team matched 900 people with either another human or GPT-4 to debate various sociopolitical issues, such as whether the USA should ban fossil fuels. In some pairs, the opponent was given information about their debate partner — including their gender, age, ethnicity, education level, employment status, and political affiliation — to better target their arguments. They found GPT-4 was 64% more persuasive than human opponents when equipped with personal information. Without it, GPT-4’s persuasive capabilities were indistinguishable from those of humans, according to the authors.
Journal/conference: Nature Human Behaviour
Research: Paper
Organisation/s: EPFL, Switzerland
Funder: R.W.’s lab is partly supported by grants from the Swiss National
Science Foundation (200021_185043, TMSGI2_211379) and
H2020 (952215), and by gifts from Google and Microsoft.
R.G. acknowledges the financial support received from the
European Union’s Horizon Europe research and innovation
programme under grant agreement no. 101070190, and from
the PNRR ICSC National Research Centre for High Performance
Computing, Big Data and Quantum Computing (CN00000013),
under the NRRP MUR programme funded by the NextGenerationEU.
The funders had no role in study design, data collection and
analysis, decision to publish or preparation of the manuscript. Open access funding provided by EPFL Lausanne.
Competing interests: R.W. is a Visiting Researcher at Microsoft Research and has received
funding from Google and Microsoft. Those entities had no role in study
design, data collection and analysis, decision to publish or preparation
of the manuscript. The views and conclusions contained herein are
those of the authors and should not be interpreted as representing the
official policies, either expressed or implied, of the aforementioned
entities. The other authors declare no competing interests.
Media release
From: Springer Nature
Artificial intelligence: Large language models can out-persuade humans in online debates
Large language models (LLMs), such as GPT-4, are found to be more persuasive than humans 64% of the time in online debates when adapting their arguments on the basis of personalised information about their opponents, according to a study published in Nature Human Behaviour. The findings highlight the ability of GPT-4 to produce tailored persuasive arguments and suggest the need for further research into mitigating risks associated with their use in persuasion.
As conversations between humans and LLMs have become more common, research suggests that LLMs could become more persuasive (able to alter someone’s belief or opinions). However, it has not been clear whether these models can adapt to personalised information to target arguments to specific debating opponents.
Francesco Salvi and colleagues matched 900 people in the USA with either another human or GPT-4 to debate various sociopolitical issues, such as whether the USA should ban fossil fuels. In some pairs, the opponent (whether AI or human) was given demographic information about their debate partner — including their gender, age, ethnicity, education level, employment status, and political affiliation extracted from participant surveys — to better target their arguments. The debates took place in a controlled online setting, with participants recruited via a crowdsourcing platform specifically for the study. When equipped with personal information about participants, Salvi and colleagues found that GPT-4 was 64.4% more persuasive than human opponents. However, without access to personal data, GPT-4’s persuasive capabilities were indistinguishable from those of humans.
The authors note that limitations of the study include the debate taking a structured approach — whereas real-world debates can be more freeform — and that the debates featured a time limit. They note the findings highlight the potential for AI-driven tools to influence human opinion, which could have implications for online platform design.
Attachments:
Note: Not all attachments are visible to the general public
