Do chatbots have distinct, reliable personalities?

Embargoed until: Publicly released: 2024-10-09 10:01

International

Photo by Jonathan Kemper on Unsplash

The personality of some chatbots is more reliable and consistent than others, according to international researchers. To investigate how well seven of the most popular large language models interact with humans, the researchers put them through personality testing at two different time points. They say Llama 3 and GPT-4o were more likely to display the same general personality traits consistently than GPT-4 and Gemini, and chatbots with a consistent personality were likely to seem more agreeable and conscientious and less Machiavellian. The researchers say for artificial intelligence language bots to be safe, they need to have a reliable personality that suits the humans interacting with them.

Media release

From:

Large Language Models (LLMs) like GPT-4, Llama 3, and others are increasingly popular for their human-like interaction capabilities. This study evaluates the temporal stability and inter-rater agreement of LLM responses to personality tests. Findings show that LLMs like Llama 3 and GPT-4o exhibit higher consistency, while models such as GPT-4 and Gemini display variable reliability over time. For traits showing fair agreement, LLMs depicted a prosocial profile with elevated agreeableness, conscientiousness, and lower Machiavellianism. Consistent, prosocial traits in LLMs are vital for their societal impact, ensuring stability and AI safety in human-AI interactions.

Attachments

Note: Not all attachments are visible to the general public. Research URLs will go live after the embargo ends.

Research The Royal Society, Web page The URL will go live after the embargo ends

Journal/
conference: Royal Society Open Science

Research:Paper

Organisation/s: University of Novi Sad, Serbia

Funder: This paper is realised with the support of the Ministry of Science, Technological Development and Innovation of the Republic of Serbia, according to the Agreement on the realisation and financing of scientific research.

Media Contact/s

Contact details are only visible to registered journalists.