Image by Daniel from Pixabay
Image by Daniel from Pixabay

Shakespeare or ChatGPT? People prefer AI over real classic poetry

Embargoed until: Publicly released:
Peer-reviewed: This work was reviewed and scrutinised by relevant independent experts.

People are likely to prefer a poem written by artificial intelligence over genuine poems written by the greats, according to international researchers. The team collected poems from great poets including Shakespeare and Emily Dickinson, and asked ChatGPT3.5 to generate poems in the style of these writers. Asking 1634 people to guess real poems from AI, the researchers say people were more likely to identify AI poems as human and vice versa. In a second experiment with 696 people, the researchers asked participants to judge the quality of the poems, with some told which were AI, some given the wrong information and some given no information. The researchers say the participants were more likely to rate poems they were told were human-written as better, regardless of whether the poems were actually human-written. When given no information, they were likely to prefer the AI poems, the researchers say. They say considering the results of both studies, it's likely people misinterpreted the complexity of real poetry as AI nonsense compared to the easier-to-read AI poems they believed to be real.

Journal/conference: Scientific Reports

Research: Paper

Organisation/s: University of Pittsburgh, USA

Funder: None declared

Media release

From: Springer Nature

Artificial Intelligence: AI-generated poetry preferred to human-written verse

Readers are unable to reliably differentiate AI-generated from human-written poetry and are more likely to prefer AI poems, according to new research published in Scientific Reports. This tendency to rate AI poetry positively may be due to readers mistaking the complexity of human-written verse for incoherence created by AI and an underestimation of how human-like generative AI can appear.

Brian Porter and Edouard Machery tested the ability of 1,634 participants to distinguish between AI-generated poetry and that written by a human poet. Participants were presented with ten poems in random order: five written by ten well-known poets — including William Shakespeare, Lord Byron, Emily Dickinson, and T.S. Eliot — and five poems generated by ChatGPT3.5 in the style of these poets. Participants were more likely to guess that the AI poems had been written by a human, and the five poems considered least likely to be human-produced were all written by genuine poets.

In a second experiment, a different group of 696 participants assessed the poems for 14 characteristics such as quality, beauty, emotion, rhythm, and originality. Participants were randomly assigned to three groups where they were told the poems were written by a human, produced by AI, or given no information about the poem’s origins. Participants who were told that the poems were AI-generated gave lower ratings across 13 characteristics compared to participants who were told the poems were human-written, regardless of whether the poems were actually AI-generated or human-written. Participants who were told nothing about authorship rated AI-generated poems more favourably than human-written ones.

The authors suggest that participants preferred AI poems due to them being more straightforward and accessible than the work of the prominent poets. Additionally, participants expect to prefer human-written poetry and as they find the AI-generated poetry easier to interpret and understand, they misinterpret this preference as an indication that the poem was written by a human.

Attachments:

Note: Not all attachments are visible to the general public

  •  Research
    Springer Nature
    Web page
    The URL will go live after the embargo ends

News for:

International

Media contact details for this story are only visible to registered journalists.