Media release

From: The BMJ

Spot the difference: can AI generate plausible Christmas BMJ titles?

AI generated titles were as attractive and entertaining to readers as real titles, but performance was enhanced by human intervention
AI could have a role in generating hypotheses or directions for future research

Artificial intelligence (AI) technology can generate plausible, entertaining, and scientifically interesting titles for potential research articles, finds a study in the Christmas issue of The BMJ.

A study of The BMJ’s most popular Christmas research articles - which combine evidence based science with light hearted or quirky themes - finds that AI generated titles were as attractive to readers but that, as in other areas of medicine, performance was enhanced by human input.

As such, the researchers say AI could have a role in generating hypotheses or directions for future research.

AI is already used to help doctors diagnose conditions, based on the idea that computer systems can learn from data and identify patterns. But can AI be used to generate worthwhile hypotheses for medical research?

To find out, the researchers used the titles of The BMJ’s 13 most read Christmas research articles of the past 10 years to prompt similar AI generated titles, which they scored for scientific merit, entertainment, and plausibility.

The 10 highest and 10 lowest scoring AI generated titles were then combined with 10 real Christmas research articles and were rated by a random sample of 25 doctors from a range of specialties in Africa, Australia, and Europe.

The results show that AI generated titles were rated at least as enjoyable (64% v 69%) and attractive (70% v 68%) as real titles, although the real titles were rated as more plausible (73% v 48%).

The AI generated titles overall were rated as having less scientific or educational merit than the real titles (58% v 39%), however this difference became non-significant when humans curated the AI output (58% v 49%).

This finding fits with previous work on AI suggesting that the best results come from combining machine learning with human oversight, say the authors.

Of the AI generated titles, among the highest for plausibility were “The clinical effectiveness of lollipops as a treatment for sore throats,” and “The effects of free gourmet coffee on emergency department waiting times: an observational study.”

The funniest AI generated title was “Superglue your nipples together and see if it helps you to stop agonising about erectile dysfunction at work,” although the authors note that this illustrates AI’s inability to see the real world application of a study and to understand if titles are offensive.

They acknowledge some limitations, but say even in the context of quirky titles such as those that appear in the Christmas issues of The BMJ, “AI has the potential to generate plausible outputs that are engaging and could attract potential readers.”

However, they stress the importance of human intervention, "a finding that mirrors the potential use of AI in clinical medicine, as decision support rather than as outright replacement of clinicians," they conclude.