Journal publisher eLife's evaluation summaries may be misinterpreted by many readers

Embargoed until: Publicly released: 2024-08-23 04:00

Australia; VIC

What does this mean?

Jump to: MultimediaAttachmentsExpert Reaction

Journal publisher eLife produces a plain-language summary for their papers evaluating their importance and strength of evidence, but an Australian study suggests many educated readers may be misinterpreting the language the summaries use. The researchers say the eLife assessments use a vocabulary with a hierarchy to determine how important they think a paper is and how strong the evidence is, but many of the words in this vocabulary are close to synonyms, potentially making it hard for a reader to see where a paper sits on the hierarchy. To test this, they asked 301 people with a doctoral or graduate degree to read a series of summaries for hypothetical papers using eLife's vocabulary, and rate their understanding of the paper's value. The researchers say most participants ranked the papers differently to how they were intended, and the participant rankings were inconsistent. Testing an alternative vocabulary developed by the researchers, the team say the participants were able to rank the papers more consistently and more accurately.