ChatGPT is getting better at citing research, but it still sometimes makes stuff up

Embargoed until: Publicly released: 2023-08-09 01:00

International

What does this mean?

Jump to: MultimediaAttachmentsExpert Reaction

ChatGPT's newest model is better at citing real scientific studies to support its advice but it still references fake articles, according to international researchers. To test the capacity of the AI chatbot to help create medical training content, the researchers put both GPT-3.5 and its newer model GPT-4 to the test - asking them questions about learning health systems, asking them to cite journal articles to back up their claim and then verifying if the sources they chose were legitimate. The researchers say 98% of the references GPT-3.5 gave were fake, while 20.6% of the references GPT-4 gave were fake. The researchers say GPT-3.5 shouldn't be used to help create medical training content and GPT-4 can only be used with humans manually verifying its claims.