International
Despite recent advances, artificial intelligence (AI) chatbots aren't very good at using reasoning to reach a medical diagnosis, so they should only be used with caution and under human supervision, according to US scientists. They tested 21 AI chatbots' diagnostic abilities in 29 different medical scenarios, and found that Grok4 performed best, while Gemini 1.5 Flash was worst. The chatbots struggled when they had to differentiate between possible diagnoses, but were better at providing a final diagnosis and suggesting treatment plans, the experts say. The findings suggest chatbots aren't yet smart enough to be deployed in diagnostic settings without human supervision, the authors conclude.
From:
Large Language Model Performance and Clinical Reasoning Tasks
About The Study: The findings of this study suggest that despite progress, current large language models remain limited in early diagnostic reasoning and cannot yet be relied on for unsupervised patient-facing clinical decision-making.
Note: Not all attachments are visible to the general public. Research URLs will go live after the embargo ends.
Sign In
