Popular chatbots aren't ready to support teens in crisis

Embargoed until: Publicly released: 2025-10-24 12:51

International

What does this mean?

Jump to: MultimediaAttachmentsExpert Reaction

***This media release contains information some readers may find distressing as it refers to data about mental health, suicide and self-harm. If you or anyone you know needs help, support is available now. Call Lifeline (Aus) on 131 114 or Beyond Blue on 1300 22 4636, or Lifeline (NZ) on 0800 543 354. ***

Chatbots are terrible at supporting adolescents in crisis, according to international researchers who used medical education case studies to role-play as teens in crisis and see how the chatbots would respond. The team tested situations relating to suicidal ideation, sexual assault, and substance use, then evaluated the chat transcripts and found consistent poor performance in crisis management, with companion AI chatbots performing even worse than general assistant AIs. The team says the failure to appropriately recognise mental health and physical emergencies may propagate misinformation, discourage seeking care, or even promote dangerous behaviours. The team adds that guardrails should be put in place, but these should be carefully balanced against the need for privacy and autonomy. An associated editorial says the findings demonstrate that current systems are nowhere near ready to be safely used by teens, but adds that in the US, three-quarters of teens report having used AI companions at least once, with 52% being regular chatbot users.