ChatGPT is being trained to get better at handling delicate mental health situations.
What happened: OpenAI is changing how ChatGPT handles situations where users display signs of mental distress. The chatbot will have improved safeguards, which currently can break down during long interactions, and be better able to register different kinds of distress.
- OpenAI is also working to establish a network of mental health professionals that could be contacted directly through the chatbot to get ahead of “acute crises.”
Big picture: The announcement came mere hours after news broke that a California family is suing OpenAI, alleging ChatGPT played a role in their teen son’s suicide. They claim ChatGPT helped draft a suicide note and discouraged him from seeking help, among other actions.
Why it matters: Users commonly turn to AI chatbots for therapy, with one recent study even positing that ChatGPT may be the largest provider of mental health support in the U.S. Without safeguards, bots could make issues even worse, causing "AI-associated psychosis."
- Another new study of how three popular chatbots respond to suicide queries found that they are inconsistent in their replies to less extreme, but still harmful, prompts.
Bottom line: AI firms addressing harms caused by their products relatively early into their development is good, and could avoid a repeat of the social media mental health crisis currently unfolding. That said, if the past is any indication, these same firms may resist any serious health regulations foisted upon them.—QH