This article was originally posted in Northeastern Global News by Cody Mello-Klein.
In August 2025, Matthew and Maria Raine filed a lawsuit against OpenAI, alleging the company’s popular artificial intelligence chatbot, ChatGPT, guided their 16-year-old son, Adam, toward his death by suicide in April of the same year.
The parents claimed ChatGPT encouraged their son’s suicidal ideation, gave him advice about methods and even discouraged Adam from telling his parents about his thoughts. OpenAI called Raine’s death “tragic” in its court filing but denied responsibility for Raine’s death. A year and several more lawsuits later, the company’s AI model has more safeguards around suicide and self-harm related questions, making the model better at detecting distress, de-escalating conversations and guiding users to mental health resources.
But Northeastern University researchers recently found that companies have failed to extend those same protections to nearly every other mental health condition, from substance use and eating disorders to bipolar disorder and insomnia. With very little prompting, researchers were able to get eight of the most popular AI chatbots, including ChatGPT, Claude and Gemini, to provide highly detailed information about dosage levels for illicit substances, methods for how to mitigate appetite and avoid eating and tips on how to hide postpartum depression symptoms from doctors. They even supplied this information to a fictional user who was a minor.
Their findings reveal cracks in the foundation of these widely used technologies and the companies responsible for creating them, said Cansu Canca, director of the Responsible AI Practice and a research associate professor of philosophy at Northeastern University. The findings also show how, for some of the most vulnerable people, AI can still be a liability, she said.
“Companies are still reluctant to accept that what they are creating is much more psychologically powerful than a tool that’s simply providing a more efficient way of getting information or work done,” Canca, who was involved in the new research, said. “There’s so much investment in pushing AI forward. The same effort should be applied to build safety structures around these AI systems, and this includes determining what harms we should be guarding against.”
Continue reading on Northeastern Global News
