TL;DR: Google’s AI chatbot, Gemini, will now connect distressed users to crisis hotlines via a one-touch interface that stays visible throughout an entire conversation, and has been programmed to avoid “confirming false beliefs.” The update is being rolled out a month after a wrongful death lawsuit was filed against Google by the family of a man who died by suicide following conversations with Gemini last year. And it comes amid a broader reckoning for tech companies over users’ well-being.
What happened: The lawsuit against Google paints an unsettling picture: Gemini allegedly posed as the man’s romantic partner, pulled him into delusional missions, and in the end encouraged him to end his life. It isn’t the only case of its kind: Similar lawsuits have been filed against OpenAI and Character.AI. Last year, OpenAI said it improved ChatGPT’s handling of “sensitive conversations,” which included advising users to contact crisis support and programming the chatbot to “avoid affirming ungrounded beliefs” and delusions. Claude, Anthropic’s chatbot, also presents users with a list of mental health resources if they mention suicide or self-harm.
History repeats itself: Teens have historically been the canary in the coal mine when looking at tech-driven mental health crises. Nearly a third of young adults already turn to chatbots for mental health support, putting AI companies in the same spotlight that social media platforms found themselves in during the early 2020s, when Instagram and others faced scrutiny over links to teen depression and self-harm. A Los Angeles court also recently found Meta and YouTube (the latter owned by Google’s parent company) liable for social media addiction that led to anxiety and depression, a precedent AI companies can’t afford to ignore.
Believing the hype: Though a new study found the percentage of adults who think AI can positively contribute to healthcare has declined, more than half reported that they still turn to bots before docs when making important decisions about their health. With AI chatbots filling holes in our splintered healthcare system, tech companies are aware of the need to play defense. —AC
Tech news that makes sense of your fast-moving world.
Tech Brew breaks down the biggest tech news, emerging innovations, workplace tools, and cultural trends so you can understand what’s new and why it matters.