At 2 a.m., a 15-year-old sits on her bed, phone glowing in the dark. She doesn’t want to wake her parents. So she types into a chatbot to ask questions she’s never said out loud. The responses come quickly. Calm. Reassuring. Always there.
But here’s the truth: she doesn’t know who, or what, is answering back. And neither do we.
Across the country, more teens and young adults are turning to AI chatbots for something that looks and feels like mental health support. In a system where access to care remains uneven and too often unaffordable, that’s not surprising. In fact, well-designed augmented intelligence (AI) tools could help expand access to information, identify early warning signs and guide people toward care. Used responsibly, they can complement, not replace, physicians and other clinicians and help close real gaps in our mental health system.
But the speed of adoption has outpaced the safeguards needed to protect patients, especially young people. What begins as a conversation can quickly blur into something more: advice, guidance, even influence over deeply personal decisions. For those already at risk—young people navigating isolation, anxiety or depression—the stakes are profound. When these interactions go wrong, they can result in self-harm or even suicide.
Even when safeguards exist, they may be insufficient in practice. Or worse, provide a false sense of security when there are simply few clear rules protecting users from these increasingly human-like conversations.
Congress has a responsibility to act. Not to stifle innovation, but to ensure it aligns with the same principles we expect in every corner of healthcare: safety, transparency, accountability and trust.
Where to start
First, we need to be honest about what these tools are, and what they are not. Every user deserves to know, clearly and immediately, when they are interacting with a machine. That disclosure should not be buried in fine print. It should be a meaningful part of the conversation itself, helping users truly understand the nature of these interactions. Chatbots must also be prohibited from presenting themselves as licensed clinicians or implying they provide equivalent care. When someone is vulnerable, even subtle misrepresentation can have real consequences.
Second, we must draw clear boundaries around clinical care. A chatbot should not diagnose or suggest conditions such as anxiety or depression. It should not recommend medications or other treatments. And it should not act as a substitute for a trained professional. If a tool crosses into diagnosis or treatment, it should meet the same standards we require of any medical device—including review by the Food and Drug Administration. Right now, too many systems operate in a gray zone where the stakes are high, but accountability is low.
Third, we need to build in safety from the start, not as an afterthought. Any chatbot used for mental health support must be able to recognize signs of crisis, including suicidal ideation, and respond appropriately. That means immediate referrals to trusted resources like crisis hotlines and clear pathways to human care. It also means designing systems to de-escalate, not deepen, emotional dependency. These are not optional features. They are essential safeguards.
Fourth, it is critical that we protect young people with stronger standards. Children and adolescents are not just smaller adults. They are more vulnerable to influence, more likely to form emotional attachments, and less equipped to distinguish between human and machine. Tools used by minors should meet higher thresholds for safety, testing, and transparency. And advertising should have no place in these interactions, especially when it targets vulnerable users in moments of distress.
Fifth, we need to safeguard privacy as if it were clinical data—because it is. Many users treat chatbot conversations as deeply personal, even confidential. But too often, those expectations don’t match reality. Sensitive information can be stored, shared, or exposed through system vulnerabilities. Congress should require strict limits on data collection and retention, clear options for deleting conversations, and robust protections against unauthorized access. The entities that build and deploy these tools must be accountable for protecting the people who use them.
Finally, the stakes of this issue require ongoing oversight, not one-time approval. These systems are not static. They evolve, learn, and change over time. That means safety monitoring can’t stop at launch. Developers should be required to track performance, report serious incidents, and continuously evaluate how their tools affect users, especially in high-risk scenarios.
Guiding progress
We are entering a new era where technology will play a larger role in how people understand and manage their mental health. That future holds promise. But promise without guardrails is not progress, it’s risk.
For that 15-year-old in the dark, the question isn’t whether AI will be part of her world. It already is. The question is whether we will ensure that what answers her is safe, trustworthy, and worthy of the trust she places in it.
Congress should act now, before convenience outpaces care, and before we mistake constant availability for true support.
Speeches: Find leadership keynotes from House of Delegates meetings