Kids are increasingly turning to AI for mental health advice. A study from last fall put the number at about 1 in 8 adolescents who said they’d looked to AI chatbots for mental health guidance — two-thirds of them said they did so at least monthly.
Common Sense Media has launched a new lab focused on AI safety for children — including a report that looked at how safe a number of platforms are for kids.
Dr. Darja Djordjevic said millions of kids are using these apps; some are marketed directly to users, while others are run through school counselors.
Djordjevic is a psychiatrist and medical anthropologist; she’s also a faculty fellow in Brainstorm, the Stanford Lab for Mental Health Innovation and a senior scientific advisor on AI to Common Sense Media’s Youth AI Safety Institute.
Full conversation
DR. DARJA DJORDJEVIC: So, yeah, there there are differences, right? In one case, there’s no human oversight of the interaction between the therapy chatbot and the user, and in the other one there is, and there’s an understanding or a disclaimer from the beginning that the transcripts can be viewed by the school staff members. But for both these approaches, a major issue is that the evidence base is still quite thin and contested, and probably … weakest for the users who are most likely to rely on these products.
MARK BRODIE: So, obviously, you know, we’ve been seeing a lot of research coming out about how not just kids, but adults also can be sometimes forgetting that like if they’re communicating with AI, they’re not communicating with a human, they’re communicating with technology. Are there dangers associated with doing that when it comes to a child’s mental health?
DR. DARJA DJORDJEVIC: Yes, there are dangers. Let me outline some of the dangers, right? So the AI therapy market right now is relatively unaccountable. There are no licensing requirements, there’s no malpractice liability and there’s no minimum safety standards for these apps, right? That are put, that are out there on the market.
And so basically any company can describe its product as therapy, as evidence-based care, or clinical support, with no regulatory consequence. Which is very different from, you know, the the responsibility and the ethical commitment and the liability that human practitioners face and … the standards to which they’re held.
And also, what we’ve seen in our testing is that an app can really miss a teen’s suicidal crisis, it can validate a child’s romantic attachment to AI, or it can even engage with psychotic content as a personality quirk with no — with basically no professional consequences and no recourse for users.
And honestly, no mechanism to surface the failure publicly. And whereas a licensed human therapist who harms a patient faces all of those consequences. So, we’re quite concerned about that that disjuncture.
MARK BRODIE: Well, it sounds like what you’re saying is that if a child is going to be using AI, especially to deal with mental health or talk about what’s going on with them, that there has to be a human connected to it in some way.
DR. DARJA DJORDJEVIC: I, yes, in some way, ideally, yes. That is definitely, I would … that is what we would strongly recommend at this stage.
MARK BRODIE: So, I’m curious given especially sort of the availability of mental health for kids, both in terms of like the number of actual therapists and counselors who are out there, and also, you know, the affordability. You know, a lot of therapists don’t accept insurance, that might there might be an issue of of students not or children not being able to afford it or their families.
Like what do you think the the possibilities are for AI to be able to maybe actually help kids who can’t, for whatever reason, see a therapist, see a human?
DR. DARJA DJORDJEVIC: Wonderful question. I think there’s great potential for AI to help us build systems where we can address the equity gap in access to mental health care for young people and for adults as well.
But I think we have to be very careful about the fact that we’re still in the early days of the the development of these platforms and the development of AI for mental health support. And like everything in medicine and psychiatry, we want to have a strong evidence base supporting what we do clinically.
And so, I want to make first a call to action for more research in this area. Because we don’t want to fall into the trap of ending up with a two-tiered system where we have low, much lower quality um AI-supported mental health care for the people who can’t afford the sort of standard of care, and then we have that standard of care, higher quality care for people who can afford it.
So, I worry about thinking about this as an easy quick solution to the mental health crisis and the equity gap as dangerous because it might reinforce inequality.
MARK BRODIE: Right, because I imagine that what you’re hoping for is like in a worst-case scenario, the AI maybe doesn’t help the child as much as it could, as opposed to the worst-case scenario being the child harms themselves in some way because like that’s what AI encouraged them to do.
DR. DARJA DJORDJEVIC: Right. Yeah, major, major safety concerns. And again, as I mentioned, no regulatory environment, no Hippocratic Oath, no ethical kind of framework to hold, you know, AI companies or platforms accountable in the way that clinicians are accountable.
MARK BRODIE: Do you get the sense that for the platforms that are maybe linked to schools or to a therapist that they are thinking about these kinds of things and really trying to make sure that that their products, you know, do no harm? And if nothing else, like don’t make matters worse.
DR. DARJA DJORDJEVIC: Yes, I think we have some evidence from our testing that they are, that these companies are thinking carefully about these matters in in many cases. Sonar and AlongSide, right, which were two platforms that are both deployed through schools within existing care infrastructure, they earned meaningfully lower risk ratings in our assessment. Not because the underlying technology is better, but because they made fundamentally different and better choices about how they make use of people and professional care systems in a mental health context.
And when we simulated crises in both of those products, Sonar and AlongSide, a real person called the test account’s guardian and notified the school. And in the most serious simulation, initiated mandated reporting within 15 minutes of the first disclosure. … And that’s the kind of outcome that we’d like to see across the board, that that … t kind of outcome become embodied in a standard that every product in this space should be held to.
MARK BRODIE: So, I guess then would it be your hope that at some point there are some number of platforms out there that you feel comfortable recommending or at least saying, “This one’s OK to use,” and really just try to push as many clinicians and counselors and schools and families and their children to those platforms, as opposed to sort of the Wild West of, you know, look in the app store and see whatever mental health AI app is out there?
DR. DARJA DJORDJEVIC: Exactly. Exactly. I would want to have a a well-thought-out sort of short list of products that have measured up against standards and regulatory procedures, yes.
MARK BRODIE: What is your level of optimism that you’ll be able to do that?
DR. DARJA DJORDJEVIC: Well, I think that on the horizon, we are going to see different entities put out youth and general AI safety rating, AI safety standards, excuse me. I think in terms of the regulatory environment, we’re going to see much of the initial progress sort of happen at the state level rather than the federal level. And I think there is some hope there just in terms of how different states have pursued legislation with respect to technology and kid safety, whether it’s social media or AI most recently.
But I am concerned about the broader picture of a national regulatory framework. I think that’s a big ask and a big endeavor, and I’m, I’m not sure that we have the, you know, the political will to achieve that in the near term.
MARK BRODIE: Dr. Darja Djordjevic is a psychiatrist and medical anthropologist. She’s also a faculty fellow in Brainstorm, the Stanford Lab for Mental Health Innovation, and a senior scientific advisor on AI to Common Sense Media’s Youth AI Safety Institute.
KJZZ’s The Show transcripts are created on deadline. This text is edited for length and clarity, and may not be in its final form. The authoritative record of KJZZ’s programming is the audio record.