When Crisis Meets Code: Navigating AI and Suicide Prevention

Richard Zwicky: Hi everyone, it's Richard Zwicky with High on Healthy, the show where we know there's no one path to healthy so we explore them all. If you haven't subscribed yet, please do so on your favorite podcast listening tool. So today we're going to talk about what happens when someone in crisis talks to a machine. AI chat bots, suicide prevention, crisis intervention, all these elements overlap and people are turning to tools rather than people. Sometimes because the access is hard to get, sometimes because they don't know how to access it, sometimes because they want the immediiacy and don't get it through the system that we normally channel through. And all these paths all come to one place. And to discuss what happens and where we go and how we can move forward, we have Christine Mutier joining us who is a psychiatrist and the chief medical officer of the American Foundation for Suicide Welcome board and healthy Christine.

Chris Yu Mout: Thanks for having me Richard, and excited about the topic we're discussing today.

Richard Zwicky: It's a powerful topic and I was having a conversation with an individual who will be a guest on the show sometime in the near future. You may or may not know Mark Kaplan who was the head of the lessons you do from UCLA and he's actually on my advisory board for my company and he's very passionate about it as I'm sure you are and any of us are about self harm. The role of chat bots, has been increasing daily. We turn to Claude, we turn to chat GBT, we turn to these tools for all of our advice. And in some areas, it's phenomenal and fantastic. And the tools are amazing when we ask them to process real data that's tangible and intelligible. But when we're dealing with emotions, it's a different story. They don't know they do learn about our tendencies but they don't know How do you approach first off how do you approach people who are turning to those tools in practice because obviously it comes up as a discussion point and we need to get clarity

Chris Yu Mout: Thanks for framing it up like that. And

Richard Zwicky: Thank you.

Chris Yu Mout: And actually Mark Kaplan was one of AFSP's funded researchers back in the day. So, he's definitely a wonderful expert. At the American Foundation for Suicide Prevention, we're really thinking about how to leverage our strengths and our platform to do some good in this space. And obviously, this is a massive in a way societal secular force that could be for bad, and probably both in terms of suicide risk and prevention. And so one way to kind of make it a little more manageable is to think about two broad categories. One are the AI LLMs that are specifically built with mental health crisis response and suicide prevention in mind. And those are in an important space. And even there'd be a lot to say. Where we're devoting more of our energy is in the broad use platform space. Thinking about the big five or seven companies and those platforms that are really highly utilized by everyday people most days and thinking about the different age groups and how people are using these broad use LLMs and where suicide risk can be detected. And obviously that's one juncture point to consider when suicidal ideiation is actually raised and the person brings up suicidal thoughts or when they're detected. But another way that we think about suicide prevention is going way upstream from the moment of acute suicidal thoughts and certainly way upstream of crisis because even suicidal thoughts can be looked at in some ways so common amongst the general population that they don't necessarily represent a crisis moment. That may sound strange to some, but people can have fleeting thoughts of ending their life as a way of their brain's solving method, thinking about various ways of coping with challenging circumstances, physical, psychological pain. So we're focused on those broader use LLMs. We're thinking about both risk detection and what should be done in those moments and also we're thinking way upstream in terms of how to tilt a person's risk far before they get to the point of even potentially being suicidal

00:05:00

Chris Yu Mout: Because there is research that speaks to all of the above as potential opportunities for suicide prevention.

Richard Zwicky: And there's much ground to cover there as we were chatting because's the aspect of are there enough markers that are intelligible through a system to pick up on that perhaps we don't in our daily lives though how to discern and differentiate between those fleeting thoughts that commonly people do have because your mind runs through every single option at different scenarios to which ones actually stand out and need to be paid attention to and then what to do about it and I think even before we get to that it becomes the challenge becomes and you touched upon this there's AI systems which are designed specifically around mental health. But all the guard rails, as much as we try, aren't always in place to protect us from the system hallucinating part of a solution or from the wrong information being presented at the wrong time. It doesn't mean it's bad information. It just means it's not the appropriate at that moment. I think stepping back where when you are dealing with people and when you're advising others how to deal with people where's that crossover line between what's healthy for people to turn for AI and when does it become unhealthy?

Chris Yu Mout: That's a great way to think about it. I think about the way many people are using AI and it's questions that are coming into their mind fromformational data and something for school or work to life and we're living quite holistically especially after the pandemic if anyone who thought they were sort of at their desk versus that we're bringing our whole person into all the settings and so people are, I think, presenting scenarios in their life, relationship issues, an argument that you just had, something you're trying to improve. And people are asking AI for advice along all those lines. And I think where it reaches a point for me that I would say here's a moment in time that we would want individuals and parents of youth and others who are involved with each other's lives to be aware of is my bias is that we are underrecognizing and underappreciating the moments when suicide risk is emerging and becoming Part of that is related to ma. Part of that is because we're not that great even at identifying when a stressful situation that's producing changes in sleep and stress and mood and emotions and thought patterns. When that becomes a clinically addressable scenario that we might call, depression or an anxiety disorder or PTSD and out of sort of normal healthy processing, but now it's in the range of needing professional and it's not to say that all of the supports and self-care and healthy strategies and healthy relationships can't help in they certainly can. But where a missing piece could be clinical treatment or a professional mental health individual or in the case of suicide prevention a trained counselor. So greatly I would wish that individuals would be thinking, okay, this is a moment when I really could take the call to 988 or in their chat function where you're going to get a trained counselor that this is what they do. They are dedicated to listening and understanding and trying to help people move from a hot moment to a cooler calm and then to follow up in terms of resources from there where they can continue on their path with a greater likelihood of reducing those risk factors, enhancing protective factors. So I think if you think about it that way in all the education we do at AFSP with gen, everyday people in a program called Talk Saves Lives, for example, we're teaching everyday people to be listening for those moments when the tone is hinting at feeling overwhelmed, they might feel like they're a burden to others, they might feel trapped in a situation. And those indicators on top of mental health conditions or changes, struggles, life circumstances where there's been a rejection or a legal issue or a financial crisis. That convergence of, shift in tone or if you're living with somebody, you're noticing sleep patterns and behavior change from their usual patterns of behavior.

00:10:00

Chris Yu Mout: All of those would signal this is a moment in time when there's a likelihood that the person's suicide risk is starting to emerge or come to a head. And again, so we would not have recommended that you have that kind of conversation with a nonhuman bot. And now of course there are some that are being designed specifically and that there's a lot of hope and opportunity there. But part of it for me is just helping everyday people understand which resources are actually a match and are safe for the scenario that they're in. I know that's obviously very ideal. People are going to continue to have these conversations and

Richard Zwicky: People are going to continue having those conversations.

Chris Yu Mout: We're seeing Right. Yeah.

Richard Zwicky: And the challenge is too big to say we have to prevent people from having these conversations with the bots and with the tools. The challenge is how do we get the tools to the place where they really do no harm that they have the appropriate guard rails in place to detect and to respond differently to individuals at different states. My work and just for full disclosure my work includes I've written and just been granted a patent which allows the system to understand the emotional state of an individual at any time to be able to look for not just any number of markers and adapt content to meet the needs of the individual. That helps, but the content still has to be constructed and the guardrails still have to be constructed to know when the person's in this state, this is what's to do, how to appro act appropriately. And all of that still requires the guidance and the education of the individual to know what to do or how to act. When we think of it from that perspective, how would somebody who is experienced suicidal ideiation interact differently with AI compared to somebody who is emotionally stable.

Chris Yu Mout: That's a good way to think of it as well. What research shows is that there are some common neurocognitive processes that tend to happen as heterogeneous as the paths to get there are that once a person is in that state of more acute or high level of suicidal ideiation and intent that there does tend to be a form of cognitive constriction and sort of a narrowing of in terms of the coping strategies that the frontal cortex can be filtering through that tends to narrow down at an earlier point in time for people who have a history of suicidal behavior than those who don't. So that some of the actual kind of baseline neurohysiology may already be a little bit different and then stress can activate at an earlier point and a more severe form of that sort of tunnel vision and cognitive constriction. So that a person may be experiencing that, but they probably usually don't serve it up with the language that might allow a platform an LLM or even a person to understand what it is that they're experiencing. They may use metaphorical language. They may talk about going home. Which is from one of the cases of tragic cases of suicide after interacting with a platform where the platform didn't seem to recognize that metaphorical language that the individual was using. So I think there are some differences in brain functioning and even this is pretty obvious, but really negative cognitive distortions that go on that are common among everyday people without mental health.

00:15:00

Chris Yu Mout: But when one is in that suicidal mind state then the ambivalence at play the desire for death as a solution to pain versus that survival in instinct and strong urge towards Hope is a very kind of universal and critically important experience we have. So that ambivalence.

Richard Zwicky: It's the power. It's the power which drives us forward and through.

Chris Yu Mout: So I mean I think the relevance here is that's fantastic that you're working on something that's very customized to an individual's state because suicide risk is highly dynamic. And so again the opportunity here is that there could be some basic things like mindfulness techniques, cognitive behavioral techniques that we see some of these being built into K12 curriculum for example. This is a universal strategy to try to build in healthier coping and sort of self-efficacy and management skills even among children. Emotion regulation and behavior regulation and so forth. And some of that could actually be infused into an LLM as a way to be sort of like a universal strategy towards suicide prevention. Perhaps as radical as that may sound, but definitely some additional training around vignettes that would be different ways that a person might indicate suicidal thinking or suicide risk that would be really useful I think to build in some of that training for LLMs. And my understanding is that there is a very old version of a sort of clinical tool called the Siri 2 that was developed in the late 80s or 90s that was intended at the time to be sort of a learning tool or a test for clinicians to see if they have the quote unquote right responses to a patient presenting in some kind of suicidal distress. That tool is actually very outdated because the science around suicide risk understanding risk and prevention over the last 40 years has progressed a lot but we didn't necessarily use those updated scientific tools and information to build the safety standards into LLMs.

Richard Zwicky: We almost started fresh and disregarded what had come before as a foundation basis upon Which also brings us to, one of the concerns when we have these conversations, we look at the world it is today and how it has been, but not where it's going. And so we talk about the fact that the LLMs can't distinguish literal intent from metaphor sarcasm or anything else and thus the AI responses will come back and may be viewed through the lens of hopelessness, shame and rejection. Part of that is because we're typing into the systems and we typing not as we speak but we're typing as a construct where and most people are not as fluent right written as verbally and also intonation is not carried forward. But we look out a 6 months, 2 years most of the systems become truly voice interactive which means that intonation becomes a critical factor. What do we need to do now to prepare for that in the future as we're designing the systems?

Chris Yu Mout: That's fantastic that you raised that there is a sub field of suicide prevention science that focuses on natural language processing and really looking for patterns that tend to cluster together and changes that a person may then exhibit wi when those brain changes and perceptual changes are happening moving closer towards suicide risk. So that kind of science could absolutely be applied here. I think some basic training certainly to even be able to detect tonal quality changes in inflection and tone of voice even if the words are the same. That would be something of great interest. Again, that's something that we're trying to do in our general education around suicide prevention with the public.

Richard Zwicky: And going down that path, we just saw the court cases where Meta and Google were fined for designing their systems to be Meta especially, right, with everything they've done. And we can learn from that past. But the economic drivers that push the AI companies forward are such that they're looking to become emotionally seductive to users.

00:20:00

Richard Zwicky: And the data shows half of all adults don't have a true close friend they can turn to and openly discuss their crisis. So, how do we avoid or how do we help people avoid creating that dangerous dependency when they don't have a viable alternative that they recognize close at hand?

Chris Yu Mout: I think you're raising something that is one of the most actionable things that we can do today in all the instances the dozen or so more that are known and reported on where there were tragic losses to suicide. Side after an interaction with an What we see in common is that they're in the tens of thousands of interactive dialogue. They are enormously long in high frequency high use kinds of dialogue that separate I think from the everyday average use of LLM. And so the dependency piece I think is huge not only from the standpoint of the natural exclusion then of interacting with other live humans and relationships in a person's life but that the seemed like the techniques that came up as patterns in those dialogues which some of the platforms may have really been doing some good work around I think this is not,

Richard Zwicky: I'm a strong believer in AI as a means to

Chris Yu Mout: It's not new information.

Richard Zwicky: Which support people. It's just where do we cross the lines?

Chris Yu Mout: And so I think if you're noticing that your family member or you yourself are experiencing this really positive, flattering kind of sycopantic feedback a lot. That is something to question and to really view with some skepticism. I think when you're there should be some never events related to AI interactions and those would include providing detailed information about lethal means for suicide even upon request for a school project or whatnot that those should not happen. The detailing out lethal methods of suicide is already known as one of the most dangerous pieces to avoid in general journalism and entertainment content. And certainly the same thing would apply here where it's a one-on-one dialogue. I think the other thing that comes to mind is when the bot encouraged the person to secrecy to keep this information sacred between the two of them and to not talk with parents or teachers or others in the person's life that is an obvious sign of a danger signal. And I think again the challenge is that for an everyday person who's using a chatbot this might be incredibly obvious but for the individuals who are at risk to begin with and may be more prone to these intense really becomes a relationship what they would experience as a very intimate important relationship if not addictive one in their life. And when I've used the term synthetic emotional intimacy, I've even encountered from people with this kind of lived experience that is kind of an offensive term to call it synthetic. And I would stand by that because it's not another human being on the end of that even if it feels very close and important and intimate.

Richard Zwicky: It's critically important because even people who are well educated and who commonly use and understand this as a tool, they get seduced by the, synthetic empathy, And even I see it in people who have full lives, who have a good circle of friends, they still turn to it and believe that when it tells them this is, something is that they've done something great, that they're thinking further down than other people. They want to believe it because it reinforces their sense of self. So, we do fall into that trap.

00:25:00

Richard Zwicky: And that's part of where the AI systems, the way they're designed, go back towards that problem of making themselves emotionally seductive. But if we take a step back, does it speak to a modern loneliness which didn't exist before and is something that's changed in our communities and societies over the last few decades and a different set of emotional needs? Or was this already and always there? We just didn't have the channel through which to explore it that AI has opened for us.

Chris Yu Mout: The way I think about this issue of loneliness and the role that AI could be playing at this moment in time, I believe that scientifically we as human beings are incredibly hardwired for connection. That's not something new at least in the modern era of human existence. Now culture change kind of trends may shift the expectations that young people that middle-aged adults the behavior at work. I think what we've seen even on social media are these viral trends towards authenticity and wanting to experience others in a more deeply and authentic way rather than the image driven and the issues that are a little bit superficial that will lead people to always feel inferior and play that comparison game. So I think there have been sort of societal trends that might shape the appearance of that very deep-seated innate need for human connection. And obviously there's a bell curve that's for humans that some people may be on a farther end where they may spend much more time alone with others but be very satisfied with something that may feel less connected than for others.

Richard Zwicky: And some people can feel completely alone in a room full of people.

Chris Yu Mout: So it's not to say that everyone is wired exactly the same way. And that I think is a very common phenomenon actually whereby because of internal states and thought processes and sense of identity and perception, an individual can in some cases feel worse and more alienated and alone when they're sensing that the people around them are in on something together and they are being otherred or they feel alienated or on the fringe. And I will say I think that's a almost universal human experience to go through at certain points in our lives. But anything that is going to accentuate that experience is not going to be good for mental health and suicide risk because that sense of being otherred especially if it becomes chronic on top of other suicide risk factors can be very problematic experiences like bullying and harassment, discrimination. I mean we look at the different populations who have higher suicide rates and not all of them but many of them are instances where the experience of that identity whether it's LGBTQ certainly other marginalized racial ethnic or rural populations that might be a bit of a different scenario but there are populations with different suicide rates and those relate to a host of things, but cultural trends can be a major one and whether it's expected to never need help and to have that be, the sort of I think very false normative message to think that we don't need each other, that our mental health if we need help, it's a sign of weakness. Those are very false and kind of old-fashioned views. But yeah, so suicide risk obviously is very complex and when you put that into this mix of loneliness and the way that our current society is shaped around human connection and now you add the availability of chat bots all very complex and I do agree with you. I think there's tremendous hope for and will and intent on the part of many different stakeholders and companies and people who are involved to actually have a continuous learning and improvement around this

00:30:00

Richard Zwicky: I think there is as well and I think there is a generalized fear because of what's happened in the last decade and a half two decades in technology that is primarily a tool of misinformation or privacy and it is access to the information that was always there. It's just framed and using natural language. So it's communicating back we're communicating to it in terms where we don't need the programming skills to run the analysis and it's responding to us in the terms we can comprehend. The information is always there. As how it's being framed is different. But people do underestimate I think the psychological and behavioral risks of improperly approaching that. And something you touched upon a minute ago to a theme that I re recurrently come back to on the show and I actually, write about is I'm the only normal person in my universe. And I actually just finished drafting a book on that. Which it basically is relational intelligence, We're all the centerpoint of that normaly and we're trying to understand everybody around us but that loneliness is something we can create for ourselves unwittingly. And when people have to wait 10 months, which is common, in the UK right now it's 22 months through the public system to get to a therapist and they're in crisis. And here in where I live, it's normal where if a young adult or teen goes to one of the suicide prevention societies and ask for help, they have a wait time and it can be 10 months. But if you're suicidal, you don't have 10 months to wait. You need somebody to help you right now. The technology that's there, how can it assist that process, right? Something tangible we can do today. Yeah.

Chris Yu Mout: I will just point out that 988 as well as the crisis text line are national resources that should be available 24/7 and you should not have to wait. Maybe minutes but not weeks to months. I think look there's research that shows the powerful positive impact that 988 is having and that the crisis text line is having. There's recent research that just came out a week or two ago that looked at trends in youth suicide rates by state by county and looked at utilization rates for 988 in those very same places and found a strong association. So much so that the researchers came to the conclusion that the launch of 988 has actually had a positive impact on reducing suicide rates among young people. Again linked with usage of 988. And having been involved with the crisis text line, I can tell you that their quality of the service they're providing is also very high. So, if I could wave a magic wand, my fantasy would be that the dialogue and the techniques that are proving effective in those crisis interactions that there could be a way to train AI on some similar modalities of kind of nuance and active listening, but again, never moving somebody towards a higher level of their own suicide risk, which can be done in a number of ways, including the provision of instructions on lethal means.

Richard Zwicky: Totally agree and I look at AI as the tool that complements not replaces because it's one thing as you deal with a therapist is they ask you to do work between sessions and it's hard for a lot of people to force themselves to sit down and do that work, but when they have an AI based tool, it can help them through that. We are out of time for the show today, Christine, but I'd like to thank you for the work you're doing with the, Foundation for Suicide Prevention and your dedication to breaking the problem, to helping overcome that crisis. If you did have one thought that you wanted listeners to leave here with regards to

00:35:00

Richard Zwicky: How to help somebody in need around them, what would it be?

Chris Yu Mout: I think it would start with having a sense of curiosity and caring non-judgmental support. The idea is if you detect that someone may be struggling in your life that opening up space for really our job in that moment is to do active listening and to invite the person to start sharing more in a way that just feels safe and not judged. And that can open up opportunities for that individual to take next steps. And you don't have to be their therapist or their doctor. Even if you are one, you're not acting in that mode with your own, loved ones and neighbors and co-workers. So I think in not shying away assuming that you're not the one to reach out, but you don't have to again have all the answers. Your job isn't to fix the problems. It's just to learn and listen and care without judgment and certainly if they're open to taking some next steps, you might be able to be helpful in those regards. But all of that is really icing on the cake. It's just about listening and caring and of course then that really starts with a broader effort in our lives to have a recognition that we're having potentially more influence about the psychological safety in the environments that we're in than we may realize. And so I might just start with that in terms of don't underestimate the power that you have in other people's lives and in shaping, how they're handling their own mental health or suicide risk.

Richard Zwicky: That's powerful. Thank you for that, Christine. And thanks for Stay on High on Healthy. And thanks to everybody for listening.

Chris Yu Mout: Thank you, Richard, for a great conversation.

Richard Zwicky: I'm Richard Zwicky, and we'll be back again with you soon.