I have worked in the treatment of mental health and substance use disorders (SUDs) for nearly 45 years as both a clinician and an organization executive. Today, I lead a large, full-continuum of care substance use disorder and mental health treatment organization known as EOSIS with services throughout the state of Minnesota.
We treat over 8,000 patients per year and “know” we provide quality care that benefits many, if not most, of them. But the nagging questions that continuously evade answers relate to a lack of quantitative objective outcome measures.
Our challenge is not unique to EOSIS; it is a pervasive obstacle in payer discussions, public policy debates and marketing conversations for providers. How to justify and differentiate the cost of care without objective measures of success is difficult, leaving patients looking for the best options vulnerable to best-funded advertising rather than quality measures or outcome statistics.
My intention in writing this piece is to push us, as providers and others, to begin determining who and how we will universally define positive clinical outcomes.
Mothers, indigenous culture healers and herbalists have historically known about remedies significantly before science could explain or quantify the impact. Their wisdom often came from experience with the impact of various interventions on symptoms and refining their protocols over generations. Non-medication aspects of mental health and SUD care have similarly developed through years of therapist theories and experiences in combination with collective experiences of patients as utilized through groups like Alcoholics Anonymous and other self-help models.
In other areas of health care, Centers of Excellence like the Mayo Clinic or Cleveland Clinic do not need clever advertising. Their statistically significant outcome studies, published in refereed scientific journals, tell patients and their caregivers that the best chance of achieving a desired clinical outcome is by trusting them with one’s care.
Defining outcomes
There are obvious obstacles to solving this often talked about challenge in behavioral health care.
The first obstacle is defining a positive clinical outcome in objective, measurable terms that can be used across settings and have a meaningful predictive qualitative value. Two distinct strategies are utilized in these discussions; the first is measurable symptom reduction such as reduced suicidal ideation; improved measures of affect or reduced experiences of hallucinations. The second are broader measures of daily functioning, such as, the ability to function in a job or school, maintain healthy interpersonal relationships, or even the ability to maintain stable housing and food security.
The most common focus of today’s outcome discussions in behavioral health care is on anecdotal stories about patient successes, often told through their own words. It is valuable to hear about a patient’s historic struggles, their experience of treatment and their subsequent improved quality of life. Particularly when the story has elements that resonate with future patients’ experiences, these efforts can be very useful.
But in fairness, every infomercial on TV starts with testimonials from people who have had success using the advertised product, despite the fact that most of the products fall short of the advertised effectiveness for the general population.
Since people are different and their diseases exhibit meaningful variation, knowing that some people benefited from a care model does little to accurately predict any individual’s likely success with the same treatment.
Any college class in introductory statistics teaches the requirements of showing statistical significance in measures of impact before predicting the result for others. Defining the patient population with specificity, ensuring that the sample analyzed is of sufficient size to show significance and controlling interfering variables that may distort the analysis, all help to define the difference between scientific study and speculative claims.
Another outcome measure of particular interest to payers is medical care utilization post-treatment compared with pre-treatment. The hypothesis being that patients post-mental health or SUD treatment should utilize fewer medical services than they did pre-treatment.
A confounding variable for SUD patients and others is that there is frequent deferred health care when people are active in their addiction and better compliance with care needs in recovery often leads to a potential paradoxical result. Additionally, there are downstream medical consequences of substance use that may surface in recovery.
Finding the right tools
The second obstacle in solving the outcome challenge in behavioral health care, after defining a positive clinical outcome, is identifying validated tools that can reliably measure the defined outcome terms.
Validating tools is a non-trivial task in itself, and therefore, we often rely on available tools that may or may not have direct applicability. The available tools are in fact sometimes used to define positive outcomes instead of working the problem in the reverse order. We have all used the PHQ-9 and its related scales to measure depression, in part because it is easy to administer, universally recognized and has the appearance of measuring elements that we believe closely relate to depression.
As useful as the PHQ-9 and other tools are for the above reasons, they have challenges. Its effectiveness assumes that a patient’s defense mechanisms won’t interfere with accurate self-reporting. So patients with a pervasive denial defense mechanism, common in SUD, may minimize their self-reporting, and patients with certain characterological disorders may inflate their reported symptoms.
The PHQ-9 provides a valuable starting place but has inherent limitations. This challenge is not unique to the PHQ-9 and is used only as an example here Most of the commonly referenced assessment tools in behavioral health have similar limitations.
A recent development in the behavioral health field is the launching of technology products designed to give organizations an infrastructure for the collection of “outcome data” and embed those tools in the daily workflow of clinicians. Some of these products connect to more robust electronic medical record systems to create a seamless collection of the desired data and aggregation for analysis and reporting purposes.
A potential challenge of these products is that they offer a myriad of tools for measurement, allowing providers to show patterns of change in scores but there are limited established definitions of positive outcome through which to indicate the importance of the changes shown. While measures of reduced depression, anxiety and suicide ideation/planning have inherent value in showing improvement, they reflect a point in time and they may not have effectiveness in predicting longer term life functioning.
My intention in writing this piece is to push us, as providers and others, to begin determining who and how we will universally define positive clinical outcomes. Despite the challenges and a level of ambivalence within the provider community, I do not believe that the market will continue to tolerate soft testimonial indications of program efficacy alone.
Some questions that may spark discussion include:
– Are definitions of outcome a natural extension of care standards like ASAM fourth edition and should similar task forces be established to develop definitions of outcome? Is there a better alternative path?
– Since the FDA approval process for medications already requires efficacy and safety studies, will the importance of medications in mental health and SUD care overshadow counseling and talk therapies if the world shifts to requiring scientifically validated clinical outcomes in mental health and SUD care?
– Will programs that adopt technology now to provide various validated tools for measuring change in symptomology have an advantage as standards for clinical outcome are established or will they need to significantly change their protocols to meet the next level of needs?
– How will programs that survive on razor-thin margins today continue to add requirements like outcome measures to their daily work? Can we find models of measurement that do not disadvantage smaller treatment providers?
– Value-based care has been slow to be adopted, in part because the criteria for defining value in mental health and SUD care remain vague and tangential to true measures of clinical outcome. What will be the role of providers and payers in defining the criteria going forward?
– In other aspects of health care, there are patient profiles that get denied organ transplants or are guided toward hospice care rather than treatment. From an outcomes perspective, are there mental health and SUD patients for whom treatment may be appropriately denied?
These are many of the questions that I think about as the CEO of EOSIS.
I don’t think there are easy answers, but I hope by writing this that I help keep the conversation going. I look forward to discussing them during my future roundtable conversation that’s part of this BHB Executive Forum piece.
Want to contribute to the BHB Executive Forum? Submit your op-ed idea here.
About the author: Lew Zeidner is the CEO of EOSIS, a company that provides evidence-based treatment, recovery support and integrated services for individuals with substance use disorders across a network of care facilities.
\