On therapist quality

Coming from a quant career, and having seen tech industry up close, it’s jarring to me how severe the problems of measuring talent and meritocracy are in the therapy world, with many implications for its workings.

Admittedly the therapist quality problem is just hard¹, I can’t on the spot name a single other occupation suffering from a comparable level of obscurity in both the quality of the inputs and of the outputs. In many professions the labor inputs are much more easily assessed, typically being some combination of IQ and a mastery of some specific knowledge base that is straightforwardly testable. In others the inputs might be more obscure, but at least the quality of the “product” is clear: who knows what makes a writer, but her novel is there for the critics and the public to check out, and similarly for many other creative professionals, or say car salesmen. In customer service it becomes a bit more like therapy, except interactions are shorter and important aspects are more visible and can be monitored, and throughput is higher so one can try to get some insight from those customer satisfaction surveys.

As for the therapists.. IQ, skills and the amount of training matter to some degree ofc, and given that’s what’s available for the public to see that’s oft what it has to get by on, preferring MDd and PhD²s, folks from better schools, with more certifications. Yet few of us, on the analytic/psychodynamic side especially, think those matter that much. A typical comment I got from practitioners when researching my career transition and schooling options is “school doesn’t matter, you learn the craft mostly from practice and supervision”. Sounds right to me, except.. Does everyone learn? I bet not, I’ve seen enough “trying to find a good therapist” horror stories to believe plenty of bad apples are out there, experienced or not. How does the public know who learned and who didn’t? No answer.

As for the output, terminating a few patients a year, their starting points too hard to assess, their ending points hard to assess, who knows if their progress was up to you abilities, or fit, or their own efforts, or their issues turning out not being so complex. And even to the extent there are valuable quality signals in patients feedback, we aren’t really collecting that data and making it available. Still, referrals, by ex-patients or colleagues who know you, might be among the better ways for a patient seeking a doctor to find somebody good, at least if they are lucky to be able to start at a relatively reliable starting point of a good friend or trusted physician or somesuch, which many less mentally healthy won’t have the privilege of having. Unfortunately in the current cultural climate (and with a lot of mentally unhealthy very disconnected from the wider society) referrals are less prominent, leaving public to look for “formal signs” or trying to read the tealeaves of therapist websites trying to figure out whom they can trust.

If this was the end of the story, “the problem is unsolvable”, there won’t be much we can do and I won’t be writing this post. I don’t think this is quite the case however. Luhrmann³ writes that everyone in her psychiatry residency cohort training in psychodynamic therapy knew who the best therapists in the cohort were. I hear similar remarks from my friends in various (better) training programs: they (claim to) know who’s good and who’s not, would happily refer to the best and don’t think the worst should be in a profession. Long-term supervisors are also highly likely to have a good idea re talents of each supervisee.

Analytic training system still retains a little bit of flexibility to kick out folks who don’t seem to “have the right stuff” (typically to another program, not out of the field), and from what I glimpse from the historical records this used to be more common, having mostly given way to “if you were admitted and put in the hours you should graduate, never mind if you’re a sociopath or a narcissist” modern attitude of only tolerating purely objective fully operationalized progression criteria. Ironic for a profession so attuned to subjectivity in which its sole claim to unique expertise lies, isn’t it.

This ofc relates to many of my favorite issues that I’ll be writing more about in the future: effective use of human judgment in society, balancing subjective and objective, right brain and left brain⁴, “10% more nepotism”. When it comes to the more narrow issue of therapist quality, the situation is something like: there probably is sufficient information to make a decently well-informed judgment about the quality of a given therapist, however it’s diffused in the minds of close colleagues, clients and supervisors and is mostly not stored, communicated or used, while the external/“objective”/”operationalized” correlates of quality are of much less value in this field than almost in any other profession.

As some of my readers might be less familiar with what a more meritocratic and “easier to measure quality” profession might look like, let me outline how things are in quant finance/tech for contrast. What makes things easier in those fields is that (I’m talking about the technical jobs) the work is very g-loaded, so all the usual signals of peak IQ are widely used and useful for hiring and selection. It’s only a mild exaggeration to say one can get hired simply by having graduated with a CS degree from Stanford, or having won a chess championship, or having published in Nature, or simply having done really well on her coding interviews.

One of my favorite examples is a very mental health troubled⁵ lady who didn’t go to college, ground online coding competitions and climbed pretty high in the rankings thanks to her IQ, and got hired by Google⁶. For an example from the opposite side, I had a misfortune at one point to work with a carelessly hired person who just wasn’t smart enough for the job, and was unable to solve issues beyond certain basic level of complexity independently, despite having a PhD⁷ and plenty of relevant theoretical knowledge. Me and our boss had to constantly babysit him and monitor the quality of his work. The difference with the therapy world is that it was very clear very quickly to the bosses and the rest of the team that the mistake was made and that the person wasn’t up to snuff.

This sort of talent being relatively easy to infer and/or assess means that people get sorted into appropriate track relatively quickly, sometimes with their first job, sometimes after the first job hop. Everyone knows what the better firms and more desirable positions are, it’s reflected in both competitiveness and compensation, firms and even teams are pretty well-sorted on IQ. It also means there’s less need to pretend formal education is of that much relevance to being able to do the job, among quants for historical and ease of talent location reasons there are still a bunch of PhDs, but hiring has been increasingly shifting to masters students or even undergrads, even more so at best firms most clear-eyed about finding talent. Tech is full of people with non-CS or even non-STEM majors, and “break in without a college degree” stories while still rare did happen a fair bit, especially during the 2010s boom years. Everyone understands most learning is done by people themselves and not in school. While formal “stamps of approval” are valuable, it’s not that uncommon for one to make their own way in simply being smart and willing to learn on their own, with a bit of hustle and luck.

Hiring is also easy. Stanford CS or Google engineer are already decent guarantees of quality, add in glancing over resume and what projects one actually did and one can do fairly decently even with such blind hiring, add in a few technical interviews and you have a fair enough idea of what you’re getting, at least in terms of technical skills and thus capacity to do individual contributor early to mid career work. Softer skill would ultimately matter more and more in the long term and at higher levels of a career, but early on they aren’t of first order importance.

Compare to therapy. Of the professionals I talked to nobody thinks that 2yr masters program or whatever limited formal licensing requirements do that much to make one a good therapist or guarantee a standard of quality. Plenty of bad therapists out there, fully licensed. Enough narcissists and sociopaths and alcoholics and other quite troubled people successfully slip through the requirements. Plenty of people who could’ve gotten better with good supervision don’t get it. Whatever some therapists’ defensive grandiosity re “we’re so good coz we’re licensed" might be, a good chunk of the public doesn’t see a random therapist as necessarily better than a random coach, and some insurance companies might feel the same way⁸.

What implications does the fact that “easy” ways of assessing therapist quality don’t work/exist, and “hard” ways (like aggregating and publicizing opinions of colleagues/supervisors/patients) aren’t implemented, have?

The most obvious one is that the public/potential patients are harmed. Largely as a result of the above issues, finding a good therapist is hard. With a differential impact harming the most vulnerable the most. If you have a decently good judgment (and/or helpful advisors) to do proper research and shopping around, orienting oneself to the few signposts like better degrees that are available, trying a few providers for a few sessions and using one’s judgment to pick the most decent one, at least you have a shot. Yet I know enough of even very smart and high-functioning people getting turned off from therapy by a bad experience or a few. Imagine being a less high-functioning and more traumatized person finally summoning the courage (and finances) to try a therapist only to be turned off if not re-traumatized by invalidation, criticism, bad listening or them talking about themselves. Actually, no need to imagine, go to r/CPTSD subreddit or any other mental health support space and read the horror stories.

Us as a community withholding or not sufficiently spreading around the inside information we have about who’s good and who’s not harms the public, and harms the most vulnerable the most. Us spreading this information to some extent using “grapevine”/referral networks differentially advantages the more privileged and more mentally healthy and thus more socially well-connected, and also some social strata over others (jewish new yorker is probably a handshake away from a brilliant psychoanalyst, immigrant techie not so much).

A standard solution to “the lemons problem”, aka difficulties for prospective “buyers” to ascertain the quality of a “product” on offer, is branding. If a given firm/brand is in the market for a long term, they have incentives to maintain their reputation, and provided clients have at least some idea at least post-factum re whether they’ve gotten something good or not, the reputation does serve the function of accumulating even fuzzy and dispersed information over time. I’m thus pleased the industry organization has been shifting a little bit in that direction in recent decades or even years, with the appearance of larger networks of therapy clinics, larger group practices as well as more recently online therapy provision platforms.

Interestingly, for these reasons I’m somewhat pessimistic about the future of Betterhelp. From what I hear it offers dubious conditions/pay, thus attracting disproportionately lower-quality clinicians with fewer other options⁹. In the short-term, especially with their aggressive marketing, this might not be so obvious, but in the long-term, especially given widespread therapists’ animosity towards them and eagerness to tarnish their reputation, I feel chances are their karma will catch up to them and they’d have negative reputation and brand value. I hear some mental health startup play the opposite strategy, offer much better terms and “bet on quality”. I’m excited to see how this plays out in the coming years. If “brand as a solution for problems of communicating the quality of the offering” works in this industry¹⁰ that would be great for both the public and the therapists. In some sense this would be getting closer to the “analytic institutes as guilds ensuring quality control” system from 60yrs back, and trying to recreate at higher levels of abstraction the benefits of the grapevine/referrals system that’s become less dominant in the recent decades, and less functional and fitting for the digitized and atomized society. Maybe it’s time to say goodbye to our artisan/craftsman system increasingly unworkable for the current industry scale, complexity and the organization of society, and embrace the “industrialization” with its disruptions but also hopefully ultimately more scalable and efficient mechanisms.

Another consequence of the quality visibility issues discussed here is the lack of meritocracy. Unlike in tech or qfin, there are no “quick break-ins” or “meteoric rises”, few 3x+ differentials in compensation right out of school reflecting one’s abilities (from the same mid-tier CS major it’s not uncommon for some grads to get $80K starting compensation working for the state or $150K working for IBM or $250K working for a hedge fund). To the extent there are differentiated outcomes, one can’t help but wonder if they are driven by one being a better therapist, or more by one being better at marketing, networking, hustling to get “hot” certifications (like IFS) earlier than others. There’s nothing wrong with non-therapeutic skills and talents, and hustle and efforts, bringing one advantages ofc, but it is not good for everybody else and the industry if those are the only things giving one an edge given near-total obscurity of the core therapist competence. One hopes that later in the career one’s good work would get one some recognition, by building a reputation among colleagues and grateful clients more eager to spread the word. Yet one wonders to what extent even at that stage that beats the benefits of non-therapeutic hustle.

As entrants to the profession are somewhat aware of all this, and seeking financial security, they are more likely to go for additional degrees and certifications even if they don’t think they have much to do with improving their work. If that’s what public, lacking better options, looks for, then that’s what we’d have to get to protect ourselves. Lose-lose.

I think talent not being properly appreciated is tragic and unfair. And creates issues for society, as it spoils incentives for could’ve-been-great-therapists to pursue this path. Or, from the opposite side, by not filtering out bad apples. The latter, and the general un-differentiatedness of therapists, creates all sorts of social perception issues. If I buy a phone for $50 vs $200 vs $500 dollars I can be pretty sure of getting what I pay for, and willing to make the purchase in accordance with my desires and my budget. In therapy just about everyone charges $200/hr¹¹ but a potential patient has no idea whether they’d be getting something truly great or quite awful for that price. The signaling value of price is lost, the virtuous cycle of better product receiving better price is broken. More talented therapists, unable to communicate their higher value, end up underpaid, less talented therapists get more than they deserve.

1. Stay tuned for another post on this where I’ll discuss what Douglas Kirsner doesn’t get in his Unfree Associations book, and Isakower’s “analyzing instrument”

2. And unfortunately not trained analysts, despite analytic training being the training regime most likely to make one into a better therapist (tho ofc I’d say that given where I sit)

3. Luhrmann, “Of Two Minds”

4. I’m yet to read Iain McGilchrist’s The Matter with Things that I kinda expect to touch on a lot of these themes, though it’s also evident between the lines of his Master and his Emissary that I did read

5. Likely prodromal schizophrenia, or at least paranoid personality disorder

6. Due to her issues she unfortunately wasn’t able to keep the job for long, which one can view as an important narrative counterbalance to this post: it’s not that “fuzzy” character stuff doesn’t matter at all in those fields, and one can make a decent argument that hiring criteria there are too objective and operationalized.. It’s not like issues of judging character don’t matter there, but they are much less salient and are typically, such extreme examples aside, can be viewed as being of secondary importance at least for the early career professionals

7. From a second-tier school

8. I’m not saying this is the only factor there, short duration of treatment and financial reasons ofc play their role. But I am saying the factoid doesn’t make sense if you think, like some therapists do, that licensing is so important and licensed therapists are so great and coaching is all snake oil. Insurance companies won’t be offering this if their advisors didn’t think it could be helpful and if they didn’t think their clients would like it.

9. And young trainees, to whom this “negative selection on quality” argument applies to a lesser extent

10. An obvious concern here is to what extent patients for whom this might be the only therapy experience they ever had can tell if they’ve gotten something good or not, and to what extent “misses” like clients who leave at the height of their negative transference would mess things up with their negative reviews. That said, those are the issues with any online reviews aggregation systems, yet to the extent there is some useful information and “valuable signal” in patients’ reviews, it would show up on net. One can wonder what incentives or issues that might create down the road, for picking easier patients, for providing more supportive and pleasant than challenging and healing therapy, etc. We’ve kinda seen all those in medicine and elsewhere. Yet I feel first order this would be a big positive for the industry.

11. NYC rates here