hardly our finest hour…

Posted on Sunday 25 September 2016

When I left the faculty in the early 1980s in the wake of the medicalization of our department after the DSM-III revolution, I didn’t think of it as leaving psychiatry [I sort of thought of it as psychiatry leaving me]. But circumstances were such that I got busy with my practice and teaching, drifting further and further from what was going on in psychiatry local [Dr. Nemeroff’s department] and psychiatry at large. It was 25 years later when two things woke me from my slumber – the Washington Irving's Rip Van Winklerevelations of widespread corruption in academic psychiatry [the KOLs] and volunteering in a clinic and being horrified at the medication regimens I found people on there. So I had a lot of catching up to do, luckily finding others who were willing to help. I think that the CROs [Contract Research Organizations], the Medical Writing Firms, that whole industry that entered the clinical trial scene must have been in its infancy about the time I was going into seclusion, because I didn’t know about any of it, though part of my reason for leaving had to do with a new administration that was keen on teaming up with PHARMA [another unfamiliar  term]. I think of the time from going into practice until five or six years into my retirement as my "Rip Van Winkle" period:

I periodically tell that story partly because I feel guilty for not noticing what was happening, and sometimes to explain why I never even heard terms like evidence-based medicine, RCTs, meta-analysis, systematic reviews, or even the word pharma until five or six years ago [I’m apparently a heavy sleeper]. This time, however, I have another reason. Research watchdog, John Ioannidis, has a new article. And it was a graph in his paper that led to my retelling that snippet of my history:
Milbank Quarterly. 2016 94[3]:485-514.

POLICY POINTS: Currently, there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta-analyses. Instead of promoting evidence-based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools. Suboptimal systematic reviews and meta-analyses can be harmful given the major prestige and influence these types of studies have acquired. The publication of systematic reviews and meta-analyses should be realigned to remove biases and vested interests and to integrate them better with the primary production of evidence.
CONTEXT: Currently, most systematic reviews and meta-analyses are done retrospectively with fragmented published information. This article aims to explore the growth of published systematic reviews and meta-analyses and to estimate how often they are redundant, misleading, or serving conflicted interests.
METHODS: Data included information from PubMed surveys and from empirical evaluations of meta-analyses.
FINDINGS: Publication of systematic reviews and meta-analyses has increased rapidly. In the period January 1, 1986, to December 4, 2015, PubMed tags 266,782 items as "systematic reviews" and 58,611 as "meta-analyses." Annual publications between 1991 and 2014 increased 2,728% for systematic reviews and 2,635% for meta-analyses versus only 153% for all PubMed-indexed items. Currently, probably more systematic reviews of trials than new randomized trials are published annually. Most topics addressed by meta-analyses of randomized trials have overlapping, redundant meta-analyses; same-topic meta-analyses may exceed 20 sometimes… Many other meta-analyses have serious flaws. Of the remaining, most have weak or insufficient evidence to inform decision making. Few systematic reviews and meta-analyses are both non-misleading and useful.
CONCLUSIONS: The production of systematic reviews and meta-analyses has reached epidemic proportions. Possibly, the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, and/or conflicted.
The abscissa on the graph goes from 1986-2014 and the ordinate goes from 0-30,000 [articles/year!], red  being systematic reviews and blue being meta-analyses [I wonder what the clinical trial graph would looks like?]. Ioannidis obviously takes a dim view of this epidemic. He’s a smart guy, so I expect he knows what he’s talking about.

But when I looked at Ioannidis’ graph, I saw something else – a historical epoch of medicine. Is it any wonder that I didn’t know about meta-analyses and systematic reviews? There weren’t any at the time I went all Rip Van Winkle! That graph parallels a profound change in medicine the age of evidence-based medicine, the age of managed care, the age of corporatization, the age of the clinical trial. Whatever you want to call it, it has been a distinct era. And it is hardly our finest hour.

Again, I don’t miss Ioannidis’ point, that many of these meta-analyses and systematic reviews can be a way for academics to rack up publications for academic advancement  without doing any original bench research or clinical studies of their own. But it’s also possible that one of the reasons for that is that research funding is so hard to come by these days – except from corporate sponsors [with strings attached]. And another reason for the burst of secondary publications might be that there’s been so much questionable research[?] in this time frame that genuinely does need a critical second [or third] look.

So at least in psychiatry, I welcome the flurry of independent meta-analyses and systematic reviews. We’ve had a pipeline of psychotherapeutic agents steadily pouring into our landscape during the time under discussion, literally changing the direction of the specialty, and we still can’t trust our literature to tell us about, either their safety or their efficacy:
The meta-analyses and systematic reviews have been our only real window into any rational understanding of these drugs. And they still have a lot to tell us. Here’s a very recent example:
by Ymkje Anna de Vries, Annelieke M. Roest, Lian Beijers, Erick H. Turner, Peter de Jonge
European Neuropsychophaemacology. 2016 Article in press.

    Cate Mullen
    September 26, 2016 | 12:26 PM

    Thanks for this.
    My concern now that some folks have been awakened on the inside of the desk and mighty work has shed light on the abuse of the past decades what about the folks sitting on the outside of the desk
    Where are the folks willing to echo Scott’s great
    – to paraphrase Oh My God ,my people.
    His diary can be read at the British Museum
    and as I recall it is opened to that last page
    When things go wayside and damage is done to many,many folks-who will be another Scott?
    He failed and his people died but he took care of them the best he could and his last words were for them
    and damaged by the academic and pharma research technocrats?
    First do no harm was not only ignored but actively ride over in fast cars and first class seating to resorts and the beautiful spaces where the gilded age folks align themselves together far from the madding crowds
    You have spoken eloquently of your experience and awakening to this tragical state of affairs
    Who will do the Hippocratic right thing and address the entire patient abuse sphere? Better now then at judgement day
    Who will take up the cause of the folks who lives have been irreparably harmed

    Cate Mullen
    September 26, 2016 | 12:28 PM

    Sorry for the tops I am Lesrning Disabled even spell check can’t help at times
    Would have been an MD if not for that very irksome life disability
    Nea culpa

    September 26, 2016 | 11:11 PM

    What a chilling remark, that “journal articles are an unreliable source of data on serious adverse effects” So, if the efficacy of antidepressants are hugely overstated (94% of published articles being positive whereas FDA data had 51% of trials being positive) and the dangers of these pills are grossly under-reported (63% of articles mention NO SAE’s), then the entire empirical foundation for these pills is cracked, shattered, and ruined.

    I am glad my doctorate is not in psychiatry, I can only imagine the angst and despair among those who trained in this field, to discover that all the research articles they wish to rely on, are grossly distorted.

    James OBrien, M.D.
    September 26, 2016 | 11:57 PM

    It seems that Gresham’s law applies to psychiatric research.

    September 27, 2016 | 11:58 AM

    Gresham’s law needs to be modified for this instance, as there is no ‘good money’ here. How about “bad money drives out necessary data’?

    James OBrien, M.D.
    September 27, 2016 | 1:00 PM

    Bad data supported by monied interests, government or publishing empires drives out good data without an agenda.

    Academia-publishing is like the matrix…you can’t see it when you’re in it.

    September 27, 2016 | 4:42 PM

    In regards:”I can only imagine the angst and despair among those who trained in this field, to discover that all the research articles they wish to rely on, are grossly distorted.”

    My observations lead me to believe that almost no clinician trained in psychiatry experiences the angst and despair you are describing. I say this not necessarily as a good or a bad thing. Rather as something that I believe does not get adequately considered and understood. I would venture that in terms of what causes psychiatrists angst, in relation to THEIR OWN clinical practice (and “their own” is key), this would not break the top 3 (top 5)? This would include those who have most vociferously decried the problems with our literature. Psychiatrists by and large do not see THEIR OWN induvidual clinical practice as impoverished by the current state of the literature, even as it is viewed as a threat to the integrity of psychiatric practice as a whole.
    I don’t see the issues discussed here as invalidating everyone’s psychiatry practice. I do see the conduct of clinical trials (and their reporting) as significantly impoverishing everyone’s practice. Including my own. However, that is very much the minority view (especially among those who have extensive clinical experience). I continue to feel that the belief that the state of the literature does not significantly impoverish (and I mean impoverish NOT invalidate) OUR OWN practice is part of why there is such limited reaction around these topics in our profession. People may be right. But right or wrong I do think the perception drives the lack of traction around the topic.

    Sandra Steingard
    September 28, 2016 | 7:21 AM

    I feel the angst and despair.

    September 28, 2016 | 12:26 PM

    angst and dispair? Not me.

    I’m mad as hell and I’m not going to take it anymore!

    September 28, 2016 | 6:19 PM

    Whether the response is angst, or righteous indignation, I stand by:”Psychiatrists by and large do not see THEIR OWN induvidual clinical practice as impoverished by the current state of the literature, even as it is viewed as a threat to the integrity of psychiatric practice as a whole.”

Sorry, the comment form is closed at this time.