European College of NeuropsychopharmacologyConference Coverageby Bruce JancinOctober 28, 2016VIENNA – Janneke A. Bastiaansen, PhD, has some advice for clinicians and researchers as they peruse the published literature on clinical trials of medication or psychotherapy for major depressive disorder: Don’t believe everything you read. “Be critical. Use your critical mind,” she urged at the annual congress of the European College of Neuropsychopharmacology.
The results of her analysis of 105 clinical trials of antidepressant drugs and 142 studies of psychotherapy indicated that the literature is rife with four types of bias: publication, outcome reporting, spin, and citation bias. “The quality of the evidence base is vital. We base our clinical decisions on what’s out there in the literature. And I think it’s really important to know that there are various biases that can color the literature,” said Dr. Bastiaansen, a psychologist at the University of Groningen, the Netherlands.
She took a closer look at 105 clinical trials of antidepressant drugs registered with the U.S. National Institutes of Health at clinicaltrials.gov. Fifty-three reported positive findings, and 52 were negative. Fifty-two of the 53 positive trials were ultimately published, as were only 25 of the 52 negative studies. That’s a sterling example of publication bias.
Upon careful scrutiny of the 25 negative trials that were published, 10 were misleadingly reported as positive studies. The investigators either switched out the prespecified primary outcome previously filed with NIH and promoted a positive secondary outcome to primary outcome status because the original primary outcome was negative, or they omitted the negative outcomes altogether. That’s outcome-reporting bias. Of the 15 published negative drug trials that were free of outcome-reporting bias, the authors of 10 of the studies employed “spin,” using phrases such as “the treatment was numerically superior.” Thus, only 5 of the 25 published negative clinical trials unambiguously reported that the studied treatment was not effective.
“Here the message is that, when you read a paper, look at the results, come up with your own conclusion, and then compare it with the conclusion of the authors, because sometimes they’ve colored it in a more positive way,” Dr. Bastiaansen said in an interview. Citation bias is the phenomenon whereby positive clinical trials are cited more frequently than published negative trials. “As a clinician, if you look at the literature and print out every paper that’s out there on a given antidepressant drug for major depression, and you look at that pile, you’ll think: ‘Ah, the literature is really strong about this treatment effect,’ because positive papers selectively cite other positive papers,” she continued.
The pharmaceutical industry takes a lot of heat for selectively burying company-sponsored negative trials, but the literature on psychotherapy for major depression is actually more opaque. “A lot of people aim their arrows at the pharmaceutical industry and say: ‘Everything’s bad about pharma,’ but actually, you see bias in every field. You see it in the trials of psychotherapy. It’s very important to know that it’s ubiquitous. The positive side of the antidepressant drug trials is that there’s this standard database [clinicaltrials.gov], and you can use it to check out what trial is published and what’s not. It’s not the case for psychotherapy trials. I think we need a mandatory registry for clinical trials of psychotherapy as well,” Dr. Bastiaansen said. Of the 142 psychotherapy studies, 49 were negative, but the abstracts of only 12 of those 49 concluded that psychotherapy was not more effective than a control.
by Andreas Ø Bielefeldt, Pia B. Danborg, and Peter C. GøtzscheJournal of the Royal Society of Medicine. 2016 109[10]:381-392.
Objective: To quantify the risk of suicidality and violence when selective serotonin and serotonin-norepinephrine reuptake inhibitors are given to adult healthy volunteers with no signs of a mental disorder.Design: Systematic review and meta-analysis.Main outcome measure: Harms related to suicidality, hostil- ity, activation events, psychotic events and mood disturbances.Setting: Published trials identified by searching PubMed and Embase and clinical study reports obtained from the European and UK drug regulators.Participants: Double-blind, placebo-controlled trials in adult healthy volunteers that reported on suicidality or violence or precursor events to suicidality or violence.Results: A total of 5787 publications were screened and 130 trials fulfilled our inclusion criteria. The trials were generally uninformative; 97 trials did not report the randomisation method, 75 trials did not report any discontinuations and 63 trials did not report any adverse events or lack thereof. Eleven of the 130 published trials and two of 29 clinical study reports we received from the regulatory agencies presented data for our meta-analysis. Treatment of adult healthy volunteers with antidepressants doubled their risk of harms related to suicidality and violence, odds ratio 1.85 [95% confidence interval 1.11 to 3.08, p = 0.02, I² = 18%]. The number needed to treat to harm one healthy person was 16 [95% confidence interval 8 to 100; Mantel-Haenszel risk difference 0.06]. There can be little doubt that we under- estimated the harms of antidepressants, as we only had access to the published articles for 11 of our 13 trials.Conclusions: Antidepressants double the occurrence of events in adult healthy volunteers that can lead to suicide and violence.
Matching the Industry of CROs and Ghost -management and -writing enterprises, there’s almost an academic-specialty growing to deconstruct these jury-rigged clinical trial reports that have descended on our literature like a hoard of locusts – particularly the psychiatric literature. Dr. Bastiaansen in the first paper gives some really good advice, however she’s speaking at a meeting of specialists and scientists, not the primary care physicians who are prescribing the medicines. I now do everything she says – look up the drugs in ClinicalTrials.gov checking for publication bias, look carefully for evidence of outcome switching, am ever alert to spin. But I’m not normal. It’s almost my new career/hobby/obsession. As a practicing doctor, no way would I be able to take the time to do that. I’d be lucky to have time to read the abstracts, much less the whole articles, way much less than look them up on ClinicalTrials.gov or Drugs@FDA.
Let me get this straight. There were 62 studies that were reported as positive for the drugs (52 that were positive, and 10 that were misleadingly reported as positive) There were another 10 that were spun as positive.
There were 52 studies that were negative for antidepressant drugs. Yet “only 5 of the 25 published negative clinical trials unambiguously reported that the studied treatment was not effective.”
So even if a physician (or anyone else) read every single journal article published of the original 105 studies, they would think there were only 5 studies that showed the pills were ineffective, and 72 studies showed that they were superior.
The issue is not that you should not ‘believe everything you read’. The real issue is that you cannot believe anything you read about antidepressant drugs in the research literature.