publication bias I – close encounters of the second kind…

Posted on Thursday 19 May 2016

Publication bias is the term for what occurs whenever the research that appears in the published literature is systematically unrepresentative of the population of completed studies. Simply put, when the research that is readily available differs in its results from the results of all the research that has been done in an area, readers and reviewers of that research are in danger of drawing the wrong conclusion about what that body of research shows…

The usual version of publication bias is the practice of simply not publishing negative clinical trials. The result is obvious. The positive trials "average out" to a falsely inflated result and the drug looks better than it actually is. We’ve pretty much caught on to that ploy by requiring all trials be registered in advance [clinicaltrials.gov], developingtools to detect missing trials [funnel plots], and doing meta-analyses that include unpublished trials [the Tamiflu saga]. A variant would be publishing Paxil Study 329 which was claimed to be positive, but not publishing two negative trials of the same drug [Paxil Study 377 and Paxil Study 701] until after the patent had run out [see paxil in adolescents: “five easy pieces”… and study 329 x – “it wasn’t sin – it was spin”…]. This post isn’t about that kind of publication bias. It’s about a getting published version of publication bias.

[Note: This particular post is kind of redundant, not too different from the last one. But it’s here because I finally figured out what I wanted to say].

I was on the RIAT team that reanalyzed Paxil Study 329 [Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence]. It was the end product of over a decade of effort by armies of people from many professions all over the world working to make it happen. It was in a high impact journal [British Medical Journal] and was extremely well received. In spite of all that preparatory work and a receptive editorial staff, writing it and getting it published was a grueling experience, roughly outlined at Restoring Study 329 [in Background], and there will be more to follow. But in spite of its success, you have to ask yourself "Why did it take such a herculean effort by an army and fourteen years to get it in print?" And for that matter, "Why was I an author – a retired adult psychiatrist who writes a blog at the edge of the galaxy?" It should’ve been authored by a chairman of child psychiatry at a prestigious medical school or the president of the American Academy of Child an Adolescent Psychiatry. Actually, Study 329 should never have been published in the first place as it is written [Efficacy of Paroxetine in the Treatment of Adolescent Major Depression: A Randomized, Controlled Trial], or should have been subsequently retracted.

And now we have another article about an antidepressant clinical trial in kids, The citalopram CIT-MD-18 pediatric depression trial: Deconstruction of medical ghostwriting, data mischaracterisation and academic malfeasance. It’s authored by a heavily credentialed multidiscipliary team. In this paper, the data comes from the sponsor’s own internal documents and focuses in on the processes involved in creating the original publication [A Randomized, Placebo-Controlled Trial of Citalopram for the Treatment of Major Depression in Children and Adolescents]. While they show the same things we documented [deviations from the a priori protocol, add-on parameters, questionable analyses, etc], they go further. Using verbatim quotes from the sponsor, they show that all of this jury-rigging of data, analysis, and presentation was done by the sponsor’s employees with the specific intent of deceiving the reader by making this negative trial look positive. And yet they had a hell of a time getting it published [see background notes]. It wasn’t turned down because it was wrong, or badly written, or poorly documented, or didn’t have proven authors. "Why did they have such a hard time getting it published?" It’s in the International Journal of Risk and Safety in Medicine, a perfectly legitimate peer reviewed medical journal, but hardly in the upper echelon. Since it was unfunded research, the journal graciously made it Open Access by waiving the fee. But thus far, it hasn’t been covered in the main stream media. "Why not?" It is  at least as important as our article, if not more-so, as it documents active fraudulent behavior on the part of the sponsor.

To my mind, this is publication bias of a second kind [see a mighty hard row…], a problem getting a contrarian take on a clinical trial published and into the discourse where it belongs. It fits the opening definition to a tee. This is a form of publication bias exerted by the publishers themselves, and may not necessarily be the result of direct or even indirect involvement of the original clinical trial’s sponsor. I’m not going to speculate further about all the motivations involved here. There’s more than enough speculation flying around about this topic already. But I do want to talk about it’s impact, and illustrate it with a contemporary example…
  1.  
    James OBrien, M.D.
    May 19, 2016 | 3:08 PM
     

    Hard to be the sober designated driver at the frat party. That’s the group dynamic at play.

    But that’s probably the person that should be listened to.

    Otherwise this can happen:

    http://arstechnica.com/science/2016/05/theranos-corrects-tens-of-thousands-of-blood-tests-voids-2-years-of-edison-results/

    9 billion dollars to nothing….this is actually far worse than Shkreli in terms of numbers.

  2.  
    1boringyoungman
    May 19, 2016 | 9:48 PM
     

    The parallels between the behavior of journal editors you are discussing here in 1bom posts and the behavior of those in other forms of media discussed in the Vanity Fair article are interesting.
    http://www.vanityfair.com/news/2016/05/theranos-silicon-valley-media
    How a cadre of academic psychiatrists (authors and editors both) have failed, and are failing, their clinical colleagues is a story that academic psychiatrists appear to have little interest in telling.

  3.  
    berit bryn jensen
    May 20, 2016 | 3:23 AM
     
  4.  
    1boringyoungman
    May 20, 2016 | 3:35 AM
     

    “But journals often have their own conflicts of interest, something Cipriani acknowledges. Ioannidis and his colleagues originally tried to publish their latest study in psychiatry journals that they thought would be more pertinent, but the reception was cold. “Some people felt pretty angry about it and many of their editors have strong ties to the industry.” Ioannidis says.”
    http://www.scientificamerican.com/article/many-antidepressant-studies-found-tainted-by-pharma-company-influence/

  5.  
    James OBrien, M.D.
    May 20, 2016 | 10:20 AM
     

    That Theranos article reminds me of the comedy Silicon Valley except it isn’t funny.

    The practice of shunning skeptics and logrolling with cheerleaders is practiced because it works, and nowhere is this more prevalent than in Washington, D.C.

    And that is why the government is the least qualified organization to solve this problem.

    I don’t know how you wake up a citizenry too distracted to care.

    Maybe we collectively don’t deserve a fair media because we don’t insist on it.

  6.  
    James OBrien, M.D.
    May 20, 2016 | 5:50 PM
     

    The Theranos debacle and this article got me thinking. General intelligence does not seem to be the remedy for hucksterism, it actually makes it easier to pull off:

    http://www.peakprosperity.com/blog/94818/death-hopium

    Psychologists understand the difference between fluid and crystallized intelligence but I think a lot more work needs to be done about book intelligence and playground intelligence, often referred to as street smarts. Which seems to be where you get a BS meter today since they don’t teach critical thinking anymore.

    I know so many bright people who are easy marks and targets. Not just in medicine. The fact that we are going through the third financial bubble in 15 years means no one is learning anything from bad experience.

    I’ve tried to come up with explanations why BS meters are set to off these days. Social media distorts reality of negative feedback? Careerism is the enemy of critical questioning? Maybe after the GI bill, college graduates had both book smarts and street smarts because they had served in the military or had done manual labor, two activities widely disdained and avoided by the intellectual classes of today.

    It really gets back to the central theme of this blog…why are our BS meters turned off when there is so much BS around us? I

  7.  
    Donald Franklin Klein
    May 20, 2016 | 7:52 PM
     

    Colleagues
    As i recall there was a difference of opinion between Aristotle(?) and Spinoza on how we deal with assertions. Aristotle said we hold them in mind and critically assess their merit. Spinoza said that we will believe an assertion until overwhelmed by facts. May have not got this exactly right but the point is clear. Apparently well based assertions require factual contradiction to disbelieve. Here the facts were concealed. The facts are now out-that the leaders of our profession remain mute testifies to some pro-concealment motivation , rather than cognitive naivete.
    Don Klein

  8.  
    James OBrien, M.D.
    May 20, 2016 | 9:21 PM
     

    We are clearly in a Spinoza bordering on Idiocracy age. Maybe in the 432 BC people didn’t have the luxury of being fooled by con man because that kind of thing would get you killed.

    I’ve noticed lately in some ways pedestrians are becoming dumber than house pets, who still have the animal sense move to the shade, stay out of puddles and look where they are going in the crosswalk instead of texting and bumbling Mr. Magoo style.

  9.  
    James OBrien, M.D.
    May 20, 2016 | 10:13 PM
     

    Speaking of Idiocracy, I need autocorrect with grammar…

  10.  
    May 20, 2016 | 10:22 PM
     
    “The facts are now out – that the leaders of our profession remain mute testifies to some pro-concealment motivation, rather than cognitive naivete.”

    Sadly true. One wonders when and how that happened?

Sorry, the comment form is closed at this time.