1 Boring Old Man

“Evidence-based medicine has been hijacked:” A confession from John Ioannidis

Retraction Watch

by Ivan Oransky

March 16th, 2016

John Ioannidis is perhaps best known for a 2005 paper “Why Most Published Research Findings Are False” … Earlier this month, he published a heartfelt and provocative essay in the the Journal of Clinical Epidemiology titled “Evidence-Based Medicine Has Been Hijacked: A Report to David Sackett.” In it, he carries on a conversation begun in 2004 with Sackett, who died last May and was widely considered the father of evidence-based medicine…

Retraction Watch: You write that as evidence-based medicine “became more influential, it was also hijacked to serve agendas different from what it originally aimed for.” Can you elaborate?
John Ioannidis: As I describe in the paper, “evidence-based medicine” has become a very common term that is misused and abused by eminence-based experts and conflicted stakeholders who want to support their views and their products, without caring much about the integrity, transparency, and unbiasedness of science…

Retraction Watch: You also write that evidence-based medicine “still remains an unmet goal, worthy to be attained.” Can you explain further?
John Ioannidis: The commentary that I wrote gives a personal confession perspective on whether evidence-based medicine currently fulfills the wonderful definition that David Sackett came up with: “integrating individual clinical expertise with the best external evidence”. This is a goal that is clearly worthy to be attained, but, in my view, I don’t see that this has happened yet. Each of us may ponder whether the goal has been attained. I suspect that many/most will agree that we still have a lot of work to do.

Retraction Watch: You describe yourself as a “failure.” What do you mean?
John Ioannidis: Well, I still know next to nothing, even though I am always struggling to obtain more solid evidence and even though I always want to learn more. If you add what are probably over a thousand rejections [of papers, grant proposals, nominations, and other sorrowful academic paraphernalia] during my career to-date, I think I can qualify for a solid failure. Nevertheless, I still greatly enjoy my work in science and in evidence-based medicine.

Retraction Watch: You say that your first grant, which you applied for 17 years ago, was “not even rejected.” Tell us about that grant.
John Ioannidis: It was a randomized controlled trial of antibiotics versus placebo for acute sinusitis. Hundreds of millions of people were treated with antibiotics without good evidence back then, and hundreds of millions of people continue to be treated with antibiotics even nowadays even though most of them would not need antibiotics. I sent in the application to a public funding agency, but have not heard back yet. Probably they felt that requesting funding for a randomized trial and not going to the industry for such funds was a joke. Many public funding agencies are accustomed to funding only research that clearly has no direct relevance to important, real-life questions, so perhaps they didn’t know where to place my application….

Most cultures have some figure who stands outside the mainstream and is allowed to say the things the rest of us can’t get away with saying. In Medieval Times, it was the Court Jester, or the Royal Fool. In our Southwestern Native American culture, they were the Pueblo and other Sacred Clowns. In classic Greek Tragedy, it was often the chorus. Such figures can be sarcastic, even openly hostile, iconoclastic, and  can generally deliver the cold hard truth [in a light-hearted way]. I think of John Ioannidis as occupying such a place in Medicine.

I know that my eyes roll when people start sprinkling terms like evidence-based medicine, best practices, clinical guidelines, etc. in their articles and presentations. I realize that those are important concepts, but they’re so frequently co-opted by other motives that my eye-rolling has almost become an involuntary reflex. But it’s his joke about public funding going to "only research that clearly has no direct relevance to important, real-life questions," that has my attention right now. As much as  we  I complain about the self-serving nature of PHARMA-funded medication research, who else is funding Clinical Trials of drugs? or simple practical studies like do antibiotics help sinus infections? I’m afraid the answer is  almost  nobody.

One place where I frequently find myself thinking about this is the RCTs done on psychiatric medications. The mandate of the FDA is primarily focused on safety, and efficacy is in the mix as a minimal standard, a 1962 afterthought: two statistically significant RCTs out of however-many-they-want-to-do, which can lead to something like the "me too" parade I was describing in the the pipeline paradigm… As I understand it, the lite efficacy testing is there to eliminate snake oil [eg inert medicinal products, the patent medicines of yore], not to inform judgements about prescribing. This is a chart the Atypicals in Schizophrenia by year of introduction, with the effect sizes from a meta-analysis added in…

… hardly representing a progression of increasing efficacy [level, or drifting south?]. One might suspect an incremental safety factor or side effect profile, but that’s not there either. The antidepressant data is harder to gather, but it looks like it degrades by perhaps even more with successive molecules [still work in progress]. The FDA just does what it’s mandated to do – two significant studies and a safety certification, usually based on short term exposure. Reform suggestions often hover around approving only drugs that offer improvement over existing drugs – but in practice that becomes the fuzziest of calls. Others suggest independent, longer running RCTs. That would just be wonderful, except these studies cost a mint, and there’s just no funding source. Sacred Clown Ioannidis’ speaks the truth. Non-industry funding goes to sexier stuff rather than such routine [but vital] independent comparisons.

We have mostly short-term RCTs done either for FDA approval or for a bit of indication creep  with their heavy dose of COI and bias. And no amount of re-analysis or meta-analysis of these early-on RCTs can really get at what we want to know – efficacy and side effects over time in the real world without the spin and sin. Managed Care isn’t any more interested in paying for ongoing research than anyone else, so MD visits remain limited in time and frequency hindering our personal learning from our own patients. It’s easy to hatch "on site" research fantasies, but "translating" such things into a workable reality is quite another matter. Thus Sackett’s evidence-based medicine can downgrade to a trope, or a cliché, or even a trick rather than “integrating individual clinical expertise with the best external evidence.”