I am so used to writing about wrong things, when something right comes along I just can’t stop thinking about it. This British Medical Journal decision to only publish Clinical Trials that have patient-level datasets available is an elegant solution to a not easily solved problem:
… Why aren’t all clinical trial data routinely available for independent scrutiny once a regulatory decision has been made? How have commercial companies been allowed to evaluate their own products and then to keep large and unknown amounts of the data secret even from the regulators? Why should it be up to the companies to decide who looks at the data and for what purpose? Why should it take legal action (as in the case of GlaxoSmithKline’s paroxetine and rosiglitazone), strong arm tactics by national licensing bodies (Pfizer’s reboxetine), and the exceptional tenacity of individual researchers and investigative journalists (Roche’s oseltamivir) to try to piece together the evidence on individual drugs?
It’s a story that has lots of twists and turns, one of those stories where the last solution turns into the next problem. When Senator Kefauver added efficacy to the FDA’s mandate to certify safety in the 1960s, who would’ve predicted that adding Randomized Clinical Trials [RTC] would turn out to be a problem? When Robert Spitzer codified psychiatric diagnoses in the DSM-III, who knew that it would add to the RTC problem from before? But that’s what happened. The FDA Approval based on RTCs in DSM-III Disorders became a ticket to commercial success, "And _____ has recently been approved for _____ Disorder by the FDA as published in the American Journal of _____" [because it separates statistically from placebo], omitting "but the patients didn’t notice and dropped out like flies." Worse, the studies were often written by the drug companies creatively and signed by the KOLs du jour. So the Clinical Trials had to be registered on clinicaltrials.gov to legitimize them but the results were rarely posted. So now we have the new TEST Act coming to insist on the clinicaltrials.gov results being posted, but it’s summaries with no primary data just like the papers that have turned out to be so distorted. There are lots of good people trying to do good things only to have their efforts incorporated into a new version of the very problem they were attempting to solve.
In ancient times, the drug companies were always around. Detail men with plastic brains and free pens. Throw-away journals with drug company articles filled our boxes. Medical journals had glossy pages of ads with snappy art work. I never saw that as a problem. It was no different than commercials on television. Sometimes something caught your eye, but mostly it was just part of the background noise of life. But then things changed, at least in psychiatry. The peer reviewed journals got thicker and were filled with clinical trial papers. The articles about the phenomenology of mental illness were replaced with reviews of medications and endless clinical trials. At first we didn’t notice they were industry financed. Then we didn’t notice the conflict of interest statements. Then we found out the authors didn’t write them. And before long, our scientific journals filled up with articles that looked like the scientific articles of yore, but read like the stuff formerly in the throw aways. Harvard, Brown, Stanford, Emory, etc. stopped meaning what it used to mean. Even editorials became suspect. For that matter, so did textbooks. In psychiatry, famous doctors became drug reps with CVs that weren’t even possible – like hundreds and hundreds of articles. The author by-line was more like celebrity endorsements than authorship.
So the reason Fiona Godlee’s BMJ announcement is so perfect is that she’s skipping all the pieces in-between and getting to the heart of the matter. If you won’t let us see your patient-level raw data to allow our peer reviewers to genuinely vet your science, we won’t consider your article. Peer reviewers have been at the same disadvantage as readers – unable to tell whether the story as presented is accurate or skillfully dolled up. With her solution in place, their work is cut out for them, but their power to adequately evaluate a study is magnified geometrically. They can look at the raw plots freed from the layers of statistics and presentation tricks that haunt the modern literature. But the greatest boon is to the reader who can pick up her journal and know that it has been checked at the most basic level by an independent expert who is not on the company payroll. Perfect.
If she’s able to start a trend, it will mean that the cost of journals will go up I think. Maybe they’ll get thinner. Maybe they’ll come out less frequently. Maybe there will be fewer of them. Great! Doctors are bombarded with information already. A decrease in quantity and an increase in quality is a double bonus. I don’t mind if there are ads all over the place and the journal looks like Times Square [before this week]. What I care about is being able to tell an ad from a piece of unbiased science. If industry wants to pay for a study, or even do the study, that’s fine with me if they collect their data properly and make it available for thorough evaluation by an independent agency. It puts the control of science back where it belongs, in the hands of the scientists instead of marketers and spin-sters. Doctors aren’t dumb. It will quickly become clear which journals are serious about peer review and which ones aren’t. Loopholes? I can’t find them yet – a good sign. So Fiona Godlee is my nominee for the Medical Editor of the Early Twenty-First Century Award. The issue now is how to enable other journals of quality follow suit…
Well done Mickey and well done Fiona
British Medical Journal
Online only
£95
€ 129
US$186
Or
iPad subscription
£9.99 – 4 week sub*
“I am a retired American psychiatrist, but I plan to subscribe to the BMJ come January. It’s a way physicians of conscience can stand for what’s right in science. And I’ll do the same for any Journal that follows the BMJ lead that’s remotely related as a way of supporting a call for data transparency.”
What a great idea.
If you figure out a way to make it explicit when you subscribe that you are doing so because of their decision let us know.