Take for example preregistration. An essential element of any credible clinical trial is a declaration of the outcome variables prior to beginning the study. The elements of the design are Randomization, Double Blinding, Preregistration of Outcomes followed by Replication of positive results with a follow-up study. Here’s a contemporary snapshot of what actually happens:
| outcome switching – The COMPare Project | |||||||||
|
|
|||||||||
| rules | It is imperative that a trial sponsor declare the outcome variables before the trial begins [see The Meaning of “Significance” for Different Types of Research, the hope diamond…, Why we need pre-registration, For Preregistration in Fundamental Research]. | ||||||||
|
|
|||||||||
| reality |
Ben Goldacre has been a tireless campaigner against the secrecy afforded the pharmaceutical industry with their clinical trial data. His AllTrials campaign has alerted us all to the problem, but it has been stymied and undermined by industry every step of the way. It’s still viable, but disappointing in that we still don’t have the data access we need. But he’s got another trick up his sleeve that seems to be moving right along in his campaign. He’s looking at how closely recent Clinical Trials adhere to preregistration, something he calls Outcome Switching: The COMPare Trials Project.
by Ben Goldacre, Henry Drysdale, Anna Powell-Smith, Aaron Dale, Ioan Milosevic, Eirion Slade, Philip Hartley, Cicely Marston, Kamal Mahtani, Carl Heneghan.
www.COMPare-trials.org, 2016.
Here again, the rule is to report on the a priori declared outcome variables. This is what Goldacre’s group reports finding on recently reported trials [major journals]:
After the data is in hand, one can do all kinds of statistical tests on the results, thumbing through all the possible outcome variables until you find something that gives you the p-value you want. This is probably the most abused rule of them all in the jury-rigging of clinical trial results. 9 ÷ 67 = 13%. That’s pitiful…
|
||||||||
And looking at the multiple attempts to insist on preregistration, compliance with the rules [some of which are actual laws] mirrors these findings…
| clinicaltrials.gov – preregistration/results | |
|
|
|
| rules |
Since 2007, the rules have been crystal clear – actual law. Trials are to be registered before they’re begun, and the results published within the first year following completion. There’s no ambiguity in what’s supposed to happen.
|
|
|
|
| reality |
If I ever have to justify the name of this blog [1boringoldman], I will probably present a monotonous list of my posts pointing out how the pharmaceutical companies [AND NIMH grant recipients] have been noncompliant with the mandates for timely PRE-registration and later posting results on clinicaltrials.gov. There’s no rational reason for their ignoring this requirement. They already have the data, and the system is simple. They’re given plenty of time. But they’ve just ignored it. This is a partial listing of my posts on this point: transparency!…, studying the studies…, what dreamer?…. non·random missing·ness…, speaking of about time!…, Beware of Shiny Objects!…, back on track…, never been imposed…, a dreamer…, wolf alert! wolf alert!…, etc. And an exempler article [among many] available on-line:.
by Jennifer E Miller, David Korn, and Joseph S Ross
BMJ Open 2015;5:e009758.
Objective: To evaluate clinical trial registration, reporting and publication rates for new drugs by: [1] legal requirements and [2] the ethical standard that all human subjects research should be publicly accessible to contribute to generalisable knowledge.
Design: Cross-sectional analysis of all clinical trials submitted to the Food and Drug Administration [FDA] for drugs approved in 2012, sponsored by large biopharmaceutical companies.
Data sources: Information from Drugs@FDA, ClinicalTrials.gov, MEDLINE-indexed journals and drug company communications.
Main outcome measures: Clinical trial registration and results reporting in ClinicalTrials.gov, publication in the medical literature, and compliance with the 2007 FDA Amendments Acts [FDAAA], analysed on the drug level.
Results: The FDA approved 15 drugs sponsored by 10 large companies in 2012. We identified 318 relevant trials involving 99,599 research participants. Per drug, a median of 57% [IQR 32–83%] of trials were registered, 20% [IQR 12–28%] reported results in ClinicalTrials.gov, 56% [IQR 41–83%] were published, and 65% [IQR 41–83%] were either published or reported results. Almost half of all reviewed drugs had at least one undisclosed phase II or III trial. Per drug, a median of 17% [IQR 8–20%] of trials supporting FDA approvals were subject to FDAAA mandated public disclosure; of these, a median of 67% [IQR 0–100%] were FDAAA-compliant. 68% of research participants [67,629 of 99,599] participated in FDAAA-subject trials, with 51% [33,405 of 67,629] enrolled in non-compliant trials. Transparency varied widely among companies.
Conclusions: Trial disclosures for new drugs remain below legal and ethics standards, with wide variation in practices among drugs and their sponsors. Best practices are emerging. 2 of our 10 reviewed companies disclosed all trials and complied with legal disclosure requirements for their 2012 approved drugs. Ranking new drugs on transparency criteria may improve compliance with legal [FDAAA] and ethics standards and the quality of medical knowledge.
 These graphs give an overview – on the left, the number of trials for each drug; – on the right, the percentage of trials for each drug that meet the criteria listed on the abscissa. Like all of this genre of studies, the compliance rates are dismal.
|
| International Committee of Medical Journal Editors | |
|
|
|
| rules |
In journal articles are forever…, I mentioned the requirements/recommendations of the International Committee of Medical Journal Editors [using the slash-mark because of their own wording]:
"Briefly, the ICMJE requires, and recommends that all medical journal editors require, registration of clinical trials in a public trials registry at or before the time of first patient enrollment as a condition of consideration for publication. Editors requesting inclusion of their journal on the ICMJE website list of publications that follow ICMJE guidance should recognize that the listing implies enforcement by the journal of ICMJE’s trial registration policy."
But being listed on the ICMJE database implies that the journal requires preregistration of Clinical Trials [registration before the study begins] as a prerequisite to considering them for publication.
|
|
|
|
| reality |
In a recent well-done study from New Zealand [Is Mandatory Prospective Trial Registration Working to Prevent Publication of Unregistered Trials and Selective Outcome Reporting?] surveying articles published since those requirements were finalized, the compliance rate was abysmal [see POM·posity…].
 |
It feels like thinking up new reform strategies is barking up the wrong tree. It would just be an exercise in giving people something else to ignore. We don’t need anything new. What we need is a cop.
Sorry, the comment form is closed at this time.