Note: POM = Primary Outcome Measures
An Observational Study of Five Psychiatry Journals That Mandate Prospective Clinical Trial RegistrationPLoS | ONEby Amelia Scott, Julia J. Rucklidge, and Roger T. MulderAugust 19, 2015
Objective: To address the bias occurring in the medical literature associated with selective outcome reporting, in 2005, the International Committee of Medical Journal Editors [ICMJE] introduced mandatory trial registration guidelines and member journals required prospective registration of trials prior to patient enrollment as a condition of publication. No research has examined whether these guidelines are impacting psychiatry publications. Our objectives were to determine the extent to which articles published in psychiatry journals adhering to ICMJE guidelines were correctly prospectively registered, whether there was evidence of selective outcome reporting and changes to participant numbers, and whether there was a relationship between registration status and source of funding.Materials and Methods: Any clinical trial [as defined by ICMJE] published between 1 January 2009 and 31 July 2013 in the top five psychiatry journals adhering to ICMJE guidelines [The American Journal of Psychiatry, Archives of General Psychiatry/JAMA Psychiatry, Biological Psychiatry, Journal of the American Academy of Child and Adolescent Psychiatry, and The Journal of Clinical Psychiatry] and conducted after July 2005 [or 2007 for two journals] was included. For each identified trial, where possible we extracted trial registration information, changes to POMs between publication and registry to assess selective outcome reporting, changes to participant numbers, and funding type.Results: Out of 3305 articles, 181 studies were identified as clinical trials requiring registration: 21 [11.6%] were deemed unregistered, 61 [33.7%] were retrospectively registered, 37 [20.4%] had unclear POMs either in the article or the registry and 2 [1.1%] were registered in an inaccessible trial registry. Only 60 [33.1%] studies were prospectively registered with clearly defined POMs; 17 of these 60 [28.3%] showed evidence of selective outcome reporting and 16 [26.7%] demonstrated a change in participant numbers of 20% or more; only 26 [14.4%] of the 181 the trials were prospectively registered and did not alter their POMs or the time frames at which they were measured. Prospective registration with no changes in POMs occurred more frequently with pharmaceutical funding.Discussion: Although standards are in place to improve prospective registration and transparency in clinical trials, less than 15% of psychiatry trials were prospectively registered with no changes in POMs. Most trials were either not prospectively registered, changed POMs or the timeframes at some point after registration or changed participant numbers. Authors, journal editors and reviewers need to further efforts to highlight the value of prospective trial registration.
[see also Is Mandatory Trial Registration Decontaminating the Psychiatric Literature? by Julia Rucklidge, Ph.D. on Mad in America].
I ended [post-it notes…] on the RIAT Initiative with:
So to my post-it notes. They add one other vital thing, the a priori study protocol [I’m 100% serious about the vital part]. Among many other things, it lays out what variables will be assessed and exactly how they will be analyzed. Most, if not all, RCT distortion involves not following the a priori study protocol [or having a biased protocol from the start]…
And then this study appears just to show us exactly why the a priori protocol is so vital. The proper sequential procedure in any Clinical Trial involves:
«declare the Primary Outcome Measures»
«register the trial including an a priori protocol»
«conduct the blinded trial»
«break the blind»
«analyze the Primary Outcome Measures according to the a priori protocol»
«report the results»
It’s pretty cut-and-dried. These investigators looked at the trials that recruited after a defined date and published 2009-2013 in the five major psychiatric journals. They compiled the start date, the registration date, the Primary Outcome Measures [POMs], and whether the declared POMs carried through to the paper itself. This is an abbreviated version of what they found.
[truncated and rearranged to fit]
And of the 60 that were preregistered and carried into the publication, only 26 POMs were used without some evidence of jury-rigging [14.1 %]! Pitiful!
And ~half of those retrospectively registered trials were registered over a year after the study began. Also pitiful!
And more…
The numbers in the published paper get somewhat confusing to follow, so here’s the summary from the Mad in America article [which is clearer]:
Twenty-one [11.6%] of the 181 studies were deemed unregistered, 61 [33.7%] were retrospectively registered, 37 [20.4%] had unclear POMs either in the article or the registry and 2 [1.1%] were registered in an inaccessible trial registry. Only 60 [33.1%] studies were prospectively registered with clearly defined POMs.
But seventeen of these 60 [28.3%] properly registered trials showed evidence of selective outcome reporting – this means that there had been changes to POMs based on a comparison of the trial registry and the publication. In total, only 26 [14.4%] of the 181 trials were prospectively registered and did not alter their POMs or the time frames at which they were measured. Prospective registration with no changes in POMs occurred more frequently with pharmaceutical funding.
The authors and sponsors bear the ultimate responsibility for this state of affairs, but the editors/journals share equally in the indictment. They’re in a position to check these things and insist that they’re correct. The journal reader has no way to see these parameters in making their judgement about the article. If that weren’t enough – these articles in their study are from a time after the level of duplicity in clinical trials of psychiatric drugs was becoming well known, well beyond the more gullible era of Study 329, Study 352, Study 15, etc. I guess the inertia of deceit is stronger than the currents of exposure and/or reform. Data Transparency means including the a priori protocol…
Parenthetically, having spent some time looking for this kind of data myself, I know that it’s hard work. A follow-up on how they located all the data for this study would help the rest of us…
Afterthought: I suppose I should’ve acknowledged the journals who got it sort of right [actually a bit of a surprise]…
Well done, Mickey. This is indeed an important issue.
I’m unhappy about the derailing of the Paxil withdrawal syndrome discussion. Here is a iatrogenic condition that may affect millions. Every single patient a doctor has on a psychiatric drug for more than a month is at risk for withdrawal syndrome — and yet few doctors, even psychiatrists, recognize it.
That the topic got deflected by a very tangential esoteric discussion is truly unfortunate.
Doctors need to become much, much better informed about the harm they might unintentionally inflict with psychiatric drug prescription. Derailing such discussion does a disservice to both the profession and to patients.
Do no harm and all that.
Thank you for taking this on yet again, Mickey, and trying twice to get the withdrawal discussion back on track.
Amen! When rural Georgia biker dudes know more about Paxil withdrawal than most doctors, there’s something wrong with the ol’ Evidence Base.