1 Boring Old Man

As a piece of their support of the movement for Data Transparency in Randomized Clinical Trials [RCTs], the BMJ featured a particular article [Restoring invisible and abandoned trials: a call for people to publish the findings] with this cover graphic [minus my added post-it notes] on their June 22, 2013 print edition:

Since then, I have learned a lot more about the details of Clinical Trial reporting and have talked about it here ad nauseum. Two of the authors, Peter Doshi and Tom Jefferson, went the extra mile in their quest to vet the studies on Roche‘s Tamiflu® for their Cochrane Collaboration review [Oseltamivir for influenza in adults and children: systematic review of clinical study reports and summary of regulatory comments] – spending literally years chasing down the raw data for the published and unpublished trials. Then they were joined by Kay Dickersin, David Healy, and Swaroop Vedula to collect even more data at large and launch the RIAT Initiative, a move supported by the BMJ and PLoS editors [see “a bold remedy”…]:

Restoring invisible and abandoned trials: a call for people to publish the findings
Analysis
British Medical Journal
^{by Peter Doshi, Kay Dickersin, David Healy, S Swaroop Vedula, and Tom Jefferson}
June 13, 2013
^{[full text on-line]}

Restoring the integrity of the clinical trial evidence base
Editorial
British Medical Journal
^{by Elizabeth Loder and Fiona Godlee [BMJ] and Virginia Barbour and Margaret Winker [PLoS Medicine]}
June 13, 2013
^{[full text on-line]}

Restoring Invisible and Abandoned Trials: A Creative Approach to a Public Good; Now a Creative Approach to Implementation is Needed
Blog
PLoS Medicine
^{By Margaret Winker and Virginia Barbour}
June 13, 2013
^{[full text on-line]}

Data Transparency proposes to reform it by skipping the system altogether. Allow independent investigators to see the same thing the system sees as soon as the blind is broken and reanalyze it, checking the work of the Sponsor/Journal. So that’s the reason for my post-its upstairs. When the RIAT paper was written, I presume they looked for the clinical study report, the electronic patient level datasets, and the completed case report forms.

Having spent some time on a RIAT Team, here two+ years later, I would revise that cover slightly. The CSR is an exhaustive report on the study, but it’s not "raw." It’s something prepared by human beings, the same human beings responsible for the published article – thereby as prey to distortion and corruption as the paper itself. What they got as electronic patient level datasets are one type of IPD [individual patient data]. It’s the compiled database derived directly from the forms filled out as the study progressed – forms called the case report forms [CRFs]. Unless one suspects fraudulent transcription, the IPD is what matters – then you need the CRFs to check with. The CSR is only useful if it contains the IPD. Otherwise, it’s just the long form of the paper you already suspect has something awry.

So to my post-it notes. They add one other vital thing, the a priori study protocol [I’m 100% serious about the vital part]. Among many other things, it lays out what variables will be assessed and exactly how they will be analyzed. Most, if not all, RCT distortion involves not following the a priori study protocol [or having a biased protocol from the start]. So remember these post-it notes when evaluating a RCT or a RIAT reanalysis: