Clinical study reports of randomised controlled trials: an exploratory review of previously confidential industry reportsBMJ Openby Peter Doshi and Tom Jefferson26 February 2013
Objective To explore the structure and content of a non-random sample of clinical study reports [CSRs] to guide clinicians and systematic reviewers.Search strategy We searched public sources and lodged Freedom of Information requests for previously confidential CSRs primarily written by the industry for regulators.Selection criteria CSRs reporting sufficient information for extraction [‘adequate’].Primary outcome measures Presence and length of essential elements of trial design and reporting and compression factor [ratio of page length for CSRs compared to its published counterpart in a scientific journal].Data extraction Data were extracted on standard forms and crosschecked for accuracy.Results We assembled a population of 78 CSRs [covering 90 randomised controlled trials; 144,610 pages total] dated 1991–2011 of 14 pharmaceuticals. Report synopses had a median length of 5 pages, efficacy evaluation 13.5 pages, safety evaluation 17 pages, attached tables 337 pages, trial protocol 62 pages, statistical analysis plan 15 pages and individual efficacy and safety listings had a median length of 447 and 109.5 pages, respectively. While 16 [21%] of CSRs contained completed case report forms, these were accessible to us in only one case [765 pages representing 16 individuals]. Compression factors ranged between 1 and 8805.Conclusions Clinical study reports represent a hitherto mostly hidden and untapped source of detailed and exhaustive data on each trial. They should be consulted by independent parties interested in a detailed record of a clinical trial, and should form the basic unit for evidence synthesis as their use is likely to minimise the problem of reporting bias. We cannot say whether our sample is representative and whether our conclusions are generalisable to an undefined and undefinable population of CSRs.
After a few years being preoccupied with evaluating the documents from a single Clinical Trial [Paxil Study 329] for a RIAT project [A Milestone in the Battle for Truth in Drug Safety, Restoring Study 329], I made this diagram trying to clarify things for myself. The upper part is clear – the a priori PROTOCOL and the SAP [Statistical Analysis Plan] – are evaluated by the Institutional Review Board [IRB] and, and if approved become the directives for the Clinical Trial. That part’s blinded [dark glasses]. Once the blind is broken, the Raw Data as Case Report Forms [CRFs] are assembled and organized. They’re transcribed into data tables known collectively as the IPD [Individual Participant Data]. Notice there aren’t any glasses there [yet] because these nuclear document have rarely ever been seen. Then someone uses the IPD to write an exhaustive CSR [Clinical Study Report]. Later all of this becomes the highly condensed published ARTICLE usually under the byline of academic authors. I put the glasses there because it’s seen and unblinded, but notice they’re "rose colored glasses." The important part is how to get untinted glasses in the IPD and CRF quadrant where they’re needed, without the "rose coloring" of conflict of interest:
Doshi and Jefferson explain some of why this eyes-on view of the Raw data is so hard to get hold of. The issue is that the meaning and requirements of CSR varies from company to company and agency to agency – and it’s not at all clear whether the CSR must contain the IPD [ergo, Raw Data] or not. In the study we looked at [Paxil Study 329], the IPD was listed in Appendices to the CSR, but in GSK’s initial release – the Appendices were nowhere to be found in the 2004 posting, only appearing 8 years later  at the insistence of Peter Doshi working with the NY Attorney General. And apparently, there’s a lot of variability. So if this is a remote interest, by all means read their paper [maybe even if it isn’t, because this at the heart of the Data Transparency question].