1 Boring Old Man

With all the reports of the NIH/FDA reforms coming out yesterday, I thought I’d dabble in the details of the various elements being discussed. Usually, one would start with an abstract discussion of how such a thing as clinicaltrials.gov came to be and what it was intended to accomplish, but this time I think it makes more sense to briefly start with what it is concretely, and then move to the loftier narrative. As the philosophers sometimes said: "existence precedes essence."

While it’s discussed almost like it’s some complex governmental agency, clinicaltrials.gov is just a very large, online, searchable database of clinical trials using human subjects [see clinicaltrials.gov]. It’s only a registry, not unlike the bridal registries filled out by prospective brides as the big day approaches. Its contents are all entered by the trial sponsors at various points along the process of doing their trial. Once you locate the trial you’re looking for, you’re offered several different ways to view the contents that relate to that specific trial:

The opening screen [full text view] shows the information gathered when a trial is registered. It tells what’s being studied and why, who’s in charge of the trial, and who’s paying for it. It briefly defines the primary and secondary outcome parameters and how they’ll be analyzed at the end. There’s a section about eligibility and another identifying the location[s] of the study sites. There are lots of dates: when it was registered, when it was completed, etc [notably, it usually doesn’t say when the study actually began].

There’s another window into the same contents [tabular view] with a completely different layout. I’m usually visiting this site to see if they played by the rules, and so I find this view more helpful. For one thing, they report what the outcome variables were when the study was originally registered AND when the study was completed [for some reasons mentioned below, this isn’t always as helpful as it ought to be]. I think this view is the one most useful to scientists, physicians, and trialists – less "fishing around" required.

The results database has been the most disappointing aspect of this enterprise, not because of its structure, but because it has been essentially ignored – even in trials where it is legally mandated. Compliance percentages have been in the teens, even for NIMH/NIH funded trials, and close to zero for the contested studies I’ve tried to look at. The structural layout is fine – the results of the primary and secondary Outcomes and the Adverse Events. But it has way too often simply said No Study Results Posted.

Clicking the link above the tabbed menu [History of Changes], you get a log of all the times it has been updated along the way, and a link that says "Continue to the history of changes for this study on the ClinicalTrials.gov Archive Site." That leads to a screen that lets you scroll through the changes made during the trial. It’s a more primitive database·y interface, but it’s an invaluable vetting tool for exploring the changes, particularly changes in the Outcome Parameters along the way. It’s for the hardy among us.

This was a really great idea, collecting all the data about a trial in a simple public database. There are some important omissions like the date the first patient started the trial or who is actually conducting the trial [which CRO]. Some parts have been degraded over time like the specifics about the study sites. And requirements have changed over time too, like who was required to use it to report results. As government databases go, the design and interface get the job done. But it has been a flop for one simple reason – they didn’t do it! Garbage In, Garbage Out [or in this case, Nothing In, Nothing Out].

Some trials weren’t registered until after they were completed [instead of before they started as they were supposed to]. Most were late. The results database was largely ignored – true for both commercial and government or foundation funded trials. It’s really a shame, because it’s a great idea. I think it’s actually a better way to report the results from clinical trials than journal articles. As I’ve said endlessly, these trials are not research, they’re product testing. So what we need to know are the basics – the results of the prespecified Outcomes and a compilation of the Adverse Events. Those extra words in the journal articles are often rhetoric – part of a smoke screen or a sales pitch – and frequently, some of the essentials have gone missing.

• Toward a New Era of Trust and Transparency in Clinical Trials
• Trial Reporting in ClinicalTrials.gov — The Final Rule
• New federal rules target woeful public reporting of clinical trial results
• HHS Issues New Rules To Open Up Data From Clinical Trials