important work…

Posted on Saturday 29 August 2015

Well look here. Those two guys from in the details…, Tom Jefferson and Peter Doshi, just popped up again. I didn’t know we’d hear from them again within the week! And they brought a new friend. I know that the topic of Clinical Study Reports [CSRs] isn’t the sexiest of blog topics, but important things are like that – hiding down in the cracks or behind a bush. These Clinical Trials and their reporting are in the absolute eye of a storm that threatens to over-run medical care with the influence of commercial interests. Not that commercial forces are intrinsically evil, but they’re hardly self regulating and need to be held in check by strong ethical and scientific watchdogs. So the work of people like Jefferson and Doshi is a vital piece of the quality of healthcare that will be delivered far downstream from their own labs and computers:
British Medical Journal
by Khaled El Emam, Tom Jefferson, Peter Doshi
27 Aug, 15

In late 2010, the European Medicines Agency [EMA] became the first regulator in history to promulgate a freedom of information policy that covered the release of manufacturer submitted clinical trial data. Under a separate, new policy [policy 070], the EMA will take an additional step and create a web based platform for sharing manufacturers’ clinical study reports [CSRs] upon a decision being made on a marketing authorization application or its withdrawal.

CSRs contain significant details that are often missing in journal publications of the same trials—for example, details pertaining to patient relevant outcomes and adverse events—and are an important new tool for those engaged in research synthesis. While the policy anticipates that the agency will require individual participant data [IPD] to also be shared, the EMA has not yet committed to a final timeline for this.

But as the EMA works towards finalizing its guidance on the anonymization of CSRs, some companies and industry initiated guidance may be promoting practices that would diminish the value of the data the regulator ultimately distributes. For example, one recent industry guidance favors the redaction and removal of significant standard content in CSRs, ostensibly in an effort to have simple rules for anonymizing these documents. This includes the removal of patient narratives [for example, of serious adverse events and patient dropouts]; line listings [tables of individual level information about participants]; and the redaction of all patient demographics, dates of birth, and other items such as event or assessment dates.

Simple rules have the advantage of being easy to understand and do not require much sophistication to implement. Unfortunately, the major disadvantage is the resulting extensive information loss across the board. CSRs are already written without the use of directly identifying personal information, and maintaining as much of the original information in the CSRs is important to be able to perform accurate analysis—for example, to evaluate the risk of bias of trials.

Thus far, the EMA’s draft guidance has erred towards less redaction of already partially anonymized CSRs, and away from blanket removal and redaction. It instead advocates a more nuanced risk analysis in compliance with recommendations from EU data protection authorities in order to maximize scientifically useful information in the CSR. The suggested approaches for further anonymization include selective masking/redaction, randomization, and generalization techniques…
When the European Medicines Agency announced that they would begin to put the Clinical Study Reports in the public domain, an enormous shot in the arm for Data Transparency. The pharmaceutical industry has mounted an equally enormous campaign to undermine that promise [remember the Abbvie/Intermune suits? see the timeline at ema data transparency…]. And Jefferson and Doshi are in the middle of that game – fighting to stop the EMA from backing down under pressure. They’re worrying here that if there’s not a strong demand for this data, momentum might be lost:
Today, demand for data is an important driver of investments in clinical trial data sharing infrastructure, and it is debatable whether demand is growing as rapidly as some have expected or hoped. The GSK initiated portal reports 104 valid requests [as of end of June 2015]; Project Data Sphere reports 900 authorized users; and the EMA, under its 2010 freedom of information based policy, has released over 2 million pages of regulatory documents. While this access has resulted in some publications and high profile research, such as the Cochrane review of neuraminidase inhibitors, one hopes for more…
I understand their concern, having spent a couple of years working with a RIAT team on just such a  project. As I’ve said, it’s one thing to lobby for data transparency, but quite another to know what to do with it once you’ve got it.

For the moment, there’s no infrastructure or funding support for such enterprises, and it’s certainly a lot of work. We were on our own. But it was plenty rewarding and well worth the effort. It’s the perfect kind of project for graduate students and junior faculty who need a challenge that will flex a wide range of analytic skills with a definable output. I’m sure I’ve learned as much doing this as back during my jurassic age in an NIH fellowship. I can even imagine "watch-dogging" unpublished or questionable studies and reevaluating them as coming into the domain of some group like the Cochrane Collaboration or a similarly structured  independent academy.

But the point right now isn’t how to incorporate increased Data Transparency into some future formal scheme, because we don’t yet really even have that kind of access. Right now, all we can do is support the persistence of people like these "Tamiflu guys," Jefferson and Doshi, in their important work. The EMA has postponed the quest for their posting of the IPD [Individual Participant Data] which is required for a thorough vetting of trials, and we can only hope that the fight for that data will be waged with the same fervor…

30 November 1995
    August 30, 2015 | 7:43 PM

    Hey, that portal is pretty cool! Any of you guys kick the tires and take it out for a test run?

    Okay, the UI is a little clunky– like, it doesn’t remember any of the previous search criteria if it doesn’t find anything, so you have to re-enter them, but not really that hard to use. Very scattershot collection of studies, too. There’s nothing for fluoxetine, for example though there are over 70 results for paroxetine.

    Maybe there’s something I’m not understanding, but… why is GSK tracking the number of requests if the data that’s available is so spotty? Would a lot of serious researchers be using this thing right now if the available information is still so limited?

    Sure, it’s fun for rookies and students like me who don’t really know what they’re looking for, and are just poking around to see if they turn up anything interesting. But when I search for something on this database, is GSK counting that and is that supposed to be part of a metric that someone is using to determine how valuable the database is? Maybe I’m not understanding correctly, but that would be kind of strange.

    Not ready for prime time yet, but it is SO cool, at least for the casual user who wants to get an overview. (That’s how it all starts– as “casual users”– the first bag is free.) Anyone who hasn’t checked it out yet should run some queries, fight your way down to the “Study Details” and have a look around. Just the fact that the data seems to be laid out in similar ways for the different studies makes it easier to digest. And reading the inclusion and exclusion criteria is revealing as well.

    I think I’ll send a link for this to my Psychopharm and Research and Writing professors. Thanks!

    September 2, 2015 | 5:24 PM

    We need this more than ever:

    The FDA Is Basically Approving Everything. Here’s The Data To Prove It

Sorry, the comment form is closed at this time.