Posted on Tuesday 7 January 2014

I’m not in love with monotony, but monotony is why my daughter named this blog 1boringoldman. At the time, my monotony was political, but I got converted to the academic·pharmaceutical alliance in psychiatry and the damage done along the way. For a few years, my monotony had to do with my catching up on all the ins and outs of the problem – the distorted clinical trials, the publication bias, the antics of the KOL culture, and the chess games played by the pharmaceutical interests [and some other stuff]. But I seem to have landed on a new monotony. There’s something major that can be done about all of this, and that’s to sidetrack one of the major mechanisms by which it has been maintained – distorted clinical trial reporting. So, in the spirit of reporting on that, this from the current Editor in Chief of The Cochrane Library. I liked it because it gives a window into the magnitude of the Cochrane enterprise:
Huffington Post UK
by David Tovey

Most people assume that when they visit their doctor they will receive impartial and expert advice based on the best current research. This expectation forms the basis for much of the trust invested in doctors and other health professionals. In practice, the medical literature expands at a speed such that even the most committed clinician could not expect to keep abreast of it all. Many doctors read a sample of the highest profile journals in their speciality, but even this is insufficient. Research published in such journals usually represents a small sample of that conducted, and is skewed towards those studies that showed the treatment in question in a favourable light.

To overcome these problems, the past 30 years has seen the growth of a different form of research article: one that looks to summarise all the high quality research, not just the small proportion that has made it into The Lancet or BMJ. These articles are known as "systematic reviews" and the largest international producer of these is a network of researchers started 20 years ago in Oxford but now spread across 120 countries and including over 30,000 individuals. The organisation is called Cochrane, after a British researcher of the same name, and Cochrane Reviews are widely seen as representing the highest quality evidence on which health care decisions can be based.

The problem is that only about half the studies completed are published in journals at all. Despite the fact that this problem has been recognised for decades it has not been effectively tackled despite many attempts to do so. This failure has meant that health professionals and the public frequently have overly optimistic impressions of how effective treatments are. This is not simply an academic issue – it means that people have died or become ill as a consequence of misplaced advice from their health professionals. The problem was highlighted recently by the publication of Ben Goldacre’s book, "Bad Pharma". This was followed up by the setting up of the AllTrials campaign by Goldacre, Sense about Science, the BMJ, Cochrane and many others. The campaign calls for public and full disclosure of the methods and results of all research involving patients, on all treatments in current usage – a fairly modest and reasonable request you might think.

Everyone has heard of Tamiflu, a treatment developed by an iconic drug company, Hoffman-La Roche, to prevent and treat "flu". The reason for the drug’s fame is that in the 2000s, many governments responded to a recommendation from the World Health Organisation [WHO] to stockpile the treatment, for use in a potential influenza pandemic. The Public Accounts Committee has assessed the cost of stockpiling the treatment in the UK alone as being £424million. The WHO recommendation was based on the publication of a paper that summarised 10 trials and appeared to show that Tamiflu prevented severe complications of influenza. To be fair, the Cochrane Review team looking at this question was among those persuaded [albeit briefly] by this evidence, until a Japanese researcher pointed out that only a fraction of the research had been published. It proved to be a milestone.
Cochrane’s 2010 review did in fact positively review Tamiflu based on the data they had. To their credit, when they came to know what a small piece of the puzzle they had, they raised a stink. The 2010 Review has been withdrawn, replaced by a 2012 Review which focuses on what they don’t know. As Dr. Godlee noted, they had only [incomplete] data on 15 of Roche’s 74 studies:
The Cochrane team, led by Dr Tom Jefferson, realised that the feedback was crucial and set out to piece together a full picture of the studies conducted irrespective of whether the studies had been published in medical journals or not. They also approached the investigators of the studies that had been published and established that none of these had access to the trial data -which was held by Hoffman-La Roche. The Cochrane team therefore asked repeatedly for the full information and were refused despite a number of private and public assurances to the contrary. The researchers decided to pull together a comprehensive analysis of all the studies and data submitted to regulatory organisations such as the US Food and Drug Administration [FDA], European Medicines Agency [EMA] and in the UK the Medicine and Healthcare products Regulatory Agency [MHRA] and National Institute for Health and Care Excellence [NICE]. The team realised that the extent of the missing data was far greater than could have been imagined – indeed the unpublished studies represented a large majority of the research completed, including the largest single trial of Tamiflu. Also, there were inconsistencies between the "Clinical Study Reports" [CSRs] submitted to the regulators and the published study reports.

They therefore decided to produce a systematic review of data held in the CSRs, and to ignore published study reports – the first [and still only] time this had been undertaken within Cochrane. This is also a major undertaking given that they estimate that there are over 150,000 pages of data involved. The first version of this review was published in February 2012. This review noted that despite the assurances, Roche had still not provided access to all relevant data, and concluded that there remained uncertainty as to whether Tamiflu was effective in reducing the complications of influenza.
I don’t know if this was the beginning of people understanding how big the problem of missing studies actually was or how it was being used as a tool in drug marketing. I didn’t. I was aware that "bad" studies like Seroquel Study 15 [15 years of study 15 [and counting]…] and that negative studies like Paxil Studies 377 and 701 had been withheld. I even knew that FDA Approval required two supporting studies, no matter how many had actually been done. But it hadn’t occurred to me that this was a common "strategy" [but it should've]. So the Cochrane team is now in its fourth year of going at Roche everyway but Sunday to get the data to analyze. The House of Commons committee listened well to the story about Tamiflu and came out with a very direct approach the problem [a river…, streams…]:
It is therefore not surprising that the Public Accounts Committee [PAC] has registered its concerns and its strong recommendation that all data from clinical trials should be made public. The conclusion that millions of people have been exposed to a treatment, at enormous cost to the public purse, despite the fact that independent researchers have been unable to verify it as being effective or safe, should trouble us all. That despite all the billions of public money spent worldwide on health research we are in this position of ignorance of the effects of most of the treatments taken every day is a true scandal of our times. The Public Accounts Committee report is a major endorsement of the Alltrials campaign. The Committee’s recommendations, if followed through, will lead to better understanding, by health professionals and the public, of the effects of medical treatments – and to more rational and safe healthcare decisions.

And what of the Cochrane Review on Tamiflu? In recent months both Roche and another drug company, Glaxo Smith Kline [GSK] have provided access to the full clinical trial data on two anti-influenza drugs, including Tamiflu, although with some information partially redacted. The research team, which receives financial support from the UK National Institute for Health Research, is expecting to complete its analysis in early 2014 and then, perhaps, we will know whether the billions invested globally in Tamiflu was worth the cost.
Several points: While the Cochrane reviews are a godsend, they come to us from the good graces of volunteer scientists or people who can drum up outside support. I read recently that they don’t have a COI policy themselves, and that could be a problem. I briefly dipped into the controversial new recommendations about Statins [an exempler…, a major force…, but I should’ve…]. Many, including me, think that the newer recommendations for Statin use are inflated, and they’re based on a recent Cochrane review. So it’s probably a mistake to count on Cochrane to keep us on the right track forever. That was Dr. Godlee’s point in the interview below [saying it again…]. The raw data needs to be available for independent investigators as well. That’s why I had such a negative reaction to the recent article [Secure use of individual patient data from clinical trials] that suggested a compromise of making raw data available to trialists like the Cochrane Group. As much as I respect one of the authors, Iain Chalmers who was a co·founder of Cochrane, I smell potential danger down the road [the wisdom of the Dixie Chicks…].

I’m sitting here looking at my used copy of the 650 page Cochrane Handbook of Systematic Reviews and Interventions, and it reminds me of a few other things to say. The Cochrane reviews aren’t always easy to read. They are really into both detailed and systematic so they can be an undertaking at times. I’m not complaining, just warning the uninitiated. They’ve come up with some marvelous techniques for analyzing and comparing results like funnel plots and forest plots [see examples in a breath of fresh air…]. But depending on the author, they can be encyclopedic. Another thing I like about them is that they sometimes reach tentative conclusions. They don’t claim to be the word, a self-skepticism I respect, but it can be off-putting to the more concrete among us.

But there can be little question that for the moment, Cochrane is a giant leap forward – a welcome bit of scientific rigor in a world to often contaminated by fantasy and dangerous games. And as for my monotony, my daughter is no fool – she was onto something. I can beat a topic to death at times…
    January 7, 2014 | 3:06 PM

    “Glaxo Smith Kline [GSK] have provided access to the full clinical trial data on… Tamiflu, although with some information partially redacted.”
    Have I go this right, finally the full release of the trial data, except for some not relevant to transparency personal data? I didn’t realise that.
    And by the way, appreciate your persistence, not boring!

    January 7, 2014 | 4:02 PM

    Thanks alot for your persistance in researching and divulgating about the pharma-academic issues. It takes a lot more than patience to dig into such a mountain of dirt. I would call it passion. Your work has lighted up my brain. Keep “boring” us ;)

    January 7, 2014 | 5:34 PM

    Years ago, I wrote the Cochrane people about the garbage in-garbage out issue. So there.

    January 10, 2014 | 8:39 AM

    I was puzzled by the suggestion that Cochrane may not have a COI policy, so I contacted Dr David Tovey (Editor in Chief, The Cochrane Library, and Deputy CEO). Here is his reply:


    We do indeed have conflict of interest and commercial sponsorship policies. They are publicly available online at the Cochrane Editorial and Publishing Policy webpages. The confusion might arise because we are in the process of ratifying an updating of the policies, so for perhaps another week or so we are in some form of limbo.

    I hope that is helpful

    Best wishes


Sorry, the comment form is closed at this time.