1 Boring Old Man

Data Diving
^{What lies untapped beneath the surface of published clinical trial analyses
could rock the world of independent review}
The Scientist
By Kerry Grens
May 1, 2012

… But even clinical study reports include some level of synthesis. At the finest level of resolution are the raw, unabridged, patient-level data. Getting access to either set of results, outside of being trial sponsors or drug regulators, is a rarity. Robert Gibbons, the director of the Center for Health Statistics at the University of Chicago, had never seen a reanalysis of raw data by an independent team until a few years ago, when he himself was staring at the full results from Eli Lilly’s clinical trials of the blockbuster antidepressant Prozac.

FDA, time, Gibbons had questioned the belief that antidepressants are linked to an increased risk of suicide. Previous meta-analyses by independent reviewers on suicidal thoughts and behaviors among people taking the drugs had for the most part relied on summary data, Gibbons says. At a meeting at the Institute of Medicine a few years ago, Gibbons spoke with a senior investigator at Eli Lilly and brought up the idea of doing a full workup of the original data. Much to his surprise, shortly after the meeting Gibbons was in possession of the numbers. “We haven’t seen anybody get these kinds of data,” he says. He decided to push his luck. Gibbons had served as an expert witness for Wyeth, and he approached attorneys for the pharmaceutical company to ask if they would also share data from trials of the company’s antidepressant Effexor. “They got back to me, and they were agreeable to provide all their adult data,” he recalls.

Gibbons and his colleagues went to work reanalyzing the data. “Everything was exquisitely well documented,” he says. The raw data allowed them to take into account each person’s depression severity and to determine individual outcomes rather than averages. Their results, published earlier this year, ended up bucking much of the published literature on antidepressants. For one, they found no link between Prozac and suicide risk among children and young adults. And secondly, they found that Prozac appeared to be more effective in youth, and antidepressants far less efficacious in the elderly, than previously thought. “I think these kinds of analyses and the discrepancies in the findings are good reason to be concerned about our reliance on traditional meta-analyses,” Gibbons says. Although some of his results reflect negatively on the drugs, others are clearly very positive. There’s been an understanding for some time that publication bias is a real occurrence, and that it often favors the drug…

May Fools’ Day
by David Healy
May 3, 2012

Following the long-standing tradition, dating back at least to Chaucer, of playing practical jokes on May 1, The Scientist clearly thought it would be a good idea to show the outside world that science doesn’t always have to be stuffy and picked the appropriate day to demonstrate the point [http://the-scientist.com/2012/05/01/data-diving/]. Sadly, the joke has gone unnoticed, which defeats the point. Practical jokes of this type need to be celebrated. Hopefully this belated recognition will bring some comfort to the author and editors, and encourage them to continue bringing this important tradition into an arena that desperately needs it. To celebrate May 1, the journal ran an article called Data Diving, ostensibly on a subject that is at the heart of healthcare debates today – access to clinical trial data. In the presence of full and unfettered access, scientific data will supposedly sing out clear and true [Chant clear – they might have said in Chaucer’s day]. Without access, companies can claim their trials show whatever the company wants them to show. Without access, companies get to charge whatever they want for their products…

The centerpiece of Data Diving, the joke, features the work of Robert Gibbons, who has supposedly had unfettered access to patient level data from the trials of Lilly’s Prozac and Wyeth’s Effexor [see Coincidence a fine thing]. Just as Doshi and Jefferson’s access overturned a myth – that Tamiflu has a significant clinical effect, so also Gibbons’ access to the “data” seemingly has overturned myths. In this case, those pesky myths that antidepressants don’t really work and especially that they cause suicide. It was, it turns out, lack of access to the data in the first place that led us to these mistaken beliefs. Companies don’t engage in conspiracies, we are being told, they are masters of the cock-up, and if given a choice of feet to shoot themselves in will opt for both feet. It needs independent academics like Robert Gibbons to  wade in and put a stop to their self-injurious behavior.

The give away that this is a May Fool’s Day joke is that very few articles in recent years in the psychiatric or any other literature  have received such withering critique as the Gibbons’ articles to which The Scientist refers – see for example the sequence from April 14 to 16 by boringoldman. The journal’s fact checkers would never have missed this, and by picking such an egregious piece are clearly letting us in on the joke:
    an anatomy of a deceit 1… introduction
    an anatomy of a deceit 2… the fog comes on little cat feet
    an anatomy of a deceit 3… readin’ and writin’ and ‘rithmetic
    an anatomy of a deceit 4… the letter
    an anatomy of a deceit 5… the purloined letter
    an anatomy of a deceit 6… anticipated and forestalled
There is high parody here in that of course Gibbons didn’t have access to anything the original authors of the Prozac papers didn’t already have access to, in that the original authors were all company people with presumably even fuller access than Gibbons was later given. Gibbons has also managed to avoid incorporating or otherwise handling data in the public domain that could be readily accessed anyway that show incontrovertibly that Prozac and Effexor can not only trigger suicide but that on balance the harms outweigh any benefits. By ignoring relevant accessible data in favor of data no-one else has access to, there is here an almost complete inversion of the standard access to data argument. In the new spirit of openness and perhaps to continue the joke, The Scientist might consider asking Dr Gibbons to make his data fully available on the journal’s website…

I was a late arrival to awareness of the systematic deceit in our psychiatric scientific literature, so most of the corrupted studies I’ve looked at have been in retrospect. I came into this one almost on the day Gibbons’ first article was published and have read the media coverage that accompanied his findings "hot off the press." It feels different when it’s happening in real-time – darker. Unless my own analysis of these articles is in left field [along with everyone else’s], the authors have to know themselves that they are promoting a jury-rigged piece of science – and Dr. Gibbons is promoting it a lot. I can’t figure any way to fold the  paper and conclude that their results are simply in error. It appears to me for all the world like conscious artifice – thus my title, an anatomy of a deceit.

^{There may be an unexpected sequel to this joke. A scandal brewing over the Gibbons’ articles – some unreconstructed academics it seems just don’t “get it”. Not only are Gibbons’ data inaccessible they complain but it seems criticism of the Gibbons’ paper is out of bounds. The Archives of General Psychiatry has steadfastly refused to publish letters from a number of groups pointing out the lethal flaws in Gibbons papers.}

FDA Hearing

September 20, 1991

…
I would suggest to you that I have as little confidence in these anecdotal reports as I do in the anecdotal report of Teicher, and that, in fact, there is no substitute for controlled prospective double-blind clinical trials…

In conclusion, there is simply no scientific evidence whatsoever, no placebo-controlled double-blind study that has established a cause-and-effect relationship between antidepressant pharmacotherapy of any class and suicidal acts or ideation.

As Drs. Potter and Fawcett have suggested, limiting the availability of antidepressants could have a very profound adverse effect in terms of increasing the morbidity and, in fact, mortality associated with untreated depression.

Dr. Charles Nemeroff,
Professor and Chair, Department of Psychiatry,
Emory University, Atlanta Georgia