This is one of the reasons for requiring investigators to declare primary outcome variables before doing the study. Once you have the data in hand, you can play around with your data and the statistical package on your fancy computer and find something that looks the way you want it to look, even if the study is a bust – as was Study 329. This example is amateur night material if you’ve got the data, but can only be inferred but not nailed down with close reading of the paper without it. To repeat myself, "If you torture the statistics long enough, they’ll tell you anything." I’ve already shown another example in the adverse effects data. In the paper, hostility and emotional lability were buried in a table:
But they had a more data available than they let us see, and hostility and suicidality were in the serious range. This looks a whole lot more ominous when you can plot the numbers:
This is what the FDA does when they analyze the results, because the company is required to submit their raw data. And the Pharmaceutical companies know that, explaining this GSK position paper from shortly after they got the results of Study 329 [October 1998]:
I realize that by going over and over the results from Study 329, I’m pushing my luck, and becoming 1reallyboringoldman. But I’m determined to make this point crystal clear, even if I’m my only reader. That GSK position paper says everything needed to critique what has been wrong with the recent era of psychiatric Clinical Trials [and if you read the third from the last sentence, GSK had picked the abberent HAM-D total score < 8 before Sally Laden ever got to work]. What they can’t get through the FDA [even with its minimalistic standards], they publish in a peer reviewed journal and promote off-label use, handing out the reprints [see J&J Deal With The States ‘Is Huge’: Friede Explains]. It’s what’s wrong with ghost-writing. If they’re going to play with the numbers until they can figure out a way to make them reach their forgone conclusion, they need to do it themselves instead of hiring a "hit-man" like Sally Laden to do it for them. It’s their sleep we want disrupted [ghost-writers sleep just fine]. It’s the reason that the raw data needs to be published so that those of us who suspect misbehavior can do an audit, much as the I.R.S. audited Al Capone. I use these mobster examples because this kind of hanky-panky is scientific crime – in this case, crime against youth and crime against physicians…
Your description of Glaxo’s thought process fits the expression HARKING – hypothesizing after results are known.
I hadn’t heard the acronym HARK [hypothesizing after results known], but I won’t forget it. It’s only really possible to get away with HARKing if no one else has access to the raw data. I guess LARK would do too [lying after the results known]. Thanks for the term…
Transparency for Clinical Trials — The TEST Act
Jeffrey M. Drazen, M.D.
N Engl J Med 2012; 367:863-864August 30, 2012
“This legislation is important. The bill requires that any trial that could be used to support an application for FDA approval be registered in ClinicalTrials.gov and that the results be reported in a timely fashion. It requires that early-phase trials (those in which a drug is initially tested in humans) be registered. Thus, these trials will become public knowledge.”
“We can make progress in medicine only if people are willing to put themselves at risk to test new diagnostic and therapeutic approaches. To recognize and reward these participants, and in keeping with the Declaration of Helsinki, clinical trials should be conducted in the open, with full public knowledge of the question asked, the intervention tested, and the results obtained. The TEST Act is another step toward this end, and we strongly support it.
Selling bulk reprints is a profit center for a journal, see http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3133891/ :
“Additionally, reprint sales can be a financially attractive source of revenue for some publishers, which has the potential to generate about $700,000 from a single journal article.”
(Apologies if you mentioned this elsewhere, Dr. Mickey.)
Martin Keller retired from Brown without explanation at the end of June http://www.pharmalot.com/2012/09/controversial-paxil-study-author-retires-from-brown/