In the early days of computing, I lived on the leading edge of hardware and software. The advances were just such exciting stuff. But over the ensuing decades, that has reversed. I actively ride the trailing edge – still use Windows XP and early Linux. I groan when I am forced to update something. So I was slow to twitter, and rarely tweet, requiring help from my daughter to get the blog to auto·tweet [and none of those italicized words have yet entered my everyday vocabulary]. But I have to admit that it’s a pretty clever idea. I wouldn’t have found that last UK Parliamentary hearing without twitter. And then there’s stuff like the tweet from Neuroskeptic above.
While fundamental neuroscience is not my area, the issues raised in this post are pertinent to a important debate on the table. In my college days, I was a math type, but statistics weren’t part of that. It was that sure thing math like calculus and topology. The statistics came later in immunology, but I wasn’t using them to see significance. It was molecular biology and the significance was visible from 100 yards – not part of the question. It was using alalysis of variance to factor out the contributions and interactions of various parameters – plenty enough to keep a Friden calculator sounding like an assault rifle or filling boxes of punch cards. 
My brother-in-law, the experimental psychologist across town, was doing that other kind of statistics. It seemed mighty soft to me, and I tended to be skeptical of differences I couldn’t see with my proverbial naked eye.
But then along came retirement with a reason to learn about the modern clinical trials and the softer side of statistics was all around me [speaking of trailing edges]. So I found myself grappling with small differences where p-values and effect sizes were of primary importance and regressions didn’t refer to neurotic processes. And there was something kind of new for me – misusing math on purpose to make tons of money – as in to sell expensive pharmaceuticals.
One of the obvious ways to clean up the horrible mathemangled clinical trials is to insist that the raw subject level data be available – data transparency. That’s the part Ben Goldacre, the Cochrane Collaboration, and AllTrials petition are all about. The PHARMA offering is to have pseudo-impartial boards decide if the proposal of some research justifies access. I see that as smoke and mirrors. We don’t want to see their numbers to do further research. We want to check their work to see if they’re cheating. So it’s not yet resolved.
Neuroskeptic and COGSCI are on another tack altogether. It has to do with pre·registration – declaring how the study will be done and analyzed up front in order to block the kind of data-play after the fact that haunts the clinical trial literature. In this tweet, Neuroskeptic is talking about a work in progress to respond to COGSCI [Sebastiaan Mathôt], but I’m heavily recommending reading the prequels The black swan and The pros and cons of pre·registration in fundamental research by Mathôt and Interview: Raising the Scientific Standard by Neuroskeptic. Beyond the issue of tightening scientific standards by pre·registration, these articles give insights into the meanings of our statistical analyses that I found both helpful and understandable [emphasis on understandable].
The best part of all of this is that the problem is on the table. They’re bickering about how to do something rather than whether there’s a something that needs doing, which is where we’ve been stuck for a long time. Kudos to the muckrakers who stuck with watch·dogging and debunking the dodgy studies we’ve been fed for years. Without them, this debate wouldn’t even be happening.
” is to insist that the raw subject level data be available – data transparency. That’s the part Ben Goldacre, the Cochrane Collaboration, and AllTrials petition are all about. ”
I know you already know this, but I think it is worth explicitly keeping in mind that, for better or for worse, raw subject level data is NOT what alltrials is all about.
From http://www.alltrials.net/wp-content/uploads/2013/01/Missing-trials-briefing-note.pdf :
“Data on individual patients who have participated in clinical trials
This is a separate issue but also important. ”
Dr. Goldacre has been explicit that they have are not demanding this as part of alltrials. I suspect, at least in part, to deprive PHARMA of the main argument they are trying to use to derail the transparency issues as a whole.
As you also know, BMJ’s policy is much more explicitly targeted at the granular transparency you are talking about.
This is not to detract from your overall points, or the importance of supporting alltrials.
I think Goldacre the fact that Goldacre is NOT attempting to impune all drug companies and all researchers, but is rather trying to privilege certain positive behaviors in these players is key to his approach. And, a deep philosophical difference between him and Healy, and perhaps some commenters on this blog.
He appears to be aiming to have patients, and perhaps physicians, base a lot of whether or not they use a particular drug on whether or not that company has earned a reputation for transparency. Both overall and in the handling of that particular drug.
If alltrials is successful in introducing and maintaining that message in the consciousness of patients, and perhaps physicians, and can sufficiently educate them to distinguish a pig even when it’s wearing lipstick, then that will be a game changer. One that will not require change in institution that have historically gown little interest in fixing this problem.
In other words: Dr. Healy may be right that GSK is trying to game the system and game alltrials. But, if alltrials really does succeed in makin true transparency a factor in whether a patient shows a preference for one drug over another then GSK will suffer mightily with their new drug rollouts if they are in alltrials just for show. Particularly when small drug companies have a drug that really DOES work well and they break ranks and allow more data access because their findings can support it.
The only person who has enough of a stake to sustain caing about this based on enlightened self-interest is the person taking the drug. Or giving it to their kids. They will judge who and what they trust and this leverages the decisions in the rest I the system.
Saying that practically no drug companies, and their allied researchers, can ever be trusted does not allow for that in the same way.
I’m not sure Dr. Goldacre’ approach will work, but it’s possible value should be considered.
The Greenwald link is particularly relevant here:
http://1boringoldman.com/index.php/2013/04/20/up-to-honesty/#comment-242705
Combine these kind of approaches with:
Let’s say there had been a competitor to Roche’s infamous flu drug, let’s say WynetteFlu. If you can achieve a critical mass of patients who would say “Hell No” at taking that Roche product over the other one, specifically based on the shell game Roche is trying to play, then you impact the system in a new way.
I wish alltrials would encourage patients to ask their own physicians:
Have you signed alltrials?
If the answer is no, educate them. Next time they see you, ask them again:
Have you signed alltrials?
If not:
Why don’t they care that there could be that much data missing on the drugs they could potentially prescribe you?
If they don’t care enough about that to even put in the level of effort needed to sign on to the alltrials campaign what does that tell you as a patient in terms of your trust in them?
Teach your own doctor(s) about alltrials and remind them of why it matters. If he/she really doesn’t seem to care, what does that teach you about them? About their care of you? #alltrials
Does that fit on to a tweet?