This is cross posted from Dr. David Healy’s web site with his permission…
The Letter below from Marty Keller and colleagues was sent to many media outlets, to retraction watch, and to professional organizations on Wednesday. Paul Basken from the Chronicle for Higher Education asked me for a response which I sent about an hour after receiving the letter. This response is from me rather than the 329 group. This and other correspondence features and will feature on Study329.org.
One quick piece of housekeeping. Restoring Study329 is not about giving Paroxetine to Adolescents – its about all drugs for all indications across medicine and for all ages. It deals with standard Industry MO to hype benefits and hide harms. One of the best bits of coverage of this aspect of the story yesterday was in Cosmopolitan.
Letter From Keller et al
Dear
Nine of us whose names are attached to this email (we did not have time to create electronic signatures) were authors on the study originally published in 2001 in the Journal of the American Academy of Child and Adolescent Psychiatry entitled, “Efficacy of paroxetine in the treatment of adolescent major depression: a randomized controlled trial,” and have read the reanalysis of our article, which is entitled, “Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence”, currently embargoed for publication in the British Medical Journal (BMJ) early this week. We are providing you with a brief summary response to several of the points in that article that which with we have strong disagreement. Given the length and detail of the BMJ publication and the multitude of specific concerns we have with its approach and conclusions, we will be writing and submitting to the BMJ’s editor an in-depth letter rebutting the claims and accusations made in the article. It will take a significant amount of work to make this scholarly and thorough and do not have a time table; but that level of analysis by us far exceeds the time frame needed to give you that more comprehensive response by today.
The study was planned and designed between 1991-1992. Subject enrollment began in 1994, and was completed in 1997, at which time analysis of the data commenced. The study authors comprised virtually all of the academic researchers studying the treatment of child depression in North America at the time. The study was designed by academic psychiatrists and adopted with very little change by GSK, who funded the study in an academic / industry partnership. The two statisticians who helped design the study are among the most esteemed in psychiatry. The goal of the study designers was to do the best study possible to advance the treatment of depression in youth, not primarily as a drug registration trial. Some design issues would be made differently today — best practices methodology have changed over the ensuing 24-year interval since inception of our study.
In the interval from when we sat down to plan the study to when we approached the data analysis phase, but prior to the blind being broken, the academic authors, not the sponsor, added several additional measures of depression as secondary outcomes. We did so because the field of pediatric-age depression had reached a consensus that the Hamilton Depression Rating Scale (our primary outcome measure) had significant limitations in assessing mood disturbance in younger patients. Accordingly, taking this into consideration, and in advance of breaking the blind, we added secondary outcome measures agreed upon by all authors of the paper. We found statistically significant indications of efficacy in these measures. This was clearly reported in our article, as were the negative findings.
In the “BMJ-Restoring Study 329 …” reanalysis, the following statement is used to justify non-examination of a range of secondary outcome measures:
Both before and after breaking the blind, however, the sponsors made changes to the secondary outcomes as previously detailed. We could not find any document that provided any scientific rationale for these post hoc changes and the outcomes are therefore not reported in this paper.
This is not correct. The secondary outcomes were decided by the authors prior to the blind being broken. We believe now, as we did then, that the inclusion of these measures in the study and in our analysis was entirely appropriate and was clearly and fully reported in our paper. While secondary outcome measures may be irrelevant for purposes of governmental approval of a pharmaceutical indication, they were and to this day are frequently and appropriately included in study reports even in those cases when the primary measures do not reach statistical significance. The authors of “Restoring Study 329” state “there were no discrepancies between any of our analyses and those contained in the CSR [clinical study report]”. In other words, the disagreement on treatment outcomes rests entirely on the arbitrary dismissal of our secondary outcome measures.
We also have areas of significant disagreement on the “Restoring Study 329” analysis of side effects (which the author’s label “harms”). Their reanalysis uses the FDA MedDRA approach to side effect data, which was not available when our study was done. We agree that this instrument is a meaningful advance over the approach we used at the time, which was based on the FDA’s then current COSTART approach. That one can do better reanalyzing adverse event data using refinements in approach that have accrued in the 15 years since a study’s publication is unsurprising and not a valid critique of our study as performed and presented.
A second area of disagreement (concerning the side effect data) is with their statement, “We have not undertaken statistical tests for harms.” The authors of “Restoring Study 329” with this decision are saying that we need very high and rigorous statistical standards for declaring a treatment to be beneficial but for declaring a treatment to be harmful then statistics can’t help us and whatever an individual reader thinks based on raw tabulation that looks like a harm is a harm. Statistics of course does offer several approaches to the question of when is there a meaningful difference in the side effect rates between different groups. There are pros and cons to the use of P values, but alternatives like confidence intervals are available.
“Restoring Study 329” asserts that this paper was ghostwritten, citing an early publication by one of the coauthors of that article. There was absolutely nothing about the process involved in the drafting, revision, or completion of our paper that constitutes “ghostwriting”. This study was initiated by academic investigators, undertaken as an academic / industry partnership, and the resulting report was authored mainly by the academic investigators with industry collaboration.
Finally the “Restoring Study 329” authors discuss an initiative to correct publications called “restoring invisible and abandoned trials (RIAT)” (BMJ, 2013; 346-f4223). “Restoring Study 329” states “We reanalyzed the data from Study 329 according to the RIAT recommendations” but gives no reference for a specific methodology for RIAT reanalysis. The RIAT approach may have general “recommendations” but we find no evidence that there is a consensus on precisely how such a RIAT analysis makes the myriad decisions inherent in any reanalysis nor do we think there is any consensus in the field that would allow the authors of this reanalysis or any other potential reanalysis to definitively say they got it right.
In summary, to describe our trial as “misreported” is pejorative and wrong, both from consideration of best research practices at the time, and in terms of a retrospective from the standpoint of current best practices.
Martin B. Keller, M.D.
Boris Birmacher, M.D.
Gregory N. Clarke, Ph.D.
Graham J. Emslie, M.D.
Harold Koplewicz, M.D.
Stan Kutcher, M.D.
Neal Ryan, M.D.
William H. Sack, M.D.
Michael Strober, Ph.D.
Response
In the case of a study designed to advance the treatment of depression in adolescents, it seems strange to have picked imipramine 200-300mg per day as a comparator, unusual to have left the continuation phase unpublished, odd to have neglected to analyse the taper phase, dangerous to have downplayed the data on suicide risks and the profile of psychiatric adverse events more generally and unfortunate to have failed to update the record in response to attempts to offer a more representative version of the study to those who write guidelines or otherwise shape treatment.
As regards the efficacy elements, the correspondence we had with GSK, which will be available on Study329.org as of Sept 16 and on the BMJ website, indicates clearly that we made many efforts to establish the basis for introducing secondary endpoints not present in the protocol. GSK have been unwilling or unable to provide evidence on this issue, even though the protocol states that no changes will be permitted that are not discussed with SmithKline. We would be more than willing to post any material that Dr Keller and colleagues can provide.
Whatever about such material, it is of note that when submitting Study 329 to FDA in 2002, GSK described the study as a negative Study and FDA concurred that it was negative. This is of interest in the light of Dr Keller’s hint that it was GSK’s interests to submit this study to regulators that led to a corruption of the process.
Several issues arise as regards harms. First, we would love to see the ADECs coding dictionary if any of the original investigators have one. Does anyone know whether ADECs requires suicidal events to be coded as emotional lability or was there another option?
Second, can the investigators explain why headaches were moved from classification under Body as a Whole in the Clinical Study Report to sit alongside emotional lability under a Nervous System heading in the 2001 paper?
It may be something of purist view but significance testing was originally linked to primary endpoints. Harms are never the primary endpoint of a trial and no RCT is designed to detect harms adequately. It is appropriate to hold a company or doctors who may be aiming to make money out of vulnerable people to a high standard when it comes to efficacy but for those interested to advance the treatment of patients with any medical condition it is not appropriate to deny the likely existence of harms on the basis of a failure to reach a significance threshold that the very process of conducting an RCT will mean cannot be met as investigators attention is systematically diverted elsewhere.
As regards RIAT methods, a key method is to stick to the protocol. A second safeguard is to audit every step taken and to this end we have attached a 61 page audit record (Appendix 1) to this paper. An even more important method is to make the data fully available, which it will be on Study329.org.
As regards ghostwriting, I personally am happy to stick to the designation of this study as ghostwritten. For those unversed in these issues, journal editors, medical writing companies and academic authors cling to a figleaf that if the medical writers name is mentioned somewhere, s/he is not a ghost. But for many, the presence on the authorship line of names that have never had access to the data and who cannot stand over the claims made other than by assertion is what’s ghostly.
Having made all these points, there is a point of agreement to note. Dr Keller and colleagues state that:
“nor do we think there is any consensus in the field that would allow the authors of this reanalysis or any other potential reanalysis to definitively say they got it right”.
We agree. For us, this is the main point behind the article. This is why we need access to the data. It is only with collaborative efforts based on full access to the data that we can manage to get to a best possible interpretation but even this will be provisional rather than definitive. Is there anything that would hold the authors of the second interpretation of these data (Keller and colleagues) back from joining with us the authors of the third interpretation in asking that the data of all trials for all treatments, across all indications, be made fully available? Such a call would be consistent with the empirical method that was as applicable in 1991 as it is now.
David Healy
Holding Response on Behalf of RIAT 329
Heel diggers will dig in their heels.
I guess we should be thankful in the age of “thug” science that Healy et al didn’t get sued or prosecuted under RICO for dissent:
http://dailycaller.com/2015/09/17/scientists-ask-obama-to-prosecute-global-warming-skeptics/#ixzz3m6i5gZfj
Where are the women? Is heel digging a Y-linked trait?
Kiwi in the responses on Healy’s blog makes an excellent point…six women Ph.Ds and M.D.s whose names were on the original study did not join the nine men who wrote the letter. Would love to hear their take.
That’s an interesting observation. Where are the women?
One of the women has died – Elizabeth Weller, from cancer a few years ago. She was a no-nonsense clinical investigator who contributed patients to the study but who was probably dialed out of access to the data. I suspect that was true for a number of the listed academic authors. Back in those days the practice was for the corporation to declare ownership of the data. At best, individual investigators might get access to data just from their own site. I was always uncomfortable with the notion that site investigators were granted scientific authorship. It may have seemed like a win-win arrangement but some of these listed authors I am sure wish they had never consented to be so trusting.
Just as a general point of strategy, another approach would have been to open the response with a direct assault on Keller’s weakest and most astonishing statement:
“The RIAT approach may have general “recommendations” but we find no evidence that there is a consensus on precisely how such a RIAT analysis makes the myriad decisions inherent in any reanalysis nor do we think there is any consensus in the field that would allow the authors of this reanalysis or any other potential reanalysis to definitively say they got it right.”
I thought Mickey answered this best in his post on Thursday:
“In our RIAT Team’s reanalysis of Study 329, we had decided to follow the a priori protocol, which meant sticking to the protocol defined outcome variables and ignoring those later exploratory variables…”
RIAT used the scientific method, the authors of the study… well, they wound up doing something else.
Keller seems to be throwing up a barrage of different criticisms of the report, as if all of them were equally important. There’s a very natural impulse to respond in kind, to jump down that rabbit hole.
Yet another approach would be to simply not respond to Keller’s letter at all. The report is very strong, it has generated tremendous interest and publicity. Keller letter is weak; he sounds defensive and circumlocutious.
The danger is that policymakers, legislators, students, and busy clinicians might be lulled into the illusion that this is an academic debate about arcane points in methodology.
If I were doing PR for GSK right now, or if my name were on the original study? That’s exactly what I would want anyone following this debate to believe.
The next obvious step is for any journalist is to interview the original authors who elected not to circle the wagons.
The letter fails no matter what rhetorical rationalizations he employs simply because not everyone under scrutiny is on board. Maybe the women have chosen to go John Dean. Their silence speaks louder than his BS.
Plus, throw in a smear or two about COI involving David Healy and Jon Jureidini. You’re right – Keller is just throwing chaff into the discussion.
What are the characterological traits of the personality disordered?
It’s what you expect from people like these authors of 329.
Forget the silence from other authors, what about the pervasive silence from most in psychiatry in general!?
And to bring it to the present tense, why are Psychiatrists still writing for paxil irregardless of children or adults??!!
Joel Hassman>>> I wish I was trained in handing out psychiatric disorders according to DSM-5, I bet you I could file all authors of study 329 (the original) under some.
Yes, why doesn’t the psychiatrist on “main street USA” speak up about how they’ve been duped? (or elsewhere in the World) Why are they quiet? Because many of them are just as fooled as we.
Yes, why are we still allowing the prescription of Paxil?
Not to deny those who say they felt better on them, but to prevent more people from suffering their harms.
We would infact save lives that way.
Who is stopping any conscientious, ethically minded psychiatrist from entering this debate via a public forum? That is the only nuance here that gives one cause to hope for change. This scandal is in the public domain– from day one.
Would be great to see psychiatrist’s going to media and to political leaders demanding that this be appropriately addressed as the threat to public safety– that it most definitely is.
The voices of the insiders. All who have used on-line forums to connect and validate each other’s perceptions– Where are these voices?
It would be a strong statement of validity and a quick remedy to redirect the public from the deception that this is an academic debate.
I can’t imagine any news or media source of substance who would deny psychiatrists their opportunity to weigh in on this.
Perhaps, the forum for putting this right is outside of the APA, the regulatory agencies,academic psychiatry, the pharmaceutical companies? This has been my long held belief ;that accessing both political and criminal justice elements in our society is the way forward. I am merely stating the obvious– none of these established entities have responded with even the slightest acknowledgement of a problem within its ranks.
The brilliance of study329.org speaks to lived experience working inside of a corrupt system. I speak for a large group of activists when I say that at this juncture, the only *good* psychiatrist is one who is using his/her credentials to give more credence to the need to criminalize these acts — by psychiatrists and pharma–.
Strictly speaking, what other action could a *good* psychiatrist take?
Overidealization and devaluation is the relationship pattern of a hysteric–rinse, lather, repeat. Organized psychiatry’s loving embrace of the next quasiproven great thing is it’s version of “my newest alcoholic boyfriend won’t hurt me”. It’s the most exciting time ever according to the Psychiatric Times, despite this admission:
http://www.psychiatrictimes.com/blogs/couch-crisis/serotonin-how-psychiatry-got-over-its-high-school-crush
So why does institutional academic psychiatry choose this immature attitude toward new treatments over something more scientific and mature, like, um, guarded optimism and rational skepticism? See my posts about ketamine.
It’s not like this pattern began and ended with SSRIs. See EMDR. recovered memories, etc.
At the end of the day, most of us are complicit to the demise of mental health in general, Psychiatry more specifically, but, medicine as a whole has been damaged irrevocably by what this society is based on today. It is been a collaborative of providers, patients, politicians, insurers, other allied professionals who work in healthcare, and even the families of the patients, they all are part of the problem
We are a quick fix nation, people have near zero frustration tolerance, have little ability to be empathetic, much less attentive to what’s going on around them, and people have no desire to responsibly problem solve. They all want a pill that reinforces one is fixed yesterday, & I think a lot of the people who are anti psychiatry invariably practice this crap and then try to project onto us, the providers.
So, as I’ve said before, the Michael Jackson “song man in the mirror”, many in this society need to look in the mirror and accept it is a flawed and imperfect view before you start complaining how everybody else is the bad guy.
We have become a nation of bad guys because we have no clue that we are part of a big picture. Nope, too many just look in the mirror and think the image is the picture. The politicians and celebrities many worship speak volumes for what they represent.
But, it doesn’t take a rocket scientist to figure this one out, does it?!
Keller 2015: “Restoring Study 329” asserts that this paper was ghostwritten, citing an early publication by one of the coauthors of that article. There was absolutely nothing about the process involved in the drafting, revision, or completion of our paper that constitutes “ghostwriting”. This study was initiated by academic investigators, undertaken as an academic / industry partnership, and the resulting report was authored mainly by the academic investigators with industry collaboration.
Keller 1999: https://industrydocuments.library.ucsf.edu/drug/docs/#id=rhfw0217
The prosecution rests.
Mickey already identified one of the female coauthors…I didn’t put this together at first because I forgot about this June article…but this is really really bad:
http://1boringoldman.com/index.php/2015/06/02/deeply-troubling/
What the hell could the AACAP membership be thinking here? Any clue how bad this looks?
“We are a quick fix nation, people have near zero frustration tolerance, have little ability to be empathetic, much less attentive to what’s going on around them, and people have no desire to responsibly problem solve. They all want a pill that reinforces one is fixed yesterday, & I think a lot of the people who are anti psychiatry invariably practice this crap and then try to project onto us, the providers.”
Problem solving how we got to be a nation hooked on the idea that pills can fix everything– including problems most of us needed to be told about– for our own good, is just one route to becoming anti-psychiatry. The most popular course now has its roots in the posts here over the past week. Many more concluding that Psychiatry cannot reform itself– and society cannot afford to tolerate the risks. You have to be really out of touch to miss the significance of this latest scandal being so readily accessible to the public.
This comment reads like PR from the APA’s problem solving think tank.
Yeh, a Bunch of celebrity worshipping anti-psychiatry zealots , trying to push the blame off on psychiatry, rather than admit it’s their fault they have been harmed, even killed by dangerous drugs prescribed for reified disorders. Yeh, that’s the ticket, turn the spotlight on the scumbags who have colluded to force psychiatrists to instantly fix even children in this horrendous health care system that, still makes millionaires of top academic psychiatrists— Oops!
No matter what group you want to blame, it’s funny how the worst of this scandal comes down to that which only a prominent academic psychiatrist can do–, and this is not what any society wants,
Suggesting providers take a stand that regains lost trust. Just a friendly suggestion, because our society is merely ignorant. Ignorance is treatable. It is already responding to the work of the RIAT team and those who see public education as a priority for changing what psychiatry just doesn’t seem to see as such a big deal.
Miss Higgins, at the risk of closing this thread, you need to be called on how clueless you really are at the end of the day.
If you think I’m a covert operative for Psychiatry to continue the status woe that it is, then, you just reinforce that regardless of what someone writes, as long as he or she has an MD after the name, they are complicit with a Psychiatry agenda.
And that’s why I think most of you anti Psychiatry folks are the characterological crowd I know you to be.
So, continue to sell your false message to everybody, some here buy it hook line and sinker, others, like me, know you to be the fraud you really are.
Sorry Mickey, this woman has tried to hijack my blog in the past, and I have banned her, so, she thinks she can attack me here as a sequel or an alternative?
Just remember folks, the characterological person only thinks in black and white, there is no gray. Thus, there can be no psychiatrist who can have a moderate or middle ground opinion. And then, if the thread is not closed, watch Miss Higgins react wildly to this!
Challenging the ideas and theories put forth in written statements posted on public forums is by no means a personal attack on the writer– whether the writer is the host of the blog, or just another commenter.
Wild accusations and character aspersions don’t address the issues raised in my comment on the quote I cited—. But it does bring up another issue.
Psychiatric diagnosis- labeling employed as a means to denigrate those whose opinions and perceptions threaten the status quo. Here, (above) this practice can be viewed as a venue to express open hostility toward those in a specific diagnostic category, (Axis II) which, by written accounts from this author (above) comprises most of our society, and all of those who oppose status woe that is psychiatry.
I can’t think of a single example from any other medical specialty , of a doctor who uses his professional credential as a weapon of mass destruction…