It’s always funny when small children try to play hide-and-go-seek by covering their eyes, but when grown-ups do it, it loses its charm. That’s what Dr. Andres Martin’s has done in his response to Dr. Juriedini’s request that the Journal of the American Academy of Child and Adolescent Psychiatry retract the 2001 Study 329 article. They had a perfect chance to do the right thing. They declined to take it.
|
Thank you for your Letter to the Editor, submitted July 20, 2012, regarding Keller et al., 2001. Following the June 27, 2012 settlement between GlaxoSmithKline and the U.S. Department of Justice, the Journal’s editorial team undertook a thorough evaluation of the article, the legal settlement, and related materials. The authors of the article were contacted and asked to respond to the questions and concerns raised by the settlement. After a comprehensive and extensive review, the Journal editors found no basis for retraction or other editorial action.
Due to the nature of the concerns and serious consideration given to the situation, the evaluation process was quite lengthy, and we appreciate your patience while the editorial team conducted its review. The inquiry is considered complete, and as such, your letter will not be published in the Journal. |
I had also contacted the American Academy of Child and Adolescent Psychiatry prior to their recent yearly meeting. Rather than write the Journal, I wrote the outgoing president, the incoming President, and contacted the Ethics Committee. All responded cordially and assured me convincingly that the matter was under review. So I think I was more hopeful than most about what they would do. My logic was that the American Academy of Child and Adolescent Psychiatry itself was responsible for its official journal, but as we now see, it remained in the hands of the journal.
"The authors of the article were contacted and asked to respond to the questions and concerns raised by the settlement." It goes without saying that contacting the authors seems an odd way to go about an investigation, particularly these authors. For example, there’s a deposition of Martin Keller about this study available on the Internet. It’s 125 pages long, but easy to summarize: "If you think I did something wrong, you’re wrong because I’ve never done any wrong things, and I don’t specifically remember anything I’ve ever done." That may sound facetious, but if you have the stomach to read it through, you’ll agree with my assessment. It’s maddening. Neal Ryan, the second author is heading up the Back to the Future Project which maps the road ahead for the AACAP [see this comment]. Karen Dineen Wagner, Boris Birmaher, and Graham Emslie continue to grind out articles about psychopharmacology in children and are prominent in AACAP affairs. I doubt that anyone on the author list was excited about the embarrassment of a retraction, but we already knew that. So did the Editor, Dr. Andres Martin.
In this particular case, there are no facts in question. It was a negative trial, declared negative by the people who did it. The paper was ghost-written and reviewed by the sponsor before any of the twenty two authors ever saw a manuscript. The science was jury-rigged to imply a positive outcome where none was supported using well-documented sleight of hand. None of that is speculative. And the article has been a centerpiece for court settlements worth billions of dollars. Yet the Journal of the American Academy of Child and Adolescent Psychiatry says "the Journal editors found no basis for retraction or other editorial action."
Dr. Goldacre,
If journal editors find that the affair of the 2001 Keller Journal of the American Academy of Child and Adolescent Psychiatry paper provides no basis for any editorial action, much less retraction, then what ever will?
You write in your book “Bad Pharma” (as published in Great Britain) on pages 204-205, in the Section “Trials that change their main outcome after they’ve finished”, in “Chapter 4: Bad Trials”:
“For one final illustration of what this means in practice, I shall return to paroxetine and the studies that were conducted in children. Remember, when an area of medicine is subject to some kind of litigation, documents often become available to researchers that would otherwise be hidden from view, allowing them to identify problems, discrepancies and patterns that would not normally be detectable. For the most part these are documents which should always be in the public domain, but are not. So paroxetine may not be worse than any other drug for this kind of mischief (in fact, as we have seen from the study just described, outcome switching happens across the board): it’s simply one of the cases about which we have the most detail.
In 2008 a group of researchers decided to go through the documents opened up by the litigation over paroxetine, and examine how the results of one clinical trial – ‘trial 329’ – had been published.26 As last as 2007 systematic reviews were still describing this trial as having a positive result, which is how it was reported in publications of its results. But in reality that was completely untrue: the original protocols specified two primary outcomes and six secondary ones. At the end of the trial there was no difference between paroxetine and placebo for any of these outcomes. At least nineteen more outcomes were also measured, making twenty-seven in total. Of those, only four gave a positive result for paroxetine. These positive findings were reported as if they were the main outcomes.
It would be tempting to regard the reporting of trial 329 as some kind of freak episode, an appalling exception in an otherwise sane medical world. Tragically, as the research above demonstrates, this behaviour is widespread.
So widespread, in fact, that there’s room for a small cottage industry, if there are any academics feeling brave enough to pursue the project. Someone somewhere needs to identify all the studies where the main outcomes have been switched, demand access to the raw data, and helpfully, at long last, conduct the correct analyses for the original researchers. If you choose to do this, your published papers will immediately become the definitive reference on these trials, because they will be the only ones to correctly present the pre-specified trial outcomes. The publications from the original researchers will be no more than a tangential and irrelevant distraction.
I’m sure they’ll be pleased to help.”
“Notes” from “Chapter 4: Bad Trials” on page 394:“26 Jon N. Jureidini, Leemon B. McHenry, Peter R. Mansfield. Clinical trials and drug promotion: Selective reporting of study 329. International Journal of Risk & Safety in Medicine 20 (2008) 73-81 DOI 10.3233/JRS-2008-0426”
This is how the paper and Jureidini’s 2003 letter (and the authors’ reply) currently appears to anyone searching Pubmed:http://www.ncbi.nlm.nih.gov/pubmed/11437014
http://www.ncbi.nlm.nih.gov/pubmed/12707552
Here is a further discussion based on the original data which, as you can see, resides on this blog:http://1boringoldman.com/index.php/2012/08/29/the-lesson-of-study-329-data-transparency/
http://1boringoldman.com/index.php/2012/09/04/the-lesson-of-study-329-naked-emperors-fractious-queens/
http://1boringoldman.com/index.php/2012/09/06/the-lesson-of-study-329-were-only-as-sick-as-our-secrets/
You write on page 295, in the Section “Ghostwriting,” in “Chapter 6: Marketing”:“What happens once a paper is in progress? For this, let’s switch to another study, on an antidepressant called paroxetine. You can read all of these documents and more at the Drug Industry Document Archive, build by the University of California, San Francisco, to house materials released during legal cases involving the pharmaceutical industry.83 Professor Martin Keller of Brown University is discussing the content of ‘his’ paper with a PR person working for the drug company GSK: ‘You did an superb job with this, thank you very much. It is excellent. Enclosed are some rather minor changes from me.’84
The ghostwriter gets back to him with everything nicely organized and ready to go, because, of course, the academic must be the one who sends the paper to the journal.85 You will remember the earlier description of what a time-consuming hassle it is for an academic to pull a paper together and submit it to a journal themselves. When you’re working with GSK, this is all rather more straightforward: ‘Please retype on your letterhead and revise as you like.’”
“Notes” from “Chapter 6: Marketing” on page 403:
83 Drug Industry Document Archive [Internet]. [cited 2012 March 24]. Available from” http://dida.library.ucsf.edu/
84 Drug Industry Document Archive – Search Results [Internet]. [cited 2012 March 24]. Available from: http://dida.library.ucsf.edu/tid/anu38h10
85 Ibid.
You write on pages 342-343, in “Afterword: Better Data”:“So I am confident that you will agree, from the evidence set out in this book, that these are systemic problems; and that it would be shameful, or even dishonest, simply to dismiss them. What’s more, where the evidence is lacking – and this isn’t often – I have been clear, and I have set out what work is needed to fill those gaps. For example: lectures from key opinion leaders paid by industry are one of the most significant ways in which qualified doctors are educated today, and two decades ago ‘mystery shopper’ research found that these lectures are systematically biased. The fact that this work hasn’t been repeated in the past five years should be a source of shame for the industry and for my profession. It’s not a cause for celebration, and it certainly doesn’t exonerate anyone.
As their next tactic, we can be sure – because we’ve watched them do this already – that people from industry will point to their guidelines. Look at all these miles and kilograms of rules, these vast offices filled with regulators: this is one of the most closely monitored industries in the world, they will say, drowning in red tape. But we have proved, I think, that these regulations simply do not do their job. The rules on registering trials were ignored; the FDA rules on posting results within a year have only been obeyed for a fifth of trials; the ICMJE regulations on ghostwriting – absurdly – permit ghostwriting; and so on. These regulations have been tested, and they have been shown to fail.
But the most dangerous tactic of all is the industry’s enduring claim that these problems are all in the past. This is deeply harmful because it repeats the insult of all the fake fixes we have seen throughout this book: and it repeats the insult of all the fake fixes we have seen throughout this book: and it is this recurring pattern of flat denial that allows the problems to persist.
The clearest window onto this strategy comes from the industry’s response to its most recent public scandal. In July 2012, GSK received a $3 billion dollar fine for civil and criminal fraud, after pleading guilty to a vast range of charges around unlawful promotion of prescription drugs, and failure to report safety data. The full list of charges and evidence in vast – you can browse it all at the Department of Justice website – but the methods they used will be very familiar to you by now.
GSK bribed doctors with gifts and hospitality; it paid doctors millions of dollars to attend meetings, and to speak at them, in lavish resorts; it used, in the justice department’s own words, ‘sales representatives, sham advisory boards, and supposedly independent Continuing Medical Education (CME) programs.’ It withheld data on the antidepressant paroxetine.”
Dr. Goldacre, the official response of the editorial team of the JAACAP speaks for itself.
It would seem hard to catalyze change if even the most egregious, and most rigorously documented, examples in the literature can continue to be deemed by editors to not be worthy of any action on the part of the originating journals.
What message does it send if the response provided in Dr. Martin’s letter is considered acceptable current standard practice?
What message does it send if the debate of such a response is not provided a broader forum?
It goes without saying that contacting the authors seems an odd way to go about an investigation, particularly these authors.
It happens not only in psychiatry. Many years ago, when synthetic insulin was being forced on an unsuspecting diabetic population, a reasonable request to our elected representative yielded almost identical results. Informing this representative that not ALL diabetics could easily or successfully switch from natural to synthetic insulin, and that their very lives were at risk; and asking him to investigate the possibility that ALL natural insulins might soon disappear brought the following result from his office. “We contacted Eli Lilly and they assured me that natural insulins would remain available (an outright lie), and that they had no reports of serious adverse events from users of synthetic insulin (another lie), and that synthetic insulin was identical to natural insulin (a final lie).” Investigation complete–no worries!
The examples set by editorial decision making at JAACAP seem of particular importance in light of the role the editor-in-chief plays in shaping child psychiatry, and in how much they represent the scientific voice of the main professional society for child psychiatrists:
JAACAP is the premier journal in child mental health.
AACAP Presidents have characterized the role of the editor as “the pathfinder, steward, and trail maker for the science of child and adolescent psychiatry” and “”the keeper of science” for child and adolescent psychiatry.”
AACAP by-laws indicate that editors of JAACAP serve 5 year terms with the one-time possibility of an additional 5 year renewal. Therefore, if Dr. Martin is (has been?) renewed for another 5 year term by the AACAP, he will hold that role until 2018. As he had said: “Mina Dulcan, M.D. has set the precedent and I will seek to follow in her example.”
From the original announcement:
WASHINGTON, June 22 /U.S. Newswire/ — The American Academy of Child and Adolescent Psychiatry (AACAP) named Andres Martin, M.D., M.P.H. the next editor of the Journal of the American Academy of Child and Adolescent Psychiatry (JAACAP). Dr. Martin is currently one of the JAACAP’s assistant editors and will become editor in 2008.
“We are pleased to have Dr. Martin become the “keeper of science” for child and adolescent psychiatry,” says AACAP President Thomas Anders, M.D. “Dr. Martin has demonstrated creativity and diligence throughout his career, both in his clinical and editorial roles. His high standards will steer the JAACAP on a proven course of excellence to even greater heights.”
….
“I am thrilled to have an opportunity to serve a publication that has been so closely associated with distinction in research. Mina Dulcan, M.D. has set the precedent and I will seek to follow in her example,” says Dr. Martin. “The JAACAP is the authority in scholarship on children’s and adolescents’ mental illnesses, and bringing the publication forward to serve the changing needs of families and clinicians is a wonderful challenge.”
Dr. Martin becomes editor in January 2008. Mina Dulcan, M.D., has been Editor of JAACAP since 1998. Under Dr. Dulcan’s tenure, the JAACAP has been ranked first in the Science Citation Index’s impact factor for children’s mental health and in the top three of 68 journals ranked in pediatrics. She oversaw the addition of the online version of the journal and the implementation of a web-based system for manuscript submission and peer review.
…
Representing over 7,400 child and adolescent psychiatrists nationwide, the American Academy of Child and Adolescent Psychiatry is the leading authority on children’s mental health. AACAP members actively research, diagnose, and treat psychiatric disorders affecting children, adolescents, and their families. Visit AACAP at http://www.aacap.org
/© 2006 U.S. Newswire 202-347-2770/
The assessment by AACAP’s leadership of Dulcan’s 1997-2007 tenure (which would have included a renewal in 2002 around the time of her original responses to Jureidini when he first began to raise questions about study 329 in the JAACAP 10 years ago) included the following:
“Her decisions and leadership were always thoughtful and measured. She chose her battles well, stood her ground, and let the science shine through.”
http://psycnet.apa.org/psycinfo/2007-18374-001
It appears that Dr. Dulcan and Dr. Martin have provided the mainstay of the AACAP’s official response to questions raised about Study 329 and as such can be considered to represent the views of AACAP’s membership. I am aware of no one represented by AACAP or JAACAP who has questioned the belief that Study 329 and the resultant 2001 paper require no further editorial comment than that provided by Dr. Martin in his recent letter to Dr. Jureidini and that of Dr. Dulcan in 2004 in the following exchange with a reporter:
SJ: Do you have no regrets about publishing the study?
Mina Dulcan: I don’t have any regrets about publishing. It generated all sorts of useful discussion which is the purpose of a scholarly journal. The purpose of a scholarly journal is not to tell people what to do. The purpose of a scholarly journal is to put out the data…
SJ: Are you aware that 329 was ghost written? Does it worry you, do you think it matters?
MD: Well, certainly if I were an author I would not put my name on anything that I didn’t feel was accurate. I can’t speak to what those authors, to the extent, how much they saw the data. Someone can write something and you may or may not agree with it. The fact that someone puts the words together may be a good thing or a bad thing depending on what the words are…
If anyone represented by AACAP and JAACAP feel that the scholarship one should expect from JAACAP, and from the peer reviewed literature in medicine as a whole, differs in important ways from the example set by the handling of the 2001 Keller paper, and that that merits further substantive commentary, I have never seen it expressed.
I concur with 1BOM that it would provide a service to show that medicine itself can express substantively such differing and higher expectations for the future. In ways that would withstand the scrutiny of individuals such as Dr. Goldacre. And, as importantly, withstand the scrutiny that a wider set of stakeholders may eventually bring.
Seeing the leadership of AACAP and JAACAP embrace the true spirit of the message of Dr. Goldacre’s message would be an important start, even if they choose to not comment on earlier practices. If such statements indicating a new way forward are present in Dr. Martin’s response they are hard to discern.
Dr. Goldacre puts forward “But the most dangerous tactic of all is the enduring claim that these problems are all in the past. This is deeply harmful because it repeats the insult of all the fake fixes we have seen throughout this book: and it is this recurring pattern of flat denial that allows the problems to persist.”
What happens to the impact of that message if simply engaging in “flat denial,” with vague references to extensive and serious consideration, turns out to be such an effective strategy?
In 2004 the authors were contacted by GSK about publishing material that might cast doubt on Study 329 and the 2001 paper.
Here is the correspondence between the main academic authors:
Correspondence involving Ryan, Keller, Wagner (Emslie?) responding to GSK’s possible interest in distancing themselves from Study 329’s Initial Presentation of Information.(2004)
http://www.healthyskepticism.org/files/docs/gsk/paroxetine/study329/20040204RyantoKellerStrober.pdf
http://www.healthyskepticism.org/files/docs/gsk/paroxetine/study329/040514KellertoCarp.pdf
http://www.healthyskepticism.org/files/docs/gsk/paroxetine/study329/20040413KellertoRyanetal.pdf
Keller states in the June 2004 to Ryan, Strober, Emslie, and Wagner:
“we are not without leverage if we are dissatisfied and want to play a little hardball of our own.”
“if we want to fight that battle we will have to go to senior management. I am willing to do that if we agree that after the changes are made to this draft we are still hung out to dry.”
“so that it is 100% clear in this paper that there is no way to read it and think that 329 is being criticized and that it was not written with complete integrity and accuracy given the data we had and should have had as investigators, on the part of investigators and our collegues from SK who worked on it.”
“otherwise we would look foolish, naive, incompetent, or “biased” (the most likely accusation that will be made) to present things in a way that was favorable to SK, disregarding our responsibility to the proper scientific method, to the public, children and their families.”
In 2012 Dr. Martin writes that “The authors of the article were contacted and asked to respond to the questions and concerns raised by the settlement.”
We know that in the end after extensive consideration GSK did not comment unfavorably.
Obviously we do not have access to the correspondence that ensued.
We know that in the end after extensive consideration JAACAP did not comment unfavorably.
Is this apparent focus on never conceding any question of past mistakes really the standard of science and scholarship people want to promote? The best way forward?
Beyond the desires of GSK marketing, was the data there to promote the use of Paxil as first line in adolescent depression? Ahead of Prozac and Zoloft? It can easily be argued that that is the primary purpose served by the 2001 paper. Had it been presented as a negative study, as opposed to being presented this way:
“CONCLUSIONS: Paroxetine is generally well tolerated and effective for major depression in adolescents.”
http://www.ncbi.nlm.nih.gov/m/pubmed/11437014/
Is it not most likely that the result would have been to pick Prozac or Zoloft ahead of Paxil for a severely depressed adolescent. Other than for GSK, would that have been a problematic outcome?
“that there is no way to read it and think that 329 is being criticized and that it was not written with complete integrity and accuracy given the data we had and should have had as investigators, on the part of investigators and our collegues from SK who worked on it.” That correspondence between Keller, Ryan, Wagner, et al pretty much says it all. Again, is that the standard people want for the field? That the standard bearers want for the field?
Dr. Goldacre,
It would be very informative if someone were to organize a thoughtful and collegial debate about the legacy of Study 329 and the 2001 paper with Drs. Martin, Ryan, and Drell on one side and Drs. Godlee, Jureidini, and Goldacre on the other. This coming close to 10 years after the original letters about 329 were published in JAACAP. Given that Dr. Ryan’s research work group will be in October 2013, as part of Back to Project Future, presenting its recommendations for the direction of child psychiatry research for the next 10 years it could not come at a better time. Indications are that a main focus will be on staging of very young children with the goal of prophylactic paychopharmacology prior to the development of full symptoms. Through primary care providers. Drs. Ryan and Martin in their respective roles will have powerful voices in how that work plays out over the next 5-10 years.
From the email between Drs. Keller and Ryan and other authors: “that there is no way to read it and think that 329 is being criticized and that it was not written with complete integrity and accuracy”
Let us say we presume that Drs. Ryan and Martin consider 329 and the resultant 2001 JAACAP paper to have been “written with complete integrity and accuracy,” and that there is no basis whatsoever for further editorial comment.
If that can stand without triggering further discussion in larger forums, including those that would reach their peers, then what does that say about what is coming in the next 5-10 years? What does that say to those who may agree with Dr. Goldacre but wonder if he is encouraging a hopeless quest? That engaging in a public, transparent, substantive, and reciprocal (and those are key) thoughtful and collegial discussion can so easily be avoided in an example that was arguably one of the main poster children for the need for transparency?
I believe that Dr. Goldacre is interested in more than just selling books and hope I am not just being naive.
This was from a recent review of Dr. Goldacre’s book:
“Goldacre’s work, then, is a lightning conductor for much that has come before. But more than that, Big Pharma is a campaigning work, designed to catalyse a movement and leverage access to the corridors of power. Goldacre has also taken to social media to reinforce the message of his book, keeping the momentum building.
So far this strategy has worked spectacularly well. The Emperor’s New Drugs was every bit as damning as Big Pharma, but it might as well have been published in Antarctica to an audience of penguins for the impact it had on the general public. Big Pharma, on the other hand, has been unanimously endorsed and promoted by the British broadsheet newspapers.”
http://theconversation.edu.au/peer-review-bad-pharma-by-ben-goldacre-9940
Until journals and professional organizations feel a need to engage around these issues it seems clear that they will simply avoid discussing them and/or with trumpet fake fixes. It is hard then to believe that the next 10 years will being something substantively different than the last.
Except, perhaps, that the children being impacted may be much younger.