Posted on Friday 14 October 2016
a crutch for the crab
by Lisa Schwartz, Steven Woloshin, Eugene Zheng, Tony Tse, and Deborah ZarinAnnals of Internal Medicine. 2016 165:421-430.
Background: Pharmaceutical companies and other trial sponsors must submit certain trial results to ClinicalTrials.gov. The validity of these results is unclear.Purpose: To validate results posted on ClinicalTrials.gov against publicly available U.S. Food and Drug Administration [FDA] reviews on Drugs@FDA.Data Sources: ClinicalTrials.gov [registry and results database] and Drugs@FDA [medical and statistical reviews].Study Selection: 100 parallel-group, randomized trials for new drug approvals [January 2013 to July 2014] with results posted on ClinicalTrials.gov [15 March 2015].Data Extraction: 2 assessors extracted, and another verified, the trial design, primary and secondary outcomes, adverse events, and deaths.Results: Most trials were phase 3 [90%], double-blind [92%], and placebo-controlled [73%] and involved 32 drugs from 24 companies. Of 137 primary outcomes identified from ClinicalTrials.gov, 134 [98%] had corresponding data at Drugs@FDA, 130 [95%] had concordant definitions, and 107 [78%] had concordant results. Most differences were nominal [that is, relative difference <10%]. Primary outcome results in 14 trials could not be validated. Of 1927 secondary outcomes from ClinicalTrials.gov, Drugs@FDA mentioned 1061 [55%] and included results data for 367 [19%]. Of 96 trials with 1 or more serious adverse events in either source, 14 could be compared and 7 had discordant numbers of persons experiencing the adverse events. Of 62 trials with 1 or more deaths in either source, 25 could be compared and 17 were discordant.Limitation: Unknown generalizability to uncontrolled or cross-over trial results.Conclusion: Primary outcome definitions and results were largely concordant between ClinicalTrials.gov and Drugs@FDA. Half the secondary outcomes, as well as serious events and deaths, could not be validated because Drugs@FDA includes only "key outcomes" for regulatory decision making and frequently includes only adverse event results aggregated across multiple trials.
But they just didn’t do it. Most trials were registered but often long after they started, and the ClinicalTrials.gov Results Database was the loneliest database on the Internet. There was no enforcement, even from Journal Editors who also claimed to require it. I’ve collected a few of the REFERENCES below that document the problems from several different angles. But even done right [Registered a priori with Results submitted], there are still difficulties. So this system created to promote public transparency just hasn’t worked, and the consequences have become the stuff of legends. Recently, the NIH, ClinicalTrials.gov, and the FDA rolled out a set of reforms designed to correct the things in the ClinicalTrials.gov system. While the effort is appreciated, it was primarily focused on compliance and enforcement – more a commitment to actually do what was supposed to happen in 2007, but uninformed by the multiple real problems in the system today. For example, nobody checks the submitted information, and the deadlines for submission mean that, at best, the information comes after it’s actually needed. On-time is too-late.
There is no reason for any lag time in posting the needed information at the time of Registration or Submission. They have it at hand, and we need it. Editors, Peer Reviewers, readers, everyone has the right to see the results certified by the responsible party. That responsible party is the FDA. They are part of the problem in colluding with industry to keep the raw data secret. Perhaps they have no choice. But that sure doesn’t apply to the a priori Protocol or the Protocol-directed Results. So sign the The Petition now and send it to everyone you can think of. It’s time for Congress to act!
by Mathieu S, Boutron I, Moher D, Altman DG, and Ravaud P.JAMA. 2009 302:977-984.
Context: As of 2005, the International Committee of Medical Journal Editors required investigators to register their trials prior to participant enrollment as a precondition for publishing the trial’s findings in member journals.Objective: To assess the proportion of registered trials with results recently published in journals with high impact factors; to compare the primary otcomes specified in trial registries with those reported in the published articles; and to determine whether primary outcome reporting bias favored significant outcomes.Data Sources: MEDLINE via PubMed was searched for reports of randomized controlled trials [RCTs] in 3 medical areas [cardiology, rheumatology, and gastroenterology] indexed in 2008 in the 10 general medical journals and specialty journals with the highest impact factors.Data Extraction: For each included article, we obtained the trial registration information using a standardized data extraction form.Results: Of the 323 included trials, 147 [45.5%] were adequately registered [ie, registered before the end of the trial, with the primary outcome clearly specified]. Trial registration was lacking for 89 published reports [27.6%], 45 trials [13.9%] were registered after the completion of the study, 39 [12%] were registered with no or an unclear description of the primary outcome, and 3 [0.9%] were registered after the completion of the study and had an unclear description of the primary outcome. Among articles with trials adequately registered, 31% [46 of 147] showed some evidence of discrepancies between the outcomes registered and the outcomes published. The influence of these discrepancies could be assessed in only half of them and in these statistically significant results were favored in 82.6% [19 of 23].Conclusion: Comparison of the primary outcomes of RCTs registered with their subsequent publication indicated that selective outcome reporting is prevalent.
by Joseph S. Ross, Gregory K. Mulvey, Elizabeth M. Hines, Steven E. Nissen, and Harlan M. KrumholzPLoS Medicine. 2009 6: e1000144.
Background: ClinicalTrials.gov is a publicly accessible, Internet-based registry of clinical trials managed by the US National Library of Medicine that has the potential to address selective trial publication. Our objectives were to examine completeness of registration within ClinicalTrials.gov and to determine the extent and correlates of selective publicationMethods and Findings: We examined reporting of registration information among a cross-section of trials that had been registered at ClinicalTrials.gov after December 31, 1999 and updated as having been completed by June 8, 2007, excluding phase I trials. We then determined publication status among a random 10% subsample by searching MEDLINE using a systematic protocol, after excluding trials completed after December 31, 2005 to allow at least 2 y for publication following completion. Among the full sample of completed trials [n=7,515], nearly 100% reported all data elements mandated by ClinicalTrials.gov, such as intervention and sponsorship. Optional data element reporting varied, with 53% reporting trial end date, 66% reporting primary outcome, and 87% reporting trial start date. Among the 10% subsample, less than half [311 of 677, 46%] of trials were published, among which 96 [31%] provided a citation within ClinicalTrials.gov of a publication describing trial results. Trials primarily sponsored by industry [40%, 144 of 357] were less likely to be published when compared with nonindustry/nongovernment sponsored trials [56%, 110 of 198; p<0.001], but there was no significant difference when compared with government sponsored trials [47%, 57 of 122; p=0.22]. Among trials that reported an end date, 75 of 123 [61%] completed prior to 2004, 50 of 96 [52%] completed during 2004, and 62 of 149 [42%] completed during 2005 were published [p=0.006].Conclusions: Reporting of optional data elements varied and publication rates among completed trials registered within ClinicalTrials.gov were low. Without greater attention to reporting of all data elements, the potential for ClinicalTrials.gov to address selective publication of clinical trials will be limited.
Despite Law, Fewer Than One In Eight Completed Studies Of Drugs And Biologics Are Reported On Time On ClinicalTrials.govby Michael R. Law, Yuko Kawasumi, and Steven G. MorganHealth Affairs. 2011 30:2338-2345.
Clinical trial registries are public databases created to prospectively document the methods and measures of prescription drug studies and retrospectively collect a summary of results. In 2007 the US government began requiring that researchers register certain studies and report the results on ClinicalTrials.gov, a public database of federally and privately supported trials conducted in the United States and abroad. We found that although the mandate briefly increased trial registrations, 39 percent of trials were still registered late after the mandate’s deadline, and only 12 percent of completed studies reported results within a year, as required by the mandate. This result is important because there is evidence of selective reporting even among registered trials. Furthermore, we found that trials funded by industry were more than three times as likely to report results than were trials funded by the National Institutes of Health. Thus, additional enforcement may be required to ensure disclosure of all trial results, leading to a better understanding of drug safety and efficacy. Congress should also reconsider the three-year delay in reporting results for products that have been approved by the Food and Drug Administration and are in use by patients.
by Deborah A. Zarin, Tony Tse, Rebecca J. Williams, Robert M. Califf, and Nicholas C. IdeNew England Journal of Medicine. 2011 364:852-860.
Background: The ClinicalTrials.gov trial registry was expanded in 2008 to include a database for reporting summary results. We summarize the structure and contents of the results database, provide an update of relevant policies, and show how the data can be used to gain insight into the state of clinical research.Methods: We analyzed ClinicalTrials.gov data that were publicly available between September 2009 and September 2010.Results: As of September 27, 2010, ClinicalTrials.gov received approximately 330 new and 2000 revised registrations each week, along with 30 new and 80 revised results submissions. We characterized the 79,413 registry and 2178 results of trial records available as of September 2010. From a sample cohort of results records, 78 of 150 [52%] had associated publications within 2 years after posting. Of results records available publicly, 20% reported more than two primary outcome measures and 5% reported more than five. Of a sample of 100 registry record outcome measures, 61% lacked specificity in describing the metric used in the planned analysis. In a sample of 700 results records, the mean number of different analysis populations per study group was 2.5 [median, 1; range, 1 to 25]. Of these trials, 24% reported results for 90% or less of their participants.Conclusions: ClinicalTrials.gov provides access to study results not otherwise available to the public. Although the database allows examination of various aspects of ongoing and completed clinical trials, its ultimate usefulness depends on the research community to submit accurate, informative data.
How often do US-based human subjects research studies register on time, and how often do they post their results? A statistical analysis of the Clinicaltrials.gov databaseby Christopher J GillBMJ Open. 2012 2:e001186
Context: The Food and Drug Administration Modernization Act of 1997 [FDAMA] and the FDA Amendment Act of 2007 [FDAAA], respectively, established mandates for registration of interventional human research studies on the website clinicaltrials.gov [CTG] and for posting of results of completed studies.Objective: To characterise, contrast and explain rates of compliance with ontime registration of new studies and posting of results for completed studies on CTG.Design: Statistical analysis of publically available data downloaded from the CTG website.Participants: US studies registered on CTG since 1 November 1999, the date when the CTG website became operational, through 24 June 2011, the date the data set was downloaded for analysis.Main outcome measures: Ontime registration [within 21 days of study start]; average delay from study start to registration; proportion of studies posting their results from within the group of studies listed as completed on CTG.Results: As of 24 June 2011, CTG contained 54?890 studies registered in the USA. Prior to 2005, an estimated 80% of US studies were not being registered. Among registered studies, only 55.7% registered within the 21-day reporting window. The average delay on CTG was 322 days. Between 28 September 2007 and June 23 2010, 28% of intervention studies at Phase II or beyond posted their study results on CTG, compared with 8.4% for studies without industry funding [RR 4.2, 95% CI 3.7 to 4.8]. Factors associated with posting of results included exclusively paediatric studies [adjusted OR [AOR] 2.9, 95% CI 2.1 to 4.0], and later phase clinical trials [relative to Phase II studies, AOR for Phase III was 3.4, 95% CI 2.8 to 4.1; AOR for Phase IV was 6.0, 95% CI 4.8 to 7.6].Conclusions: Non-compliance with FDAMA and FDAAA appears to be very common, although compliance is higher for studies sponsored by industry. Further oversight may be required to improve compliance.
by Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, and Platts-Mills TF.British Medical Journal. 2013 347:f6104.
Objective: To estimate the frequency with which results of large randomized clinical trials registered with ClinicalTrials.gov are not available to the public.Design: Cross sectional analysisSetting: Trials with at least 500 participants that were prospectively registered with ClinicalTrials.gov and completed prior to January 2009.Data Sources: PubMed, Google Scholar, and Embase were searched to identify published manuscripts containing trial results. The final literature search occurred in November 2012. Registry entries for unpublished trials were reviewed to determine whether results for these studies were available in the ClinicalTrials.gov results database.Main Outcome Sources: The frequency of non-publication of trial results and, among unpublished studies, the frequency with which results are unavailable in the ClinicalTrials.gov database.Results: Of 585 registered trials, 171 [29%] remained unpublished. These 171 unpublished trials had an estimated total enrollment of 299,763 study participants. The median time between study completion and the final literature search was 60 months for unpublished trials. Non-publication was more common among trials that received industry funding [150/468, 32%] than those that did not [21/117, 18%], P=0.003. Of the 171 unpublished trials, 133 [78%] had no results available in ClinicalTrials.gov.Conclusions: Among this group of large clinical trials, non-publication of results was common and the availability of results in the ClinicalTrials.gov database was limited. A substantial number of study participants were exposed to the risks of trial participation without the societal benefits that accompany the dissemination of trial results.
by Jessica E. Becker, Harlan M. Krumholz, Gal Ben-Josef, and Joseph S. RossJAMA. 2014 311:1063-1065.
The 2007 Food and Drug Administration [FDA] Amendments Act expanded requirements for ClinicalTrials.gov, a public clinical trial registry maintained by the National Library of Medicine, mandating results reporting within 12 months of trial completion for all FDA-regulated medical products. Reporting of mandatory trial registration information on ClinicalTrials.gov is fairly complete, although there are concerns about its specificity; optional trial registration information is less complete.1- 4 To our knowledge, no studies have examined reporting and accuracy of trial results information. Accordingly, we compared trial information and results reported on ClinicalTrials.gov with corresponding peer-reviewed publications.Methods: We conducted a cross-sectional analysis of clinical trials for which the primary results were published between July 1, 2010, and June 30, 2011, in Medline-indexed, high-impact journals [impact factor ≥10; Web of Knowledge, Thomson Reuters] and that were registered on ClinicalTrials.gov and reported results. For each trial, we assessed reporting of the following results information on ClinicalTrials.gov and corresponding publications and compared reported information in both sources: cohort characteristics [enrollment and completion, age/sex demographics], trial intervention, and primary and secondary efficacy end points and results. Results information was considered concordant if the described end point, time of ascertainment, and measurement scale matched. Reported results were categorized as concordant [ie, numerically equal], discordant [ie, not numerically equal], or could not be compared [ie, reported numerically in one, graphically in the other]. For discordant primary efficacy end points, we determined whether the discrepancy altered study interpretation. Descriptive analyses were performed using Excel [version 14.3.1, Microsoft].Results: We identified 96 trials reporting results on ClinicalTrials.gov that were published in 19 high-impact journals. For 70 trials [73%], industry was the lead funder. The most common conditions studied were cardiovascular disease, diabetes, and hyperlipidemia [n = 21; 23%]; cancer [n = 20; 21%]; and infectious disease [n = 19; 20%]. Trials were most frequently published by New England Journal of Medicine [n = 23; 24%], Lancet [n = 18; 19%], and JAMA [n = 11; 12%]. Cohort, intervention, and efficacy end point information was reported for 93% to 100% of trials in both sources [Table 1]. However, 93 of 96 trials had at least 1 discordance among reported trial information or reported results.
Among trials reporting each cohort characteristic and trial intervention information, discordance ranged from 2% to 22% and was highest for completion rate and trial intervention, for which different descriptions of dosages, frequencies, or duration of intervention were common.
There were 91 trials defining 156 primary efficacy end points [5 trials defined only primary safety end points], 132 [85%] of which were described in both sources, 14 [9%] only on ClinicalTrials.gov, and 10 [6%] only in publications. Among 132 end points described in both sources, results for 30 [23%] could not be compared and 21 [16%] were discordant. The majority [n = 15] of discordant results did not alter trial interpretation, although for 6, the discordance did [Table 2]. Overall, 81 of 156 [52%] primary efficacy end points were described in both sources and reported concordant results.
There were 96 trials defining 2089 secondary efficacy end points, 619 [30%] of which were described in both sources, 421 [20%] only on ClinicalTrials.gov, and 1049 [50%] only in publications. Among 619 end points described in both sources, results for 228 [37%] could not be compared, whereas 53 [9%] were discordant. Overall, 338 of 2089 [16%] secondary efficacy end points were described in both sources and reported concordant results.Discussion: Among clinical trials published in high-impact journals that reported results on ClinicalTrials.gov, nearly all had at least 1 discrepancy in the cohort, intervention, or results reported between the 2 sources, including many discordances in reported primary end points. For discordances observed when both the publication and ClinicalTrials.gov reported the same end point, possible explanations include reporting and typographical errors as well as changes made during the course of the peer review process. For discordances observed when one source reported a result but not the other, possible explanations include journal space limitations and intentional dissemination of more favorable end points and results in publications.
Our study was limited to a small number of trials that were not only registered and reported results, but also published in high-impact journals. However, because articles published in high-impact journals are generally the highest-quality research studies and undergo more rigorous peer review, the trials in our sample likely represent best-case scenarios with respect to the quality of results reporting. Our findings raise questions about accuracy of both ClinicalTrials.gov and publications, as each source’s reported results at times disagreed with the other. Further efforts are needed to ensure accuracy of public clinical trial result reporting efforts.
|Nobel Prize Literature 2016||50th Anniversary|
I’ve run out of ways to say how far away from mainstream medicine I drifted for the last 20 years of my medical career, or how much coming back to it felt like waking up 25 years later in a strange land. Every time I say that, it sounds melodramatic and exaggerated – but that’s not at all how it feels. For example, I had never directly heard the term evidence-based medicine used to describe a discipline or a movement. As a matter of fact, one of my earlier blogs was about encountering a particularly concrete example of EBM [Evidence-Based Medicine] and being dumbfounded by the article [Barriers to implementation of a computerized decision support system for depression: an observational report on lessons learned in "real world" clinical settings]. I ended up writing a little series of blogs trying to figure it out [evidence-based medicine I… 28 Jan 2011]. Ironically, it was in that same article that I encountered the first example of a multi-paragraph declaration of conflicts of interest [which was even more dumbfounding!].
Back then, I drew a little diagram to illustrate what I was reading. A structured interview [SCID] lead to a diagnosis [DSM-IV]. Enter that into Dr. Madhukar Trivedi‘s computer program and it spit out an evidence-based treatment informed by the latest RTCs [Randomized Clinical Trials]. The evidence-based treatments for depression provided were either the second generation antidepressants or CBT [Cognitive Behavior Therapy]. And then you went round and round the process hopefully iterating towards success. The actual article I was reading had Dr. Trivedi‘s reflections about why the clinicians ignored his system unless he was actively looking over their shoulder. I had no difficulty at all understanding that myself [but I’ll fight the temptation to say why yet again]. But that was then, and this is now.
Something else I encountered for the first time on awakening – the Cochrane Collaboration and their Systematic Reviews. Founded in 1993, it’s a virtual army of thousands of volunteer scientists who publish comprehensive meta-analyses of RCTs including an evaluation of the scientific rigor of each trial. It’s an invaluable resource that counteracted some of my horror on discovering so many jury-rigged RCTs corrupting the medical literature, particularly in my specialty of psychiatry.
Archibald Leman Cochrane [12 January 1909 – 18 June 1988] was a Scottish doctor noted for his book Effectiveness and Efficiency: Random Reflections on Health Services. His advocacy of randomized controlled trials eventually led to the development of the Cochrane Library database of systematic reviews, the establishment of the UK Cochrane Centre in Oxford and the international Cochrane Collaboration.
Who would want to argue with the idea of looking for a scientific way to answer questions in medicine? Or asking "what is the evidence?" when someone suggests some course of action. What’s the alternative? But who could’ve imagined that simple ideas like Randomized Clinical Trials [RCTs] or Evidence-Based Medicine [EBM] could’ve become vehicles for so much corruption in some segments of medicine? creating a virtual superhighway for the commercial contamination if our literature. This article from the University of Oslo Medicine/Humanities faculty filled in a piece of that story that I didn’t really know much of anything about. And it’s an important history. I don’t know that this piece will usher in a Renaissance, as it’s hidden away in a think journal behind a pay wall, but it is certainly clarifying and worth putting some effort into getting hold of [I’m going to write the authors to see is there’s a free full-text version available for general consumption]. The abstract only paints some broad strokes…
by Clemet Askheim, Tony Sandset, and Eivind EngebretsenMedical Humanities Online First, 6 October 2016
Abstract: Over the last 20 years, the evidence-based medicine [EBM] movement has sought to develop standardised approaches to patient treatment by drawing on research results from randomised controlled trials [RCTs]. The Cochrane Collaboration and its eponym, Archie Cochrane, have become symbols of this development, and Cochrane’s book Effectiveness and Efficiency from 1972 is often referred to as the first sketch of what was to become EBM. In this article, we claim that this construction of EBM’s historical roots is based on a selective reading of Cochrane’s text. Through a close reading of this text, we show that the principal aim of modern EBM, namely to warrant clinical decisions based on evidence drawn from RCTs, is not part of Cochrane’s original project. He had more modest ambitions for what RCTs can accomplish, and, more importantly, he was more concerned with care and equality than are his followers in the EBM movement. We try to reconstruct some of Cochrane’s lost legacy and to articulate some of the important silences in Effectiveness and Efficiency. From these clues it might be possible, we argue, to remodel EBM in a broader, more pluralistic, more democratic and less authoritarian manner.
In the days before television and other devices, some satirists used the comic strip for their commentaries – none more effectively than Walt Kelly who drew Pogo. Pogo possum and Kelly’s other anthropomorphized denizens of the Okefenokee Swamp delighted us with political satire and commentary on the human condition for a quarter century. There was only one character, Pup Dog, who remained on all fours as a playful puppy and only said "wurf." But there was a period, where he said, "Poltergeists make up the principal type of spontaneous material manifestation" – a nonsense Tautology that translates to "ghosts are ghosts."
To be honest, these short term clinical trials don’t really need much writing by anyone at all. Some simple tables would do just fine: a list of the primary and secondary outcome variable defined before the study started with their results; the subjects’ demographics and drop-out rates; some comparative tables of adverse events; and a special table for severe adverse events. Write a few sentences at the end summarizing the conclusions. How their assertion that "In fact, the contributions and expertise of medical writers working with authors is associated with more complete reporting of the results of clinical trials and higher quality content" fits into that is unclear [and highly unlikely]. The notion of the hired-on ghostwriters and the hired-on KOLs working hand in hand to produce a value-added article is a fabulist scenario. Neither has seen the data or run the numbers, so the whole picture they’re painting in their response to Matheson is a fantasy. We have a few opened windows into the ghostwriting process via subpoenaed internal emails and documents from various lawsuits. The general pattern is for the ghostwriter to get some kind of summary document from the Sponsor to turn into a first draft. After it’s tweaked by the Sponsor, it’s cleared for the "authors" to take a look at – hardly the royal road to "higher quality content."
"ISMPP’s long-standing position is that “ghostwriting” – defined by Laine and Mulrow in 2005 as individuals who wrote the paper but are not acknowledged’ – is an unacceptable practice…"
"ISMPP fully supports the role of professional medical writers and the complete and transparent disclosure of their involvement in medical publications and the source of their funding."
"… My analysis shows, however, that modest transparency and small print disclosure, while better than nothing, paradoxically provides an apparatus for subtle misattribution, concealment and spin. My article discusses this as a cultural problem within medicine for which many stakeholders are responsible. It is important not to lay the blame at industry’s door alone. As Stretton rightly points out, it is journals that make authorship rules, and decide how disclosures should be presented to their readers. Table 2 of my article provides some useful suggestions to the editorial community."
"More importantly, however, while I applaud Stretton’s and Weigel’s commitment to transparency, I remind them that the publications trade remains unacceptably secretive. Numerous articles are produced for journal publication every year, scientific platforms and publications plans are developed, academics are recruited to participate as authors, trade writers draft manuscripts, and substantial payments pass from corporations to agencies, but little of this activity is in the public domain. Indeed, the process has been characterized as ‘ghost management’. Commercial secrecy has no place in an open scholarly discourse…"
This affirmation of continued commercial secrecy on these aspects of medical journal literature – made directly to the medical community itself, in a leading medical journal – will be of broad ethical and policy concern. The joint statement ends by claiming the trade has been “working towards” transparency and disclosure for 15 years. There is clearly a long road yet to travel…
12 September 2016: from Al Weigel – President and CEO, International Society for Medical Publication Professionals [ISMPP]
… ISMPP’s long-standing position is that “ghostwriting” – defined by Laine and Mulrow in 2005 as “individuals who wrote the paper but are not acknowledged” – is an unacceptable practice. ISMPP fully supports the role of professional medical writers and the complete and transparent disclosure of their involvement in medical publications and the source of their funding. In fact, the contributions and expertise of medical writers working with authors is associated with more complete reporting of the results of clinical trials and higher quality content.Over more than a decade, disclosure of the role of medical writers [and their funding] in contributing to medical publications has progressed to become standard practice today. The involvement of medical writers, statisticians, and others are made fully transparent to journal editors and peer reviewers, and ultimately to the readers of the published literature, along with the authors’ disclosures and potential conflicts of interest. Through their interactions with authors, medical writers are important contributors to the timely and complete dissemination of clinical data in the medical literature…
15 September 2016: from Serina Stretton, Cindy Hamilton, Jackie Marchington, Art Gertel, Julia Donnelly – Global Alliance of Publication Professionals [GAPP]
… At the core of Matheson’s article are claims that there is no distinction between ghostwriters and professional medical writers, and that the pharmaceutical industry uses medical writers to insert commercial messages into the peer-reviewed literature. We completely agree with and support the International Society for Medical Publication Professionals [ISMPP] response to Matheson’s claims that there has been no “rebranding of ghostwriting” by industry and that rather, “there has been a positive evolution of transparency and completeness in medical publications reporting industry research.” As GAPP has stated consistently, professional medical writers are NOT ghostwriters because they ARE transparent about their involvement, are strictly prohibited from ghostwriting, and work within ethical publication guidelines that require authors to contribute to and control the content at every step of the publication process…
21 September 2016 from Alastair Matheson21 September 2016 from Alastair Matheson
Weigel and Stretton both object to my article by emphasizing their trade’s commitment to “transparency”. My analysis shows, however, that modest transparency and small print disclosure, while better than nothing, paradoxically provides an apparatus for subtle misattribution, concealment and spin. My article discusses this as a cultural problem within medicine for which many stakeholders are responsible. It is important not to lay the blame at industry’s door alone. As Stretton rightly points out, it is journals that make authorship rules, and decide how disclosures should be presented to their readers…Numerous articles are produced for journal publication every year, scientific platforms and publications plans are developed, academics are recruited to participate as authors, trade writers draft manuscripts, and substantial payments pass from corporations to agencies, but little of this activity is in the public domain. Indeed, the process has been characterized as ‘ghost management’. Commercial secrecy has no place in an open scholarly discourse. Since this literature has been placed within the public academic forum with the goal of informing treatment decisions, readers and their patients have a right to know exactly what commercial considerations were at work in its development, and who was paid by whom to create the material…
30 September 2016 from Alison M Rapley, President, European Medical Writers Association
The International Society for Medical Publication Professionals (ISMPP) and Global Alliance of Publication Professionals [GAPP] have already provided detailed responses to the issues raised in the article by Alastair Matheson. The American and European Medical Writers Associations [AMWA and EMWA] fully support these responses which were prepared by medical communications leaders who have been on the forefront of assessing and establishing ethical and robust professional standards. These standards included input from editors of first-tier peer-reviewed medical and scientific journals. There was no pharmaceutical industry review or input into these rapid responses. Clearly Dr Matheson’s interpretation of what constitutes ethical transparency differs from that of these major medical communications organizations representing thousands of professionals working around the globe, as well as differing from that of the International Committee of Medical Journal Editors [ICMJE] who represent more than 500 medical journals.The undersigned organizations have been working towards transparency and disclosure for more than 15 years and invite all interested parties with a stake in the future of effective and ethical medical publications to tender constructive suggestions directly to them.
Alison Rapley – President, European Medical Writers Association
Brian Bass – Past President, American Medical Writers Association
Al Weigel – President and CEO, International Society for Medical Publication Professionals
Serina Stretton – Global Alliance of Publication Professionals
British Medical Journal – Rapid Responseby Alastair MathesonOctober 7, 2016
The debate following my article has culminated in a very rare, and unaccommodating, joint public statement on transparency by the three foremost trade associations of the commercial publications industry – the International Society for Medical Publication Professionals [ISMPP] and the European and American Medical Writers Associations [AMWA and EMWA].They have been joined in this statement by the Global Alliance of Public Professionals [GAPP], the most active advocacy group for the trade. In this statement, these groups have given their official response to a series of specific transparency requests which I made in order to clarify where the trade stood on secrecy and transparency in its development of medical journal articles. The trade chose to meet none of these requests, indicating its commitment to continued secrecy on these aspects of the medical publications business.
In particular, the trade did not agree to reveal any details of the commercial publications plans, scientific platforms, product positioning and key messages underpinning its past, and future, journal articles; nor did it agree to release details about its recruitment and use of “key opinion leaders”; nor details of intellectual property rights; nor any indication of how much money its companies receive from industry clients to plan and develop these articles. All this information is to be remain withheld from the readers of this literature and their patients. Most remarkably, the trade did not agree even to disclose exactly what it has published. It did not agree to provide simple lists of its published output; it did not agree to disclose any secret ghostwriting it may be aware of in the published journal literature; and it did not agree to encourage writers to disclose such articles, or to waive their confidentiality agreements if they do so.This affirmation of continued commercial secrecy on these aspects of medical journal literature – made directly to the medical community itself, in a leading medical journal – will be of broad ethical and policy concern. The joint statement ends by claiming the trade has been “working towards” transparency and disclosure for 15 years. There is clearly a long road yet to travel…
So I was nosing around looking for something to keep me from perseverating endlessly about the NIH/ClinicalTrials.gov/FDA business and our Petition, and in my twitter feed, Dinah [Shrink Rap] had tweeted a New York Times article about the political affiliations of different medical specialties [Your Surgeon Is Probably a Republican, Your Psychiatrist Probably a Democrat]. Looking at it, I had a hunch of sorts, so I googled "medical specialty salary", and immediately got the Medscape Physician Compensation Report 2016. The lists had close to the same specialties, so I plotted all that were on both lists against each other [mean salaries versus political affiliation]. Voila` – data! Other than the outliers [Cardiologists and Family Practitioners], not bad.
Is there a which-came-first? the chicken or the egg relationship here? What does it mean? I can make up stuff, but I haven’t got a clue. I was just trying to distract myself from The Petition and from my wife’s leaving for a trip this afternoon to South America [via Miami now that Hurricane Matthew has moved up the coast]. But here’s the thing. It didn’t work. I wasn’t distracted like I planned. When I clicked on the spreadsheet trend line menu item, I didn’t bother to do the stats to see how well the line fit the points. Why? Because this is an Exploratory bit of fluff. I didn’t have a hypothesis to even test, just some playing around and had a hunch. And statistical testing is essentially meaningless when you’re just messing around with hunches and such. They’re for Hypothesis Testing under strictly defined conditions. So see, my mind is still on statistical testing and clinical trials – even when I’m horsing around.
As silly as my example is, it’s actually a notch above some of the sheenanigans that have gone on in the serious world of drug testing. In a number of these clinical trials, it’s obvious that somebody cheated – meaning they looked at the data after the study was done, or peeked at it through the blind while it was ongoing and selected outcomes that showed what they wanted to see. And that’s simply impossible to ever prove. So that’s the reason the a prori Protocol and Statistical Analysis Plan must be concretely certified before the study begins. It’s the only way to prevent that kind of cheating [outcome switching].
Likewise, one needs to see the actual results of the prespecified parameters and methods to evaluate whether the reported results match the reported conclusions. As much as this system has been abused, it’s hard for me to imagine leaving loopholes in place. The FDA is tasked with protecting and monitoring the American formulary. Insuring the accuracy of the results of these clinical trials in terms of efficacy AND toxicity as they are reported in academic journals is near the top of the list.
This petition asks Congress to require the U.S. Food and Drug Administration (FDA) and the National Institutes of Health (NIH) to coordinate so as to prevent misreporting of clinical trials.
It is by now no secret that medical journals often publish biased reports of clinical trials funded by industry. These biased reports exaggerate benefits and minimize harms of new medical products (drugs and medical devices). Corporations succeed in this practice by sending different information and statistical analyses to the NIH than what they sent to the FDA. Medical journals, which rely on NIH data, have no way to detect these discrepancies. Both the FDA and the NIH know that this happens, but they do nothing, so the misleading claims escape regulation. We aim to close this loophole.
SPECIFIC STEPS TO RECONCILE FDA DATA AND NIH DATA
Trials Registration: Our petition first asks Congress to require coordination between FDA and NIH in the registration of clinical trials. Already, these must be pre-registered with the FDA and with the NIH (through the NIH public site called ClinicalTrials.gov). The second of those steps is often ignored or delayed. We ask Congress to require posting of the study protocol and detailed plan of statistical analysis with ClinicalTrials.gov at the time of registration with the FDA (current rules for the NIH site allow a delay of 1-3 years beyond the date of final data collection). The key elements of trials registration are to declare the a priori study protocol and the a priori statistical analysis plan before the study actually begins. That includes specification in advance of the primary and secondary outcome measures by which efficacy and safety will be assessed. Any other outcome measures must be clearly labeled as exploratory or non-protocol. The FDA then shall post a notice on ClinicalTrials.gov confirming or disconfirming that the a priori protocols and plans of analysis registered at both sites are identical.
Reporting Results: We also ask Congress to require timely publication of trials results on the NIH site and certification by the FDA that the information placed on the public NIH site agrees with the results that the FDA itself reviewed. Congress shall require that the FDA shall (i) post the FDA’s analyses on ClinicalTrials.gov as soon as the product is approved or disapproved for the requested indication; (ii) verify by notice whether corporate analyses reported on the NIH site are consistent with the a priori registered statistical analysis plan; (iii) verify by notice whether Results posted on ClinicalTrials.gov are concordant with Results determined by the FDA review; (iv) post on ClinicalTrials.gov a notice of any requested follow-up Phase 4 trial; (v) notify both the responsible party and ClinicalTrials.gov if the FDA determines that any corporate information placed on ClinicalTrials.gov conflicts with the data on file at the FDA; and (vi) notify both the responsible party and ClinicalTrials.gov if the FDA determines that any requested Phase 4 trial was not registered on ClinicalTrials.gov. Any publication or presentation or brochure or press release shall be required to display links to the trial’s registration on ClinicalTrials.gov and to any applicable FDA notices.
Expected Benefit: This petition requires both the FDA and NIH to meet their obligation to protect the public safety through genuine coordination of their joint oversight responsibilities. When FDA and NIH coordinate in these ways, then editors and reviewers at medical journals, and other stakeholders, can verify the fidelity of reported protocols and analyses. These provisions will allow stakeholders to determine whether any secondary analyses that use modified data sets or that address unplanned secondary questions have been properly identified.These steps will prevent the widespread manipulation of in-house corporate statistical analyses that are the basis for misleading reports in the medical journals. Positive analyses based on the a priori protocol and statistical analysis plan may be clinically actionable, as determined by the FDA. However, positive in-house corporate statistical analyses based on modified outcome measures or protocol changes cannot be regarded as clinically actionable, even though that is implied in medical journal reports, e.g., for off-label us es and unapproved patient groups such as children. These changes will for the first time allow stakeholders to recognize such claims for what they are.
Perhaps, rather than talk about what the NIH and FDA propose doing about it, we should frame the problem that needs fixing. Because of the PHARMA Sponsors’ keeping clinical trial data secret [proprietary, intellectual property] and the FDA, the only entity with access, we are only privy to the published journal articles for our information about clinical trials. And it is increasingly apparent that there has been widespread corruption in the public reporting – leaning towards exaggerated efficacy and minimization of harms. While it’s hard to prove because it happens behind closed doors, a common tool of distortion involves outcome switching – changing reported outcome parameters to fit the results. For this and other more scientific reasons, the outcome parameters and methods of analysis must be declared before the trial begins.
How could we be sure that the a priori declaration of parameters and methods is accurate? Ask the FDA to certify their accuracy. In most cases, they already have the documents as part of an FDA registration. How could we know that the resulrs posted are accurate? Ask the FDA to certify that they match those submitted to the FDA. They are the only agency with access to the data itself, and have evaluated it as part of their approval process.
But that’s not what the NIH/ClinicalTrials.gov/FDA did [see what to do? the final rule?…]. The NIH and ClinicalTrials.gov wrote the final rule, which is essentially what was in place before except they clarified a few ambiguities and promised to enforce it this time around. The FDA was supportive but changed nothing. It’s going to take direct legislation to get changes like the ones suggested above in place – mandated by Congress. Those are the exact changes that are in our petition [Stop False Reporting of Drug Benefits & Harms by Making FDA & NIH Work Together]. Please take a look and sign it.
From the STAT article [New federal rules target woeful public reporting of clinical trial results]:
- Federal officials said last year that they wouldn’t penalize offenders before clarifying the reporting rules. Now that this is happening, “I’m expecting a flood of trials to get registered,” FDA Commissioner Robert Califf told reporters during a conference call. “It would be pretty hard to hide that you are doing a clinical trial or hide the result. … I’m very optimistic about this.”
- Yet not a single researcher or trial sponsor has been fined or penalized, STAT reported — though the Food and Drug Administration had the authority to fine violators more than an aggregate $25 billion since the law went into force in 2008.
- Neither agency will add such staff, according to Califf, whose agency oversees private sector reporting, and NIH Director Francis Collins, whose agency is responsible for enforcement regarding NIH grant recipients as well as NIH staff scientists.
- Califf said enforcement instead would be folded into the FDA’s existing bioresearch monitoring program, which collects information related to clinical trials during company inspections.
- “Typically, when the FDA believes there may be a violation of the law, it sends a warning letter” for voluntary correction, Califf said. He expects this method to handle most problems, and promised that the public will have “a good window of what’s going on” in compliance actions.In the NPR article [HHS Issues New Rules To Open Up Data From Clinical Trials],
- "This has been a very opaque world up until to now," Food and Drug Commissioner Robert Califf told reporters during an embargoed phone conference Thursday. "These are tremendous changes."
- For studies that are regulated by the FDA, companies can face fines of $10,000 a day if they don’t comply with the existing rules. Those fines have not been imposed, however, and the FDA will not have additional funding to hire enforcement staff, Califf said. "I really believe that it won’t take much to get people to comply with this once they realize how serious this is."And I wasn’t in love with this one either:
- Both agencies will automate compliance checks to ensure trials are properly registered and results reported, and contact institutions as needed to provide all the data required to bring their trials into compliance with the law.
And in spite of being impressed that the NIH/ClinicalTrials.gov final rule shows signs of some real effort and investigation, I question their solutions. There are two points in the life cycle of a clinical trial where data is entered into ClinicalTrials.gov – Registration and Results: