As I tried to say in proxies…, while I can find plenty to criticize about the Clinical Research industry, I’ve realized that the problems come after the blind is broken, not before – that thing called publication bias. I mentioned two studies along the way by the same group about publication bias in psychiatric studies. Here are some blurbs. If you don’t recall them, take a look at the graphs either in the papers or my posts.
[see gone missing…]:
by Erick H. Turner, Annette M. Matthews, Eftihia Linardatos, Robert A. Tell, and Robert RosenthalNew England Journal of Medicine. 2008 358:252-260.
Results:Among 74 FDA-registered studies, 31%, accounting for 3449 study participants, were not published. Whether and how the studies were published were associated with the study outcome. A total of 37 studies viewed by the FDA as having positive results were published; 1 study viewed as positive was not published. Studies viewed by the FDA as having negative or questionable results were, with 3 exceptions, either not published [22 studies] or published in a way that, in our opinion, conveyed a positive outcome [11 studies]. According to the published literature, it appeared that 94% of the trials conducted were positive. By contrast, the FDA analysis showed that 51% were positive. Separate meta-analyses of the FDA and journal data sets showed that the increase in effect size ranged from 11 to 69% for individual drugs and was 32% overall.
[see gone missing…]:
by Erick H. Turner, Daniel Knoepflmacher, and Lee ShapleyPLoS Medicine. 2012 9[3]:e1001189.
Methods and Findings: FDA Drug Approval Packages for eight second-generation antipsychotics—aripiprazole, iloperidone, olanzapine, paliperidone, quetiapine, risperidone, risperidone long-acting injection [risperidone LAI], and ziprasidone—were used to identify a cohort of 24 FDA-registered premarketing trials. The results of these trials according to the FDA were compared with the results conveyed in corresponding journal articles. The relationship between study outcome and publication status was examined, and effect sizes derived from the two data sources were compared. Among the 24 FDA-registered trials, four [17%] were unpublished. Of these, three failed to show that the study drug had a statistical advantage over placebo, and one showed the study drug was statistically inferior to the active comparator. Among the 20 published trials, the five that were not positive, according to the FDA, showed some evidence of outcome reporting bias. However, the association between trial outcome and publication status did not reach statistical significance. Further, the apparent increase in the effect size point estimate due to publication bias was modest [8%] and not statistically significant. On the other hand, the effect size for unpublished trials [0.23, 95% confidence interval 0.07 to 0.39] was less than half that for the published trials [0.47, 95% confidence interval 0.40 to 0.54], a difference that was significant.
They just don’t publish what they don’t want seen, doll up what they do publish and hide the rest. Yesterday, in the BMJ we have another example. It’s not from psychiatry but from medicine at large. It’s a review of large studies [> 500 subjects] that looks at publication – when they do it and whether they followed the law about posting the results on clinicaltrial.gov [they didn’t]:
by Christopher W Jones, Lara Handler, Karen E Crowell, Lukas G Keil, Mark A Weaver, and Timothy F Platts-MillsBritish Medical Journal. 2013 347:f6104.
Objective To estimate the frequency with which results of large randomized clinical trials registered with ClinicalTrials.gov are not available to the public.Setting Trials with at least 500 participants that were prospectively registered with ClinicalTrials.gov and completed prior to January 2009.Data sources PubMed, Google Scholar, and Embase were searched to identify published manuscripts containing trial results. The final literature search occurred in November 2012. Registry entries for unpublished trials were reviewed to determine whether results for these studies were available in the ClinicalTrials.gov results database.Main outcome measures The frequency of non-publication of trial results and, among unpublished studies, the frequency with which results are unavailable in the ClinicalTrials.gov database.Results Of 585 registered trials, 171 (29%) remained unpublished. These 171 unpublished trials had an estimated total enrollment of 299 763 study participants. The median time between study completion and the final literature search was 60 months for unpublished trials. Non-publication was more common among trials that received industry funding (150/468, 32%) than those that did not (21/117, 18%), P=0.003. Of the 171 unpublished trials, 133 (78%) had no results available in ClinicalTrials.gov.Conclusions Among this group of large clinical drug trials, non-publication of results was common and the availability of results in the ClinicalTrials.gov database was limited. A substantial number of study participants were exposed to the risks of trial participation without the societal benefits that accompany the dissemination of trial results.
Everything in red is outrageous, but the part that’s beyond outrageous is "Of the 171 unpublished trials, 133 (78%) had no results available in ClinicalTrials.gov." It was also stupid. The results database on ClinicalTrials.gov is data transparency lite. Had they been compliant with that requirement, maybe they wouldn’t be fighting the demand for all of the data that they’re contending with right now. But that mistake was theirs to make and it’s too late now. That opportunity has passed. As much as I’ve written and learned about this clinical trial story, I still can’t get my mind around "Of the 171 unpublished trials, 133 (78%) had no results available in ClinicalTrials.gov."
The ultimate issue is; how does this translate into medical practice. While I found the Trayvon Martin reference unnecessary in this link, I do agree with the premise that we are using medications as a quick easy way to move children through our medical system.
http://brodyhooked.blogspot.com/2013/10/an-epidemic-out-of-control-poor.html
Taken as a whole, and being a little paranoid, will the ACA with its financially mandated yearly physicals, EMR interconnectivity, and a growing NSA type data mining rush going to start a process where the government mandates treatments?
My wife and I, with good insurance and disposable income, have both suffered relentless demands that we be medicated and tested for no other reason that to support the medical system and prevent undefined “future” events. The unrelenting pressure would be laughable if not for being promoted by those who truly believe there is merit to their position.
Using a fact based argument only generates a great deal of hostility and “You don’t understand” response from doctors.
The patient doctor interaction in this example should not be considered unique or limited to pediatrics. Rapid fire questions, accusatory tones, and an air of superiority are common markers of many doctor’s office visits today.
Back to my paranoia, a recent local article highlighted how the TSA now reads licensee plates entering the local airport, checks passport numbers for trip profiles, and runs names against data bases looking for outstanding warrants for such things as financial crimes.
The NSA seems to have listened in on the Pope, has an interest in the personal phone calls of the Chancellor of Germany, while today I read an article where the US is criticizing them for a strong growing economy.
From the Oct. 30 WSJ opinion piece The Outrage Arrives by Holman W. Jenkins Jr. we find this:
“Democrats at least are consistent. Back in 1993, during the fight over HillaryCare Mrs. Clinton explained Democratic reasoning to the House GOP Leader Denny Haster. If Americans are allowed too much discretion over how they spend their health-care dollars, Mrs. Clinton said, “We just think people will be focused on saving money and they won’t get the care for their children and themselves that they need …
“The money has to go to the federal government because the federal government will spend that money better.”
….Mr. Obama says he cares about your incentive to get preventative care or test that you may not get if they don’t appear to involve a free lunch.”
What will you do now that the IRS can check to see if you got that free physical, and filled your prescriptions? What a Brave New World we will live in.
Steve Lucas
I would be glad for an explanation of the “rapid response” given by Lent and Out to “Non-publication of large randomized clinical trials” above.
I had an article published on a high quality Australian online media site today. I hope the AllTrials campaign is successful and clears the mess up. The campaign has virtually all medical and health related organisations in the UK behind it. But many organisations in the USA, Australia and elsewhere need to also give it their support if it is going to be successful as only total international support and coverage shall work.
See – https://theconversation.com/making-all-clinical-data-public-is-vital-for-better-medical-care-19755
Mickey,
I am confused.
Is there no legal obligation to publish the results of a registered trial on clinicaltrials.gov within a time limit after completion?
I assumed this was so, otherwise I couldn’t see how the alltrials campaign would get around the 4th amendment’s right to be protected “against unreasonable searches and seizures”, which is what forced disclosure of ‘unpublished information’ would be.
http://www.archives.gov/exhibits/charters/bill_of_rights_transcript.html
I know most of the Pharmaceutical Companies are convicted felons, which limits their rights somewhat, but I’m really confused if that’s the legal basis or what now?
Tin Can,
I’m confused too. I think that business of reporting the results to ClinicalTrials.gov within a certain time limit [one year] is actually a LAW. But it’s enforcement has been zero. And the journals don’t even really require registration or results being posted. Even NIMH funded studies are not often posted, at least results. STAR*D is a prime example. The non-compliance is legendary [bring on the hoops! at least the ones that matter…].
I think the legal part is the Freedom of Information Act. Since they publish a paper as a proxy for the information, they must provide access to the information that backs up the proxy. Since the information is used for drug approval, it must be available. I’m not sure of the legalities, actually.
My own way of thinking about this is that if you are going to have access to Approval by the FDA and if you are going to have access to the medical literature for publication, these are certifications that must be backed up by available information.
But maybe I don’t quite get your question.
Mickey, thanks for the response, Law isn’t my field either. I guess I’ll have to look into it. I was just curious if any laws had been broken in the first place by not reporting the completed trial results.
I’ll look into the Clincicaltrials.gov thing, and post if i find anything interesting.
Colleagues
Thanks for your sustained attention. I probably have made some mistakes below. Corrections welcome.
Would like some discussion or references to the following issues. My point is that most of these issues do not seem to be even on the horizon of public concern but follow quite naturally from the EMA actions and the Ombudsman support for overriding public health concerns.
1 The EMA stand is that patient level clinical trials data should be available to the public, at the point of marketing approved (EMA,FDA,ETC) drugs. Apparently the costs are born by the industry sponsoring that drug.
Since this is a public good, that does not directly improve the industry balance sheet , should the costs be born by the government agency? Does that entail government or industry responsibility for ensuring that the protocol and data are readily comprehended, complete, easily searched, etc?
2 The EMA stand is currently judicially on hold since it is contested as infringing “trade secrets or intellectual property or similar notions regarding property”. However the European Union Ombudsman has declared that the public health issue is of overriding importance. This argument seems to support a wide range of corollaries.
Apparently USA law is quite explicit in protecting trade secrets. Is there any precedent in USA judicial procedure for overriding relevant legislation on the grounds of “public health/welfare ?”
3 Is there a similar EMA type of responsibility regarding drugs approved in the past?. Who is responsible?
4 Who bears a similar EMA type of responsibility for safety or efficacy investigation and publication regarding common off-label usage?
5 Off-label usage is generally considered a public good ,enhancing medical practice. Publication or communications regarding safety or efficacy of such practices are also generally considered contributing to public health or welfare–except when sponsored, supported, etc by the medication sponsoring industry–since this is held likely to be biased.
Apparently -please correct me if I am wrong or incomplete–this was the basis for several recent industry fines since industry supported off label practices to increase profitability, without concern for public health impact.
6 I am not arguing against this judgement–but note that it does not affect the ubiquitous off label practices except by withdrawing industrial support. The off label practices continue without review for either safety or efficacy. There seems to me a problematic disconnect between the general belief that off label practice is a good enhancement to medical practice (although data regarding safety or efficacy is sparse or absent) and the notion that transparently self seeking industry support for an off label practice places the public at sustantial risk —-particularly because of the absence of public education regarding the absence of relevant safety or efficacy knowledge of this practice. Pardon this convoluted statement.
7 This seems to me to call for an illuminating, independent, safety/efficacy investigation–but this is not mandated or funded .
8 Should this be a government responsibility? More broadly, it raises the issue of responsibility for assuring the safety and effectiveness of all therapeutic practices–including surgery, psychotherapy, etc.
9 It is immediately obvious that this will not fly–but is there some incremental approach to this ultimate goal–or is that desirable??
10 In my view, the AllTrials goals are admirable and it probably make tactical sense to narrowly focus on this issue. However, It may make sense to also present the big picture– indicating that the AllTrials goals address only a small part of our ignorant practices, and are hardly revolutionary, given this context .