"There were approximately 600 clinical trials mentioned in the documents we gathered; owing to redactions, most of these trials could not be identified. However, in some cases, key information was not redacted from the documents, allowing us to identify 101 trials in which at least one clinical trial site received an OAI grade on an inspection."
OAI = Official Action Indicated
A few years back when I started looking into the Clinical Trials literature, I developed a solid case of Cognitive Dissonance. I couldn’t get my mind around how much research misconduct and deceitful science I found in plain view, and how much more people who knew a lot more about these things than I did were able to show me. After a few years, I got kind of jaded, used to reading articles looking for the various ways that things were being distorted. I acclimated to my disillusionment and expected the worst. When I first read this article below, I saw it as just more of the same. But then something told me to read it again. It is of much greater import than I saw the first time around:
Out of Sight, Out of Mind, Out of the Peer-Reviewed Literatureby Charles Seife, MSJAMA Internal Medicine. 2015 175[4]:567-577.
IMPORTANCE Every year, the US Food and Drug Administration [FDA] inspects several hundred clinical sites performing biomedical research on human participants and occasionally finds evidence of substantial departures from good clinical practice and research misconduct. However, the FDA has no systematic method of communicating these findings to the scientific community, leaving open the possibility that research misconduct detected by a government agency goes unremarked in the peer-reviewed literature.OBJECTIVES To identify published clinical trials in which an FDA inspection found significant evidence of objectionable conditions or practices, to describe violations, and to determine whether the violations are mentioned in the peer-reviewed literature.DESIGN AND SETTING Cross-sectional analysis of publicly available documents, dated from January 1, 1998, to September 30, 2013, describing FDA inspections of clinical trial sites in which significant evidence of objectionable conditions or practices was found.MAIN OUTCOMES AND MEASURES For each inspection document that could be linked to a specific published clinical trial, the main measure was a yes/no determination of whether there was mention in the peer-reviewed literature of problems the FDA had identified.RESULTS Fifty-seven published clinical trials were identified for which an FDA inspection of a trial site had found significant evidence of 1 or more of the following problems: falsification or submission of false information, 22 trials [39%]; problems with adverse events reporting, 14 trials [25%]; protocol violations, 42 trials [74%]; inadequate or inaccurate recordkeeping, 35 trials [61%]; failure to protect the safety of patients and/or issues with oversight or informed consent, 30 trials [53%]; and violations not otherwise categorized, 20 trials [35%]. Only 3 of the 78 publications [4%] that resulted from trials in which the FDA found significant violations mentioned the objectionable conditions or practices found during the inspection. No corrections, retractions, expressions of concern, or other comments acknowledging the key issues identified by the inspection were subsequently published.CONCLUSIONS AND RELEVANCE When the FDA finds significant departures from good clinical practice, those findings are seldom reflected in the peer-reviewed literature, even when there is evidence of data fabrication or other forms of research misconduct.hat·tip to Elia…
This report required a lot of dedicated sleuthing. They were looking for Clinical Trial Site inspections that received an OAI rating [official action indicated] which means there was a serious problem. So, first off, "Who are they?" The author is Charles Seife is a journalism professor at New York University. And "they" are Professor Seife and his journalism students. The full text of the medical article isn’t on-line, but there’s a fine version in Slate:
FDA buries evidence of fraud in medical trials. My students and I dug it up.Slateby Charles SeifeFeb 9, 2015… Much of my research has to do with follies, foibles, and fraud in science, and I knew that the FDA wasn’t exactly bending over backward to correct the scientific record when its inspectors found problems during clinical trials. So as part of my investigative reporting class at New York University, my students and I set out to find out just how bad the problem was — and how much important information the FDA was keeping under wraps. We didn’t have to search very hard to find FDA burying evidence of research misconduct. Just look at any document related to an FDA inspection. As part of the new drug application process, or, more rarely, when the agency gets a tipoff of wrongdoing, the FDA sends a bunch of inspectors out to clinical sites to make sure that everything is done by the book. When there are problems, the FDA generates a lot of paperwork — what are called form 483s, Establishment Inspection Reports, and in the worst cases, what are known as Warning Letters. If you manage to get your hands on these documents, you’ll see that, most of the time, key portions are redacted: information that describes what drug the researcher was studying, the name of the study, and precisely how the misconduct affected the quality of the data are all blacked out. These redactions make it all but impossible to figure out which study is tainted. My students and I looked at FDA documents relating to roughly 600 clinical trials in which one of the researchers running the trial failed an FDA inspection. In only roughly 100 cases were we able to figure out which study, which drug, and which pharmaceutical company were involved. [We cracked a bunch of the redactions by cross-referencing the documents with clinical trials data, checking various other databases, and using carefully crafted Google searches.] For the other 500, the FDA was successfully able to shield the drugmaker [and the study sponsor] from public exposure…
And these weren’t minor infractions. They were fraud, big time fraud [from the JAMA]:
Case 3
A researcher was caught falsifying documents in a number of trials, in part because those falsifications led to the death of a patient undergoing treatment in a clinical trial comparing 2 chemotherapy regimens. The researcher had falsified laboratory test results to hide the patient’s impaired kidney and liver function, and the first dose of the treatment proved to be fatal.The researcher pleaded guilty to fraud and criminally negligent homicide and was sentenced to 71 months in prison. Although this episode is described in detail in FDA documents as well as court documents, none of the publications in the peer-reviewed literature associated with the chemotherapy study in which the patient died have any mention of the falsification, fraud, or homicide. The publications associated with 2 of the 3 other studies for which the researcher falsified documents also do not report on the violations.
And here’s the bottom line question. Why does the FDA keep stories like this secret? [from the JAMA]:
The FDA does not typically notify journals when a site participating in a published clinical trial receives an OAI inspection, nor does it generally make any announcement intended to alert the public about the research misconduct that it finds. The documents the agency discloses tend to be heavily redacted. As a result, it is usually very difficult, or even impossible, to determine which published clinical trials are implicated by the FDA’s allegations of research misconduct. The FDA has legal as well as ethical responsibilities regarding the scientific misconduct it finds during its inspections. When the agency withholds the identity of a clinical trial affected by scientific misconduct, it does so because it considers the identity to be confidential commercial information, which it feels bound to protect. However, failing to notify the medical or scientific communities about allegations of serious research misconduct in clinical trials is incompatible with the FDA’s mission to protect the public health…
First off, we are in Professor Seite’s and his investigative journalism students’ debt. I’m not sure that any of us bloggers could have pulled this off. They pored over the data sources with a fine tooth comb to match the reports of failed FDA Inspections with the published Clinical Trials, against a gradient of FDA attempts to hide the connection by redaction. So we know that’s just the tip of an iceberg. These are not just the kinds of research misconduct we’ve found all over the place – subtle methods to accentuate the positive and eliminate the negative. These are examples of outright fraud being actively kept out of the public eye by the FDA under the guise of protecting CCI [Confidential Commercial Information]. CCI my ass! This is simply criminal behavior that probably ought to disqualify the trials altogether. Data Transparency won’t do anything about this. So in its zeal to protect this precious [and largely mythical] CCI, the FDA, a public agency whose existence rests on protecting the public, becomes an active co-conspirator in criminal fraud. Back when I tried to look at the origins of this notion of proprietary ownership of data and Confidential Commercial Information, I was surprised that it was not based on any laws. It rested on interpretations of Trade Agreements [a crushing setback…, except where necessary to protect the public…]. But as this article makes very clear, it has been used to keep all kinds of unsavory behavior from the medical community and the public at large [from the Slate article]:
The sworn purpose of the FDA is to protect the public health, to assure us that all the drugs on the market are proven safe and effective by reputable scientific trials. Yet, over and over again, the agency has proven itself willing to keep scientists, doctors, and the public in the dark about incidents when those scientific trials turn out to be less than reputable. It does so not only by passive silence, but by active deception. And despite being called out numerous times over the years for its bad behavior, including from some very pissed-off members of Congress, the agency is stubbornly resistant to change. It’s a sign that the FDA is deeply captured, drawn firmly into the orbit of the pharmaceutical industry that it’s supposed to regulate. We can no longer hope that the situation will get better without firm action from the legislature. The FDA wants you to take it on faith that its officials have the public’s best interest at heart. Justification through faith alone might be just fine as a religious doctrine, but it’s not a good foundation for ensuring the safety and effectiveness of our drugs. After all, the whole point of science-based medicine is to keep us from having to make a leap of faith every time we swallow a pill.
My personal belief is that many decades ago we saw data being shown in a good light. This polishing the apple soon led to outright fraud and when no one person paid a price the price became the cost of doing business.
Today we have a revolving door between government and the private sector. While this can bring needed expertise it has been used to corrupt the system. Wording is now parsed to eliminate liability and we see such things as speed becoming a weight loss product again after being declared unacceptable decades ago, but under the new guidelines obesity is an over riding condition.
The suburban doctor’s office has become a retail outlet for both the drug companies and hospitals selling additional services. Challenge this system and you are given the terrible mark of being noncompliant. Heavens, turn down a test that your insurance company will pay for and you are trying to put the poor, poor doctor out of business.
Decline or look for alternative medications, especially a generic, and you are a knuckle dragger not willing to take advantage of the latest and greatest pharma has to offer.
Follow blogs such as Health Care Renewal and fight off pharma shills as outrageous drug company behavior is countered with diversion and other old psychological and sales techniques.
The FDA has become no better than the snake oil salesmen it was created to stop.
Steve Lucas
It’s not the FDA’s fault. The entire US government has been acting in the interest of business expansion for 35 years, since the Reagan administration. This has become ingrained throughout US culture.
Not the FDA’s fault, not the KOLCHOs fault, not pharma influence’s fault but the fault of a guy who hasn’t been President in 26 years. And whose influence has continued for the past six by appointees who are the ideological opposites…uh…OKAY.
That would be like blaming 9/11 on Gerald Ford.
The FDA is not only idiotic in what they allow but also what they ban. Like 23andme.com for health data.
Can we stop with the incessant agendas and focus on reality?
@Steve
I am not sure what you are getting at. Are you saying that patients want tests and the latest drugs? Pretty much all of my patients are quite happy to not get unnecessary lab tests and take generic medications after I spend some time discussing my reasoning with them. They also tend to be comfortable not taking a medication for something that is best treated with a lifestyle change.
I am sorry if I misunderstood your post. It has been a long day 🙂
Joseph,
I, and most patients, are with you. The problem is in my area medicine has become such a business that practicing medicine in the manner you described is not allowed. We have a dominant hospital that owns a large number of practices and they have declared these to be retail outlets with a major function of the doctors employed to up code, and send patients to the hospital for any test paid for by insurance.
Request a generic and some doctors will refuse. There are so many side deals with everything from free lunches to hospitals having contracts with drug companies that patients are left in the cold. One elderly lady on a fixed income was told by her physician that if her family loved her they would pay for her branded medications even though generics were available.
I am sorry I was not clear, but my frustration with medicine is growing as week after week we find more and more acts that should be categorized as criminal and no person held accountable. Government agencies that should be protecting our interest are instead being run by those whose loyalty lay with outside commercial interest.
We need more doctors like you, and fewer of the corporate doctors who are very quick to point to their patient count, or gross billings.
Steve Lucas
Nice points above, and yes, when our colleagues start seeing their “brothers” going to court over felony charges, it might actually get some to pause and reflect if their choices are so “magnanimous and altruistic”, or just greedy and self serving. Again, time to take the business model out of medicine, as there is no real overlap with patient care and profit margins.
Being silent and complicit is not a defense, colleagues!