1 Boring Old Man

So I was nosing around looking for something to keep me from perseverating endlessly about the NIH/ClinicalTrials.gov/FDA business and our Petition, and in my twitter feed, Dinah [Shrink Rap] had tweeted a New York Times article about the political affiliations of different medical specialties [Your Surgeon Is Probably a Republican, Your Psychiatrist Probably a Democrat]. Looking at it, I had a hunch of sorts, so I googled "medical specialty salary", and immediately got the Medscape Physician Compensation Report 2016. The lists had close to the same specialties, so I plotted all that were on both lists against each other [mean salaries versus political affiliation]. Voila` – data! Other than the outliers [Cardiologists and Family Practitioners], not bad.

Is there a which-came-first? the chicken or the egg relationship here? What does it mean? I can make up stuff, but I haven’t got a clue. I was just trying to distract myself from The Petition and from my wife’s leaving for a trip this afternoon to South America [via Miami now that Hurricane Matthew has moved up the coast]. But here’s the thing. It didn’t work. I wasn’t distracted like I planned. When I clicked on the spreadsheet trend line menu item, I didn’t bother to do the stats to see how well the line fit the points. Why? Because this is an Exploratory bit of fluff. I didn’t have a hypothesis to even test, just some playing around and had a hunch. And statistical testing is essentially meaningless when you’re just messing around with hunches and such. They’re for Hypothesis Testing under strictly defined conditions. So see, my mind is still on statistical testing and clinical trials – even when I’m horsing around.

As silly as my example is, it’s actually a notch above some of the sheenanigans that have gone on in the serious world of drug testing. In a number of these clinical trials, it’s obvious that somebody cheated – meaning they looked at the data after the study was done, or peeked at it through the blind while it was ongoing and selected outcomes that showed what they wanted to see. And that’s simply impossible to ever prove. So that’s the reason the a prori Protocol and Statistical Analysis Plan must be concretely certified before the study begins. It’s the only way to prevent that kind of cheating [outcome switching].

Likewise, one needs to see the actual results of the prespecified parameters and methods to evaluate whether the reported results match the reported conclusions. As much as this system has been abused, it’s hard for me to imagine leaving loopholes in place. The FDA is tasked with protecting and monitoring  the American formulary. Insuring the accuracy of the results of these clinical trials in terms of efficacy AND toxicity as they are reported in academic journals is near the top of the list.

EXECUTIVE SUMMARY

PURPOSE
This petition asks Congress to require the U.S. Food and Drug Administration (FDA) and the National Institutes of Health (NIH) to coordinate so as to prevent misreporting of clinical trials.

JUSTIFICATION
It is by now no secret that medical journals often publish biased reports of clinical trials funded by industry. These biased reports exaggerate benefits and minimize harms of new medical products (drugs and medical devices). Corporations succeed in this practice by sending different information and statistical analyses to the NIH than what they sent to the FDA. Medical journals, which rely on NIH data, have no way to detect these discrepancies. Both the FDA and the NIH know that this happens, but they do nothing, so the misleading claims escape regulation. We aim to close this loophole.

SPECIFIC STEPS TO RECONCILE FDA DATA AND NIH DATA

Trials Registration: Our petition first asks Congress to require coordination between FDA and NIH in the registration of clinical trials. Already, these must be pre-registered with the FDA and with the NIH (through the NIH public site called ClinicalTrials.gov). The second of those steps is often ignored or delayed. We ask Congress to require posting of the study protocol and detailed plan of statistical analysis with ClinicalTrials.gov at the time of registration with the FDA (current rules for the NIH site allow a delay of 1-3 years beyond the date of final data collection). The key elements of trials registration are to declare the a priori study protocol and the a priori statistical analysis plan before the study actually begins. That includes specification in advance of the primary and secondary outcome measures by which efficacy and safety will be assessed. Any other outcome measures must be clearly labeled as exploratory or non-protocol. The FDA then shall post a notice on ClinicalTrials.gov confirming or disconfirming that the a priori protocols and plans of analysis registered at both sites are identical.

Reporting Results: We also ask Congress to require timely publication of trials results on the NIH site and certification by the FDA that the information placed on the public NIH site agrees with the results that the FDA itself reviewed. Congress shall require that the FDA shall (i) post the FDA’s analyses on ClinicalTrials.gov as soon as the product is approved or disapproved for the requested indication; (ii) verify by notice whether corporate analyses reported on the NIH site are consistent with the a priori registered statistical analysis plan; (iii) verify by notice whether Results posted on ClinicalTrials.gov are concordant with Results determined by the FDA review; (iv) post on ClinicalTrials.gov a notice of any requested follow-up Phase 4 trial; (v) notify both the responsible party and ClinicalTrials.gov if the FDA determines that any corporate information placed on ClinicalTrials.gov conflicts with the data on file at the FDA; and (vi) notify both the responsible party and ClinicalTrials.gov if the FDA determines that any requested Phase 4 trial was not registered on ClinicalTrials.gov. Any publication or presentation or brochure or press release shall be required to display links to the trial’s registration on ClinicalTrials.gov and to any applicable FDA notices.

Expected Benefit: This petition requires both the FDA and NIH to meet their obligation to protect the public safety through genuine coordination of their joint oversight responsibilities. When FDA and NIH coordinate in these ways, then editors and reviewers at medical journals, and other stakeholders, can verify the fidelity of reported protocols and analyses. These provisions will allow stakeholders to determine whether any secondary analyses that use modified data sets or that address unplanned secondary questions have been properly identified.

These steps will prevent the widespread manipulation of in-house corporate statistical analyses that are the basis for misleading reports in the medical journals. Positive analyses based on the a priori protocol and statistical analysis plan may be clinically actionable, as determined by the FDA. However, positive in-house corporate statistical analyses based on modified outcome measures or protocol changes cannot be regarded as clinically actionable, even though that is implied in medical journal reports, e.g., for off-label us es and unapproved patient groups such as children. These changes will for the first time allow stakeholders to recognize such claims for what they are.