I am personally drawn to the mathematical/statistical analyses employed in a properly conducted Randomized Double Blind Clinical Trial [RCT], and would like to think that if we had access to the raw data from Clinical Trials [Data Transparency], we could generate an accurate efficacy and safety profile for our drugs based that data. That was the general gist of the first ending to this story. In that scenario, the carefully analyzed RCT would become the gold standard for evaluating medications. Certainly, the RCT is, by Congressional Decree, already the gold standard for the FDA’s Approval process.
There are other ways to look at this [see hearing voices… and something essential betrayed…]. While the RCT may be a reasonable way for the FDA to evaluate new drugs for approval, that doesn’t necessarily translate to a standard for clinical practice [see this comment by Dr. Bernard Carroll and Pharmageddon by Dr. David Healy]. Subjects are screened for exclusions. Subjects are recruited and often paid whereas Patients are help-seeking. The RCTs are short-term [6-8 weeks] while Patients are on medications much longer [these days, sometimes indefinitely]. Subjects are seen weekly for a while [metrics and questionnaires]. Patients are seen for brief med-checks at infrequent intervals. Subjects are assessed using sensitive standardized scales. Patients are asked, "How’s it going?" Lots of differences.
Those are concrete differences between the RCTs that we read about in our journals and the actual practice of medicine. But there’s another glaring difference. The RCTs are done by the company that has the patent on the drug, and who obviously wants to sell it widely for a maximal price. This blog and many others are filled with examples where the RCT is selectively-analyzed/reported to achieve that end. For example, all of the studies of Atypical Antipsychotic Augmentation of Treatment Resistant Depression studies reported in this series were statistically significant, but the Strength of the Effect for the primary outcome variables wasn’t reported. Here are the values extracted in the Spielman et al meta-analysis for those outcomes and the values I got using their same methodology [Cochrane Handbook] in the Geodon® and Rexulti® reports [also showing the IDS-SR Effect Sizes where available]:
| SOURCE | MADRS/HAM-D [d] | IDS-SR [d] | ||
|
|
||||
| Spielmans et al |
0.32 | 0.14 | ||
| Ziprasidone |
0.25 | – | ||
| Brexpiprazole 1mg-3mg |
0.29 | 0.16 | ||
| Brexpiprazole 2mg |
0.29 | 0.20 | ||
The observer rated MADRS/HAM-D numbers are in the weak to moderate range, but the self-rated scales are dramatically near-null – Statistically Significant, but trivial Effect from the Subject’s perspective. So with Data Transparency, willing independent analysts, and cooperative journal editors we could correct a lot of the misbehavior and get all the information.
But these short-term industry run "Approval" RCTs really are still nothing more than a starting place for understanding either efficacy or safety, even at their best. Perhaps the analogy of a model airplane to a Jumbo Jet might be a more reasonable way to conceptualize their place. The real test is when the drugs are put into general use by Clinicians and Patients. And we often don’t hear about those results until well down the line when the side effects and harms start showing up in court-rooms. Patients have plenty of time in waiting rooms getting "screened" or watching tv getting "detailed" by DTC ads, but very little time in the med-check world of Managed Care getting carefully evaluated before or after prescriptions are written. And the time allotted for reporting is… well it just isn’t in the program. That clown at the World of DTC Marketing in the last post [when pigs can fly…] has plenty of suggestions about how we can talk to our patients about the provocative questions raised by his silly ads when they "ask the doctor", but apparently doesn’t know that we already don’t even have time to do an adequate medical interview much less what he suggests.
I’m grateful to you Dr. Mickey for taking the time to plow through these Anti-Psychotic Augmentation studies and expose the holes in them. They seem to be made almost entirely of holes sometimes!
The studies last from three to six weeks; the “investigators” hand out everything from benzos to chloral hydrate to help the study drug stay “well tolerated”; and the patient surveys show NO improvement … Then some “medical communications agency” slaps together an article strewn with improbable words like “robust” and “significant.” And most docs only read the abstract anyway. They might as well rely on the TV commercial …
At RxISK we hear fairly regularly from earnest Healthcare Professionals who worry that patients will be unduly alarmed by a site like ours, which by definition hears mainly from people with negative or unusual experiences. I’ve always felt that grown-ups should be treated like grown-ups, and good doctors should help us put the risks in context, not hide them from our innocent little eyes. Thanks for the vote of confidence.