A recent discussion thread explored the topic of clinical intuition based on a physician’s experience versus reliance on RTCs [Randomized Clinical Trials], the gold standard of evidence-based medicine. Right now, that’s a front burner issue, particularly in clinical psychiatry. Dr. David Healy’s recent book, Pharmageddon, is an eye-opener, and a useful resource for understanding the history of the prominence of RTCs in modern medicine. Dr. Ben Goldacre also has a new offering coming soon, Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients, which I hope lives up to his prequels, which are stellar.
But there is a third kind of knowledge that guides many clinicians: fashion. Whatever is on the field’s radar is often what steers individual clinicians. We don’t abandon agents, such as the barbiturates or the amphetamines, because RCTs have ruled them ineffective; we abandon them because it is no longer fashionable to prescribe them (and doing so can expose one to legal action). Diagnoses such as melancholia go off the boards not because they have been demonstrated not to exist, but because such terms have become unfashionable, receding in the face of “major depression.” This third kind of knowledge — fashionable knowledge — is worse than either of the first two because it has no scientific footing at all, and because clinicians are often obliged to deny the reality of their own experiences in order to conform to it.
And speaking of fashionable, Randomized Clinical Trials have been fashionable in recent history in psychiatry, but have provided an open conduit for pharmaceutical company manipulation leading to a crisis level epidemic of overmedication in psychiatry and medicine. And the prevalence of RTCs of dubious scientific validity have produced another crisis of their own – an understandable crisis of confidence in the practice of psychiatry. One antidote to the suspicious mindset that now greets psychiatric RTCs is the Meta-Analysis, a study of studies. For example, a recent Journal of the American Medical Association reports on a meta-analysis of the clinical trials of "off-label" atypical antipsychotics use:
Off-label Atypical Antipsychotics
by Bridget M. Kuehn
Journal of the American Medical Association. 2012 308:1198.
Efficacy and Comparative Effectiveness of Atypical Antipsychotic Medications for Off-Label Uses in Adults
A Systematic Review and Meta-analysis
by Alicia Ruelaz Maher, MD; Margaret Maglione, MPP; Steven Bagley, MD; Marika Suttorp, MS; Jian-Hui Hu, MPP; Brett Ewing, MS; Zhen Wang, MS; Martha Timmer, MS; David Sultzer, MD; and Paul G. Shekelle, MD, PhD
Journal of the American Medical Association. 2011 306[12]:1359-1369.
[Full Text On-Line]
Clinical trial data as a public good.
by Rodwin MA and Abramson JD.
Journal of the American Medical Association. 2012 308[9]:871-2.
Knowledge of the benefits and risks of prescription drugs is based mainly on published reports of clinical trials, yet the medical literature may present an incomplete and potentially biased sample of clinical trials. Trials with positive results generally are published more frequently than studies that conclude that a new drug poses greater risks or is no more effective than standard therapy or a placebo. Furthermore, some articles may distort trial findings by omitting important data or by modifying prespecified outcome measures. Lack of access to detailed information about clinical trials can undermine the integrity of medical knowledge.To increase transparency, the International Committee of Medical Journal Editors decided in 2004 that their journals would not publish results of a clinical trial unless the trial was registered prior to patient enrollment. The committee stated that registries should include data specified by the World Health Organization, although these data elements do not provide a complete picture of the clinical trials. Since 2007, US law has required researchers to register phase 2 and higher trials of drugs and biologicals on the ClinicalTrials.gov website if there is a trial site in the United States or if the trial is part of a US Food and Drug Administration (FDA) investigational new drug application. Researchers are typically required to post key results within a year of completing data collection, but studies of off-label drug uses (ie, uses other than those described in an FDA-approved drug label) are allowed 3 years to post trial results. However, actual trial registration falls short of requirements. A review of 323 articles found that nearly 28% of the trials were unregistered. Among articles with adequately registered trials, 31% had discrepancies between outcomes reported in the registration and in the published report. Moreover, no authority checks whether registration information is accurate. Even more important, current law does not require registration of sufficient information to ensure accuracy, completeness, or reasonable interpretation of the findings.
Current policy does not consider a practical, inexpensive solution: mandatory disclosure of the standardized Clinical Study Report (CSR) for all clinical trials involving FDA-approved drugs. The FDA follows the International Conference on Harmonization Standards for Registration of Pharmaceuticals for Human Use, which requires submission of a CSR (with specified content and format) when reporting clinical trials to governmental authorities. The CSR summarizes the trial, clinical end points, methods, key data, and data analysis. The CSR includes “statistical description, presentations … tables and figures … with appendices containing the protocol, sample case report forms, investigator related information, information related to the test drugs/investigational products including active control/comparators, technical statistical documentation, related publications, patient data listings, and technical statistical details such as derivations, computations, analyses, and computer output etc.”
A CSR includes the most pertinent information about a clinical trial in an easily analyzed format. Drug manufacturers already produce these reports to meet international and national regulatory requirements. Making CSRs publicly available would not be expensive, yet disclosure would promote research integrity, medical knowledge, and public health. Furthermore, CSRs are more likely to be reliable than other summaries. Drug manufacturers submit CSRs to public authorities when they seek marketing approval and cannot alter or delete data without potentially jeopardizing their relationships with regulatory agencies and risking criminal prosecution…
1BOM,
Studies, even with full data transparency, will never take the place of a physician’s personal clinical experience in the great majority of instances. Even if every RCT from now on strongly recommended against the use of atypically in most patients, the pull of individual physician’s experiences, and the pull to use every tool at their disposal would leave much of practice unchanged. If physicians weren’t under the impression from their own experience that these approaches work they would have stopped using them. If psychiatry doesn’t come up with a way to filter those experiences more accurately the overuse of medication will continue. The problems in psychiatry that the RCTs were meant to address remain unchanged. The RCTs made it worse by allowing people to pretend that the problems had been addressed, and by becoming these grotesque experimercials. You and others are addressing the latter. But at the end of the day, even great RCTs are a false promise given limitations they can never overcome. As long as physicians can’t organize their personal experience in a way that allows them to better recognize when their pharmaceutical intervention has done little to nothing to enhance the positive change occurring secondary to other factors. Because few to no psychiatrists will ever stop using a pharmaceutical intervention they believe from their own experience is helpful to someone, no matter what anybody else says.
There is also the issue of test subjects and the checklists used to determine whether or not someone is “depressed” or whatever label the drug manufacturer wants to aim their study at. Many of the test subjects treat clinical trials like a job and learn what to say to get into a particular study.
Annonymous,
If psychiatry has become that front-loaded, what you say may be true. See a patient, pick a treatment, then send the patient into the ether or to brief infrequent med checks. In that situation, there is no mutative clinical experience, in fact, no real clinical experience at all. If you don’t follow patients, it becomes treatment by personal algorithm as you suggest, un-influenced by clinical feedback to organize experience and/or expertise.
In a heavily managed environment, that state of affairs is potentially the rule rather than the exception. If it is to be the only environment of mental health care, you are correct and there is no solution. And as we all know there are many more problems in that situation than just clinicians practicing with no access to clinical feedback.
But rather than lamenting the problem as being due to something like psychiatrist stubborn-ness or narcissism, I think it more reasonable to fight for psychiatrists to actually be able to fully evaluate and follow their patients, even those who choose to do medication management for patients with other therapists. Jury-rigged pharmaceutical trials are hardly the only thing wrong with mental health care today, but from my perspective, they’re wrong enough to go after with a vengeance.
We went from a situation where psychiatrists and psychiatric hospitalizations ate up too much time and dollars for too little yield. The long-term therapies and long-term hospitalizations are radically diminished, and mostly exist in only in the non-reimbursable sector. Now, we’re on the other side of the fence. We can’t hospitalize even the most desperately ill, nor adequately follow our outpatients – either to treat them properly or to learn from experience. I doubt psychiatrists are any more hard-headed than any other physicians. They’re just working in an impossible system. I see that as just another important fight, and I’m only one blog wide.
Even with all of that, I don’t share your cynicism. I do, however, share your frustration. But when I retired, I wouldn’t have even written this blog. There wouldn’t be anything much to report on, and no readers. But we now live in a time of paradigm exhaustion, and that’s a time when things can change [in either direction]. Naive or not, I still feel comforted by these winds of change…
Wiley,
That is, indeed, a big problem that isn’t addressed by data transparency. We’d love to rely on studies in “help seeking” subjects rather than volunteers/customers. Some CROs are even proposing “on line” Clinical Trials. Arrgh!
1BOM, Thank you for taking the time to share your thoughts. It is much appreciated. You are likely right to maintain your current focus, and I sincerely hope that your optimism turns out to be vindicated. Thanks again.
A commentary has just appeared in JAMA that addresses several of these concerns. See:
Bringing diagnosis into the quality and safety equations.Graber ML, Wachter RM, Cassel CK.
JAMA. 2012 Sep 26;308(12):1211-2. No abstract available.
PubMed ID: 23011708
1BOM,
I am commenting to thank you for the incredible work you are doing here. It is depressing but extremely valuable and this has become a place I check into reguarly to get an honest, thoughtful view of psychiatry.