a comfort to this old man…

Posted on Thursday 27 September 2012

A recent discussion thread explored the topic of clinical intuition based on a physician’s experience versus reliance on RTCs [Randomized Clinical Trials], the gold standard of evidence-based medicine. Right now, that’s a front burner issue, particularly in clinical psychiatry. Dr. David Healy’s recent book, Pharmageddon, is an eye-opener, and a useful resource for understanding the history of the prominence of RTCs in modern medicine. Dr. Ben Goldacre also has a new offering coming soon, Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients, which I hope lives up to his prequels, which are stellar.

In the best of all possible worlds, clinical experience and the information from RTCs should not necessarily be a dichotomy, rather complementary sources that mutually inform one another. But this is not the world of Voltaire’s Candide. The nursing home doctor who puts everyone on atypical antipsychotics every night invoking clinical experience and the industry financed, ghost-written, experimercial RTCs make mincemeat out of any rational discussion of this topic. In the comments, Dr. Shorter added a third factor to clinical experience and evidence-based studies, fashion:
But there is a third kind of knowledge that guides many clinicians: fashion. Whatever is on the field’s radar is often what steers individual clinicians. We don’t abandon agents, such as the barbiturates or the amphetamines, because RCTs have ruled them ineffective; we abandon them because it is no longer fashionable to prescribe them (and doing so can expose one to legal action). Diagnoses such as melancholia go off the boards not because they have been demonstrated not to exist, but because such terms have become unfashionable, receding in the face of “major depression.” This third kind of knowledge — fashionable knowledge — is worse than either of the first two because it has no scientific footing at all, and because clinicians are often obliged to deny the reality of their own experiences in order to conform to it.
Great point. And I would also add morality. There’s a long history of various medications prescribed for anxiety symptoms reaching into antiquity. The most recent wave was the Benzodiazepines – effective but habituating and/or addicting. They’ve fallen out of fashion, but more than that, their use has become seen as immoral – in the range of "street drugs." But anxious patients in their many flavors have hardly disappeared, and I suspect that a segment of the cohort now being given atypical antipsychotics are the patients formerly prescribed Valium or Xanax. Atypical antipsychotics became fashionable, but not immoral – the modern "nerve pill."

And speaking of fashionable, Randomized Clinical Trials have been fashionable in recent history in psychiatry, but have provided an open conduit for pharmaceutical company manipulation leading to a crisis level epidemic of overmedication in psychiatry and medicine. And the prevalence of RTCs of dubious scientific validity have produced another crisis of their own – an understandable crisis of confidence in the practice of psychiatry. One antidote to the suspicious mindset that now greets psychiatric RTCs is the Meta-Analysis, a study of studies. For example, a recent Journal of the American Medical Association reports on a meta-analysis of the clinical trials of "off-label" atypical antipsychotics use:

Off-label Atypical Antipsychotics
by Bridget M. Kuehn
Journal of the American Medical Association. 2012 308:1198.
It refers to this Agency for Healthcare Research and Quality [AHRQ] report:
Efficacy and Comparative Effectiveness of Atypical Antipsychotic Medications for Off-Label Uses in Adults
A Systematic Review and Meta-analysis
by Alicia Ruelaz Maher, MD; Margaret Maglione, MPP; Steven Bagley, MD; Marika Suttorp, MS; Jian-Hui Hu, MPP; Brett Ewing, MS; Zhen Wang, MS; Martha Timmer, MS; David Sultzer, MD; and Paul G. Shekelle, MD, PhD
Journal of the American Medical Association. 2011 306[12]:1359-1369.
[Full Text On-Line]
A bit of efficacy here and there. An uncomfortable side effect profile. But certainly nothing that would support the wide use of these medications in clinical practice. It’s worth reviewing even if you just look at the pictures and tables. Atypical antipsychotics are no panacea for anything – not even close. But my topic isn’t really the atypicals, it’s more about the issue of RTCs and meta-analyses in general. We can’t just rely on meta-analyses to evaluate the RTCs that increasingly haunt our journals, as helpful as they can be. I’ll spare you any more comments about the Gibbons’ meta-analysis of antidepressants. That point has been made, at least to my satisfaction [an anatomy of a deceit 1… etc] – the point being that meta-analyses can be manipulated too. But there are other problems. Meta-analyses come late in a drug’s life. In the case of this AHRQ report, atypicals have been widely prescribed "off-label" for well over a decade [as blockbusters], so the report is more a latter-day confirmation than a warning. And meta-analyses deal with the reported findings, so whatever distortions are in the originals get passed "upstairs." There is a growing consensus that what we need instead is full data transparency. Rather than relying on just meta-analyses of reported information, we need re-analysis, the ability to bring a whole new set of unbiased eyes to the complete raw data-set from a clinical trial. It’s the point of Ben Greenacre’s TED talk. It was the point of my obsession with the Paxil Study 329 raw data [the lesson of Study 329: the basics… etc]. It’s the point of the TEST Act, and the recent action of the European version of the FDA [EU agency lifts lid on drug data secrets]. And it’s the point of this thoughtful article, in a recent Journal of the American Medical Association:
Clinical trial data as a public good.
by Rodwin MA and Abramson JD.
Journal of the American Medical Association. 2012 308[9]:871-2.

Knowledge of the benefits and risks of prescription drugs is based mainly on published reports of clinical trials, yet the medical literature may present an incomplete and potentially biased sample of clinical trials. Trials with positive results generally are published more frequently than studies that conclude that a new drug poses greater risks or is no more effective than standard therapy or a placebo. Furthermore, some articles may distort trial findings by omitting important data or by modifying prespecified outcome measures. Lack of access to detailed information about clinical trials can undermine the integrity of medical knowledge.

To increase transparency, the International Committee of Medical Journal Editors decided in 2004 that their journals would not publish results of a clinical trial unless the trial was registered prior to patient enrollment. The committee stated that registries should include data specified by the World Health Organization, although these data elements do not provide a complete picture of the clinical trials. Since 2007, US law has required researchers to register phase 2 and higher trials of drugs and biologicals on the ClinicalTrials.gov website if there is a trial site in the United States or if the trial is part of a US Food and Drug Administration (FDA) investigational new drug application. Researchers are typically required to post key results within a year of completing data collection, but studies of off-label drug uses (ie, uses other than those described in an FDA-approved drug label) are allowed 3 years to post trial results. However, actual trial registration falls short of requirements. A review of 323 articles found that nearly 28% of the trials were unregistered. Among articles with adequately registered trials, 31% had discrepancies between outcomes reported in the registration and in the published report. Moreover, no authority checks whether registration information is accurate. Even more important, current law does not require registration of sufficient information to ensure accuracy, completeness, or reasonable interpretation of the findings.

Current policy does not consider a practical, inexpensive solution: mandatory disclosure of the standardized Clinical Study Report (CSR) for all clinical trials involving FDA-approved drugs. The FDA follows the International Conference on Harmonization Standards for Registration of Pharmaceuticals for Human Use, which requires submission of a CSR (with specified content and format) when reporting clinical trials to governmental authorities. The CSR summarizes the trial, clinical end points, methods, key data, and data analysis. The CSR includes “statistical description, presentations … tables and figures … with appendices containing the protocol, sample case report forms, investigator related information, information related to the test drugs/investigational products including active control/comparators, technical statistical documentation, related publications, patient data listings, and technical statistical details such as derivations, computations, analyses, and computer output etc.”

A CSR includes the most pertinent information about a clinical trial in an easily analyzed format. Drug manufacturers already produce these reports to meet international and national regulatory requirements. Making CSRs publicly available would not be expensive, yet disclosure would promote research integrity, medical knowledge, and public health. Furthermore, CSRs are more likely to be reliable than other summaries. Drug manufacturers submit CSRs to public authorities when they seek marketing approval and cannot alter or delete data without potentially jeopardizing their relationships with regulatory agencies and risking criminal prosecution…

[This quoted piece is from the publicly available on-line offering]. I personally find this growing chorus of people saying that it’s time to stop simply lamenting the fact that our journals have been allowed to fill up with junk science and begin to do what’s necessary to stem the tide downright uplifting. As a young doctor, I saw the medical literature as something like the Library at Alexandria in ancient history. And in my medical lifetime, a big piece of it caught on fire. These beginning efforts to restore its integrity are long overdue and a comfort to this old man…
    September 27, 2012 | 2:01 PM

    Studies, even with full data transparency, will never take the place of a physician’s personal clinical experience in the great majority of instances. Even if every RCT from now on strongly recommended against the use of atypically in most patients, the pull of individual physician’s experiences, and the pull to use every tool at their disposal would leave much of practice unchanged. If physicians weren’t under the impression from their own experience that these approaches work they would have stopped using them. If psychiatry doesn’t come up with a way to filter those experiences more accurately the overuse of medication will continue. The problems in psychiatry that the RCTs were meant to address remain unchanged. The RCTs made it worse by allowing people to pretend that the problems had been addressed, and by becoming these grotesque experimercials. You and others are addressing the latter. But at the end of the day, even great RCTs are a false promise given limitations they can never overcome. As long as physicians can’t organize their personal experience in a way that allows them to better recognize when their pharmaceutical intervention has done little to nothing to enhance the positive change occurring secondary to other factors. Because few to no psychiatrists will ever stop using a pharmaceutical intervention they believe from their own experience is helpful to someone, no matter what anybody else says.

    September 27, 2012 | 2:31 PM

    There is also the issue of test subjects and the checklists used to determine whether or not someone is “depressed” or whatever label the drug manufacturer wants to aim their study at. Many of the test subjects treat clinical trials like a job and learn what to say to get into a particular study.

    September 27, 2012 | 3:24 PM


    If psychiatry has become that front-loaded, what you say may be true. See a patient, pick a treatment, then send the patient into the ether or to brief infrequent med checks. In that situation, there is no mutative clinical experience, in fact, no real clinical experience at all. If you don’t follow patients, it becomes treatment by personal algorithm as you suggest, un-influenced by clinical feedback to organize experience and/or expertise.

    In a heavily managed environment, that state of affairs is potentially the rule rather than the exception. If it is to be the only environment of mental health care, you are correct and there is no solution. And as we all know there are many more problems in that situation than just clinicians practicing with no access to clinical feedback.

    But rather than lamenting the problem as being due to something like psychiatrist stubborn-ness or narcissism, I think it more reasonable to fight for psychiatrists to actually be able to fully evaluate and follow their patients, even those who choose to do medication management for patients with other therapists. Jury-rigged pharmaceutical trials are hardly the only thing wrong with mental health care today, but from my perspective, they’re wrong enough to go after with a vengeance.

    We went from a situation where psychiatrists and psychiatric hospitalizations ate up too much time and dollars for too little yield. The long-term therapies and long-term hospitalizations are radically diminished, and mostly exist in only in the non-reimbursable sector. Now, we’re on the other side of the fence. We can’t hospitalize even the most desperately ill, nor adequately follow our outpatients – either to treat them properly or to learn from experience. I doubt psychiatrists are any more hard-headed than any other physicians. They’re just working in an impossible system. I see that as just another important fight, and I’m only one blog wide.

    Even with all of that, I don’t share your cynicism. I do, however, share your frustration. But when I retired, I wouldn’t have even written this blog. There wouldn’t be anything much to report on, and no readers. But we now live in a time of paradigm exhaustion, and that’s a time when things can change [in either direction]. Naive or not, I still feel comforted by these winds of change…

    September 27, 2012 | 3:32 PM


    That is, indeed, a big problem that isn’t addressed by data transparency. We’d love to rely on studies in “help seeking” subjects rather than volunteers/customers. Some CROs are even proposing “on line” Clinical Trials. Arrgh!

    September 27, 2012 | 4:47 PM

    1BOM, Thank you for taking the time to share your thoughts. It is much appreciated. You are likely right to maintain your current focus, and I sincerely hope that your optimism turns out to be vindicated. Thanks again.

    Bernard Carroll
    September 27, 2012 | 5:33 PM

    A commentary has just appeared in JAMA that addresses several of these concerns. See:
    Bringing diagnosis into the quality and safety equations.Graber ML, Wachter RM, Cassel CK.
    JAMA. 2012 Sep 26;308(12):1211-2. No abstract available.
    PubMed ID: 23011708

    September 28, 2012 | 12:08 AM

    I am commenting to thank you for the incredible work you are doing here. It is depressing but extremely valuable and this has become a place I check into reguarly to get an honest, thoughtful view of psychiatry.

Sorry, the comment form is closed at this time.