1 Boring Old Man

Decline in placebo-controlled trial results suggests new directions for comparative effectiveness research.
^{by Olfson M and Marcus SC.}
Health Affairs. 2013 32[6]:1116-1125.

^{The Affordable Care Act offers strong support for comparative effectiveness research, which entails comparisons among active treatments, to provide the foundation for evidence-based practice. Traditionally, a key form of research into the effectiveness of therapeutic treatments has been placebo-controlled trials, in which a specified treatment is compared to placebo. These trials feature high-contrast comparisons between treatments. Historical trends in placebo-controlled trials have been evaluated to help guide the comparative effectiveness research agenda. We investigated placebo-controlled trials reported in four leading medical journals between 1966 and 2010. We found that there was a significant decline in average effect size or average difference in efficacy [the ability to produce a desired effect] between the active treatment and placebo. On average, recently studied treatments offered only small benefits in efficacy over placebo. A decline in effect sizes in conventional placebo-controlled trials supports an increased emphasis on other avenues of research, including comparative studies on the safety, tolerability, and cost of treatments with established efficacy.}

1. National Trends in the Office-Based Treatment of Children, Adolescents, and Adults With Antipsychotics. Archives of General Psychiatry. 2012 69[12]:1247-1256.
2. National Trends in the Antipsychotic Treatment of Psychiatric Outpatients With Anxiety Disorders. American Journal of Psychiatry 2011 168:1057-1065.
3. Proportion Of Antidepressants Prescribed Without A Psychiatric Diagnosis Is Growing. Health Affairs [2011] 30:1434-1442.
4. National trends in the outpatient diagnosis and treatment of bipolar disorder in youth. Archives of General Psychiatry. 2007 Sep;64[9]:1032-1039.

I graphed some of their tabular data to make it easier to see [all were significant]. On the left is the percent of Clinical Trials where PHARMA funding was acknowledged. This figure is confusing as that only became a requirement during the study period [NEJM 1985, JAMA 1989, BMJ 1994, Lancet 1994]. On the right, the number of authors rose from a median of 2-3 to 8-20 over the study period! Who knew?

The left graph below shows the median number of sites used in the clinical trials. The trend to multiple sites is a striking phenomenon of the recent past, I presume reflecting the ascendency of of the Clinical Research Organizations. Similarly, in the center the median number of study subjects has soared over the last two decades [chasing significance]. And on the right, another way of showing the falling effect size – the mean odds ratio.

And finally some miscellaneous stats from their tables. In this study the % of non-US studies, studies with insignificant Primary Outcomes, and drop-out rates did not change significantly. Intent-to-treat analysis means all subjects that start the study are included including drop-outs etc [corrected for missing data]. Studies reporting number of subjects screened and ITT analysis increased significantly.

^{[reformatted]}

Substantial investments are made in clinical trials. In the United States, for example, more than $100 billion is spent each year on biomedical research, with most of the funding devoted to clinical trials.  Concern over a slowdown in the discovery of innovative medical treatments has prompted calls for new public policies to stimulate drug development, including incentives for novelty drugs. During difficult economic times, however, private research sponsors may be tempted to take a cautious approach that favors incremental research focused on follow-on or “me too” drugs that offer little additional efficacy over more speculative high-risk/high-reward strategies.

The Affordable Care Act mandates a new national comparative effectiveness research agenda. Its great promise is to build the scientific foundation of clinical practice. Yet forty years of declining effect sizes in traditional placebo-controlled trials underscores the increasing challenge of discovering breakthrough interventions. Where established treatments achieve comparable efficacy, comparative effective research can help identify the safer, less expensive, and better-tolerated alternative. By placing clinical decision making on firmer footing, it is hoped that comparative effectiveness research will help patients receive the best possible care and save them from unnecessary risks, inconvenience, and costs.

I obviously liked this study – well-done, timely, well-analyzed. I’m not sure I’m through digesting it, but several things occurred to me. First, my focus has been on the deceit in the psychiatric clinical trial world and I tend to see the problems with clinical trials as localized to psychiatry. This study widens that lens to all of medicine. I was surprised by that. Besides deceit, I’ve also tended to blame the falling effect sizes on the Clinical Research Industry using recruited subjects often in far-away lands with culturally diverse subgroups. So I wouldn’t have expected the same decline in trials of biomarker-confirmed physical diseases. But there it is.

It’s the rare clinical trial article that doesn’t comment on the small effect sizes these days – blaming the placebo effect or other karmic forces. And this paper makes it very clear that they are escalating the number of subjects [requiring multiple sites] to chase significance with smaller effect sizes. But I find it impossible to look at this data and conclude anything except that, on the whole, they’re testing weaker and weaker drugs.

^{hat tip to Pharmagossip…}

Declining Differences in Response Rates with Antidepressants versus Placebo: A Modest Proposal for Another Contributing Cause.
^{by Preskorn SH.}
Journal of Psychiatric Practice. 2013 19:227-233.

^{This column discusses declining differences in response rates between sequentially introduced selective serotonin reuptake inhibitors [SSRI] and placebo. Although discussions of this phenomenon in the literature have largely focused on increasing placebo response rates, the author proposes that another factor may be responsible. That factor is an order effect, meaning that response rates have been declining as a function of the number of SSRIs on the market when the next SSRI is in development. The rationale is that the pool of potential clinical trial participants likely to respond to a drug with this mechanism of action [MOA] becomes progressively smaller with the introduction of each new agent with the same MOA, because many patients will already have been treat- ed and responded to an earlier member of the class. This phenomenon is not limited to the SSRIs but generalizes to any class of treatments that shares the same MOA.}

^{hat tip to Helge…}