1 Boring Old Man

A manifesto for reproducible science [PDF]

by Marcus R. Munafo, Brian A. Nosek, Dorothy V. M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware, and John P. A. loannidis

Nature: Human Behavior. Published 10 January 2017. Open.

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.

^{[abbreviated and reformatted from the paper]}

From my perspective, there’s nothing more important in Medicine right now than reclaiming the academic medical literature from its captivity by the paramedical industries and others who are called stakeholders. But the problem in academic science is bigger than just Medicine. In the other fields, it goes under the name, The Reproducibility Crisis.

No New Antidepressants in Sight Despite Growing Need, Experts Warn

Reuters

By Kate Kelland

January 11, 2017

LONDON — It is likely to be at least 10 years before any new generation of antidepressants comes to market, despite evidence that depression and anxiety rates are increasing across the world, specialists said on Wednesday. The depression drug pipeline has run dry partly due to a "failure of science" they said, but also due to big pharma pulling investment out of research and development in the neuroscience field because the profit potential is uncertain. "I’d be very surprised if we were to see any new drugs for depression in the next decade. The pharmaceutical industry is simply not investing in the research because it can’t make money from these drugs," Guy Goodwin, a professor of psychiatry at the University of Oxford, told reporters at a London briefing.

Andrea Cipriani, a consultant psychiatrist at Oxford, said such risk aversion was understandable given uncertain returns and the approximately billion dollar cost of developing and bringing a new drug to market. "It’s a lot of money to spend, and there’s a high rate of failure," Cipriani said. Treatment for depression usually involves either medication, some form of psychotherapy, or a combination of both. But up to half of all people treated fail to get better with first-line antidepressants, and around a third of patients are resistant to relevant medications.

There were many attempts to enhance efficacy – sequencing, combining, augmenting with a variety of other drugs. Non-responders were said to have Treatment Resistant Depression, discussed almost as if it represented a unique entity. Multiple markers were queried looking for something that would predict the right drug – called Personalized Medicine. Practitioners and patients alike kept their eyes on the future – what’s coming down the pipeline. And there was a vague sense that the newer drugs were improvements over the earlier offerings, though that’s hard to justify in retrospect. Somewhere in there, the notion developed that the incidence of depression was rising rapidly, although that was hard to put together with the predominant view that depression was a biological-?-genetic entity. And the scientific basis for that escalating prevalence is hard to pin down.

And then in the summer of 2012, the Pharmaceutical companies threw in the towel and began to shut down their R&D programs for CNS drugs. They’d run out of candidates ["me too drugs"]. A great wail was heard throughout the land. There were conferences and task forces – much rhetoric and blaming. The NIMH seemed to have a new idea about how to jump-start drug development  every month. Multiple schemes were proposed to lure PHARMA back into the game. And all eyes turned to the search for something "novel" to keep things alive [eg Ketamine and its derivatives].

DEPRESSION RATES RISING

The experts said that since the current generation of SSRI [selective serotonin reuptake inhibitors] antidepressants – including Pfizer’s blockbuster Prozac [fluoxetine] – are widely available as cheap generics, there is reluctance among health services to fund expensive new drugs that may not be much better. That is partly because existing medications, while by no means perfect, are quite effective in more than half of patients, the specialists said, and partly because in this condition in particular, placebo can have a massive impact. That makes it difficult, they explained, to show that a new drug is working above and beyond a positive placebo response and an already effective generation of available drugs.

Depression is already one of the most common forms of mental illness, affecting more than 350 million people worldwide and ranking as the leading cause of disability globally, according to the World Health Organization. And rates are rising. Glyn Lewis, a professor of psychiatric epidemiology at University College London, cited data for England showing a doubling in prescriptions for antidepressants in a decade, to 61 million in 2015 from 31 million in 2005.

In the United States too, more people than ever are taking antidepressants. A study in the Journal of the American Medical Association [JAMA] in 2015 found that prevalence almost doubled from 1999 to 2012, rising to 13 from 6.9 percent. Yet several major drug companies including GlaxoSmithKline and AstraZeneca have scaled right back on neuroscience R&D in recent years, citing unfavorable risk-reward prospects.

Goodwin said the absence of a drug development pipeline was also due to lagging scientific research into what is really happening in the brains of those who do and do not respond to current antidepressants. "It’s partly a failure of science, to be frank," said Goodwin. "Scientists have to … get more of an understanding about how these things actually work before we can then propose ways to improve them."

With all due respect to Dr. Goodwin, his pronouncement might’ve worked in the 90s [the Decade of the Brain] or the 2000s [the Research Agenda for the DSM-V]. But after thirty years, this argument itself has run out of mojo too. The scientists have scienced themselves silly trying to do what he suggests without much success. They’ve certainly gone through a small fortune in the process. The marketeers have had more success, raking in a beyond-modest fortune in the process. But this train is pulling into the station, its journey’s almost done. 

A supernova is a stellar explosion that briefly outshines an entire galaxy, radiating as much energy as the Sun or any ordinary star might emit over its lifespan. This astronomical event occurs during the last stages of a massive star’s life, whose dramatic and catastrophic destruction is marked by one final titanic explosion concentrated in a few seconds, creating a "new" bright star that gradually fades from sight over several weeks or months.

[click image for info]

Will the SSRIs have the same kind of fate as SN2014J? evaporating into the ether? I doubt it. At least not any time soon. They’re still useful in clinical practice when used carefully and in moderation. I expect the short acting ones will gradually disappear because of their heightened withdrawal profiles. And hopefully the others will be used in a more time limited way. And then, maybe we can get around to reworking our diagnostic system to bring it closer to clinical reality.

We’ll see… and speaking of shiny objects:

ASASSN-15lh [supernova designation SN 2015L] is a bright astronomical object. Initially thought to be a superluminous supernova, it was detected by the All Sky Automated Survey for SuperNovae [ASAS-SN] in 2015 in the southern constellation Indus. The discovery, confirmed by ASAS-SN group with several other telescopes, was formally described and published in a Science article led by Subo Dong at the Kavli Institute of Astronomy and Astrophysics [Peking University, China] on January 15, 2016. In December 2016, another group of scientists raised a hypothesis that ASASSN-15lh might not be a supernova. Based on observations from several stations on the ground and in space [including Hubble], these scientists proposed that this bright object might have been "caused by a rapidly spinning supermassive black hole as it destroyed a low-mass star". ASASSN-15lh, if a supernova, would be the most luminous ever detected; at its brightest, it was approximately 50 times more luminous than the whole Milky Way galaxy with an energy flux 570 billion times greater than the Sun….

[click image for info]

When our group assembled to do our RIAT analysis of Paxil Study 329, we already had access to a wealth of raw data from that clinical trial thanks to the hard work of many other people who came before us. So we had the a prori Protocol, the Statistical Analysis Plan, the CSR [Clinical Study Report], and the IPD [Individual Participant Data] – all available in the Public Domain as the result of various Legal Settlements and Court Orders. The only thing we didn’t have – the CRFs [Case Report Forms] – the actual raw forms the raters used to record their observations during the study. But we felt that we needed them too. We had good reason to question the system originally used to code the Adverse Events, and felt it was important to redo that part from scratch using a more modern and widely validated system.

About that time, the European Medicines Agency [EMA] had announced that it was going to release all of its regulatory data. AllTrials was pressing for "all trials registered, all trials reported." I was researching on what authority the data was being kept proprietary in the first place, and finding nothing much except convention and inertia. What was being called Data Transparency was in the air, and it was an exciting prospect.

And then the pharmaceutical companies seemed to do a turnabout. GSK had just been hit with a $3 B fine, in part over Study 329, and they were one of the first to sign on to AllTrials. But as things developed, what they offered was something different from what a lot of us reeally wanted, at least what I wanted. By that time, I wasn’t a rookie any more and I’d vetted a number of industry funded, ghost written, psychopharmacology drug trials  turned into journal articles. I can’t recall a one of them was totally straight. So I wanted to see what the drug company saw – the a priori Protocol and Statistical Analysis Plan, the IPD, and the CRFs – the raw data. And the reason wasn’t to do any new research. It was to check their reported work, to do it right by the book, to stop the cheating.

And so with much fanfare, what the drug companies rolled out was something else – Data Sharing. They pretended that what we wanted was access to their raw data so we could do further new research – and that they were being real mensches to let us see it. They set up independent boards to evaluate proposals for projects. If we passed muster, we could have access via a remote desktop – meaning we couldn’t download the data. We could only see it online. All we could download were our results, if approved. In this scenario, they are generously sharing the data with us, avoiding duplication and wastage or some such, and the remote access portal protects the subjects’ privacy. They maintained control and ownership. What we wanted was Data Transparency to keep them honest, to stop them from publishing these fictional photo-shopped articles, to stop the cheating.

So our RIAT group submitted a request to their panel, and when they asked for a proposal, we didn’t make one up. We played it straight and told them why. After some back and forth, we submitter the Protocol from the original Study 329, and to their credit, they granted our request. The remote access system actually worked, but working inside of it was a complete nightmare [we called it "the periscope"]. The CRFs came to around 50,000 pages, and we could only look at them one page at a time! But that’s another story and it’s available in detail at https://study329.org/. The point for this post is that the call for Data Transparency got turned into something very different – Data Sharing. That’s called "SPIN." Instead of being on the hot-seat for having published so many distorted clinical trial reports – carefully crafted by professional ghost-writers – they portrayed themselves as heros, generously allowing outsiders to use their data for independent research. Sleight of hand extraordinaire!

So what does this have to do with the New England Journal of Medicine, and editor Jeffrey Drazen, and Data Transparency versus Data Sharing, and a bully pulpit? A lot – some mentioned in this series from in April 2016.

• wolf alert! wolf alert!…
• the making of « wolf alert! wolf alert!… » part 1
• the making of « wolf alert! wolf alert!… » part 2
• the making of « wolf alert! wolf alert!… » part 3

Data Sharing

by Dan L. Longo, and Jeffrey M. Drazen

New England Journal of Medicine. 2016  374:276-277.

[full text on-line]

The aerial view of the concept of data sharing is beautiful. What could be better than having high-quality information carefully reexamined for the possibility that new nuggets of useful data are lying there, previously unseen? The potential for leveraging existing results for even more benefit pays appropriate increased tribute to the patients who put themselves at risk to generate the data. The moral imperative to honor their collective sacrifice is the trump card that takes this trick…

A second concern held by some is that a new class of research person will emerge — people who had nothing to do with the design and execution of the study but use another group’s data for their own ends, possibly stealing from the research productivity planned by the data gatherers, or even use the data to try to disprove what the original investigators had posited. There is concern among some front-line researchers that the system will be taken over by what some researchers have characterized as “research parasites”…

Revisiting the Commercial–Academic Interface
• Revisiting the Commercial–Academic Interface
  May 7, 2015 | Jeffrey M. Drazen, M.D.
• Reconnecting the Dots — Reinterpreting Industry–Physician Relations
  May 7, 2015 | Lisa Rosenbaum, M.D.
• Understanding Bias — The Case for Careful Study
  May 14, 2015 | Lisa Rosenbaum, M.D.
• Beyond Moral Outrage — Weighing the Trade-Offs of COI Regulation
  May 21, 2015 | Lisa Rosenbaum, M.D.

• Richard Lehman’s journal review
  [NEJM 21 May 2015 Vol 372]
  British Medical Journal
  May 20, 2015 | Richard Lehman
• Say It Ain’t So: Logical Fallacies in Defense of Conflicts of Interest … in the New England Journal of Medicine?
  Healthcare Renewal
  May 21, 2015 | Roy Poses, M.D.
• Justifying conflicts of interest in medical journals: a very bad idea
  British Medical Journal
  June 2, 2015 | Robert Steinbrook, Jerome P Kassirer, and Marcia Angell
• Revisiting the commercial-academic interface in medical journals
  The New England Journal of Medicine goes on an ill advised journey
British Medical Journal
June 2, 2015 | Elizabeth Loder, Catherine Brizzell, and Fiona Godlee
• Offline: The BMJ vs NEJM — lessons for us all
  The Lancet
  June 6, 2015 | Richard Horton
• Conflict of interest: forward not backward
  British Medical Journal
  June 11, 2015 | Fiona Godlee

Then, someone sent me a link to this month’s The Large Pharmaceutical Company Perspective about several heroic PHARMA adventures. I noticed it was from a series called The Changing Face of Clinical Trials, so I ran down the rest of the series and read them all. And then I found some other NEJM Clinical Trial Offerings offerings in 2016. 

The Changing Face of Clinical Trials
1. Clinical Trials Series
   June 2, 2016 | J. Woodcock and Others
   With this issue, we launch a series of articles that deal with contemporary challenges that affect clinical trialists today. Articles will define a specific issue of interest and illustrate it with examples from actual practice, as well as bring additional history and color to the topic.
2. Comparative Effectiveness Studies and Patient Care
   June 2, 2016 | L.D. Fiore and P.W. Lavori
   Investigators use adaptive trial designs to alter basic features of an ongoing trial. This approach obtains the most information possible in an unbiased way while putting the fewest patients at risk. In this review, the authors discuss selected issues in adaptive design.
3. Adaptive Designs for Clinical Trials
   August 4, 2016 | I. Ford and J. Norrie
   Investigators use adaptive trial designs to alter basic features of an ongoing trial. This approach obtains the most information possible in an unbiased way while putting the fewest patients at risk. In this review, the authors discuss selected issues in adaptive design.
4. Pragmatic Trials
   August 4, 2016 | I. Ford and J. Norrie
   In pragmatic trials, participants are broadly representative of people who will receive a treatment or diagnostic strategy, and the outcomes affect day-to-day care. The authors review the unique features of pragmatic trials through a wide-ranging series of exemplar trials.
5. The Primary Outcome Fails — What Next?
   September 1, 2016 | S.J. Pocock and G.W. Stone
   When the primary outcome of a clinical trial fails to reach its prespecified end point, can any clinically meaningful information still be derived from it? This review article addresses that question.
6. Considerations When the Primary Outcome Is Positive
   September 8, 2016 | S.J. Pocock and G.W. Stone
   When a clinical trial reaches its primary outcome, several issues must be considered before a clinical message is drawn. These issues are reviewed in this article.
7. Data Monitoring Committees — Expect the Unexpected
   October 6, 2016 | D.L. DeMets and S.S. Ellenberg
   Randomized clinical trials require a mechanism to safeguard the enrolled patients from harm that could result from participation. This article reviews the role of data monitoring committees in the performance of randomized clinical trials.
8. Lessons from Clinical Trials Involving Hypertension
   November 3, 2016 | M.A. Pfeffer and J.J.V. McMurray
   Ethical issues can arise in the design and conduct of clinical trials. Using the trials that set the stage for our current treatment of hypertension, the authors show how the changing treatment landscape raised ethical problems as these trials were undertaken.
9. The Large Pharmaceutical Company Perspective
   January 5, 2017 | M. Rosenblatt
   The former chief medical officer of a large pharmaceutical company addresses the issue of complexity and how it affects the performance of clinical trials.

The Final Rule
• Trial Reporting in ClinicalTrials.gov
  September 16, 2016 | D.A. Zarin and Others
  The final rule for reporting clinical trial results has now been issued by the Department of Health and Human Services. It aims to increase accountability in the clinical research enterprise, making key information available to researchers, funders, and the public.

History of Clinical Trials
1. Lessons from the History of RCTs
   June 2, 2016 | L.E. Bothwell and Others
2. Medicine, Monopoly, and the Premodern State
   July 14, 2016 | A. Rankin and J. Rivest
3. Emergence of the RCT
   August 11, 2016 | L.E. Bothwell and S.H. Podolsky
4. Clinical Trials, Healthy Controls, and the IRB
   September 15, 2016 | L. Stark and J.A. Greene

Data Sharing
1. Collaborative Clinical Trials
   March 3, 2011 | A.J. Moss, C.W. Francis, and D. Ryan
2. Pragmatic Trials — Guides to Better Patient Care?

   May 5, 2011 | J.H. Ware and M.B. Hamel
3. The Prevention and Treatment of Missing Data in Clinical Trials
   October 4, 2012 | R.J. Little and Others
4. Preparing for Responsible Sharing of Clinical Trial Data
   October 24, 2013 | M.M. Mello and Others
5. Data Sharing, Year 1 — Access to Data from Industry-Sponsored Clinical Trials
   November 27, 2014 | B.L. Strom and Others
6. Transparency and the European Medicines Agency — Sharing of Clinical Trial Data
   December 25, 2014 | S. Bonini and Others
7. The Proposed Rule for U.S. Clinical Trial Registration and Results Submission
   January 8, 2015 | D.A. Zarin, T. Tse, and J. Sheehan
8. Sharing Individual Patient Data from Clinical Trials
   January 15, 2015 | J.M. Drazen
9. Adaptive Phase II Trial Design
   July 7, 2015 | D. Harrington and G. Parmigiani
10. Sharing Data from Cardiovascular Clinical Trials
    August 4, 2015 | The Academic Research Organization Consortium for Continuing Evaluation of Scientific Studies — Cardiovascular (ACCESS CV)
11. Toward Fairness in Data Sharing
    August 4, 2015 | The International Consortium of Investigators for Fairness in Trial Data Sharing
12. The Yale Open Data Access (YODA) Project
    August 4, 2015 | H.M. Krumholz and J. Waldstreicher
13. Data Sharing
    January 21, 2016 | Dan L. Longo, and Jeffrey M. Drazen
14. Strengthening Research through Data Sharing
    August 4, 2016 | E. Warren
15. Data Sharing at a Crossroads
    September 22, 2016 | F. Rockhold, P. Nisen, and A. Freeman
16. Incentives for Clinical Trialists to Share Data
    September 22, 2016 | B. Lo and D.L. DeMets
17. Toward a Shared Vision for Cancer Genomic Data
    September 22, 2016 | R.L. Grossman and Others
18. Data Sharing — Is the Juice Worth the Squeeze?
    October 27, 2016 | B.L. Strom and Others

Medicare Payment for Behavioral Health Integration

by Matthew J. Press, M.D., Ryan Howe, Ph.D., Michael Schoenbaum, Ph.D., Sean Cavanaugh, M.P.H., Ann Marshall, M.S.P.H., Lindsey Baldwin, M.S., and Patrick H. Conway, M.D.

New England Journal of Medicine. December 14, 2016

DOI: 10.1056/NEJMp1614134

[full text on-line]

For example, under CoCM, if a 72-year-old man with hypertension and diabetes presents to his primary care clinician feeling sad and anxious, the primary care team [primary care clinician and behavioral health care manager] would conduct an initial clinical assessment using validated rating scales. If the patient has a behavioral health condition [e.g., depression] and is amenable to treatment, the primary care team and the patient would jointly develop an initial care plan, which might include pharmacotherapy, psychotherapy, or other indicated treatments. The care manager would follow up with the patient proactively and systematically [using a registry] to assess treatment adherence, tolerability, and clinical response [again using validated rating scales] and might provide brief evidence-based psychosocial interventions such as behavioral activation [which focuses on helping people with mood disorders to engage in beneficial activities and behavior] or motivational interviewing. In addition, the primary care team would regularly review the patient’s care plan and status with the psychiatric consultant and would maintain or adjust treatment, including referral to behavioral health specialty care as needed.

"an initial clinical assessment using validated rating scales. If the patient has a behavioral health condition [e.g., depression] and …"

"[again using validated rating scales] and might provide brief evidence-based psychosocial interventions such as…"

1. adjectives saying that something is evidence-based [meaning positive clinical trials]: validated rating scales, evidence-based psychosocial interventions, evidence based psychotherapy, indicated treatments, guideline approved this and that, etc. The gist of things is that only group-certified interventions are valid…
2. traditional language is de-psychiatrized and de-medicalized: behavioral health care manager, behavioral health condition, rating scales, behavioral health specialty. behavioral activation.
3. strict control, limiting choices and duration of anything: [now we get to brief] – particularly any face to face contact with psychiatrists.
4. adverbs implying contientiousness and industry: proactively and systematically [using a registry]

Just a couple of observations. In every example, the cases are universally lite – unlikely reaching anyone’s standards for mental illness proper. I didn’t really know what behavioral activation and motivational interviewing were. I watched a few youtube videos and looked at several trials of the latter [and there have been many] with widely varying results. They’re behavior modification interview techniques. But the main thing I took away from thinking about this hadn’t occurred to me before. In a way, I’ve already done this myself for over thirty years. I did it in the clinic where I’ve been working for the last eight. And when I was on the faculty, I did it somewhere in some affiliated facility on most days. The med students, or residents, or staff saw the patients and presented them. Sometimes I said "fine." Sometimes we talked about the case. And sometimes I saw the patient myself. I expect most psychiatrists have done this kind of thing at some point in their career for years. But I’m absolutely sure I wouldn’t do this one.

The secret to being able to supervise other clinicians in a situation like the one described here isn’t some encyclopedic knowledge of medications, or diagnoses. It’s in being able to get to know how to read the clinician you’re supervising. Early on, I expect I saw most cases a given resident presented. But as I got to know them, I learned when I could trust what I was hearing, versus when something wasn’t right. One almost never knows what’s out of whack from such a presentation, just that you hear yourself saying, "Let’s go take a look." It’s a skill that comes with experience and a lot of it. Of course, the best trainees say, "I don’t know what’s going on with this case. Would you "take a look"? But others don’t know, and so [1] you "take a look" and then [2] try to help them figure out why they were off track, what they missed.

So I guess I know the reason I’m absolutely sure I wouldn’t do this one. That whole system being described up there is designed to keep me out of the room the patient’s in. They don’t need me to help them with behavioral activation and motivational interviewing. They know how to do that certainly better than I. And most of the time, the Primary Care Docs don’t need me to pick an antidepressant. One learns that kind of thing quickly. What they need is someone who has spent a lifetime around suicidal and psychotic patients; who has actually seen most of those unusual medical cases that masquerade as mental illness that most doctors only heard about in medical school; someone who has missed a few diagnoses along the way and knows the dire consequences, is acutely aware when something smells funny. And in this proposed Integrative system, I’m not the one who gets to say, "I need to see this person." Usually, I’m hearing about the case from a care coordinator who may be giving me second-hand information. And even with stable outpatients who aren’t getting better, I wouldn’t have much of a clue how to get the case on track without either seeing the patient, or making sure they’re being seen by someone who really knows the ropes and doesn’t speak new·speak.

Note: When I started writing this post, it wasn’t at all clear to me where I was headed. It’s been that way every time I run across a Collaborative Care article or reference. My reaction has been visceral. It wasn’t until somewhere around that lightbulb up there that I finally could put some words to my reaction. Reading the various versions of Collaborative Care, I always feel the same. But now I can at least make sense of why I respond so negatively. In the system as proposed, I can’t do my job – the actual job assigned to me. I can’t be in charge of saying "Let’s go take a look." And if I can’t do that, there’s really no reason for me to be there in the first place.

^{hat tip to James O…} 

Adult Utilization of Psychiatric Drugs and Differences by Sex, Age, and Race.

by Moore TJ and Mattison DR

JAMA Intern Medicine. Dec 12, 2016. [Epub ahead of print]

"Long-term use was defined as 3 or more prescriptions filled in 2013 or a prescription started in 2011 or earlier…"

"Most psychiatric drug use reported by adults was long term, with 84.3% [95% Cl, 82.9%-85.7%] having filled 3 or more prescriptions in 2013 or indicating that they had started taking the drug during 2011 or earlier. Differences in long-term use among the 3 drug classes were small. The long-term users filled a mean [SE] of 9.8 [0.19] prescriptions for psychiatric drugs during 2013…"

"These data show 1 of 6 US adults reported taking psychiatric drugs at least once during 2013, but with 2- to 3-fold differences by race/ethnicity, age, and sex. Moreover, use may have been underestimated because prescriptions were self-reported, and our estimates of long-term use were limited to a single survey year…"

"Among adults reporting taking psychiatric drugs, more than 8 of 10 reported long-term use…"

This report by Moore and Mattison well documents what I found returning to general psychiatry. I still find the figures staggering, but the one that makes the least sense is that these medications are being taken long term. Depression, even in its mest malignant format is time limited for the most part. There’s evidence that in some depressions, maintenance medication can be a relapse preventive, but hardly in 80% of the cases. All of this has happened in a period where psychiatry has been telling itself and the rest of the world that it’s medicalizing, but there’s nothing about those figures that’s medical. It’s contaminated by profiteering, plain and simple, and at the expense of patients who’ve come for help.

"… insanity is doing the same thing over and over again expecting different results"

"… baby with the bathwater"

Use the active voice.
The active voice is usually more direct and vigorous than the passive:
     I shall always remember my first visit to Boston.
This is much better than
     My first visit to Boston will always be remembered by me.
The latter sentence is less direct, less bold, and less concise. If the writer tries to make it more concise by omitting "by me,"
     My first visit to Boston will always be remembered,
it becomes indefinite: is it the writer, or some person undisclosed, or the world at large, that will always remember this visit?

One encounters patients who appear to live their lives in the domain of the passive voice. Things just happen in the world. Things happen to them [usually bad or disappointing things], but there’s no agent causing them. And invariably, they leave out their own participation in the things that happen. This was once known as the Fate Neurosis. While it may keep them from blaming others, or keep them from shouldering blame themselves, it’s part of a long-suffering view of life that has a sticky persistence that maintains their dysphoria [and often drives their therapists and acquaintances to distraction]. One of the goals of their psychotherapy is to help them see their own part in making things happen, even if it’s negative – to help them see a world in which they are actors rather than victims of obscure forces like fate, destiny, or bad luck. Whodunit? is of major importance in understanding anything that happens to these people [often times theydunit].

Oddly, my mind goes down this path when reading some of the language used to describe the various sources of bias in Clinical Trial reporting. There’s a long-suffering quality to the lamentations, as if we are victims of a maleficant  universe. It’s in the language we use. Publication Bias refers to trials with unfavorable outcomes that don’t get published. That italicized phrase happens to be an example of the use of the passive voice in that the actor who didn’t publish the study is missing in action – literally. Selective Reporting? Somebody did the selecting. And so it goes. The culprit isn’t in the language. All these sheenanigans that have so garbled our Clinical Trial literature aren’t mistakes, or sloppiness, or something overlooked, or random acts of a perverse deity. They’re not coming from incompetent or poorly trained statisticians. They’re the conscious, motivated acts of a person or persons who’ve got something very specific in mind. And again, the important question is whodunit?

We all know that these distorted trial reports are motivated actions. The goal is to exaggerate efficacy and downplay toxicity, to sell these drugs, but that knowledge doesn’t make it into our descriptive language or our policies. We routinely relate to them in the passive voice, but then wrack our brains trying to think up things that might respond to their happening rather than stopping people from doing them in the first place. We request minimal information and give industry a long time to provide it. Then we don’t levy fines when the required information doesn’t show up. We lament the things that are happening and rarely go after the agents except to extract inadequate fines long after the fact. Many say it won’t stop until we start sending people to prison, but that just hasn’t happened, in part because it’s so difficult to prosecute and often impossible to prove.

Instead of chasing instances where various biases have colored  the reported results after the fact, we could face the reality that distortion and non-compliance are the the expected response. The a priori protocol and declared outcome variables are unlikely to be available a priori. So we could say that no trial can begin until the outcome variables are posted in the registration section on ClinicalTrials.gov. Why not? They’re already available from the Institutional Review Board submission. Similarly, we could say no FDA review of an NDA will be initiated until the Results Database is publicly available filled out on ClinicalTrials.gov for all submitted trials. Why not? They’re being submitted to the FDA so they’re available. Why not submit them to the rest of us?

So, picking up from explanation would be welcome… and looking at the enrollments in RAP-MD-01, RAP-MD-02, and RAP-MD-03 that seem so high in those Rapastinel Clinical Trials. The way trialists pick their sample size is to do a Power Analysis. Using the Standard Deviation from a previous similar study, then pick the difference in means between the control and experimental values you would consider meaningful, and it spits out a sample size. Here’s the formula:

These studies use Montgomery-Asberg Depression Rating Scale [MADRS], and I can’t find any data to give me a Standard Deviation for Rapastinel using MADRS, nor do I know that scale well enough to select a mean difference. But, if you’ll allow me a bit of numeric mojo, I can use the Z Scores for p=0.05 and an 80% power [both standard], si I simplified the equation. I noticed that it has a clause that contains the formula for the Cohen’s d Effect Size so I substituted and come up with a formula that calculates the sample size for the group for any given Effect Size [could that be right?]. The table on the right gives some example values:

Looking at the group sample sizes for RAP-MD-01, RAP-MD-02, and RAP-MD-03, we have 700÷2 = 350, 2333÷3 = 778, and 1556÷2 = 778 respectively. Applying the formula, that gives Cohen’s d values of 0.21, 0.14, and 0.14. So if my formula works, using Cohen’s rough guidance [0.25 is a weak Effect, 0.50 is moderate Effect, and 0.75+ is a strong Effect] these studies are powered to detect statistically significant differences at Effect Sizes that are for all intents and purposes no Effect at all, clinically insignificant. By this read, these studies are dramatically overpowered.

And as to the question, why three different clinical trials that are close to identical? That one’s easy. They need two statistically significant RCTs to be eligible for an FDA Approval. Instead of doing a trial, and if it’s positive, trying to replicate it, they are doing them simultaneously because it’s faster. Sinilarly, why so many sites? I really speeds up recruitment.And how are they going to come up with 4589 depressed subjects that are treatment failures [<50% response to an SSRI] and get this study done in two years? I haven’t a clue. But the point is clear, this is a race to the finish line, to be the first company to have a Ketamine-like product on the market.

Surprise/Flash: So I just went to ClinicalTrials.gov to look up something about RAP-MD-04 that I was about to write about, and there’s another whole trial! posted on Christmas Day! It’s called Long-term Safety Study of Rapastinel as Adjunctive Therapy in Patients With Major Depressive Disorder [RAP-MD-06]. And this is what I had just typed, "But those things aren’t the most alarming piece of this suite of clinical trials. The fourth trial [RAP-MD-4] is a longer term clinical trial looking at relapse prevention…" I was about to talk about the potential harms. They were talking about an extension study where they were giving an intravenous drug weekly that is a kissing cousin to Ketamine, a hallucinogen. They’ve reported that Rapastunel isn’t a hallucinogen based on one published single shot trial. Now they’re going to give it weekly for up to two years as a preventative, but there’s no contingency for any continued monitoring over the long haul. And Flash, there’s appears a safety analysis. Thought broadcasting? I hope not. Anyway, here’s the new study…

… which looks for all the world like yet another trial using that same cohort. Is that kosher? Basing multiple trials on a single group of subjects? I guess the right thing to do at this point is to back off until Allergan settles down and lands on a scenario that suits them. Is this their first CNS drug trial? They’re sure making one fine mess of things so far…