I had sworn off vetting any clinical trials or meta-analyses for a while, but sometimes something special comes along. Neuroskeptic has already had his way with this one, but it had some features that seemed worthy of note. It’s a failed study [and how were they going to write JNJ-18038683 on a pill anyway?]:
Translational evaluation of JNJ-18038683, a 5-HT7 receptor antagonist, on REM sleep and in major depressive disorder
by Pascal Bonaventure, Christine Dugovic, Michelle Kramer, Peter De Boer, Jaskaran Singh, Sue Wilson, Kirk Bertelsen, Jianing Di, Jonathan Shelton, Leah Aluisio, Lisa Dvorak, Ian Fraser, Brian Lord, Diane Nepomuceno, Abdellah Ahnaou, Wilhelmus Drinkenburg, Wenying Chai, Curt Dvorak, Steve Sands, Nicholas Carruthers, and Timothy W. Lovenberg
Journal of Pharmacology and Experimental Therapeutics
Published on-line May 8, 2012. [full text on-line]
In rodents, 5-HT7 receptor blockade has been shown to be effective in models of depression and to increase the latency to REM sleep and decrease REM duration. In the clinic, the REM sleep reduction observed with many antidepressants may serve as a biomarker. We report here the preclinical and clinical evaluation of a 5-HT7 receptor antagonist, JNJ-18038683. In rodents, JNJ-18038683 increased the latency to REM sleep and decreased REM duration and this effect was maintained after repeated administration for 7 days. The compound was effective in the mouse tail suspension test. JNJ-18038683 enhanced serotonin transmission, antidepressant-like behavior, and REM sleep suppression induced by citalopram in rodents. In healthy human volunteers, JNJ-18038683 prolonged REM latency and reduced REM sleep duration demonstrating that the effect of 5-HT7 blockade on REM sleep translated from rodents to humans. Like in rats, JNJ-18038683 enhanced REM sleep suppression induced by citalopram in humans, although a drug-drug interaction could not be ruled out. In a double blind, active- and placebo-controlled clinical trial in 225 patients suffering from major depressive disorder, neither treatment with pharmacologically active doses of JNJ-18038683 or citalopram separated from placebo indicating a failed study lacking assay sensitivity. A post hoc analyses using an enrichment window strategy, where all the efficacy data from sites with an implausible high placebo response [placebo group MADRS<= 12] and from sites with no placebo response [MADRS>=28] are removed, there was a clinically meaningful and statistically significant difference between JNJ-18038683 and placebo. Further clinical studies are required to characterize the potential antidepressant efficacy of JNJ-18038683.
One would think, looking at that graph, that you might just put JNJ-18038683 back on the shelf and move on to something else. This Clinical Trial dates back to 2007 and changed more times than Elizabeth Taylor changed husbands. Neuroskeptic noted that they seemed to change the name of the active comparator from escitalopram to citalopram back and forth in the text. That was about the only thing that didn’t change during the Clinical Trial according to clinicaltrial.gov [it was always escitalopram though it was published as citalopram]. And it was an all in house show – all 21 authors were JNJ employees.
For starters, notice that they used all the graph pretty-fication tricks: mean change instead of raw scores, standard error rather than standard deviation bars, leaving off that it was last observation carried forward [LOCF] corrected. Standard fare. Recall that a positive study is when a drug and the active comparator separate from placebo. A negative study is when a drug doesn’t separate but the active comparator does. And a study where neither drug nor active comparator separates from placebo is a failed study – money down the drain and usually an indictment of the state of the clinical research industry and their recruitment methods. But this study added something new to me – a population enrichment window strategy. Says Neuroskeptic:
Ouch! But it gets better. Unhappy that JNJ-18038683 bombed, Janssen reached for their copy of the Cherrypicker’s Manifesto. This is a new statistical method, proposed by fellow Pharma company GSK in a 2010 paper, which consists of excluding data from study centres with a very high [or very low] placebo response rate.Anyway, after applying this "filter" JNJ-18038683 seemed to do a bit better than placebo, but the benefit over placebo still wasn’t statistically significant – with a p value of 0.057, the wrong side of the sacred p=0.05 line [on page 33].
Yet Page 33’s "trend towards statistical significance" magically becomes "significant" – in the Abstract:
[with] a post hoc analyses (sic) using an enrichment window strategy… there was a clinically meaningful and statistically significant difference between JNJ-18038683 and placebo.Well, no, there wasn’t actually. It was only a trend. Look it up.
With his usual style, Neuroskeptic explains this technique he names the Cherrypicker’s Manifesto. Here’s the original from GSK:
A new population-enrichment strategy to improve efficiency of placebo-controlled clinical trials of antidepressant drugs.
by Merlo-Pich E, Alexander RC, Fava M, Gomeni R.
Clinical Pharmacology Therapeutics. 2010 88[5]:634-642.
The rate-limiting factor in the discovery of novel antidepressants is the inefficient methodology of traditional multicenter randomized clinical trials (RCTs). We applied a model-based approach to a large clinical database (five RCTs in major depressive disorder (MDD), involving 1,837 patients from 124 recruitment centers) with two objectives: (i) to learn about the role of center-specific placebo response in RCT failure and (ii) to apply what is learned to improve the efficiency of RCTs by enhancing the detection of treatment effect (TE). Sensitivity analysis indicated that center-specific placebo response was the most relevant predictor of RCT failure. To reduce the statistical "noise" generated by centers with nonplausible, excessively high/low placebo responses, we developed an enrichment-window strategy. Clinical trial simulation was used to assess the enrichment strategy applied before the standard statistical analysis, resulting in an overall reduction in failure of RCTs from ~50 to ~10%.
They’re trying to control for "bad clinical research centers" by dropping them based on "bad" "placebo effect" curves. Stand up, walk around your chair, and then consider for a moment what they are talking about. The "placebo effect" is a phenomenon of the modern psychiatric drug clinical trials. Subjects get better just by being in a Double Blind Randomized Placebo Controlled Clinical Trial. And the longer the Clinical Research Industry is in existence, the better they get on placebos. This technique proposes to drop centers where the subjects don’t do well on placebos and the ones where they do too well on placebos. The reason to walk around the chair is to break the spell and realize how incredibly bizarre this all is. They have the Mixed Method or the Last Observation Carried Forward methods to deal with drop-outs. Now they propose adding a drop-centers-with-atypical-placebo-responses method. In this case, when they apply this correction, they end up with p=0.057 for JNJ-18038683 and p=0.353 for citalopram [or escitalopram? whichever they actually used]. So that almost significant p=0.057 becomes significant.
So you want to study the effect of JNJ-18038683 as an antidepressant. Here’s what you do. You farm out your study of 225 subjects to 27 different Clinical Research Centers for recruitment [an average of 8.3 subjects per center, with an average of 2.7 subjects in each group at each center on either placebo, JNJ-18038683, or escitalopram citalopram an active comparator]. You medicate them and administer a MADRS questionnaire weekly for 8 weeks. Then you collect your numbers and begin to play with them. You correct for drop-outs [LOCF] and you pretty up your graph. From across the room, it’s a dud. The escitalopram citalopram active comparator didn’t work, nor did JNJ-18038683. You run your statistics and it still didn’t work. So now you give the new population-enrichment strategy a whirl. Neither escitalopram citalopram active comparator nor JNJ-18038683 worked, but JNJ-18038683 got close. So you say…
A post hoc analyses using an enrichment window strategy, where all the efficacy data from sites with an implausible high placebo response [placebo group MADRS<= 12] and from sites with no placebo response [MADRS>=28] are removed, there was a clinically meaningful and statistically significant difference between JNJ-18038683 and placebo. Further clinical studies are required to characterize the potential antidepressant efficacy of JNJ-18038683.
…ignoring the fact that the population-enrichment strategy didn’t bring the escitalopram citalopram active comparator into significance, so even with all of that sleight of hand, it’s still a failed study or something like that. If you glanced at the graph before you started reading this, you already knew you wouldn’t recommend JNJ-18038683 to a depressed friend just from the graph, without reading a single word.
Well done. You can’t help laughing at this stuff, and especially at the straight faced solemnity with which this garbage is trotted out. Between you and Neuroskeptic, you nailed them.
Will someone please tell the editor in chief of JPET that his distinguished journal has a black eye? Maybe he could request a retraction, and while he’s at it he might want to blacklist the reviewers of this manuscript – permanently!
Heh. I would love to know what the real story behind this trial is. How on earth could they get citalopram / escitalopram mixed up? I mean I could see a preclinical scientist or a bureaucrat making that mistake quite easily, but not a psychiatrist, and you’d hope that a psychiatrist would have been in charge of a clinical trial of an antidepressant… it seems not…
I guess when you’ve got 21 authors, it’s hard to keep such things straight…
Citalopram, escitalopram, whatever — it didn’t separate from placebo either.
I just love “the inefficient methodology of traditional multicenter randomized clinical trials”. Over at the Cochrane Collaboration they are celebrating International Clinical Trials Day – am hoping this isn’t part of the party.
So, the Cherrypicker’s Manifesto was the work of Maurizio Fava, among others. Why are we not surprised? Based at Massachusetts General Hospital/Harvard Medical School, Dr. Fava must come close to the record for the number of corporate relationships he maintains. Below is the complete list from a current publication (PubMed ID 22077211). This appears to surpass even Charles Nemeroff’s record at the peak of his enmeshment with drug, device, and contract research organization industries. It is not hard to see how an academic who is involved with industry to this extent would be motivated to find ways to help his corporate clients put lipstick on the pig, like the contrived “enrichment window†strategy. I have a hard time imagining that the FDA would ever accept that idea. Dr. Fava’s faculty colleague at HMS, the ultra pharmabooster Thomas P. Stossel, would approve, however. Another name for this activity is gaming the system. What is it with Harvard, anyway?
Maurizio Fava:
Research Support: Abbott Laboratories; Alkermes, Inc.; Aspect Medical Systems; AstraZeneca; BioResearch; BrainCells Inc.; Bristol-Myers Squibb; Cephalon, Inc.; Clinical Trials Solutions, LLC; Covidien; Eli Lilly and Company; EnVivo Pharmaceuticals, Inc.; Forest Pharmaceuticals, Inc.; Ganeden Biotech, Inc.; GlaxoSmithKline; Johnson & Johnson Pharmaceutical Research & Development; Lichtwer Pharma GmbH; Lorex Pharmaceuticals; Novartis AG; Organon Pharmaceuticals; PamLab, LLC.; Pfizer Inc.; Pharmavite® LLC; Roche; RTC Logic, LLC; Sanofi-Aventis US LLC; Shire; Solvay Pharmaceuticals, Inc.; Synthelabo; Wyeth-Ayerst Laboratories.
Advisory/Consulting: Abbott Laboratories; Affectis Pharmaceuticals AG; Amarin Pharma Inc.; Aspect Medical Systems; AstraZeneca; Auspex Pharmaceuticals; Bayer AG; Best Practice Project Management, Inc.; BioMarin Pharmaceuticals, Inc.; Biovail Corporation; BrainCells Inc; Bristol-Myers Squibb; CeNeRx BioPharma; Cephalon, Inc.; Clinical Trials Solutions, LLC; CNS Response, Inc.; Compellis Pharmaceuticals; Cypress Pharmaceutical, Inc.; Dov Pharmaceuticals, Inc.; Eisai Inc.; Eli Lilly and Company; EPIX Pharmaceuticals, Inc.; Euthymics Bioscience, Inc.; Fabre-Kramer Pharmaceuticals, Inc.; Forest Pharmaceuticals, Inc.; GenOmind, LLC; GlaxoSmithKline; Gruenthal GmbH; Janssen Pharmaceutica; Jazz Pharmaceuticals, Inc.; Johnson & Johnson Pharmaceutical Research & Development, LLC.; Knoll Pharmaceuticals Corp.; Labopharm Inc.; Lorex Pharmaceuticals; Lundbeck Inc.; MedAvante, Inc.; Merck & Co., Inc.; Methylation Sciences; Neuronetics, Inc.; Novartis AG; Nutrition 21; Organon Pharmaceuticals; PamLab, LLC.; Pfizer Inc.; PharmaStar; Pharmavite® LLC.; Precision Human Biolaboratory; Prexa Pharmaceuticals, Inc.; PsychoGenics; Psylin Neurosciences, Inc.; Ridge Diagnostics, Inc.; Roche; RCT Logic, LLC; Sanofi-Aventis US LLC.; Sepracor Inc.; Schering-Plough Corporation; Solvay Pharmaceuticals, Inc.; Somaxon Pharmaceuticals, Inc.; Somerset Pharmaceuticals, Inc.; Synthelabo; Takeda Pharmaceutical Company Limited; Tetragenex Pharmaceuticals, Inc.; TransForm Pharmaceuticals, Inc.; Transcept Pharmaceuticals, Inc.; Vanda Pharmaceuticals, Inc.; Wyeth-Ayerst Laboratories.
Speaking/Publishing: Adamed, Co; Advanced Meeting Partners; American Psychiatric Association; American Society of Clinical Psychopharmacology; AstraZeneca; Belvoir Media Group; Boehringer Ingelheim GmbH; Bristol-Myers Squibb; Cephalon, Inc.; Eli Lilly and Company; Forest Pharmaceuticals, Inc.; GlaxoSmithKline; Imedex, LLC; MGH Psychiatry Academy/Primedia; MGH Psychiatry Academy/Reed Elsevier; Novartis AG; Organon Pharmaceuticals; Pfizer Inc.; PharmaStar; United BioSource, Corp.; Wyeth-Ayerst Laboratories.
Equity Holdings: Compellis.
Royalty/patent, other income: Patent for SPCD and patent application for a combination of azapirones and bupropion in MDD, copyright royalties for the MGH CPFQ, SFI, ATRQ, DESS, and SAFER.
Maurizio Fava runs MGH Clinical Trials Network & Institute (CTNI), a research-for-hire center.
Is it true that Giovanni Fava, outspoken psychiatry critic and author of Long-Term Treatment with Antidepressant Drugs: The Spectacular Achievements of Propaganda http://content.karger.com/ProdukteDB/produkte.asp?doi=10.1159/000056279 is Maurizio’s brother?
Yes, my understanding is that they are brothers. Go figure.
Hmm. Freud must be rolling over in his grave. Sibling rivalry and unresolved Oedipal (narcissistic) striving are responsible for all of this?
Actually, that might be what Freud is thinking, but Melanie Klein, in a nearby grave, is writhing over the vicissitudes of ENVY and its relationship to the Favas…
Nice one Mickey. And to keep the Kleinian roll going, I guess there is not much GRATITUDE between the bros?
What is it with Harvard?
It’s got a big blind spot, among other issues.
I think these last few comments are unnecessary and they do not enhance the tone of this blog. None of these commenters knows either of the brothers Fava. I have had a cordial correspondence with Giovanni Fava for several years, and I have never crossed paths with Maurizio Fava. I have no reason to think there is ill will between the brothers. As a matter of fact, if you look at Maurizio Fava’s publications there are several that appeared in his brother’s journal, Psychotherapy and Psychosomatics. Let’s please focus on the professional issues, not on amateur psychodynamic speculations about people you have never met.
Excuse me Dr. Carroll . . . but we were joking. And please sir…don’t call me an amateur. With all due respect, you need to lighten up.
Moreover, Dr, Carroll YOU were the one who said of the Fava brothers relationship: GO FIGURE. So I did. And tongue and cheek, I did.
Tom, I agree… unprofessional would have been a more accurate term than amateur. As for the ‘Go figure” – that was a comment on the paradox, not an invitation to speculate on psychopathology. I apologize the the Fava brothers for my careless style.
I have found Giovanni Fava to be a very concerned and kind correspondent.
Well all I can do is reiterate that I was joking. Cyber postings lack tonality so I can understand that people might have misinterpreted my comments. I don’t know the Fava brothers, and even if I did, I would never presume to make any “professional” assessment of their relationship. Period. So if the Fava brothers were insulted by my irreverent humor, I apologize to them. After all, that’s the “professional” thing to do.