not research…

Posted on Saturday 14 January 2017

I spent a day with the article in the last post [A manifesto for reproducible science]. It lived up to my initial impression and I learned a lot from reading it. Great stuff! But my focus here is on a particular corner of this universe – the industry-funded Clinical Trial reports of drugs that have filled our medical journals for decades. And I’m not sure that this manifesto is going to add much. Here’s an example of why I say that:

Looking at one of the clinical trial articles of SSRIs in adolescents, there was something peculiar [Wagner KD. Ambrosinl P. Rynn M. et al. Efficacy of sertraline in the treatment of children and adolescents with major depressive disorder, two randomized controlled trials. JAMA. 2003;290:1033-1041.]. What does it mean "two randomized controlled trials"? Well it seems that there were two identical studies that were pooled for this analysis. Why? They didn’t say… The study was published in August 2003, and there were several letters along the way asking about this pooling of two studies. Then in April 2004, there was this letter:

    To the Editor: Dr Wagner and colleagues reported that sertraline was more effective than placebo for treating children with major depressive disorder and that it had few adverse effects. As one of the study group investigators in this trial, I am concerned about the way the authors pooled the data from 2 trials, a concern that was raised by previous letters critiquing this study. The pooled data from these 2 trials found a statistically marginal effect of medication that seems unlikely to be clinically meaningful in terms of risk and benefit balance.

    New information about these trials has since become available. The recent review of pediatric antidepressant trials by a British regulatory agency includes the separate analysis of these 2 trials. This analysis found that the 2 individual trials, each of a good size [almost 190 patients], did not demonstrate the effectiveness of sertraline in treating major depressive disorder in children and adolescents.

    E.Jane Garland, MD, PRC PC
    Department of Psychiatry
    University of British Columbia

So the reason they pooled the data from the two studies appears to be that neither was significant on its own, but pooling them overpowered the trial and produced a statistically significant outcome [see power calculation below]. Looking at the graph, you can see how slim the pickings were – significant only in weeks 3, 4, and 10. And that bit of deceit is not my total point here. Add in how Dr. Wagner replied to Dr. Garland’s letter:

    In Reply: In response to Dr Garland, our combined analysis was defined a priori, well before the last participant was entered into the study and before the study was unblinded. The decision to present the combined analysis as a primary analysis and study report was made based on considerations involving use of the Children’s Depression Rating Scale [CDRS] in a multicenter study. Prior to initiation of the 2 pediatric studies, the only experience with this scale in a study of selective serotonin reuptake inhibitors was in a single-center trial. It was unclear how the results using this scale in a smaller study could inform the power evaluation of the sample size for the 2 multicenter trials. The combined analysis reported in our article, therefore, represents a prospectively defined analysis of the overall study population…

This definition ["well before the last participant was entered into the study and before the study was unblinded"] is not what a priori means. It means "before the study is ever even started in the first place." And that’s not what prospective means either. It also means "before the study is ever even started in the first place" too. She is rationalizing the change by redefining a priori’s meaning.

The problem here wasn’t that Pfizer, maker of Zoloft, didn’t have people around who knew the ways of science. If anything, it was the opposite problem. They had or hired people who knew those science ways well enough to manipulate them to the company’s advantage.

  • Why did they have two identical studies? Best guess is that they were going for FDA Approval, and in a hurry. You need two positive studies for FDA Approval.
  • Why would they decide to pool them somewhere along the way? Best guess is that things weren’t going well and pooling them increases the chance of achieving significance with a smaller difference between drug and placebo.
  • How would they know that things weren’t going well if the study was blinded? You figure it out. It isn’t that hard.
  • Why would they say that a priori means "well before the last participant was entered into the study and before the study was unblinded" when that’s not what it means? That isn’t that hard to figure out either.
  • So why not just say that they cheated? Because I can’t prove it [plausible deniability]

I’m not sure that the industry-funded Clinical Trials of drugs should even be considered research. They’re better seen as product testing. And the whole approach should reflect that designation. Everyone involved is biased – by definition. The point of the enterprise isn’t to answer a question, it’s to say this in whatever way you can get there:
Conclusion The results of this pooled analysis demonstrate that sertraline is an effective and well-tolerated short-term treatment for children and adolescents with MDD.
And the only way to insure that the outcome parameters aren’t changed is to require preregistration with a date-stamped certified Protocol and Statistical Analysis Plan on file before the study begins – a priori. What if they change their minds? Start a new study. Product testing may be science, but it’s not research. And we may have more oversight on our light-bulbs and extension cords than we have on our medications.

And after all of that, the Zoloft study is still in Dr. Wagner’s repertoire at the APA Meeting some 13 years later…
by Aaron Levin
June 16, 2016

… As for treatment, only two drugs are approved for use in youth by the Food and Drug Administration [FDA]: fluoxetine for ages 8 to 17 and escitalopram for ages 12 to 17, said Wagner. “The youngest age in the clinical trials determines the lower end of the approved age range. So what do you do if an 11-year-old doesn’t respond to fluoxetine?” One looks at other trials, she said, even if the FDA has not approved the drugs for pediatric use. For instance, one clinical trial found positive results for citalopram in ages 7 to 17, while two pooled trials of sertraline did so for ages 6 to 17.

Another issue with pediatric clinical trials is that 61 percent of youth respond to the drugs, but 50 percent respond to placebo, compared with 30 percent among adults, making it hard to separate effects. When parents express anxiety about using SSRIs and ask for psychotherapy, Wagner explains that cognitive-behavioral therapy [CBT] takes time to work and that a faster response can be obtained by combining an antidepressant with CBT. CBT can teach social skills and problem-solving techniques as well. Wagner counsels patience once an SSRI is prescribed.

A 36-week trial of a drug is too brief, she said. “The clock starts when the child is well, usually around six months. Go for one year and then taper off to observe the effect.” Wagner suggested using an algorithm to plot treatment, beginning with an SSRI, then trying an alternative SSRI if that doesn’t work, then switching to a different class of antidepressants, and finally trying newer drugs. “We need to become much more systematic in treating depression,” she concluded.
    January 14, 2017 | 12:20 PM

    So a 36 week trial of a pill is too brief, but when the parents express a desire for therapy, they are shut down, and told that junior needs to swallow a pill. If that does not work, then another type of pill. If that does not work, then another type of pill, where two or three of those are tried, and then to yet another type of pill. The parents desire for an intervention that is effective in treating mood disorders and has NO negative side effects is ignored, in favor of a treatment that does not work, has many side-effects on the developing brain and that actually leads to much higher relapse rates than placebo (see my previous comment on pro-depressants).
    Sadly, psychotherapy does not have billions of dollars with which to buy the opinion of academic psychiatrists.

    January 14, 2017 | 1:41 PM

    I’ll have to say that Dr. Wagner takes the cake when it comes to double-talk. And that graph is downright embarrassing! In her prime, her name was on every child and adolescent clinical trial around. Then she jumped on the Bipolar Child wagon and wrote for the Psychiatric Times. She’s now Chief of Child and Department Chairman in Galveston and president of the American Academy of Child and Adolescent Psychiatry. Her career was apparently unphased by being on Senator Grassley’s list.

    berit bryn jensen
    January 15, 2017 | 5:16 AM

    …a toxic shift in medicine/psychiatry:

    Sadly, the toxic shift is spreading in Europe too through the combined efforts of Big Pharma, the American Chamber of Commerce and organized denial of psychiatric organizations.

    berit bryn jensen
    January 15, 2017 | 6:28 AM

    Nobel Prize winner Angus Deaton stated clearly what is wrong. The American health care system “seems optimally designed for rent seeking and very poorly designed to improve people’s health.” And nothing is going to get better until we tackle that problem head on” … rent seeking=greed

    January 15, 2017 | 12:16 PM

    “Product testing may be science, but it’s not research.”
    This was from before my time, but would be curious as to whether or not the study of clinical psychopharmacology in the 60s and 70s, which feel like an era of greater wisdom an less hollow “visible expertise”, followed this model. My preliminary impression is that the researchers from that era treated the work with a dynamic approach that was beyond fixed product testing. Yet managed to produce work of greater benefit to clinicians that visible experts such as Wagner. Very open to correction by commentators here who actually worked in that era.

    Cate Mullen
    January 15, 2017 | 12:32 PM

    1boringoldman. Look into the work of Dr. Loren Moser and his same thinking peers. His academic history and trajectory is compelling.
    In my adolescence, I worked with developmentally disabled and multi disabled children. It was during the era of the breaking down of state institutions and at times we had children who had gone form the state institutions to for profit nursing. homes. This is worth a book in and of itself.
    We were in charge of them for the day so were aware if medication was an issue. I can only remember one child who was known to be on medication. We were surprised and not sure how to view the medicating. I assume he was on Ritalin, because the word was he was on a stimulant but that it fact helped him calm down. He was one of the kids that really, really needed to be watched. And there were others. He was the only child and it was a group of about forty that was medicated. We handle them well with only a few issues and everyone always went home safe and sound. Not a professional or research based comment but real life in those times.

    January 15, 2017 | 1:31 PM

    There is also the reverse experience. Namely, of care providers seeing patients stuck in more institutionalized care who benefitted greatly from medication. And, sometimes dos not have the opportunity to benefit from that medication in an era where some psychiatrists held a strong bias towards psychoanalytically (in isolation) based care.
    I appreciate your insights. I would like to highlight that my comment was more around regarding measurement of the impact of medication on patients through the lens of “research” not solely “product testing.” That I agree with 1bom around much of what needs to be addressed in the nature of drug study and publication of those findings, but perhaps differ around the point that rigid adherence to the “product testing” lens is the best one.
    Some of this may be a matter of terminology. But I get the impression that there was a degree of flexibility (and learning as one went along) in the use of medication in psychiatric residential/impatient settings in the 60s and 70s that was important to what was learned. And, that the rigid adherence to the “product testing”, here is our treatment protocol and we won’t deviate for the next 3 years no matter what (start a separate study if you want), approach was not what was being done then. Thought that is based on very preliminary impressions on my part.

    January 15, 2017 | 2:21 PM

    I think this is who you are referencing?

    Maybe I can illustrate my question more through a specific example: ketamine. There is enough we don’t understand about how to structure this treatment that I would rather this be approached from an iterative clinical research lens than from a fixed product testing perspective. ALONG WITH true transparency and data access. Treating this from a fixed product testing model seems to come at a cost to efficiency and real life relevance and I don’t mean just to the pharmaceutical company.

    Cate Mullen
    January 15, 2017 | 3:27 PM

    1boringoldman Medicine is an art not all and completely centered around science. Get out out of your head and spend some time with your heart. Have you ever worked or volunteered in a nursing home or state institution? Have you ever spend a large amount of time daily interacting with either the developmentally disabled populations or those in the psychiatric system. Daily, I mean eating lunch with them, playing games,, dressing them etc. ?
    As member of such an esteemed profession who will be be seeing Down’s Syndrome’s folks with Alzheimer’s. A professional with excellent ethics will need to delve deep into a good comprehension of what these folks families are dealing with whether they live at home, in a group home in assisted living, or nursing home.. Many, many community institutions use medication solely for ease of care.
    What needs to be stated is that there is another option of using medication wisely and sparingly and allowing for good care for all patients in all medical settings to actual – my goodness – improvement.
    Your profession is concerned with humans and the human world. Think on that for awhile – humanity and read some Hannah Arendt

    January 15, 2017 | 3:55 PM

    Using medication wisely to my way of thinking involves as much head as heart. I believe there can sometimes be a false dichotomy struck around this. There are various factors that go into using medication wisely. One factor need not be to the exclusion of others. Sorry if my particular focus in this thread, and response to your comment, implied that one factor is all I think is important. However. I do believe the question I am asking had independent importance and I maintain my more focused interest in it in this particular thread.

Sorry, the comment form is closed at this time.