where’s the beef?…

Posted on Wednesday 14 May 2014

I really have a distaste for grant applications. I did some when I was an NIH Fellow in the bronze age and much later wrote periodic training grants. I get it why one has to fill out all that stuff and look endlessly at the endless guidelines, but getting it doesn’t erase the tedium of doing it. I am discovering that reading such things is an equally odious task, having tried yesterday to make some sense out of the NIMH RAISE Studies I mentioned earlier [a fabrication?…]. While I’m at it, I apologize for the disjointedness of that last post. I started out with something in mind, but every time I looked something up, some new confusion reared its head. So the post wandered as I wandered from place to place trying to figure things out. This post is a second look at the material from that post.

First off, I declare that I’m a supporter of both the efforts to identify people who are going to have psychosis in advance, though I question whether the existing methods being proposed are on the right track. And I also believe that early intense intervention in FSE [First Schizophrenic Episode] is an imperative. So my interest in the RAISE Programs isn’t antagonistic, I just want to be sure that what’s being proposed is rational and likely to get us somewhere useful. So first, here is what I could find out about the grants:


NIMH Director’s blog
By Thomas Insel
December 29, 2009

From the extraordinary funding opportunities presented by the passage of the American Recovery and Reinvestment Act of 2009 [Recovery Act] to significant new investments in research and resource infrastructure — this has been a remarkable year for our Institute. I would like to reflect with you on how the work of 2009 has prepared us for the year ahead…

Finally, I would like to highlight a research project launched this year that seeks to fundamentally change the way schizophrenia is treated: the Recovery After an Initial Schizophrenia Episode [RAISE] project. The RAISE project, which is also supported through Recovery Act funds, will develop and test innovative and coordinated intervention approaches in the early stages of the illness when symptoms may be most responsive to treatment. The study is designed to alter the long-term disability that can result from schizophrenia and help ensure that people with the disease can lead productive, independent lives. Importantly, the interventions being tested will be designed from the outset to be readily adopted in real-world health care settings and quickly put into practice.
and here are the grants:


[reformatted to fit]

OD stands for Office of the Director, in this case I presume that the Director is allocating the funds from the American Recovery and Reinvestment Act [ARRA]. An Aside: This was the stimulus money after the Great Recession of 2008, I can’t resist showing their prediction graph from that time with what actually happened added [in red].

So with the ARRA windfall, Dr. Insel funded two RAISE [Recovery After an Initial Schizophrenia Episode] grants: RAISE Connection [271200900020C-1-0-1] and RAISE ETP [271200900019C-0-0-1]. Dr. Jeffrey Lieberman was Principle Investigator on RAISE Connection – a two site effort looking to recruit >370 subjects. It looks as if they planned to have two treatment arms assigned randomly at both sites. Dr. John Kane [1][2] was the Principle Investigator on RAISE ETP with 33 sites with a projected enrollment of >400 also with two arms, but each site was assigned to a specific arm. While the NIMH grant description for both was the same, the clinicaltrials.gov descriptions differed and they followed a remarkably different revision history.


And these are the two studies that were implemented from these grants:

RAISE Connection: Clinical Trial NCT01216891 was registered on October 4, 2010 with two arms:

  1. Behavioral: Multi-element, team-oriented treatment
    Arm Label: Team-based treatment:
    Participants will receive treatment and services based on their needs coordinated by a small team that is led by a clinical coordinator. Services that are available to the participants include social skills training, medication treatment to address symptoms, education and employment advising, and substance use treatment.
  2. Behavioral: Connections to community-based treatment and services
    Arm Label: Facilitated community-based treatment:
    A care specialist will work with participants to identify and link to treatment and service options that are available in the community. These options can include finding a therapist, obtaining ongoing medication treatment to address symptoms, joining a support group, and getting a job coach.
It was logged as recruiting on 01/14/2011, then recorded as Active, not recruiting on 02/18/2011. Then on 06/22/2011 there were a number of changes:

  • Dr. Lieberman was replaced as the responsible party by Susan Ellis PhD.
  • The target enrollment was dropped from 370 to 80, and shortened.
  • The active comparator was dropped [now only one arm]:
    "The Connection Program aims to to assess the effectiveness of a Team-based intervention for individuals with a first psychotic episode, observing outcomes over time for our study participants. When tracking outcomes, the Connection Program will make comparisons with what is known about the natural history of untreated first episode of psychosis as well as usual care outcomes from other experimental studies…"
  • The study was logged in as recruiting again.
  • Obviously, it was no longer randomized or blinded as there was only one group.
Thereafter, Lisa Dixon MD replaced Susan Essock PhD, and on 04/16/2013, the enrollment was dropped to 65. The study was completed on 12/23/2013.

RAISE ETP [Extended Treatment Program]: Clinical Trial NCT01321177 was registered on January 27, 2011 with two arms:

  1. Behavioral: Integrated Treatment
    Arm Label: Integrated Treatment:
    Integrated program of treatments and services delivered by a coordinated team of providers that includes:
    • education about schizophrenia and its treatment for the participants and their family members
    • medication for symptoms and preventing relapse that uses a computerized decision support system
    • strategies for managing the illness and building personal resilience
    • help getting back to school or work using a supported employment/education model
  2. Behavioral: Connections to community-based treatment and services
    Arm Label: Facilitated community-based treatment:
    A care specialist will work with participants to identify and link to treatment and service options that are available in the community. These options can include finding a therapist, obtaining ongoing medication treatment to address symptoms, joining a support group, and getting a job coach.
  3. Standard mental health treatments and services offered at the local agency that may include:
    • medication for symptoms and preventing relapse
    • psychosocial therapy which may include a range of behavioral treatments and supportive services
    • Case management

It was logged as recruiting on 03/22/2011 and as Active, not recruiting on 12/27/2012.


If you’ve made it through this tediousness, you can see that something fairly dramatic happened along the way. RAISE Connection got off to the first start but shortly after they started recruiting, they stopped. Then a few months later, it was as if it just shut down. There was a new P.I.; they dropped the Active Comparator; they radically downsized the projected enrollment; then sauntered along with changing investigators and sponsors; and they ended up with only 65 subjects. At the same time RAISE Connection went on ice, RAISE ETP got started and barreled ahead full bore. I can locate nothing that tells me what happened, but I think it matters some. RAISE Connection was awarded ~$10 M and basically shut itself down when RAISE ETP got started. So I think it’s legitimate to ask, Where did that $10 M go?

There have been two windfall moments in this story. First, the availability of ARRA funding that got both studies started, granted by the Office of the Director outside the usual NIMH channels. One study, RAISE Connection, seems to have fizzled [Where did that $10 M go?]. The other appears to be perking along. The next windfall was the recent availability of SAMHSA Block Grant opportunities for the States to implement FSE programs. There is a complex and detailed document, Evidence-Based Treatments for First Episode Psychosis: Components of Coordinated Specialty Care, from both RAISE studies on the NIMH site that summarizes all the information the NIMH has extracted to pass on to the States applying for these block grants listing all of their various supplementary materials. I’m not up to taking it on just yet, but I did look at the acknowledgements at the end:
Portions of this report have been drawn from the final reports of the RAISE Connection Program [Contract No. HHSN271200900020C] and interim progress reports of the RAISE Early Treatment Program [Contract No. HHSN271200900019C]. We thank the following subject matter experts for their careful review of this document, insightful feedback, and helpful editorial suggestions: Lisa Dixon, Susan Essock, and Howard Goldman [RAISE Connection Program]; John Kane, Nina Schooler, Delbert Robinson, Jean Addington, Mary Brunette, David Penn, and Patricia Marcy [RAISE Early Treatment Program]; Kristin Cadenhead, Barbara Cornblatt, Keith Nuechterlein, Diana Perkins, and Scott Woods. Finally, we thank Patrick McGorry [AO, M.D., Ph.D, FRCP, FRANZCP] for generously sharing resources developed in Australia to support broad implementation of evidence-based treatment approaches for FEP, as well as lessons learned from the national roll out of coordinated specialty care programs in that country.

Dr. Insel’s blog post, Director’s Blog: From Research to Practice, focused of the rapidity of the NIMH translation of the RAISE Connection study to the SAMSA Block Grants for the States to implement FSE/FPE programs. This is a relative huge initiative similar to the EPPIC program in Australia. First off, the RAISE Connection study is absolutely nothing to write home about, and we don’t have access to "the final reports of the RAISE Connection Program … and interim progress reports of the RAISE Early Treatment Program" to allow us to know if the results of these programs justify their being a template for such a major undertaking. It might be the greatest idea in the world or it might be a major dud and a massive waste of resources.

Again, I am on the side of a well funded program for an intense focus on the early days of psychosis including psychosocial interventions. If we’re going to have an impact on the course of this illness, that’s the time in the course of things to get started – and those patients need a lot more that just medications. I don’t question the credentials of either Dr. Lieberman or Dr. Kane to lead that effort. But this is a time for data transparency – and what we have is too little to help us decide for ourselves. Are  we moving ahead based on solid evidence, or because the money’s available? I can’t determine the answer to that so far with what we have to look at, and the days for this kind of expensive conclusion without public access to the the data justifying that conclusion should be behind us. I would call on Dr. Insel to make those documents mentioned above available for all of us to see – and while he’s at it, answering the questions about Raise Connection – "Where did that $10 M go?"
  1.  
    Steve Lucas
    May 14, 2014 | 9:45 AM
     

    Does that 65 number represent just a group of people that may or may not reflect the desired traits or is it just 65 people. So we spend $10M on a group this size and expect what?

    Steve Lucas

  2.  
    May 14, 2014 | 10:26 AM
     

    Steve,

    I’m not sure I’m right about that, but that’s sure how it looks. I think we have a right to know how that study played out. There’s all this talk about accountability these days, but “where’s the beef?”

  3.  
    anonymous
    May 14, 2014 | 11:35 AM
     

    Mickey, I believe that the Connection program’s withdrawal was due to RAISE funding being slashed, as sometimes happens with multi-year grants since all promised funding is provisional depending on NIMH budget status. Thus, $10M didn’t go anywhere–some smaller amount went as far as it could go, and the rest ebbed back into the NIMH (or to the federal government). Feel free to email for more info.

  4.  
    Bernard Carroll
    May 15, 2014 | 9:53 AM
     

    Are government bureaucrats like Thomas Insel the right people to be determining the direction of important research? Absolutely not, even if they have the seeming concurrence of research KOLs – whose advice is anything but disinterested. As a matter of fact, I am having trouble recognizing an incisive, focused scientific question in the RAISE program descriptions.

    Sadly, the RAISE fiasco is not an exception. Another example is the Emory-GlaxoSmithKline-NIMH Collaborative Mood Disorders Initiative 5U19MH069056 (original Principal Investigator Charles Nemeroff). You can read the details here. It’s an ongoing drama.

  5.  
    May 16, 2014 | 6:11 PM
     

    I think the research question of RAISE ETP was whether a coordinated approach with specific interventions (family therapy, individual therapy, evidenced based support for prescribers, supported employment, team coordination) would result in better outcomes than treatment as usual. I think these kinds of studies are valid but they are difficult to do. They did achieve enrollment goals – ~ 400 people enrolled. At the very least it will give us a window on current treatment practices in the US. They have centers from across the country. I am interested, for instance, in drugs used and doses used.

  6.  
    May 16, 2014 | 9:08 PM
     

    Sandra,

    I’m interested in all those things too. But I’m also interested in how these different treatment modalities are going to be implemented in this particular population with such unique needs. They’re in the process of saying this is the template to launch all over the country, and it may well be. But I think they owe us more access to the specifics and the results from RAISE before barreling ahead. This is something we need to do well, not just do.

Sorry, the comment form is closed at this time.