1 Boring Old Man

DSM-5 Field Trials: What Was Learned
Psychiatric Times
By James Phillips, MD
January 8, 2013

^{With DSM-5 now approved by the APA Board of Trustees — and, to the dismay of this reader, all discussion removed from the DSM-5 Web site—how are we to evaluate the results of the field trials for the end product? I suggest beginning with the short piece published in Psychiatric News, “What We Learned from DSM-5 Field Trials.” Authors David Kupfer and Helena Kraemer wrote, “We ultimately tested the criteria for 23 disorders. The question we asked was a straightforward one: In the hands of regular clinicians, assessing typically symptomatic patients in no different way than they would during everyday practice, was a particular disorder reliable?”}

^{In responding to this statement, let me begin with a confession of ignorance. Since I was not a participant in the field trials—neither for DSM-5 nor for previous versions of the manual—I didn’t appreciate that the major purpose of the field trials was to test reliability of the diagnostic constructs. I labored under a naïve illusion that since the ongoing crisis in the DSM has to do with diagnostic validity, field testing would somehow involve working on validity of new diagnoses. The “Guidelines for Making Changes to DSM-V” did hold out a little hope for work on validity with the statement: “Although some preliminary ‘convergent validity’ may be obtained in the field test [clinicians find the criteria fit syndromes expressed by real patients], a full test of the validity of these criteria will occur after DSM-V is published.”}

^{A little history}

^{Let me give a quick reminder of the relevant history. The goal of DSM-III was to establish diagnostic reliability through operational definitions and diagnostic criteria. It was assumed that psychiatric science would fill in and establish validity for the DSM-III diagnostic constructs. Thirty years later, as we were preparing for DSM-5, everyone acknowledged the great shock of the previous 3 decades:  diagnostic validity for the DSM-III/IV diagnostic categories was in shambles. As the DSM-5 leaders wrote in the 2002 white paper, “A Research Agenda for DSM-V”:}

^{In the more than 30 years since the introduction of the Feighner criteria by Robins and Guze, which eventually led to DSM-III, the goal of validating these syndromes and discovering common etiologies has remained elusive. Despite many proposed candidates, not one laboratory marker has been found to be specific in identifying any of the DSM-defined syndromes. Epidemiologic and clinical studies have shown extremely high rates of comorbidities among the disorders, undermining the hypothesis that the syndromes represent distinct etiologies. Furthermore, epidemiologic studies have shown a high degree of short-term diagnostic instability for many disorders. With regard to treatment, lack of treatment specificity is the rule rather than the exception.}

^{The authors’ response to this crisis was a vague appeal for a “paradigm shift”:}

^{All these limitations in the current diagnostic paradigm suggest that research exclusively focused on refining the DSM-defined syndromes may never be successful in uncovering their underlying etiologies. For that to happen, an as yet unknown paradigm shift may need to occur. Therefore, another important goal of this volume is to transcend the limitations of the current DSM paradigm and to encourage a research agenda that goes beyond our current ways of thinking to attempt to integrate information from a wide variety of sources and technologies.}

^{In the ensuing years, advances in neuroscience and genetics were still not validating the DSM-IV categories. The perfectly reasonable directives for the Work Groups outlined in the Guidelines mentioned above could not accomplish much without clear support from basic science. In the face of this limitation, the DSM-5 leadership moved to dimensional measures as one solution to the validity problem. In 2009 Regier and colleagues wrote:}

^{The single most important precondition for moving forward to improve the clinical and scientific utility of DSM-V will be the incorporation of simple dimensional measures for assessing syndromes within broad diagnostic categories and supraordinate dimensions that cross current diagnostic boundaries. Thus, we have decided that one, if not the major, difference between DSM-IV and DSM-V will be the more prominent use of dimensional measures in DSM-V.}

^{As the dimensional measures were being challenged [they were dismissed by the Board of Trustees in May of 2012], Kupfer and Regier moved to the NIMH Research Domain Criteria project as the ultimate salvation of DSM-5—almost implying that the RDoC project, developed independent of DSM-5 and the DSM categories because the categories don’t work for research, was somehow developed in conjunction with the DSM work:}

^{A logical extension of those discussions, as detailed in our Research Agenda articles, is the Research Domain Criteria (RDoC) initiative recently launched by the National Institute of Mental Health … This NIMH objective is consistent with our research planning conferences and conclusions, which underscored our commitment to examining evidence from neurobiology and assessing the readiness of proposed revisions for DSM-5. We are pleased with the work on RDoC that is being undertaken, and we believe this initiative will be very informative for subsequent versions: DSM-5.1, DSM-5.2, and beyond…It is important to emphasize that DSM-5 does not represent a radical departure from the past, nor does it represent a radical separation from the goals of the RDoC.}

^{Back to 1980?}

^{So there we were, waiting for the appearance of DSM-5 in May, and being reassured that those new diagnoses that have survived the chopping block of the Board of Trustees and the Scientific Review Committee have been field tested and proven to be reliable. Isn’t this where we were in 1980 with the appearance of DSM-III—now another handful of new diagnoses with demonstrated reliability and minimal evidence of scientific validity? Since we still don’t have adequate validity for the existing ton of DSM-IV diagnostic categories, is there any reason to think that the new handful won’t suffer the fate of the DSM-III/IV categories—heterogeneity of presentation, fuzzy boundaries, excessive comorbidity, and failure of neuroscientific and genetic foundation. And we really can’t count on the RDoC to save the day for DSM-5.1, DSM-5.2, and beyond, given that the RDoC researchers are not working with DSM categories and have no expectation that the constructs for which they are seeking disruptions of neural circuitry will match up with existing diagnostic categories. If you check the working grid of RDoC, you won’t find a column for “DSM diagnosis.”}

^{What did we learn from the DSM-5 field trials?}

^{We learned that with diagnostic criteria, it’s not that hard to assure diagnostic reliability. And that we now have such reliability for another small group of new diagnoses. But didn’t we already know from DSM-III that diagnostic reliability is fairly easy? How can one not respond to Kupfer and Kraemer: is that it?}

^{"I didn’t know whether to laugh or cry. Laugh — because there is no way task force and work group members can be made to refrain from discussing the developing DSM-V with their colleagues. Cry — because this unprecedented attempt to revise DSM in secrecy indicates a failure to understand that revising a diagnostic manual — as a scientific process — benefits from the very exchange of information that is prohibited by the confidentiality agreement. Such exchange of information with colleagues was encouraged with the development of DSM-III, DSM-III-R, and DSM-IV."}

In Dr. Phillips’ take on the DSM-5, the Task Force started with a criticism, a "crisis". The system hadn’t lived up to the expectations of Drs. Robins and Guze, the primo neoKraepelinians. After thirty years, we still had no evidence that the DSM Disorders were discrete biomedical entities – no confirming biological markers, no lab work, nothing to validate the syndromes – "diagnostic validity for the DSM-III/IV diagnostic categories was in shambles."

So they spent years pursuing their research agenda looking for the elusive validators, having a series of mega-conferences, but nothing appeared. So they went to an add in diagnostic idea – cross cutting dimensions, adding in a parallel system to refine diagnoses. It was rejected by the Trustees [and anyone else who read much about it]. Having failed to solve the crisis, they deferred to the RDoC project of the NIMH which proposes to classify human subjectivity by technology [genetics, neuroimaging, neurocognitive studies]. As Dr. Phillips points out, this project neither arose from nor addresses DSM-5 diagnoses. So Dr. Phillips concludes that other that a few new categories that made it through the Field Trials, there’s nothing much there that wasn’t in the DSM-IV.

What about that crisis? the one where the "diagnostic validity for the DSM-III/IV diagnostic categories was in shambles"? From their point of view, the DSM-III/IV diagnostic categories must’ve been in shambles on the day they were written for the very first time. Their pseudo-crisis simply confirms what we’ve lived for the last thirty years. Maybe Dr. Spitzer believed that the "DSM-III was atheoretical with regard to etiology or pathophysiological process" [and maybe not considering how he handled depression], but the body psychiatric has steadfastly treated all mental illness as biological and all treatment as biological since the DSM-III appeared. The other revisions [DSM-III 1980, DSM-IIIR 1988, DSM-IV 1994] have stuck to the atheoretical meme [though the possibility of finding "specific psychological causes" was in the range of zero in the system as written].