Forest Plots are a useful way to display information when multiple items are compared by a single measurement variable – in this case, the DSM-5 Diagnoses [ordinate, y-axis] are compared by the kappa inter-rater reliability variable [abscissa, x-axis] from the DSM-5 Field Trials. These first two plots show the kappa values separated by diagnosis and site [with 95% Confidence limits as the horizontal line]. The adults are on top with children and adolescents below:
In the lead-up to the presentation of the Field Trial results, there was something of a debate between Dr. Frances, DSM-IV leader, and the DM-5 leaders. The issues were:
- The Task Force cancelling one of the two planned sets of Field Trials
- The actual interpretations of the kappa values
- Whether the DSM-IV and DSM-5 kappas were comparable
Below, these plots are color coded to show the difference in their standards [the yellow band being "questionable"]. Again Adults on top, children below:
I’ve said all I need to say about the results of the Field Trials. They look even less reliable in kids than adults, and the adult results didn’t look very good. Musing about that, I re-read the conclusion to the article. This was right at the end:
DSM-5 Field Trials in the United States and Canada, Part II: Test-Retest Reliability of Selected Categorical Diagnoses and Analytic Approaches
by Darrel A. Regier, William E. Narrow, Diana E. Clarke, Helena C. Kraemer, S. Janet Kuramoto, Emily A. Kuhl, and David J. Kupfer
American Journal of Psychiatry. 2012 October 30, AJP in Advance
"… Emil Kraepelin, who pioneered the separation of schizophrenic and affective psychoses into separate diagnostic groups in 1898, noted later in a 1920 publication— prescient in its anticipation of a current polygenetic environmental interaction model of mental disorders—that the strict separation of these categorical diagnoses was not supported. We are now coming to the end of the neo- Kraepelinian era initiated in the U.S. by Robins and Guze with a renewed appreciation of both the benefits and limitations of a strict categorical approach to mental disorder diagnosis.The ultimate goal is to build on the progress achieved with categorical diagnoses by continuing with longitudinal follow-up of patients with these diagnoses, incorporating cross-cutting dimensional measures judiciously into the diagnoses where they prove useful, and in some cases recommending simple external tests [such as a cognitive test for mild neurocognitive disorder] that might improve the reliability and move toward a more mature scientific understanding of mental disorders. A noted philosopher of science, Carl Hempel, observed that “although most sciences start with a categorical classification of their subject matter, they often replace this with dimensions as more accurate measurements become possible”.
Clinicians think dimensionally and adjust treatments to target different symptom expressions in patients who may have the same categorical diagnosis. The intent of DSM-5 is to provide a diagnostic structure that will more fully support such dimensional assessments with diagnostic criteria revisions, specifiers, and cross-cutting symptom domain assessments. The goal is to support better measurement-based care and treatment outcome assessment in an era when quality measurement and personalized medicine will require new diagnostic approaches."
As I read that first paragraph, I felt like I understood something that has bothered me through this whole process. I haven’t understood why the DSM-III and DSM-IV framers and the DSM-5 Task Force have been at such odds. I got the trees – the various issues that have come up along the way and I’ve had plenty to say about that. But I think I missed the forest. It’s so easy to get caught up in the details and miss the big picture, then when it becomes a bit clearer, you wonder why you didn’t see it in the first place. This is the forest: "We are now coming to the end of the neo-Kraepelinian era." The framers of the DSM-5 don’t believe in the neo-Kraepelinian Revolution of 1980 any more. That explains a lot of things:
-
Why they locked Robert Spitzer out of their process
-
Why the’ve never really responded to Dr. Frances
-
Why they’ve occupied themselves with quirky diagnoses
-
Why they didn’t really revise the current manual
-
Why they’re willing to accept these lousy Field Trials
-
Why they keep talking about things like cross-cutting and dimensional
-
Why they’ve barreled ahead in the face of such negativity
The framers of the DSM-5 are over it. I don’t know why I didn’t notice that. They said it in their research agenda book ten years ago. They wanted to put psychiatry on a genetic/biological basis and it didn’t work. They apparently felt they had to fight the good fight to do that – shut out the reactionaries. Well they fought the good fight, ignored a lot of the refinements needed in the DSM-IV, and waited for something that would move psychiatry in the direction they wanted expected it to move – the something never came so they turned their hopes to the RDoC at the NIMH. In spite of their un-success in moving the DSM-5 in the direction they wanted expected it to go, hope springs eternal:
The intent of DSM-5 is to provide a diagnostic structure that will more fully support such dimensional assessments with diagnostic criteria revisions, specifiers, and cross-cutting symptom domain assessments. The goal is to support better measurement-based care and treatment outcome assessment in an era when quality measurement and personalized medicine will require new diagnostic approaches.
I’ve known they had a bad case of future think, but it didn’t occur to me that the framers of the DSM-5 had given up on the very system they were in charge of revising, or that they saw the neo-Kraepelinian era as being at its end. If that wasn’t true before they started, it’s guaranteed to be true now. I think this is an example of the proverb, "don’t put the cart before the horse". And in doing that, they’ve made one hell of a mess…
Ha! I told you so. As Dick Cheney might have said, “Diagnoses don’t matter.”