Preliminary results are mixed for the recently completed field trials for the upcoming Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), according to the first public presentation of the findings here at the American Psychiatric Association’s (APA’s) 2012 Annual Meeting. Diagnostic criteria for autism spectrum disorder, posttraumatic stress disorder (PTSD), and attention-deficit/hyperactivity disorder (ADHD) in children were found to be very reliable. Conditions that did not do well included major depressive disorder (MDD), in adults and in children, and general anxiety disorder (GAD). According to Darrel Regier, MD, vice-chair of the DSM-5 task force, the poor scores for MDD may be attributable to "co-travelers," such as PTSD, major cognitive disorder, or even a substance use disorder, which often occur concurrently with depression. "Patients often don’t come in a single, simple diagnosis in clinical practice," Dr. Regier told Medscape Medical News. "If you have a patient with PTSD and major depression, the one that will probably get the most attention is PTSD. And that’s what we found in our study." He added that analysis of the field trials is ongoing and that the results presented at the meeting are preliminary.Members of the task force said they hope to publish the full results "within a month." However, the third and final public comment period for the manual opened last week and ends on June 15. Although the entire period is 6 weeks long, the public may only have 2 weeks to comment after the publication of the field trials’ findings. From all accounts, the manual is still on track for publication right before next year’s APA Annual Meeting in San Francisco…
It’s a long article with lots of quotes, mostly explaining [away] the results. It had some more kappas, so I updated my table. The table has all the data from all the articles I’ve seen so far:
| Disorder | DSM-5 (95% CI) | DSM-IV | ICD-10 | DSM-III |
|
|
||||
| Major neurocognitive disorder | .78 (.68 – .87) | — | .66 | .91 |
| Autism spectrum disorder | .69 (.58 – .80) | .85 | .77 | -.01 |
| Post traumatic stress disorder | .67 (.59 – .74) | .59 | .76 | .55 |
| Child attention deficit disorder | .61 (.51 – .72) | .59 | .85 | .50 |
| Complex somatic disorder | .60 (.41 – .78) | — | .45* | .42* |
| Bipolar disorder | .54 (.43 – .65) | — | .69 | — |
| Oppositional defiant disorder | .41 (.21 – .61) | .55 | — | .66 |
| Major Depressive Disorder (in adults) | .32 (.24 – .40) | .59 | .53 | .80 |
| Generalized anxiety disorder | .20 (.02 – .36) | .65 | .30 | .72 |
| Disruptive mood dysregulation disorder | .50 (.32 – .66) | — | — | — |
| Schizophrenia | .46 | .76 | .79 | .81 |
| Mild neurocognitive disorder | .50 ( .40 – .60) | — | — | — |
| Schizoaffective Disorder | .50 | .54 | .51 | .54 |
| Mild traumatic brain injury | .46 (.28 – .63) | — | — | — |
| Alcohol use disorder | .40 (.27 – .54) | — | .71 | .80 |
| Hoarding | .59 (.17 – .83) | — | — | — |
| Binge Eating | .56 (.32 – .78) | — | — | — |
| Major Depressive Disorder (in kids) | .29 (.15 – .42) | — | — | — |
| Borderline personality disorder | .58 (.46 – .71) | — | — | — |
| Mixed anxiety/depressive disorder | .06 | — | — | — |
| Conduct Disorder | .48 | .57 | .78 | .61 |
| Antisocial Personality Disorder | .22 | — | — | — |
| Obsessive Compulsive Disorder | .31 | — | — | — |
| Attenuated Psychosis Syndrome | .46 (0-?) | — | — | — |
[* see comment]
What the DSM-5 leadership will need to account for is the falloff in diagnostic reliability (lower Kappa results) for many of the ‘bread and butter’ conditions, such as autism spectrum disorder, oppositional defiant disorder, major depressive disorder, generalized anxiety disorder, schizophrenia, and alcohol use disorder. Never mind the fancy new proposed conditions like mixed anxiety/depressive disorder and attenuated psychosis syndrome. Darrel Regier’s glib explanation doesn’t fly.
Thank you, Mickey, for a most interesting series of posts around the DSM-5 field trial results.
The field trials for the new proposed category “Complex Somatic Symptom Disorder (CSSD)” were held at Mayo. According to one of several tables in Ms Brauser’s report for Medscape, the following data have been released for “Complex somatic disorder” [sic]:
Complex somatic disorder
DSM-5 .60 (.41 – .78)
DSM-IV —
ICD-10 .45
DSM-III .42
CSSD is a new category for DSM-5 which redefines and replaces some, but not all of the existing DSM-IV “Somatoform Disorders” categories under a new rubric with a new definition and criteria. It’s a mashup of the existing categories:
Somatization Disorder
Hypochondriasis
Undifferentiated Somatoform Disorder
Pain Disorder
Following evaluation of the field trials, this new category, “Complex Somatic Symptom Disorder” is now proposed to drop the “Complex” word and be named “Somatic Symptom Disorder” and to absorb “Simple Somatic Symptom Disorder (SSSD)” – a separate diagnosis that had been introduced for the second draft, with criteria requiring fewer symptoms than CSSD and shorter chronicity.
Since CSSS (or SSD) did not exist as a category in DSM-IV, or in ICD-10 or DSM-III, it’s unclear and unexplained in the table what data for which existing somatoform disorders have been used for the kappa comparisons for this new category with data for ICD-10 and DSM-III, or how meaningful comparison between them would be.
Wow, any idea why schizophrenia took a nosedive?
I cannot find out the reliability computations used by DSM5. Which Kappa is referred to?? Note– Cohen’s Kappa required 2 fixed raters. Not the case here. The various strength assignments .2,.4…8 were quite arbitrarily taken from Cohen’s review of a number of psychological journals with regard to Cohen’s kappa. To apply them to the statistic used here is doubly arbitrary.
Relevant articles show what a difficult area this is. See Rae G , Educational and Psychological Measurement 1988 48: 367 The problem of multiple raters and the attribution of multiple diagnoses to the same patient, that may or may not be used for other patients, is not new. It is just very difficult.
Thank you for explaining the Kappa stuff. I feel enlightened! In your latest table you present figures in brackets. What do they mean?
Those are the 95% confidence limits eg 95% of values fall somewhere in there.
The various kappas have outrun my personal brain-power. From Spitzer’s original article [coauthor Joeseph Fleiss] I think this must be Fleiss’s kappa which is for use with multiple raters. Consulting that classic stats book [Wikipedia], it has a different derivation from Cohen’s. It’s apparently in both the SAS and SPSS packages. I found this reference [Fleiss, J. L. (1971) “Measuring nominal scale agreement among many raters.” Psychological Bulletin, Vol. 76, No. 5 pp. 378–382]. But instead of looking it up, I sent the stuff on to a “higher power”, my brother-in-law, a retired Statistics professor in hopes of getting my amateur brain up to speed.
Since the treatments are risky in themselves, doesn’t it make sense to err on the side of not giving a diagnosis and striving to rule out medical causes of mental dysfunction until further investigation is done? Shouldn’t that be obvious? Asserting that giving a person a medical diagnosis releases that person from stigma is too arrogant and galling to be laughable.
It seems that the insurance companies should be happy to pay for more time for evaluations that could result in less spending on drugs. Perhaps the real payoff for them is when the patients become un-insurable because of their psychiatric diagnosis.
Kappa this, kappa that. The bottom line is this: If two shrinks can rarely agree on who is depressed or anxious, then this field is in deep sh*t.
So, for Hoarding, 95% of values fall between .17 and .83. I am starting to feel that I must suffer from a Mild neurocognitive disorder…or is it Major? Can you diagnose me? What is the kappa score on this one?
To Ivan’s list (above) I would add OCD. As a clinician, I have always been struck by the unique and unfortunate pathology of OCD and the distinctiveness of its presentation. To have such a low kappa implies (a) it’s not a real disorder (unlikely), or the existing criteria are simply unreliable. I think Tom is onto something with his “deep sh*t” comment.
Which brings up another point re: Ivan’s description of “bread and butter” conditions. Are these “bread and butter” conditions because the pathology is so common and widespread? Or is it because the diagnostic criteria apply (for MDD, GAD, ASD) to such a wide swath of human experience that they become easy labels to slap on the people in our offices– particularly when we have less and less time to evaluate them?
@SteveBMD: bread and butter, nuts and bolts, take your pick of metaphors. The point is that these are the most common ways that patients come to us and so if we can’t do better with reliably diagnosing these, forget about the fancy stuff.
It really doesn’t matter. One diagnosis is as good as another. The treatment will be a succession of drugs from A to Z, or maybe Z to A, or random order, depending on how the clinician is feeling that day.