Reading this blog, it might be easy to forget that I’m a psychiatrist, because I’m so often criticizing something about the current state of affairs in psychiatry. But this particular criticism is more fundamental. In the usual blog, I’m looking for scientific misbehavior driven in most instances by misguided alliances. In this case, an official Task Force of the American Psychiatric Association didn’t misrepresent their numbers, they just moved the scale. This isn’t the work of some KOL with conflicts of interest, or a ghost-written article from a hired medical writer. They didn’t omit the downside. They just moved the whole scale to fit their desired outcome. It’s right there in their published articles for the world to see. They downgraded psychiatric diagnosis:
The forest plot on the left gives the DSM-5 Field Trial results as they would have been graded in 1980 [DSM-III] or 1994 [DSM-IV]. On the right is what they said in the articles finally published yesterday.
kappa is the index of inter-rater concordance devised by Dr. Spitzer and his statistician colleagues to be our standard for reliability, our fall-back position in the absence of objective markers for validity. In 1980, the real issue was legitimacy, and Spitzer’s use of kappa carried the day – psychiatric diagnosis was not arbitrary. A kappa of zero means observers agree only by chance alone. A kappa of one mean there’s complete agreement. The gradation between zero and one were worked out thirty years ago.
In 1974, Spitzer published a meta-analysis of studies of the DSM-II using the kappa coefficient and found reliability lacking:
He followed that with a study of his revised RDC criteria showing much improved kappa values:
These are the kappa coefficients for some of the resultant DSM-III, ICD-9, and DSM-IV Field Trials:
And this is what we’re being offered from the DSM-5 Task Force:
DSM-5 Field Trials Discredit the American Psychiatric Association
by Allen Frances
October 31, 2012
He concludes with…
It is sad that the American Journal of Psychiatry agreed to publish this sleight of hand interpretation of the remarkably poor DSM-5 field trial results. Clearly, AJP has been forced into the role of a cheerleading house organ, not an independent scientific journal. AJP is promoting APA product instead of critically evaluating it. Scientific journals all have some inherent conflicts of interest – but this is ridiculous.
The DSM-5 field trial fiasco and its attempted cover-up is more proof (if any were needed) that APA has lost its competence and credibility as custodian for DSM. A diagnostic system that affects so many crucial decisions in our society cannot be left to a small professional association whose work is profit driven, lacking in scientific integrity, and insensitive to public weal.