too many inconsistencies…

Posted on Friday 7 December 2012

Field Trial Results Guide DSM Recommendations
Huffington Post
by David J. Kupfer, M.D.
Written with Helena C. Kraemer, Ph.D.

… some DSM-5 detractors have spotlighted the six as indicative of flaws in the field trials, especially because this group included major depressive disorder and generalized anxiety disorder, two of the most commonly diagnosed conditions. The opposite is closer to the truth. Rather than discrediting the field trials, the outcome here reveals the critical value of how the trials were constructed and conducted and how we are moving forward. Ironically, both major depressive disorder and generalized anxiety disorder were tested not because they were being modified for the next manual, but because they were remaining relatively unchanged and could serve as reference disorders from the DSM-IV trials. But as part of that process two decades ago, patients were carefully screened, and participating clinicians received special training and explicit direction on how to perform evaluations. In contrast, the DSM-5 field trials accepted patients as they came and asked clinicians to work as they usually did – to mirror the circumstances in which most diagnosing takes place.

We believe the DSM-5 results represent the truer picture of the difficulty clinicians may have in reliably diagnosing both conditions, either because they often occur with other conditions or because they are accompanied by symptoms that can fluctuate greatly. Regardless of why, we acknowledge that the relatively low reliability of major depressive disorder and generalized anxiety disorder is a concern for clinical decision-making. Strategies need to be developed to address the problem as the manual evolves into a living document that incorporates revisions and additions as research and clinical practices advance. The good news is that we’re now inherently better prepared for this challenge; the DSM-5 field trials have laid the groundwork for how such strategies and future changes should be judged…
Like most practicing physicians, I had the habit of scanning the literature and paying attention to the topic, then the conclusion, then a quick read through of the graphs, tables, and conclusions. I didn’t spend a lot of time picking through the Methods, etc. Who has time for that? After I’d been retired for a while, I started looking at the literature differently because I didn’t believe a lot of what I was reading. Thus the boring aspects of this blog – piddling around in the Methods and Analysis parts. As we all know by now, the industry sponsored Clinical Trial articles counted on practicing physicians being scanners, because the jury-rigging is in the Methods and Analysis. By the time you get to the Results and Conclusions, you’ve already drunk the Kool Aid.

One of the joys of being retired is that I can do what I want to do, no matter how long it takes. So now I read articles in a different way. After the Abstract, I read the Acknowledgements, the Conflict of interest declarations, and the Authors’ affiliations. Then I read the Methods. Finally I read the article through. Sometimes I forget and have to go through it again in the right way. I would’ve thought that I was a person who would be good at vetting articles. I’m a math major with an almost Masters in Statistics; was an NIH Research Fellow in Internal Medicine; and have been a computer programmer and closet geek since the first chip hit the market. But that’s not the case. I’ve had to have a lot of help with the methodology from some very patient people [Thanks!] and still get crossed eyes when people use "statistical packages" to generate their stats. They didn’t teach Clinical Trial design in my day and I still struggle with a lot of it.

There’s one thing I have gained poring over these bizarre Clinical Trials that have become the conduit for so much corrupted science. I’m beginning to have a usable bull-shit meter. I’ve been helped along with that because there’s so much of it, so that my gullability rating is also falling. I am no longer wowed by Harvard, Stanford, or Brown [I think Biederman, Schlatzberg, or Keller]. The spin-meisters have found all the trusted markers [institution, publication record, journal reputation, authors’ degrees, etc] and exploited them to the hilt. So when I read Kupfer’s and Kraemer’s article about the DSM-5 Field Trials, my meter sounded its alarm and I went looking for Pinocchio and Jiminy Cricket graphics – a standard symbol for the evils of prevarication. Even though I’ve never had a love affair with any DSM, or for that matter the MDD diagnosis, I’ve come to believe that Reliability [Kappa] means something. As inadequate as it is, it’s our inadequate, and that rationalization up top just won’t do. If you’re going to discount it if it’s not what you expected or wanted – why even measure it?

So now that the DSM-5 is a done deal, I have been looking over the Methods of the DSM-5 Field Trials. They’re in that article no one, including me, spent much time on. I admit that I didn’t even read the abstract. It’s this one:
DSM-5 Field Trials in the United States and Canada, Part I: Study Design, Sampling Strategy, Implementation, and Analytic Approaches
by Diana E. Clarke, William E. Narrow, Darrel A. Regier, S. Janet Kuramoto, David J. Kupfer, Emily A. Kuhl, Lisa Greiner, and Helena C. Kraemer
American Journal of Psychiatry. 2012 October 30, AJP in Advance

Objective: This article discusses the design, sampling strategy, implementation, and data analytic processes of the DSM-5 Field Trials.
Method: The DSM-5 Field Trials were conducted by using a test-retest reliability design with a stratified sampling approach across six adult and four pediatric sites in the United States and one adult site in Canada. A stratified random sampling approach was used to enhance precision in the estimation of the reliability coefficients. A web-based research electronic data capture system was used for simultaneous data collection from patients and clinicians across sites and for centralized data management. Weighted descriptive analyses, intraclass kappa and intraclass correlation coefficients for stratified samples, and receiver operating curves were computed. The DSM-5 Field Trials capitalized on advances since DSM-III and DSM-IV in statistical measures of reliability [i.e., intraclass kappa for stratified samples] and other recently developed measures to determine confidence intervals around kappa estimates.
Results: Diagnostic interviews using DSM- 5 criteria were conducted by 279 clinicians of varied disciplines who received training comparable to what would be available to any clinician after publication of DSM-5. Overall, 2,246 patients with various diagnoses and levels of comorbidity were enrolled, of which over 86% were seen for two diagnostic interviews. A range of reliability coefficients were observed for the categorical diagnoses and dimensional measures.
Conclusions: Multisite field trials and training comparable to what would be available to any clinician after publication of DSM-5 provided “real-world” testing of DSM-5 proposed diagnoses.
I printed out the article when it came out, and have lately looked it over several times. It’s a doozie and I don’t know if I’m up to the task even with time and help. But there’s something wrong in there: the bizarre discordance between sites? the unreliability of the two most common and unchanged diagnostic entities? all the delays? the fact that the Method section of the abstract clarifies nothing? There are too many inconsistencies. Like those industry sponsored Clinical Trials, just talking about the Results and Conclusions isn’t enough. Something is wrong before we get there…
    December 8, 2012 | 10:43 AM

    Doesn’t this behavior kind of mirror what is going on in Syria, albeit not at a level of violence mind you, but, the DSM task force acts like a dictator and thinks it can just demand subservience and respect, yet just brutalizing the masses while the elite few figuratively sit on their couches and eat grapes!

    The irony in the analogy is, while Assad is planning to gas his assailants with Sarin or other chemical weapons, isn’t it interesting the end game agenda of the DSM5 is to chemically neuter the population, and fully expecting the minions of not psychiatry mind you, but the non psychiatric grunts of PCPs, NPs, prescribing able psychologists, and others without a mental health background to just accept the orders and sell, sell, sell, like the Don Ameche character screams in “Trading Places” at the end.

    These are more disturbing times as the new year approaches. First this DSM 5 crap for the past 6 plus months, now the CPT crap by the AMA, oh, and signed off equally by the APA from what I know, and the increasingly reckless prescribing by non psychiatric providers then dumping their messes on primary care psychiatrists. You can read about my opinion as interested at my site by tomorrow, it is nothing less than absurd how benzos are being prescribed in record numbers of late.

    Just expect the shotgun approach with DSM 5: give an antidepressant, an antimanic, an anxiolytic, an antipsychotic, and some side effect med like cogentin or benadryl for good measure. Oh, that’s what the PCPs will be doing once they get the Non-psychiatric version of DSM 5!

    December 8, 2012 | 11:06 AM

    Controversial FDA Official Tom Laughren Retires

    By Ed Silverman // December 7th, 2012 // 8:57 am

    “One of the more controversial figures at the FDA over the past decade quietly retired late last month. Tom Laughren, who was the director of the Division of Psychiatry Products in the Office of New Drugs in the Center for Drug Evaluation and Research, departed the FDA after spending nearly 20 years at the agency, an FDA spokeswoman confirms.

    A board certified psychiatrist who co-authored numerous medical papers, he oversaw FDA approval, labeling decisions and guidance documents that influenced the fate some of the best-known psychiatric drugs marketed in the US and helped make him one of the more notable figures in the field of psychiatry.”

    December 8, 2012 | 11:19 AM

    What’s the point of even talking about the quack book the DSM 5?

    DSM 6, 7, 8, 9, 10 etc. will all be quack bibles, because psychiatry is quackery and has been since its inception.

    December 9, 2012 | 12:41 PM

    The interest of the APA in publishing the DSM-5 is to get a salable product on the shelves for 2013 revenue. How it affects human beings with MDs or without is not a concern.

    December 11, 2012 | 12:34 AM

    “One of the committee’s most ambitious proposals was perhaps the least noticed: a commitment to update the book continually, when there’s good reason to, rather than once every decade or so in a giant heave. That was approved without much fanfare.”

    What is this going to look like? Has anyone noticed this?

Sorry, the comment form is closed at this time.