Back in November 2012 when I read the first of these Computerized Adaptive Test papers, I noted that it ended with:
Funding/Support: This work was supported by grant R01-MH66302 from the National Institute of Mental Health…
Additional Information: The CAT-DI will ultimately be made available for routine administration, and its development as a commercial product is under consideration…
and registered a blip, but I didn’t have any idea the story would become so complex and convoluted. Here’s my blog log as I’ve tried to follow it:
| 11/09/2012 really?… 08/12/2013 a road to nowhere… 11/21/2013 careful watching… 12/29/2013 insider trading… 01/03/2014 DSM-5 retrospective I… |
01/03/2014 DSM-5 retrospective II… 01/03/2014 DSM-5 retrospective III… 01/06/2014 royalty? … 01/11/2014 top down problem… 01/16/2014 why?… |
My last post [why? …] was a transcript of an APA Memo in response to the revelation that the authors had not declared significant Conflicts of Interest while they were involved with the DSM-5 Task Force. I found that Memo way off point and started jotting down a timeline to organize my response, and that’s where yesterday went. So here’s my timeline for your perusal. I’ve thrown in a few short quotes from the selections, but the links connect to the real material. Rather than jump in with my thoughts, I’ve put it here for you read through yourself while I think about it too. It tells the story all by itself…
| Date Public | Submitted | Accepted | Published | Disclosure | ||
|
|
||||||
| 1 | 07/01/1993 | Exactly what does the Hamilton Depression Rating Scale measure? by Gibbons RD, Clark DC, and Kupfer DJ. Journal of Psychiatric Research. 1993 27(3):259-273. "… the HDRS total score is a weak index of depressive syndrome severity. The findings provide a benchmark by which the adequacy of future results may be judged, because the multidimensional IRT model does not suffer from the statistical limitations that arise when applying traditional factor analytic methods to discrete symptom ratings…" |
||||
| 2 | 2002-2010 | COMPUTERIZED ADAPTIVE TESTING – DEPRESSION INVENTORY NIMH Project MH066302 NIH RePORTER "Total project funding amount for 9 projects is $4,958,346" |
||||
| 3 | 2002 | A Research Agenda for DSM-V edited by David J. Kupfer, Michael B. First, and Darrel A. Regier, "DSM-IV and ICD-10 are both categorical classifications or typologies, and so were all their predecessors. In principle, though, variation in the symptomatology of mental disorder could be represented by a set of dimensions rather than by multiple categories…" |
||||
| 4 | 11/01/2005 | Dimensional Models for Research and Diagnosis: A Current Dilemma in Toward a Dimensionally Based Taxonomy of Psychopathology by Kupfer, David Journal of Abnormal Psychology. 2005 114[4]:557-559. "Ultimately, new methodological strategies need to be incorporated that address both categorical and dimensional aspects of the overall diagnostic framework. These refinements will be vital in determining the extent and reality of co-occurrence of disorder and the determination of boundaries across specific disorders." |
||||
| 5 | 04/01/2006 | Kupfer appointed chair DSM-5 Task Force | ||||
| 6 | 04/01/2008 | Using Computerized Adaptive Testing to Reduce the Burden of Mental Health Assessment by Robert D. Gibbons, David J. Weiss, David J. Kupfer, Ellen Frank, et al Psychiatric Services 2008 59:361–368. "Instead of using small fixed-length tests, clinicians can create item banks with a large item pool, and a small set of the items most relevant for a given individual can be administered with no loss of information, yielding a dramatic reduction in administration time and patient and clinician burden." |
||||
| 7 | 03/27/2009 | Dr. Jane Costello resigns from the DSM-5 Child and Adolescent Disorders workgroup. In her letter of resignation, she said: "…The tipping point for me was the memo from David and Darrell on February 18, 2009, stating “Thus, we have decided that one if not the major difference between DSM-IV and DSM-V will be the more prominent use of dimensional measures inDSM-V”, and going on to introduce an Instrument Assessment Study Group that will advise workgroups on the choice of old scale measures or the creation of new ones." |
||||
| 8 | unk | Drs. Robert Gibbons and Paul Pilkonis appointed as Expert Advisors to the Instrument Assessment Study Group | ||||
| 9 | 11/29/2011 | Psychiatric Assessments Inc. incorporated in Delaware [enter File #5072041]. | ||||
| 10 | 01/23/2012 | Psychiatric Assessments Inc. incorporated in Illinois [enter File #68256313]. | ||||
| 11 | 08/31/2012 | Yehuda Cohen, a professional management executive, registers Adaptive Testing Technologies website. Mr. Cohen is featured as a principal on the corporate website. . | ||||
| 12 | 10/23/2012 | Privacy Policy posted on the website. | ||||
| 13 | 11/01/2012 | Development of a computerized adaptive test for depression. by Gibbons, Weiss, Pilkonis, Frank, Moore, Kim, and Kupfer. Archives of General Psychiatry. 2012 69[11]:1104-12. "Traditional measurement fixes the number of items administered and allows measurement uncertainty to vary. In contrast, a CAT fixes measurement uncertainty and allows the number of items to vary. The result is a significant reduction in the number of items needed to measure depression and increased precision of measurement." |
||||
| Publication [CAT-DI] | 08/19/2011 | 01/04/2012 | 11/01/2012 | no | ||
| 14 | 12/01/2012 | DSM-5 approved by the APA Trustees. "Cross-Cutting Dimensions" moved to Section III [Emerging Measures and Models]. |
||||
| 15 | 02/25/2013 | The Future Arrived by David J. Kupfer; Emily A. Kuhl; Darrel A. Regier. JAMA. 2013 309[16]:1691-1692. "The next revision of psychiatry’s Diagnostic and Statistical Manual of Mental Disorders [DSM-5] will be published in May 2013 and is the first revision of this psychiatric nomenclature in almost 2 decades…" |
||||
| Publication [DSM-5] | 02/25/2013 | no | ||||
| 16 | 03/02/2013 | Dr. Kupfer’s lecture to ACP Virginia by David J. Kupfer Dr. Kupfer’s lecture to ACP Virginia promoting the Dimensional Measures in DSM-5 [see slides 8, 17, 32, 34, & 36]. No disclosure of PAI/ATT. |
||||
| Presentation [slides] | 03/02/2013 | no | ||||
| 17 | 05/18/2013 | DSM-5 Published. | ||||
| 18 | 07/01/2013 | Computerized Adaptive Test–Depression Inventory Not Ready for Prime Time by Bernard Carroll JAMA Psychiatry. 2013 70[7]:763. "The goal of commercial development seems premature; patients risk being “assayed” against a non–gold standard. Though CAT-DI may have been an interesting statistical challenge, it lacks a solid clinimetric grounding. It is not ready for clinical use…" |
||||
| Publication [letter] | 11/19/2012 | 11/26/2012 | 07/01/2013 | yes | ||
| 19 | 07/01/2013 | Computerized Adaptive Test–Depression Inventory Not Ready for Prime Time: In Reply by Gibbons, Weiss, Pilkonis, Frank, Moore, Kim, and Kupfer JAMA Psychiatry. 2013 70[7]:763-765 "In this case, it is Carroll who has theoverwhelming conflict of interest.As developer,owner,and marketer of the Carroll Depression Scale–Revised, a traditional fixed-length test, it is not surprising that the paradigm shift described in our article would be of serious concern to him." |
||||
| Publication [reply] | 07/01/2013 | no | ||||
| 20 | 07/01/2013 | The Computerized Adaptive Diagnostic Test for Major Depressive Disorder [CAD-MDD]: A Screening Tool for Depression by Gibbons, Hooker, Finkelman, Weiss, Pilkonis, Frank, Moore, and Kupfer. Journal of Clinical Psychiatry. 2013 74[7]:669–674. "Inexpensive [relative to clinical assessment], efficient, and accurate screening of depression in the settings of primary care, psychiatric epidemiology, molecular genetics, and global health are all direct applications of the current system." |
||||
| Publication [CAD-MDD] | 12/20/2012 | 04/05/2013 | 07/01/2013 | yes | ||
| 21 | 08/09/2013 | Development of the CAT-ANX: A Computerized Adaptive Test for Anxiety by Gibbons, Weiss, Pilkonis, Frank, Moore, Kim, and Kupfer. American Journal of Psychiatry. published on-line Aug 9, 2013 "Potential applications for inexpensive, efficient, and accurate screening of anxiety in primary care settings, clinical trials, psychiatric epidemiology, molecular genetics, children, and other cultures are discussed." |
||||
| Publication [CAT-ANX] | 02/08/2013 | 05/06/2013 | 08/09/2013 | yes | ||
| 22 | 08/20/2013 | Dr. Carroll’s letter to JAMA Psychiatry This letter was provided by Dr. Carroll at my request – 1boringoldman |
||||
| [letter] | 08/20/2013 | rejected | unpublished | yes | ||
| 23 | 10/09/2013 | The Future of Mental Health Diagnostic Screening [click on page 4] PsychsTalk Blog: CME Institute by Robert Gibbons, Ellen Frank, and David Kupfer. "… The CAD-MDD and related CAT-DI and CAT-ANX share goals similar to those of the new version of DSM: they seek to improve screening and assessment of mental health disorders in a number of ways for patients, clinicians, and caregivers, including decreasing clinician and patient burden. The third section of DSM-5 is aimed at providing tools for cross-cutting and dimensional assessment, often involving patient-reported outcomes. The electronic version of the DSM-5 will allow for the development and application of many more scales and certainly better possibilities for tracking change and the effectiveness of treatment. Another objective of DSM-5 is to improve the interface with the rest of medicine, especially primary care…" |
||||
| Publication [CME] | 10/09/2013 | yes | ||||
| 24 | 11/20/2013 | Failure to Report Financial Disclosure Information by Gibbons, Weiss, Pilkonis, Frank, and Kupfer. JAMA Psychiatry. 2013 71[1]:95. "To the Editor We apologize to the editors and readers of JAMA Psychiatry for our failure to fully disclose our financial interests in an article that reported a diagnostic tool, the Computerized Adaptive Test for Depression [CAT-DI]… Lead author Robert D. Gibbons, PhD, is the president and founder of PAI, which was incorporated in Delaware in late 2011, then registered to do business in Illinois in January 2012. Dr Gibbons awarded “founder’s shares in PAI” to us, yet all 5 of us failed to report our financial interests in connection with our article and again in a Reply to Letters to the Editor regarding the article… Our submitted disclosure lacked transparency, and we regret our omission." |
||||
| 25 | 11/21/2013 | When is Disclosure Not Disclosure? Healthcare Renewal by Bernard Carroll "Here is a case study in conflict of interest [COI]. A remarkable confession has just appeared by a group of 5 prominent academics, writing in the journal JAMA Psychiatry. Having been outed to the Editors, they now admit to concealing pertinent financial information. One of the five is David J. Kupfer, MD, chairman of the DSM-5 Task Force and past chairman of the department of psychiatry at The University of Pittsburgh…" |
||||
| 26 | 12/23/2013 | DSM-5 – Dimensional Diagnoses – More Conflicts of Interest? Behaviorism and Mental Health by Phil Hickey "… why hasn’t Dr. Kupfer issued some kind of explanation for the lack of transparency? The JAMA Psychiatry letter of apology was just a stark statement of fact, which leaves a huge cloud of doubt not only over Dr. Kupfer, but also over DSM-5 and psychiatry generally…" |
||||
| 27 | 01/14/2014 | Mindy Young, MD, Speaker Memo to APA Assembly members APA Newsroom "Dr. Kupfer should have disclosed to APA his interest in PAI in 2012. Dr. Kupfer’s interest in PAI, which came after the decision had been made to include dimensional measures in DSM-5, did not influence DSM-5’s inclusion of dimensional measures for further study in Section 3. Interest in inclusion of these measures in DSM-5 began with conferences starting in 2003. If and when PAI develops a commercial product with CAT, it will not have any greater advantage than the dozens of dimensional measures currently being marketed by others." |
||||
Something else that is unclear is who is bankrolling this start-up corporation? Last summer they brought on board an executive named Yehuda Cohen who is a mover and shaker in Chicago business circles. I am sure he doesn’t work for peanuts. He set up a website that must be staffed to respond to queries from consumers and professionals. Plus, the corporate office appears to be located in prime commercial space at 217 N Jefferson #600, Chicago IL 60661; phone 312-878-6490. E-mail: info@adaptivetestingtechnologies.com (from the website: you can find a picture of the neighborhood on Google). So, they are racking up significant operating expenses already. Heaven forfend that these expenses might be covered directly or indirectly by their NIMH funding! Are the responsible administrators at NIMH and at The University of Chicago looking and auditing?
It is also unclear whether they have a sound or even a legal business plan. Where will the high powered computing needed for their expansive applications be conducted? Do they have a computing facility at the corporate office? Or do they intend to perform the computing through the NIH-supported Center for Health Statistics at The University of Chicago, which Dr. Gibbons personally directs? If so, did the University sign off on that plan? Is NIH aware of the plan?
In late November I asked NIH about the ownership of the data bases and algorithms on which the corporation relies for the business plan. I received a reply from NIH outlining the applicable federal policy and stating that “We understand that the data are deposited and made available to the community per request through the Center for Health Statistics.” That had to be what Dr. Gibbons told them. Notice that no mention was made of the algorithms. I have replied to NIH, asking them to clarify the status of the algorithms, without which the data bases alone are of little use. I also pointed out to NIH that there is no mention of this public access option in any of the publications from these authors or on the corporate website. This situation is typical of the dissembling style we have seen before from Dr. Gibbons and Dr. Kupfer – lacking in candor and transparency.
The more one looks, the more questions arise.
We are all now very aware of the Chicago Way.
Steve Lucas
I have a few questions:
1. When I think of the advantages of dimensional ratings, I think it addresses the fact that the the conditions we identify do not exist in the categorical forms we label in DSM. However, this work appears to use dimensions to map onto those categorical constructs. The advantage the authors claim is that it allows for a flexible scale that adjusts to each person’s answers.
2. Doesn’t the NIMH have rules regarding commercial development of research it supports? If so, where those rules violated?
I agree with the serious COI and lack of disclosure issues. I am trying to make sure I understand this.
Sandra, regarding the ethics questions, it’s hard to know what’s going on inside a black box. The players here are circling the wagons and making no public statements beyond the bald confession in JAMA Psychiatry November 20. The editors of JAMA Psychiatry and of JAMA are saying nothing in public. I see it as to the credit of the APA that the Speaker did communicate with the members of the Assembly, though I have disagreements with the slant and conclusions of that statement. The additional facts given today by Dr. Nardo in this post should make a re-thinking necessary within the APA.
As the authors are not being forthcoming, all the more reason for NIH and The University of Chicago to make a close examination and audit of the matter. For all we can tell, there may well be boundary violations between the academic effort sponsored by NIMH and the commercial effort sponsored by we know not who.
It’s amazing how many slimy deals there are in psychiatry.
Maybe Allen Frances has been so outraged because his hands were comparatively clean, and he knew others were making millions.
Alto, that is the kind of ungenerous statement we would have expected from Alan Schatzberg and David Kupfer.
At least I said Dr. Frances’s hands were comparatively clean!
Even his version of the DSM had no evidence base. See http://www.wired.com/magazine/2010/12/ff_dsmv/
And the scandal over PMDD…. Every hormonal stage of a woman’s life has been pathologized.
I’m just making a guess at the politics behind current events.