the consequences of the DSM-III unitary depression
being questioned about focusing on "old studies"
Peter Kramer’s snappy response to Ed Shorter’s review
… and then the thread about Lewis, Kiloh, and Parker on Lewis’ cases
Based on his MD Thesis cases, Aubrey Lewis couldn’t confirm the classic separation between neurotic and melancholic depression. I read the Lewis papers [200+ pages], and I couldn’t confirm the diagnostic dichotomy either. So I can’t fault him for his conclusion. But it’s easy to see why his dataset might not be the best choice for evaluating the distinction. These were inpatient cases admitted to the Maudsley Hospital, and they were severely ill – a skewed sample of the general cohort of depressed patients by any read [the hard cases]. So I can fault him for generalizing from this sample, and for sticking to his conclusion throughout his influential career. I think he made a mistake.
In my experience, the clinical differentiation of Melancholia from other depressive states is not so difficult as it sounds in Aubrey Lewis‘ papers. In political sabbatical…, I provided links to articles with diagnostic criteria, including a follow-up article by Gordon Parker [furthering the use of latent-class analysis] and Bernard Carroll‘s article on diagnosis [that also puts it to verse]:
"Shorter’s critique amounted to little more than a complaint that I have disregarded a controversial theory he favors. He used the review space to give a hobbyhorse a ride."
seems way, way off base to me. Whether by intention or not, Kramer perpetuates the "all depression = brain disease" agenda of the KOLs whose motives are certainly suspect.
But whether his conclusions turned out to be mistaken or not, Lewis’ articles are absolute classics – particularly the last one [Melancholia: prognostic studies and case-material, 1936] which has his 61 case narratives with almost a full journal page each [56 pages ÷ 61 cases = 0.92 pages/case]. You know they’re classics because two different solid investigators were able to find what they needed to do their sophisticated reanalyses using Lewis’ case reports. It was as if those cases had waited patiently [pun intended] for a half century for the statistical methodology and the computing power to come along that could finally analyze them.
Reading Lewis’ commentary and particularly his patient narratives just reminded me of how much I missed case reports. As physicians, it’s the plight of people taken one at a time that matters. We collect them in groups of shared syndromes to see what we can learn from their similarities, but their differences are important too. And we may not be able to find the most important things just yet, but if the narratives endure, maybe some future reader can see what we missed, or bring some new pair of glasses that finds what we were looking for. And that’s exactly what happened here. So the next time I’m asked why I’ve continued to look at "old" studies, I’m going to say, "If the data’s faithfully recorded and maintained, there’s no such thing as an old study" and start talking about the Lewis cases and their reanalysis fifty years later.
I had a related experience almost 50 years ago myself. When I started a fellowship in rheumatology in a former life, my boss was in the process of happening onto an significant finding. Using an electron microscope, he was studying the capillary morphology in rheumatologic diseases in biopsies from uninvolved muscle. And in Scleroderma [Progressive Systemic Sclerosis] the technicians couldn’t find any capillaries to look at. So he developed a technique to quantify the capillary density, and sure enough, it was dramatically diminished – no small observation for a disease characterized by generalized scarring. So my first project was the literature review of previous reports of vascular problems in Scleroderma. I doubted I’d find anything. In those pre computer search days, it was no small task and I spent months in the dusty library stacks. The results were amazing. It was everywhere, vascular lesions recorded in the meticulous old reviews of pathology in journals from every specialty, every organ system. They didn’t realize the significance of what they were seeing, but they had written down what they saw for me to find years later. They just didn’t have our electron microscope to move them that final inch. We ended up having to radically trim our reference list to only the most pertinent papers.
And that’s it, a river that runs through it. As much as I’d enjoy another good rant about why the distinction between Melancholia [Depression] and the heterogeneous category, depression matters, the real lesson in this story is that it’s the patients and their data that’s important. I think Aubrey Lewis did the best he could with the data he had. I wish he’d taken a later look at a more representative cohort, but that doesn’t detract from what he did do. His legacy, in this instance, is the careful case histories he passed on to Kiloh and Parker. And the impressive thing is that even with this skewed cohort, they were able to convincingly detect the two populations that it represents. And that’s probably the river that runs through this whole blog and others like it. It’s the patients, their narratives, and the data generated in its rawest form that endures, not the interpretations of its contemporary custodians. Just imagine how much we could learn if we had this kind of data from all the subjects in the clinical trials of antidepressants that we write about.
[Norman Maclean, author of “A River Runs Through It”]
One could make the same categorical argument about colds, flu and sinusitis, that they are all part of the same basic illness because of considerable overlaps. Except they very clearly are not and have different causes. Once one looks into the details such as duration and the presence or absence of fever, or duration, the difference becomes clear.
If you had a symptom scale for those illnesses on a 0-3 basis, you could have someone with the flu and sinusitis both scoring a 24 or whatever, but the critical difference is that the flu patient had a different duration of illness and the sinusitis patient didn’t have myalgias.
The term “mild major depression” shows how confused the establishment is. That’s kind of like “semipro Major League Baseball.”
Which gets us back to the idea of grief, common depression and major depression all of which might get you a score of 40 on the BDI but upon further review the symptoms and durations are qualitatively different.
But that type of analysis doesn’t sell drugs and polemics.
Interestingly, this can sometimes evolve the other way. Headaches used to be more strictly subcategorized than they are now.
This trilogy is a fine tribute to Aubrey Lewis, Leslie Kiloh, and Gordon Parker as clinician-researchers. Kiloh came from the Newcastle school which, under Martin Roth, developed early clinical instruments for discriminating endogenous and nonendogenous depressions, and for predicting response to ECT. The German transplants to UK during the Nazi era, such as Willi Mayer-Gross, likewise held to the binary view of clinical depressions. So, even in the UK, Lewis’s strong position did not go unchallenged. This history makes Peter Kramer’s statements all the more unreliable. He seems to have absorbed by osmosis the nihilistic position of his former boss Gerald Klerman that if we cannot make the distinctions reliably then we should not even try.
Now, 80 years on since the early Lewis papers, we see more clearly that the binary view is just a first cut at the classification of depressions. The nonendogenous group is heterogeneous, as Dr. Mickey and Donald Klein have elaborated. There are reasons also to think that the endogenous or melancholic group also is heterogeneous. For one, there are the unipolar and bipolar melancholic cases. Then there is the condition increasingly discussed today as vascular depressions. These typically are of late onset (after age 50), with a low genetic loading for major mood disorders, and MRI-defined vascular changes in the brain. These are relatively unresponsive to antidepressant medications, so they often receive ECT and then maintenance ECT.
As for multivariate classification techniques, those have progressed now well beyond factor analysis and cluster analysis that emerged in the 1960s. Max Woodbury at Duke developed a procedure called Grade of Membership or GOM analysis, which clearly reveals multiple depressive types. As we now know with some historical perspective on the Aubrey Lewis data sets, a key determinant of what we find is the representativeness of the sample that we analyze. Lewis was correct, as you say, but he and his group at the Maudsley overstated the generalizability of their results from a limited sample.
I had a few more thoughts along the way when I was reading these papers. One was about the RDoC initiative at the NIMH. It’s still not completely clear what it even is, but by inference, I think they’re trying to create a mega-database with multiple measurements including response to treatments. And the idea is to apply these various discriminating “big data” algorithms to find correlations, patterns, etc. Based on Insel’s exit comments about not having enough computing power, I gathered that it wasn’t going very well.
Something else. I thought Lewis using narrative case summaries was a good idea. You never know what you’ll want to know in the future, and simply compiling the parameters you think matter now, may make it impossible to use the data in the future because the variables you want to look at aren’t there. With narratives, they may be “extractable,”
They have no problems slicing and dicing bipolar into many subcategories. The difference is there’s money in that. Or coming up with categorical noncategories like Mood Disorder NOS, because you need a code to get paid.
I realize this is a bit off-topic for this thread, and I have not visited this website daily and do not know if anyone commented on this in a previous thread, but I wanted to share a link to a webinar held back in June which debates the quality of the evidence of decades of fMRI resesarch in psychiatry. It was stimulated by and article co-authored by Daniel Weinberger of NIMH that was published in the American Journal of Psychiatry that emphasized the problematic confounds in this type of research. Why is this important? Because neuroimaging research is one of the primary components of the RDoC project to identify brain circuits underlying psychiatric disorders. Weinberger, for those of you who do not know him, is a major schizophrenia researcher and his 1986 proposal to reframe schizophrenia as a “neurodevelopmental disorder” has become paradigmatic.
Weinberger’s bluntness in his comments in this webinar are stunning. He is the second speaker. I urge anyone with an interest in the status of neuroimaging evidence to listen to this:
It may not be that off topic. in Against Depression, Kramer explains a lot of his disease modeling on the overly simplistic idea that brain changes=brain pathology. This is also covered pretty thoroughly in Satel and Lillenfield’s Brainwashed.
For the sake of leaving a bread crumb trail that someone else down the line may want to follow, the status of fMRI evidence is being seriously challenged on many fronts. This Retraction Watch report on a recent survey of 40,000 such studies suggests that as much as 70 percent of all findings might have been false positives.
Richard Noll is right on target with his comment about the implications for RDoC. Our friend Neuroskeptic has looked into this as well, and has received extensive comments in reply. It seems the number of affected studies may be much lower than 70%, but it’s early in this developing story yet.
“Finally, we point out the key role that data sharing played in this work and its impact in the future. Although our massive empirical study depended on shared data, it is disappointing that almost none of the published studies have shared their data, neither the original data nor even the 3D statistical maps. As no analysis method is perfect, and new problems and limitations will be certainly found in the future, we commend all authors to at least share their statistical results [e.g., via NeuroVault.org (44)] and ideally the full data [e.g., via OpenfMRI.org (7)]. Such shared data provide enormous opportunities for methodologists, but also the ability to revisit results when methods improve years later.”
Until this happens, there is no reason to trust this research.
One could make the same categorical argument about colds, flu and sinusitis, that they are all part of the same basic illness because of considerable overlaps. Except they very clearly are not and have different causes. Once one looks into the details such as duration and the presence or absence of fever, or duration, the difference becomes clear.
If you had a symptom scale for those illnesses on a 0-3 basis, you could have someone with the flu and sinusitis both scoring a 24 or whatever, but the critical difference is that the flu patient had a different duration of illness and the sinusitis patient didn’t have myalgias.
The term “mild major depression” shows how confused the establishment is. That’s kind of like “semipro Major League Baseball.”
Which gets us back to the idea of grief, common depression and major depression all of which might get you a score of 40 on the BDI but upon further review the symptoms and durations are qualitatively different.
But that type of analysis doesn’t sell drugs and polemics.
Interestingly, this can sometimes evolve the other way. Headaches used to be more strictly subcategorized than they are now.
I’m going to dump on my own point by noting that the Atlanta Braves could very well be considered a semipro Major League team…
This trilogy is a fine tribute to Aubrey Lewis, Leslie Kiloh, and Gordon Parker as clinician-researchers. Kiloh came from the Newcastle school which, under Martin Roth, developed early clinical instruments for discriminating endogenous and nonendogenous depressions, and for predicting response to ECT. The German transplants to UK during the Nazi era, such as Willi Mayer-Gross, likewise held to the binary view of clinical depressions. So, even in the UK, Lewis’s strong position did not go unchallenged. This history makes Peter Kramer’s statements all the more unreliable. He seems to have absorbed by osmosis the nihilistic position of his former boss Gerald Klerman that if we cannot make the distinctions reliably then we should not even try.
Now, 80 years on since the early Lewis papers, we see more clearly that the binary view is just a first cut at the classification of depressions. The nonendogenous group is heterogeneous, as Dr. Mickey and Donald Klein have elaborated. There are reasons also to think that the endogenous or melancholic group also is heterogeneous. For one, there are the unipolar and bipolar melancholic cases. Then there is the condition increasingly discussed today as vascular depressions. These typically are of late onset (after age 50), with a low genetic loading for major mood disorders, and MRI-defined vascular changes in the brain. These are relatively unresponsive to antidepressant medications, so they often receive ECT and then maintenance ECT.
As for multivariate classification techniques, those have progressed now well beyond factor analysis and cluster analysis that emerged in the 1960s. Max Woodbury at Duke developed a procedure called Grade of Membership or GOM analysis, which clearly reveals multiple depressive types. As we now know with some historical perspective on the Aubrey Lewis data sets, a key determinant of what we find is the representativeness of the sample that we analyze. Lewis was correct, as you say, but he and his group at the Maudsley overstated the generalizability of their results from a limited sample.
I had a few more thoughts along the way when I was reading these papers. One was about the RDoC initiative at the NIMH. It’s still not completely clear what it even is, but by inference, I think they’re trying to create a mega-database with multiple measurements including response to treatments. And the idea is to apply these various discriminating “big data” algorithms to find correlations, patterns, etc. Based on Insel’s exit comments about not having enough computing power, I gathered that it wasn’t going very well.
Something else. I thought Lewis using narrative case summaries was a good idea. You never know what you’ll want to know in the future, and simply compiling the parameters you think matter now, may make it impossible to use the data in the future because the variables you want to look at aren’t there. With narratives, they may be “extractable,”
They have no problems slicing and dicing bipolar into many subcategories. The difference is there’s money in that. Or coming up with categorical noncategories like Mood Disorder NOS, because you need a code to get paid.
I realize this is a bit off-topic for this thread, and I have not visited this website daily and do not know if anyone commented on this in a previous thread, but I wanted to share a link to a webinar held back in June which debates the quality of the evidence of decades of fMRI resesarch in psychiatry. It was stimulated by and article co-authored by Daniel Weinberger of NIMH that was published in the American Journal of Psychiatry that emphasized the problematic confounds in this type of research. Why is this important? Because neuroimaging research is one of the primary components of the RDoC project to identify brain circuits underlying psychiatric disorders. Weinberger, for those of you who do not know him, is a major schizophrenia researcher and his 1986 proposal to reframe schizophrenia as a “neurodevelopmental disorder” has become paradigmatic.
Weinberger’s bluntness in his comments in this webinar are stunning. He is the second speaker. I urge anyone with an interest in the status of neuroimaging evidence to listen to this:
http://www.schizophreniaforum.org/for/live/detail.asp?liveID=99
It may not be that off topic. in Against Depression, Kramer explains a lot of his disease modeling on the overly simplistic idea that brain changes=brain pathology. This is also covered pretty thoroughly in Satel and Lillenfield’s Brainwashed.
For the sake of leaving a bread crumb trail that someone else down the line may want to follow, the status of fMRI evidence is being seriously challenged on many fronts. This Retraction Watch report on a recent survey of 40,000 such studies suggests that as much as 70 percent of all findings might have been false positives.
http://retractionwatch.com/2016/07/12/is-the-bulk-of-fmri-data-questionable/
Again, the relevance of these critiques is in their implications for the foundational logic of the RDoC project.
Just FYI. I see a fire brewing that has the potential to engulf the whole forest of psychiatric neuroimaging studies.
Richard Noll is right on target with his comment about the implications for RDoC. Our friend Neuroskeptic has looked into this as well, and has received extensive comments in reply. It seems the number of affected studies may be much lower than 70%, but it’s early in this developing story yet.
Key excerpt from the original article:
“Finally, we point out the key role that data sharing played in this work and its impact in the future. Although our massive empirical study depended on shared data, it is disappointing that almost none of the published studies have shared their data, neither the original data nor even the 3D statistical maps. As no analysis method is perfect, and new problems and limitations will be certainly found in the future, we commend all authors to at least share their statistical results [e.g., via NeuroVault.org (44)] and ideally the full data [e.g., via OpenfMRI.org (7)]. Such shared data provide enormous opportunities for methodologists, but also the ability to revisit results when methods improve years later.”
Until this happens, there is no reason to trust this research.