{"id":49121,"date":"2014-08-26T20:12:10","date_gmt":"2014-08-27T00:12:10","guid":{"rendered":"http:\/\/1boringoldman.com\/?p=49121"},"modified":"2014-08-26T23:51:01","modified_gmt":"2014-08-27T03:51:01","slug":"pixels","status":"publish","type":"post","link":"https:\/\/1boringoldman.com\/index.php\/2014\/08\/26\/pixels\/","title":{"rendered":"pixels?…"},"content":{"rendered":"\n
My last post [and wasted research dollars…<\/a>] lead me to Dr. Nemeroff’s 1984<\/font><\/strong> paper announcing that Cortictrophin-Releasing Factor [CRF] is significantly elevated in the CSF [cerebrospinal fluid] of patients with Major Depressive Disorder – a thirty year old observation that has figured heavily in his research since then – culminating in the clinical trial listed in the last post. It’s a short paper in Science and it’s been open on my desktop for several days. The graph haunts me when I look at it. Here’s the abstract and that one figure from the paper, followed by a description of the analytic methods from the paper:<\/div>\n
\n
Elevated concentrations of CSF corticotropin-releasing factor-like immunoreactivity in depressed patients. <\/a><\/div>\n
by Nemeroff CB<\/font><\/strong>, Widerlöv E, Bissette G, Wall\u00e9us H, Karlsson I, Eklund K, Kilts CD, Loosen PT, Vale W. <\/div>\n
Science<\/font><\/strong>. 1984<\/font><\/strong> Dec 14;226(4680):1342-4.<\/div>\n

The possibility that hypersecretion of corticotropin-releasing factor (CRF) contributes to the hyperactivity of the hypothalamo-pituitary-adrenal axis observed in patients with major depression was investigated by measuring the concentration of this peptide in cerebrospinal fluid of normal healthy volunteers and in drug-free patients with DSM-III diagnoses of major depression, schizophrenia, or dementia. When compared to the controls and the other diagnostic groups, the patients with major depression showed significantly increased cerebrospinal fluid concentrations of CRF-like immunoreactivity; in 11 of the 23 depressed patients this immunoreactivity was greater than the highest value in the normal controls. These findings are concordant with the hypothesis that CRF hypersecretion is, at least in part, responsible for the hyperactivity of the hypothalamo-pituitary-adrenal axis characteristic of major depression.<\/p>\n

<\/p>\n

"The results (see Fig. 1) were statistically analyzed by both parametric [analysis of variance (ANOVA) and Student-Newman-Keuls test] and nonparametric (Mann-Whitney U test) methods."<\/em><\/div>\n<\/blockquote>\n

One of the most frustrating things about papers like this is that the raw data isn’t available, even if one has the time to go over it in detail. Once again, I find myself looking at a graph that I’m told is meaningful, significant, has something to say important about a major psychiatric syndrome. And what I see looks like a trivial difference that is probably meaningless, and I even doubt significant. So I did something that I’ve been tempted to do many times. I opened it in a graphics program and reconstituted the data by measuring the pixel count to the center of each data point and using that table, the baseline, and the ordinate scale to reproduce the data. I wouldn’t recommend this on a Nobel Prize application or even in a paper, but I thought I’d give it a shot because I don’t believe the analysis is correct, or correctly done [the next paragraphs is only for the hardy]. <\/p>\n

So armed with my little made-up table, I proceeded to the analysis. It says that they used an ANOVA. That means considering the numbers as a continuous variable. In an ANOVA with four groups, first you check the whole dataset to see if there is any significance to the grouping. If there is, then you test the groups against each other to locate the significant difference. But with a small dataset like this where the assumptions of ANOVA [normal distribution] are questionable, it is more accurate to use a non-parametric statistic that only considers the ranking of each value, not its magnitude. With four groups, the drill is the same. First one tests the whole dataset to see if the grouping in significant [Kruskal-Wallis]. If it is, you test the groups against each other to find the significant differences [Mann-Whitney]. If you read their paragraph [in italics], it’s hard to figure out exactly what they did but it looks like some steps were skipped. Here’s my version using the R statistical package. <\/div>\n

<\/p>\n

The top green value [p = 0.007656<\/font><\/strong>] says that the ANOVA is significant [p<0.05]. But in the table of pairwise comparisons, it’s not the difference between NORMAL and MDD that achieves significance [p = 0.10858<\/font><\/strong>]. In the non-parametric test, the overall Kruskal-Wallis test of the table is not significant [0.1485<\/font><\/strong>]. There’s nothing there. Whether my crude method is valid or not, it sure doesn’t say this: <\/div>\n
\n
The CSF concentration of CRF-LI was significantly increased (by both methods of statistical analysis) in patients with major depression compared to either the normal controls or the patients with schizophrenia or senile dementia.<\/em>"<\/div>\n<\/blockquote>\n
My point in playing this little game is that we deserve the access to raw data<\/font><\/strong> for this very reason. This 30 year old study has been rehashed and discussed for years and has been nuclear to several grant requests, including the Clinical Trial in the previous post. It looks like a thorough vetting thirty years ago might well have put it to rest. I can’t find further studies to confirm this finding and nothing that suggests that this compound has any solid connection with PTSD. If you haven’t figured it out yet. I think this whole line of research is based on unsubstantiated speculations. <\/div>\n

As you may recall, when we looked at Dr. Nemeroff’s NYU Grand Rounds and London lecture to the Institute of Psychiatry, we were alerted to a study reported as positive that Dr. Nemeroff, himself, had reported as based on an error so the significance disappeared, yet he presented it as a valid study in those presentations [see has to stop…<\/a>]. So the best predictor of future behavior is past behavior. Now we have GSK, the VAH, and the NIMH chasing some new drug as a treatment for PTSD based on the very shakiest of speculations. Shame on him. Shame on them. And shame on journals that don’t vet questionable studies like this. <\/p>\n

Maybe we ought to say shame on me too for using a pixel count to get my numbers. But instead of that – why not support Data Transparency so I don’t have to resort to extreme measures to confirm my reaction to that graph. Like I said, this kind of silliness has to stop…<\/a><\/p>\n

\n

Whoops:<\/font><\/strong> [for the even more hardy] I left out this plot from the R package. The upper and lower borders of the "boxes" represent 25% and 75% of the points. The fact that the Means [bold horizontal lines] aren’t centered in the box points to a skewing of the data [not normally distributed], suggesting that the ANOVA is not the best choice of statistics, and that the non-parametric test is a more appropriate choice [Kruskal-Wallis]. My method of data capture is also more likely to be accurate using only the rank order.<\/p>\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

My last post [and wasted research dollars…] lead me to Dr. Nemeroff’s 1984 paper announcing that Cortictrophin-Releasing Factor [CRF] is significantly elevated in the CSF [cerebrospinal fluid] of patients with Major Depressive Disorder – a thirty year old observation that has figured heavily in his research since then – culminating in the clinical trial listed […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[2],"tags":[],"class_list":["post-49121","post","type-post","status-publish","format-standard","hentry","category-politics"],"_links":{"self":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/49121","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/comments?post=49121"}],"version-history":[{"count":24,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/49121\/revisions"}],"predecessor-version":[{"id":49145,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/49121\/revisions\/49145"}],"wp:attachment":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/media?parent=49121"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/categories?post=49121"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/tags?post=49121"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}