{"id":62969,"date":"2016-01-07T16:13:18","date_gmt":"2016-01-07T21:13:18","guid":{"rendered":"http:\/\/1boringoldman.com\/?p=62969"},"modified":"2016-01-07T22:24:20","modified_gmt":"2016-01-08T03:24:20","slug":"john-henrys-hammer-continuous-variables-iii","status":"publish","type":"post","link":"https:\/\/1boringoldman.com\/index.php\/2016\/01\/07\/john-henrys-hammer-continuous-variables-iii\/","title":{"rendered":"john henry’s hammer: continuous variables III…"},"content":{"rendered":"\n

\"Alice's<\/a>Generally, folk heros are from the past like John Henry and represent some laudable principle – like man versus machine. Back in the upside-down years of the 60s, Arlo Guthrie had a shot with his endless comical antiwar song, Alice’s Restaurant. The premise, however, was dead serious. When enough people get behind a needed change, that’s when things finally happen. We’ve seen some of that with Data Transparency and distorted Clinical Trial reports alread [though not near enough].<\/p>\n

I happen to agree with Arlo that the hard work of righting a wrong is in keeping it on the front burner [as in the Civil Rights Movement, Women’s Rights, Gay Rights], and the distortion of the scientific record by corruptly reported Clinical Trials is a wrong of sufficient magnitude to dog until it’s past history. While Psychiatry lead the pack there for a while, it’s a problem throughout medicine that needs our full attention – and I personally have come to believe that without Data Transparency<\/font>, it will never happen. One of the major frustrations with these journal articles is that you can’t get at the data itself to evaluate things yourself, at least that’s been my major frustration. Even if you have an academic connection that allows you access to the articles themselves rather than just the abstracts, they often omit key parameters that would allow you to get under the carefully crafted rhetoric even with the crude tools described in the first two john henry’s hammer<\/font><\/em> posts to see what’s going on.<\/p>\n

I’ve been monotonously preoccupied with the most recent arrival on the psychiatric scene, Brexpiprazole<\/font><\/strong> [Rexulti®<\/font><\/strong>] acting on the principle of early detection – if there’s something awry, the sooner it’s exposed the better. Looking at their Schizophrenia data, the papers are ghost-written industry productions sure enough, but they give us enough information to do a reasonable elm-tree mechanic vetting. It’s a middle-weight antipsychotic, a clone of the blockbuster Aripiprazole<\/font> [Abilifyi®<\/font>]. But when I look at the two articles introducing it as an adjunct for augmenting antidepressants in treatment resistant depression<\/font><\/em> [for which it also has FDA Approval!], it’s a different story. First off, there’s the sleight of hand Protocol infraction [see income switching…<\/a>] that I’m going to reject with no more comment and look only at the original Protocol [they call it the efficacy<\/font><\/em> population]. But when I go looking for the summary data on the primary outcome variable [change in MADRS score at 6 weeks], here’s all I find:<\/div>\n
\n
from<\/font><\/em> Adjunctive brexpiprazole 1 and 3 mg for patients with major depressive disorder following inadequate response to antidepressants: a phase 3, randomized, double-blind study.<\/a><\/div>\n

"Mean change in MADRS total score for the efficacy population also showed improvement for brexpiprazole 3 mg versus placebo (−1.52; 95% CI, −2.92 to −0.13; P = .0327) but did not reach the level of statistical significance required for multiple comparisons according to the prespecified statistical analysis. The mean improvement for brexpiprazole 1 mg versus placebo was less than that for 3 mg (−1.19; 95% CI, −2.58 to 0.20; P<\/em> = .0925)."<\/p>\n

from<\/font><\/em> Efficacy and safety of adjunctive brexpiprazole 2 mg in major depressive disorder: a phase 3, randomized, placebo-controlled study in patients with inadequate response to antidepressants.<\/a><\/p>\n

"Similar results were seen for brexpiprazole versus placebo in the efficacy population (LS mean = −8.27 vs −5.15; LS mean difference =−3.12 [95% CI, −4.70 to −1.54], P<\/em> = .0001)."<\/div>\n<\/blockquote>\n

They give us all the summary data we could ever want on all of their secondary outcome variables, but not the primary outcome variable in either article [a μ<\/font><\/strong> or so with the "good article," but no SEM<\/font><\/strong> or σ<\/font><\/strong>]. So I went to the FDA Medical Report<\/a>, hoping that since they approved the drug for this indication, they would give us more. No such luck. Maybe I missed something [and I’d appreciate someone else taking a look], but I ended up with a table that was barely populated and a mighty lonely spreadsheet:<\/p>\n\n\n\n\n\n\n\n\n
BREXPIPRAZOLE Trials Augmenting Treatment Resistant Depression<\/font><\/strong> <\/p>\n
<\/sup><\/td>\n<\/tr>\n
STUDY<\/sup><\/font><\/strong><\/td>\nDRUG<\/sup><\/font><\/strong><\/td>\nMEAN<\/sup><\/font><\/strong><\/td>\nSEM<\/sup><\/font><\/strong><\/td>\nσ<\/sup><\/font><\/strong><\/td>\nn<\/sup><\/font><\/strong><\/td>\np<\/sup><\/font><\/strong><\/td>\nd<\/sup><\/font><\/strong><\/td>\nlower<\/sup><\/font><\/strong><\/td>\nupper<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
Thase
et al<\/sup><\/td>\n		placebo<\/sup><\/td>\n –<\/sup><\/td>\n –<\/sup><\/td>\n–
<\/sup><\/td>\n		 218<\/sup><\/td>\n –<\/sup><\/td>\n–
<\/sup><\/td>\n		–
<\/sup><\/td>\n		 –<\/sup><\/td>\n<\/tr>\n
1mg<\/sup><\/td>\n <\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n 225<\/sup><\/td>\n 0.0925<\/sup><\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n<\/tr>\n
3mg<\/sup><\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n226
<\/sup><\/td>\n		 0.0327<\/sup><\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n<\/tr>\n
Thase
et al<\/sup><\/td>\n		placebo<\/sup><\/td>\n-8.27
<\/sup><\/td>\n		 –<\/sup><\/td>\n–
<\/sup><\/td>\n		 191<\/sup><\/td>\n–
<\/sup><\/td>\n		 –<\/sup><\/td>\n –<\/sup><\/td>\n–
<\/sup><\/td>\n<\/tr>\n
2mg<\/sup><\/td>\n-5.15
<\/sup><\/td>\n		 <\/sup><\/td>\n <\/sup><\/td>\n 187<\/sup><\/td>\n0.0001
<\/sup><\/td>\n		 <\/sup><\/td>\n <\/sup><\/td>\n <\/sup><\/td>\n<\/tr>\n<\/table>\n

Instead, they give us the difference between the means…<\/p>\n

(μ1<\/sub>–μ2<\/sub>)<\/font><\/strong><\/p>\n

and the 95% confidence limits – the formula for that is:<\/p>\n

CI[95%]<\/sub>=(μ1<\/sub>–μ2<\/sub>)±1.96×√((σ1<\/sub>²÷n1<\/sub>)+(σ2<\/sub>²÷n2<\/sub>))<\/font><\/strong><\/p>\n

There’s just no way I can figure to deconstruct those values, work it backwards, and end up with the summary data [even though I know they had it and used it to generate something I didn’t particularly want]. In the two articles, they give us four tables of summary data for secondary outcome variables – two with their jury-rigged numbers in the articles and two with the real numbers in the supplements, but for their declared primary outcome variables, slim pickings. <\/p>\n

Admittedly, some claim that reporting the difference in the MEANs is<\/u> an Effect Size and don’t convert it to a Standardized MEAN Difference [Cohen’s d], but I’m not in that particular some<\/font><\/em> [and they aren’t either, since the same people didn’t do that in their Schizophrenia articles from the same company with the same ghost-writing firm – even reporting the Cohen’s d in their best showing article]. So what’s a fellow who’s trying to write a guide for other elm-tree vetters going to do in this situation? Is John Henry’s Hammer<\/font><\/em> dead in the water? I say where there’s a will, there’s a way. Stay tuned. <\/p>\n

\n

When I read Spielmans et al’s meta-analysis<\/a> of the augmentation with Atypical Antipsychotics in treatment resistant depression<\/font><\/em> [see the extra mile…<\/a>], I ran across these lines:<\/p>\n

\n
"The quality of data reporting varied across studies. For continuous outcomes, effect sizes were computed from means and standard deviations when possible. When these were not provided, effect sizes were computed based on means and p<\/em>-values, or p<\/em>-values only."<\/div>\n<\/blockquote>\n
Having lost some hair over the absence of complete summary data in a number of papers, I was intrigued. So I wrote Glen Spielmans and he got right back to me with good news and bad news. The bad news was expensive software<\/font><\/em> [Comprehensive Meta-Analysis Software<\/a>]. The good news was a link<\/a>, but mostly the conviction that there was an answer. I had looked in my handy Cochrane Handbook for Systematic Reviews and Interventions<\/a> and found this [page 182 of the book version]:<\/div>\n
\n
7.7.7.2 Obtaining standard errors from confidence intervals and P values: absolute (difference) measures<\/font><\/strong><\/em><\/div>\n
… The first step is to obtain the Z value corresponding to the reported P value from a table of the standard normal distribution. A standard error may then be calculated as<\/div>\n
SE = intervention effect estimate\/Z.<\/font><\/strong><\/div>\n
As an example, suppose a conference abstract presents an estimate of a risk difference of 0.03 (P = 0.008). The Z value that corresponds to a P value of 0.008 is Z = 2.652. This can be obtained from a table of the standard normal distribution or a computer (for example, by entering =abs(normsinv(0.008\/2)<\/font><\/strong> into any cell in a Microsoft Excel spreadsheet). The standard error of the risk difference is obtained by dividing the risk difference (0.03) by the Z value (2.652), which gives 0.011.<\/div>\n<\/blockquote>\n

Still a little heady for John Henry’s hammer<\/font>, but promising. The Z value<\/a> is the area under the normal distribution curve used to compute p-values. But while looking around about the Z value, I ran across the Mother Lode – Psychometrica<\/a>, a German web site tailor made for my purposes. It has an Internet Calculator<\/a> that takes the Z value<\/font><\/strong> and the sample size [n1<\/sub>+n2<\/sub><\/font><\/strong>] and out comes Cohen’s d<\/font><\/em><\/strong>. And on top of that, I found another Internet Calculator<\/a> that produces a Z Score<\/font><\/strong> from a p-value [remember, you have to divide the p-value by 2 first!]<\/font><\/strong> requiring neither tables nor spreadsheet visits. So, armed with my two new calculators, I decided to try it out first on something where I had arrived at Cohen’s d<\/font><\/em> and the CI[95%]<\/sub> by other means. So, back to the Brexpiprazole in Schizophrenia Trials [truncated with the new stuff in green]. You’ll note that I’m using my p-values rather than the ones in the articles because one of theirs appears to be in error [Kane 1mg], and that I changed the  sign on Cohen’s d <\/font>and the CI<\/font>[95%]<\/font> <\/sub>:<\/p>\n\n\n\n\n\n\n\n\n\n\n\n
BREXPIPRAZOLE Trials in Schizophrenia<\/font><\/strong> <\/p>\n
<\/sup><\/td>\n<\/tr>\n
STUDY<\/sup><\/font><\/strong><\/td>\nDRUG<\/sup><\/font><\/strong><\/td>\nn<\/sup><\/font><\/strong><\/td>\np<\/sup><\/font><\/strong><\/td>\nd<\/sup><\/font><\/strong><\/td>\nlower<\/sup><\/font><\/strong><\/td>\nupper<\/sup><\/font><\/strong><\/td>\nZ<\/sup><\/font><\/strong><\/td>\nn1<\/sub>+n2<\/sub><\/sup><\/font><\/strong><\/td>\nnew d<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
Correll
et al<\/sup><\/td>\n		placebo<\/sup><\/td>\n178<\/sup><\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n<\/tr>\n
0.25mg<\/sup><\/td>\n87<\/sup><\/td>\n0.2976<\/sup><\/font><\/td>\n0.137<\/sup><\/font><\/strong><\/td>\n-0.120<\/sup><\/font><\/td>\n0.393<\/sup><\/font><\/td>\n1.0146<\/sup><\/font><\/td>\n265<\/sup><\/font><\/td>\n0.128<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
2mg<\/sup><\/td>\n180<\/sup><\/td>\n0.0001<\/sup><\/font><\/td>\n0.414<\/sup><\/font><\/strong><\/td>\n0.204<\/sup><\/font><\/td>\n0.623<\/sup><\/font><\/td>\n3.8906<\/font><\/sup> <\/td>\n358<\/sup><\/font><\/td>\n0.420<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
4mg<\/sup><\/td>\n178<\/sup><\/td>\n0.0007<\/sup><\/font><\/td>\n0.365<\/sup><\/font><\/strong><\/td>\n0.155<\/sup><\/font><\/td>\n0.574<\/sup><\/font><\/td>\n3.3896<\/sup><\/font><\/td>\n356<\/sup><\/font><\/td>\n0.365<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
Kane
et al<\/sup><\/td>\n		placebo<\/sup><\/td>\n180<\/sup><\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n<\/tr>\n
1mg<\/sup><\/td>\n117<\/sup><\/td>\n0.1629<\/sup><\/font><\/td>\n0.166<\/sup><\/font><\/strong><\/td>\n-0.067<\/sup><\/font><\/td>\n0.399<\/sup><\/font><\/td>\n1.3951<\/sup><\/font><\/td>\n297<\/sup><\/font><\/td>\n0.162<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
2mg<\/sup><\/td>\n179<\/sup><\/td>\n0.1488<\/sup><\/font><\/td>\n0.153<\/sup><\/font><\/strong><\/td>\n-0.054<\/sup><\/font><\/td>\n0.360<\/sup><\/font><\/td>\n1.4438<\/sup><\/font><\/td>\n359<\/sup><\/font><\/td>\n0.153<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
4mg<\/sup><\/td>\n181<\/sup><\/td>\n0.0025<\/sup><\/font><\/td>\n0.321<\/sup><\/font><\/strong><\/td>\n0.113<\/sup><\/font><\/td>\n0.529<\/sup><\/font><\/td>\n3.0233<\/sup><\/font><\/td>\n361<\/sup><\/font><\/td>\n0.322<\/sup><\/font><\/strong><\/td>\n<\/tr>\n<\/table>\n

I agree with John Henry and Arlo! Those numbers are close enough for elm tree work in my book! I had no clue if this would pan out, so my hat’s off to the Calculator makers. Why do you divide by two? For that matter, how does it work? I don’t know those answers yet. Magic? <\/p>\n

So moving right along – What about that other dataset where I didn’t know the the Effect Sizes?<\/font><\/em> [The CI[95%]<\/sub><\/font><\/strong> are from the formula in john henry’s hammer: continuous variables I…<\/a>. This one’s also truncated with the new stuff in green]:<\/p>\n\n\n\n\n\n\n\n\n
BREXPIPRAZOLE Trials Augmenting Treatment Resistant Depression<\/font><\/strong> <\/p>\n
<\/sup><\/td>\n<\/tr>\n
STUDY<\/sup><\/font><\/strong><\/td>\nDRUG<\/sup><\/font><\/strong><\/td>\nn<\/sup><\/font><\/strong><\/td>\np<\/sup><\/font><\/strong><\/td>\nZ<\/sup><\/font><\/strong><\/td>\nn1<\/sub>+n2<\/sub><\/sup><\/font><\/strong><\/td>\nd<\/sup><\/font><\/strong><\/td>\nlower<\/sup><\/font><\/strong><\/td>\nupper<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
Thase
et al<\/sup><\/td>\n		placebo<\/sup><\/td>\n 218<\/sup><\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n–<\/td>\n<\/tr>\n
1mg<\/sup><\/td>\n 225<\/sup><\/td>\n 0.0925<\/sup><\/td>\n1.6824<\/sup><\/font><\/td>\n443<\/sup><\/font><\/td>\n0.160<\/sup><\/font><\/strong><\/td>\n-0.027<\/sup><\/font><\/strong><\/td>\n0.347<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
3mg<\/sup><\/td>\n226<\/sup><\/td>\n 0.0327<\/sup><\/td>\n2.1357<\/sup><\/font><\/td>\n444<\/sup><\/font><\/td>\n0.204<\/sup><\/font><\/strong><\/td>\n0.017<\/sup><\/font><\/strong><\/td>\n0.390<\/sup><\/font><\/strong><\/td>\n<\/tr>\n
Thase
et al<\/sup><\/td>\n		placebo<\/sup><\/td>\n 191<\/sup><\/td>\n–<\/sup><\/td>\n–<\/sup><\/td>\n–<\/sup><\/td>\n–<\/sup><\/td>\n–<\/sup><\/td>\n–<\/sup><\/td>\n<\/tr>\n
2mg<\/sup><\/td>\n 187<\/sup><\/td>\n0.0001<\/sup><\/td>\n3.8906<\/sup><\/font><\/td>\n378<\/sup><\/font><\/td>\n0.409<\/sup><\/font><\/strong><\/td>\n0.205<\/font><\/strong><\/sup> <\/td>\n0.613<\/sup><\/font><\/strong><\/td>\n<\/tr>\n<\/table>\n

I doubt if anyone is much surprised that the 3mg dose in that first study had a trivial Cohen’s d – right near inert. Why else would they make it so hard to generate it? But when you take away the Protocol sleight of hand and look at the Effect Size of 0.204, that p-value of 0.0327 doesn’t look very impressive at all. And that’s the point. Of course p and d are still just numbers. But adding the Effect Size [d] to the mix definitely deepens our understanding of the reported values well beyond the p-value. If the authors don’t give an Effect Size, and the editors don’t insist on an Effect Size, we need to be able to easily generate it ourselves. <\/p>\n

As much as I’d like to rale at the FDA for that Augmentation Approval, or run down the internal workings of these last two Internet Calculators, my goal was to provide a way to vet studies by adding the Effect Sizes even if they withheld the summary values – and I think I’ve either done it or am close enough to take a breather. My mathematico-statistico-html-table-formatting<\/font><\/em> libido is all used up for today… <\/div>\n","protected":false},"excerpt":{"rendered":"

Generally, folk heros are from the past like John Henry and represent some laudable principle – like man versus machine. Back in the upside-down years of the 60s, Arlo Guthrie had a shot with his endless comical antiwar song, Alice’s Restaurant. The premise, however, was dead serious. When enough people get behind a needed change, […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[5],"tags":[],"class_list":["post-62969","post","type-post","status-publish","format-standard","hentry","category-opinion"],"_links":{"self":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/62969","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/comments?post=62969"}],"version-history":[{"count":48,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/62969\/revisions"}],"predecessor-version":[{"id":63054,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/62969\/revisions\/63054"}],"wp:attachment":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/media?parent=62969"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/categories?post=62969"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/tags?post=62969"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}