{"id":49624,"date":"2014-09-11T23:01:49","date_gmt":"2014-09-12T03:01:49","guid":{"rendered":"http:\/\/1boringoldman.com\/?p=49624"},"modified":"2014-09-12T07:48:48","modified_gmt":"2014-09-12T11:48:48","slug":"about-my-connectomes","status":"publish","type":"post","link":"https:\/\/1boringoldman.com\/index.php\/2014\/09\/11\/about-my-connectomes\/","title":{"rendered":"about my connectomes<\/em>…"},"content":{"rendered":"\n

While I haven’t thought about it very much, I made a move from the hardest of medical sciences to the softest without any transition. The first time around was in a lab with scintillation counters printing data to punch cards to feed into Fortran programs that cranked out ANOVA with p values. And then I was in the world of psychotherapy where there was little in the way of a control group [or for that matter – any groups], and validation was subjective at best – only clinical. There’s a gulf there that seems like it needs some of those connectomes Dr. Insel loves to talk about. <\/em><\/a><\/em>But apparently I’m not the only person around with that kind of connectome problem. Many of my colleagues seem to obsess about clinical trials and their p values without addressing what matters – clinical relevance. So what! if a drug group in a clinical trial statistically separates from the placebo group if the people treated don’t even notice the difference. Large groups don’t come to our offices, individuals come. So the question is how to take those numbers that are generated in a clinical trial and turn them into something that really matters to the patients and doctors that inhabit those offices. Likely almost anyone reading this already knows what I’m about to say, but I’m about to say it anyway. So this is that post you might want to skip…<\/em><\/p>\n

This is the part I want to mention from Agomelatine efficacy and acceptability revisited: systematic review and meta-analysis of published and unpublished randomised trials<\/a>. As I said above, it may be old hat to many, but it’s still on the growth edge of my own understanding – how to get those numbers into something that has to do with clinical relevance – something that matters to actual people: <\/div>\n
    <\/p>\n
    The present systematic review found that acute treatment with agomelatine is associated with a difference of 1.5 points on the HRSD. This difference was statistically significant, although the clinical relevance of this small effect is questionable. No research evidence or consensus is available about what constitutes a clinically meaningful difference in HRSD scores. Antidepressant research has recently faced the issues of [a] a large number of studies reporting negative findings and [b] a possible increase in placebo response rates, which may be caused by changes in selection of study participants and how studies are conducted. Such changes might contribute to a reduction in the likelihood of identifying drug effectiveness in antidepressant drug trials. However, even with this consideration in mind, it is plausible to agree with one of the agomelatine clinical trials that a difference of less than three HRSD points is unlikely to be clinically meaningful. Other publications have discussed a difference of two points as being clinically important, but the effect of agomelatine in our review was also below this threshold. Furthermore, it cannot be excluded that a 1.5-point difference may reflect a weak effect on sleep-regulating mechanisms rather than a genuine antidepressant effect. <\/font><\/strong><\/div>\n

    <\/p>\n

    In a recent statement, the EMA Committee for Medicinal Products for Human Use [CHMP] pointed out that, in addition to a statistically significant effect in symptom scale scores, the clinical relevance has to be confirmed by responder and remitter analyses and that ‘… results in the short-term trials need to be confirmed in clinical trials, to demonstrate the maintenance of effects’. For dichotomous outcomes, agomelatine was not superior to placebo in terms of relapse and remission rates, but was statistically superior to placebo in terms of response rates. The difference in response rates corresponds to an absolute risk difference of 6% and to a number needed to treat [NNT] of 15. Based on an analysis of regulatory submissions, which found an average difference of 16% in the response rates between common antidepressants and placebo, EMA CHMP states that this difference ‘… is considered to be the lower limit of the pharmacological effect that would be expected in clinical practice.’ Other authors considered an NNT of ten or below as clinically relevant. Clearly, the effect size in the present analysis is of doubtful clinical significance. This point is strengthened by the fact that depression is a clinical condition for which many active antidepressants are already available.<\/div>\n

    <\/span><\/font><\/ul>\n

    Over there on the right is our old friend, Mr. Normal Distribution. If you take a large sample of almost anything, the values will look like this with the measurement variable across the bottom [abscissa] and the frequency of the values on the left axis [ordinate]. So you can describe a dataset that fits this distribution [most of them] with just three numbers: μ<\/font><\/strong> [the mean, average]; n<\/font><\/strong> [the number of observations]; and σ<\/font><\/strong> [the standard deviation, an index of how much variability there is within the data]. 95% of the values fall between two standard deviations on either side of the mean. That means that measurements outside those limits have only a 5% chance of belonging to this group – thus p<\/font><\/strong><\/em> < 0.05 [p <\/font><\/strong><\/em>as in probability<\/em>]. <\/div>\n

    In the examples on the left, all three might be statistically significant but looking at the p <\/font><\/strong><\/em>value won’t tell you anything about the likelihood it will be a property noticed by individual patients. It might be simple a chemical finding that’s imperceptible, or it might be a power-house. But p<\/font><\/strong><\/em> doesn’t tell you that. Regulatory bodies, like the Food and Drug Administration [FDA], were established to keep inactive patent medicines<\/em> off the medical market, not to direct drug usage or attest to the magnitude of their effects. So the FDA insists on two well conducted Clinical Trials that demonstrate statistically significant differences between the drug and a placebo. The main task of the FDA Approval process is safety – what are the drug’s potential harms? Efficacy is a soft standard added on in 1962.<\/p>\n

    There are mathematical ways to use the data generated in a Clinical Trial to get at something more worth knowing – How strong is the effect of the drug? Just looking at the figures on the left demonstrates one such method – the Mean Difference<\/font><\/strong> between the two groups. How far apart are their μ<\/font><\/strong> values? measured in this example as units on the HAM-D Scale. That could be used to compare different studies if they all used the HAM-D [but they don’t]. Another problem: What if the variability [σ<\/font><\/strong>] of one group is very different from the other group? So they came up with a way to correct the Mean Difference<\/font><\/strong> by dividing it by the variance [of the whole sample]…<\/div>\n
    (μ1<\/sub> – μ2<\/sub>) ÷ σ1+2<\/sub><\/font><\/strong><\/div>\n
    … converting it into standard deviation units<\/em>. It’s called the Standardized Mean Difference<\/font><\/strong> or Cohen’s d<\/em> <\/font><\/strong>or sometimes something else. It can be used for divergent populations or even studies using different rating scales if they purport to measure the same parameter [eg HAM-D, MADRS, CDRS-R, K-SADS-L – all depression scales]. These are variants of measurements of the Effect Size<\/font><\/strong> – How strong is the effect of the measured difference? <\/div>\n

    There are no absolute standards for a meaningful Standardized Mean Difference<\/font><\/strong>. Cohen suggested 0.25 = weak; 0.50 = moderate; 0.75 = strong. But you could gather a bunch of statisticians and they could argue about that for the whole evening. The place where Effect Sizes<\/font><\/strong> are routinely used is in meta-analyses that compared multiple studies and\/or multiple drugs. I guess you could say it’s a powerful relativity<\/em> tool, often combined with a representation of the 95% Confidence Interval. In the example on the right from the article quoted above, the Standardized Mean Difference<\/font><\/strong> is on the Abscissa with the Confidence Intervals as the horizontal line [in this example, the 99% Confidence Intervals were used]. This is the format used in the Cochrane Systematic Reviews called a forest plot and it tells us a lot. The top 5 studies are unpublished Clinical Trials of agomelatine vs placebo. The weighted average is 0.08 in favor of agomelatine [which might as well be zero], and none are significant at the 1% level [the Confidence Interval line crosses zero]. The bottom 5 studies are published studies of agomelatine. The weighted average is 0.26 with only one significant at the 1% level. Overall, the weighted average Standardized Mean Difference<\/font><\/strong> is 0.18. In everyday language, a trivial<\/em> effect with clear publication bias<\/em>.<\/p>\n

    So why go through with all of this simplified statistical mumbo jumbo? It’s because these seasoned Cochrane meta-analizers<\/em> take a very credible stab at translating their findings into the realm of clinical relevance in the colored paragraph above. Whether you use the Standardized Mean Difference<\/font><\/strong> of 0.18 evaluated by the values I quoted above, or the HAM-D Mean Difference<\/font><\/strong> of 1.5 HAM-D units as they did, these studies may be statistically significant at p<\/font><\/strong><\/em> < 0.05 [the published ones are], but they are able to conclude that it is not a clinically relevant difference<\/u>, particularly when you look at the studies Servier<\/font> neglected to let us see. So there’s my mythical connectome<\/em> between the numeric part of my brain and the part that practices clinical medicine.<\/font><\/strong> Very satisfying. <\/p>\n

    So why don’t we see Effect Sizes<\/font><\/strong> plastered all over these Clinical Trials that have flooded our journals? They use the same data as the statistical calculations that are invariably prominently displayed. Would you publish them if your main goal was to sell agomelatine? Probably not because nobody would be very excited about either prescribing it or taking it. You’d display Effect Sizes<\/font><\/strong> and their Confidence Intervals <\/font><\/strong>if you wanted to give clinicians and patients as accurate as possible a notion of how effective the medication might be in relieving the targeted symptom – pending the later results of the reported responses in our offices where clinical medicine meets real live people in pain.<\/p>\n

    I’ve way overly-simplified some of this, probably didn’t get it 100% right, and left out how you measure Effect Sizes<\/font><\/strong> in categorical variables [eg response vs non-response] using the Odds Ratio or the NNT [Number Needed to Treat] [mentioned in the second paragraph above]. All I wanted to do is illustrate how this group is able to go beyond the simple statistical analysis found in most of these Clinical Trials by giving us some information that might help us<\/em> in the actual task at hand [us<\/em> being the clinician and a help-seeking patient]…<\/div>\n","protected":false},"excerpt":{"rendered":"

    While I haven’t thought about it very much, I made a move from the hardest of medical sciences to the softest without any transition. The first time around was in a lab with scintillation counters printing data to punch cards to feed into Fortran programs that cranked out ANOVA with p values. And then I […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[2],"tags":[],"class_list":["post-49624","post","type-post","status-publish","format-standard","hentry","category-politics"],"_links":{"self":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/49624","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/comments?post=49624"}],"version-history":[{"count":42,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/49624\/revisions"}],"predecessor-version":[{"id":49666,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/49624\/revisions\/49666"}],"wp:attachment":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/media?parent=49624"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/categories?post=49624"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/tags?post=49624"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}