{"id":59367,"date":"2015-09-17T12:00:27","date_gmt":"2015-09-17T16:00:27","guid":{"rendered":"http:\/\/1boringoldman.com\/?p=59367"},"modified":"2015-09-17T12:26:48","modified_gmt":"2015-09-17T16:26:48","slug":"study-329-viii-variable-variables-decoded","status":"publish","type":"post","link":"https:\/\/1boringoldman.com\/index.php\/2015\/09\/17\/study-329-viii-variable-variables-decoded\/","title":{"rendered":"study 329 viii &#8211; <em>variable<\/em> variables decoded&#8230;"},"content":{"rendered":"\n<p align=\"center\"><sup><em><strong>WARNING AGAIN:<\/strong> Sometimes the devil really is in the details. Here too, we need to get pretty far into the details&#8230;<\/em><\/sup><\/p>\n<div align=\"justify\" class=\"small\">The 528 page <a href=\"https:\/\/www.gsk.com\/media\/389566\/depression_329_full.pdf\" target=\"_blank\">Full study report acute<\/a> is an exhaustive narrative of the entire study including the results in much more detail than the published article &#8211; filled with charts and tables. It was released in part by court order in 2004 and in full in 2012. The sections that relate to our questions about the changing outcome variables in the published paper are scattered around. The first reference is on <a target=\"_blank\" href=\"https:\/\/www.gsk.com\/media\/389566\/depression_329_full.pdf#page=44\">page 44<\/a>:<\/div>\n<blockquote>\n<div align=\"justify\"><strong><font color=\"#990000\">Prior to opening the blind<\/font><\/strong>, the sponsor and investigators developed a plan to analyze the efficacy data.  The plan described a definition of responders and called for additional measures of effectiveness.  These included the depression items from the HAM-D and K-SADS-L instruments, and the plan provided for a status of remission&#8230;<\/div>\n<\/blockquote>\n<div align=\"justify\" class=\"small\">The reason for the discrepancy mentioned in the last post <a href=\"http:\/\/1boringoldman.com\/?p=58893\">     <\/a> is simple. <font color=\"#200020\">They just changed the outcome variables, redefining the meaning of <\/font><strong><font color=\"#990000\">a priori<\/font><\/strong><font color=\"#200020\"> in the process.<\/font> It&#8217;s intended to mean <strong><em><font color=\"#200020\">before starting the study<\/font><\/em><\/strong>. They&#8217;ve changed it to mean <strong><em><font color=\"#200020\">before the blind is broken<\/font><\/em><\/strong> [which comes at the end of the study]. They continue on <a target=\"_blank\" href=\"https:\/\/www.gsk.com\/media\/389566\/depression_329_full.pdf#page=49\">page 49<\/a>:                           <\/div>\n<blockquote>\n<div><u><strong><font color=\"#200020\">Primary Efficacy Variables<\/font><\/strong><\/u><\/div>\n<div align=\"justify\">The protocol defined the primary efficacy parameters for comparing the efficacy of each active treatment with that of placebo to be:<\/div>\n<ul>\n<li>\n<div align=\"justify\">The change from baseline in total HAM-D score at endpoint of the acute phase.<\/div>\n<\/li>\n<li>\n<div align=\"justify\">The percentage of responders at the endpoint of the acute phase.<\/div>\n<\/li>\n<\/ul>\n<div align=\"justify\">Initially the protocol defined a &quot;responder&quot; as a patient whose HAM-D at endpoint was at least 50% lower than the baseline score.  This definition was to be used as an &quot;operational&quot; criteria for entry into the continuation phase.<\/div>\n<p align=\"justify\">Prior to opening the blind, the sponsor and the investigators developed an analytical plan.  Among other issues, this agreed plan included a definition of a &quot;responder&quot; and a &quot;remission&quot; status.  The intent was to provide a robust definition of &quot;response&quot; and to describe a status of &quot;remission&quot; in order to provide a rigorous anchor point in analyzing relapses in the continuation phase.<\/p>\n<p align=\"justify\">The agreed analytical plan described a &quot;responder&quot; as a patient whose HAM-D score was 8 or less or was reduced from baseline by at least 50%.  The remission status was defined as a HAM-D score of 8 or less. The agreed analytical plan also called for the following measure of effectiveness to be included in the analysis: the 9-item depression subscale of the K-SADS-L, the depression item from both the HAM-D and the K-SADS-L, and two methodologies for analyzing the clinical global improvement score:  1) the mean scores and 2) the proportion of patients with rating of &quot;1&quot; or &quot;2&quot; (&quot;very much&quot; or &quot;much improved&quot; respectively).   The initial protocol described the K-SADS-L and CGI instruments as secondary measures.<\/p>\n<div align=\"justify\">The protocol defined as secondary measures the behavior and functional instruments. These included the Autonomic Function Checklist (AFC), the Self- Perception Profile (SPP), and  the Sickness Impact Scale (SIP).  The agreed analytical plan included a time to sustained response and various subsidiary covariate analyses of response as secondary analyses.<\/div>\n<\/blockquote>\n<p align=\"justify\" class=\"small\">If you look back at the last post, the name changing, <strong><font color=\"#990000\">depression-related variables<\/font><\/strong> and <strong><font color=\"#990000\">declared a priori&nbsp;<\/font><\/strong> make more sense. They simply cover-up the fact that they&#8217;ve changed the outcome variables by changing the meaning of <strong><font color=\"#990000\">a priori<\/font><\/strong> [without really letting you in on the change]. The terms <strong><font color=\"#990000\">depression-related variables <\/font><\/strong>and <strong><font color=\"#990000\">plan of analysis<\/font><\/strong> appear to have come from an <a href=\"http:\/\/www.healthyskepticism.org\/files\/docs\/gsk\/paroxetine\/study329\/19990309McCaffertytoLaden.pdf\" target=\"_blank\">email exchange<\/a> among the ghost writer and the two SKB authors during editing. And speaking of changes, remember that this is part of <a target=\"_blank\" href=\"https:\/\/www.gsk.com\/media\/360485\/329-AppA.PDF#page=55\">Appendix B<\/a> of that self-same Protocol:<\/p>\n<table width=\"95%\" cellspacing=\"0\" cellpadding=\"0\" border=\"0\" align=\"center\">\n<tr>\n<td align=\"left\" class=\"small\"><strong><font color=\"#200020\">II.&nbsp;<\/font><\/strong><\/td>\n<td align=\"left\" class=\"small\"><strong><font color=\"#200020\">PROTOCOL AMENDMENTS<\/font><\/strong><\/td>\n<\/tr>\n<tr>\n<td align=\"justify\" class=\"small\">&nbsp;<\/td>\n<td align=\"justify\" class=\"small\"><font color=\"#200020\">No changes to the study           protocol will be allowed unless discussed in <\/font><font>detail with the SmithKline Beecham (SB) Medical&nbsp;           Monitor and filed as <\/font><font>an           amendment\/modification to this protocol. <\/font><font>Any amendment\/modification to the protocol will           be adhered to by the <\/font><font>participating           centre (or all participating centres) and will apply to all <\/font><font>subjects following approval as appropriate by the           Ethical Review <\/font><font>Committee or           Institutional Review Board.<\/font><\/td>\n<\/tr>\n<\/table>\n<p align=\"justify\" class=\"small\">You might need to take a look at <a target=\"_blank\" href=\"http:\/\/1boringoldman.com\/images\/329-table-2.jpg\">the table<\/a> again. I&#8217;m going to resist the temptation to go on and on about how deceitful that was, since <strong><font color=\"#01717c\">Study 329<\/font><\/strong> will likely become the enduring symbol for why we strictly adhere to the <strong><em><font color=\"#200020\">before starting the study<\/font><\/em><\/strong> meaning of <strong><font color=\"#990000\">a priori<\/font><\/strong> in Clinical Trials going forward. But since we were looking to reanalyze this study, we felt obligated to look for documentation that this plan of analysis actually occurred as described in the <font color=\"#200020\">CSR<\/font>. The volumes of archived documents available from the various lawsuits yielded only the <a target=\"_blank\" href=\"http:\/\/www.healthyskepticism.org\/files\/docs\/gsk\/paroxetine\/study329\/970422teleconference.pdf\">Draft Minutes<\/a> of an April 22, 1997 Teleconference call a few months before the blind was broken. After repeated requests to <strong><font color=\"#ff0000\">GSK<\/font><\/strong>, we received another &#8211; a copy of an email documenting that an SAS programmer was coding for some of the suggested new analysis, again from the period right before the blind was broken. From the perspective of our reanalysis of the data from <strong><font color=\"#01717c\">Study 329<\/font><\/strong>, our path was clear. The point was to aim towards adding a correct analysis of the study to the medical literature, so we planned to use the outcome variables outlined by the original <strong><font color=\"#990000\">a priori<\/font><\/strong> protocol [from <strong><em><font color=\"#200020\">before starting the study<\/font><\/em><\/strong>] and to only mention the late addition non-protocol variables in passing without analysis. <\/p>\n<p align=\"justify\" class=\"small\">Parenthetically, in the three years since I first happened onto the data from <strong><font color=\"#01717c\">Study 329<\/font><\/strong>, there has been an intense debate about Data Transparency &#8211; the AllTrials Initiative, the EMA, the FDA\/NIH, Journals [BMJ], even some PHARMAs, etc. <font color=\"#200020\">But there&#8217;s anything but a consensus about what Data Transparency actually means. The CSR [Clinical Study Report] is often mentioned as the thing to release into the Public Domain for Data Transparency. Every time I read that, I cringe [see <\/font><font color=\"#200020\"><a target=\"_blank\" href=\"http:\/\/1boringoldman.com\/index.php\/2015\/08\/17\/post-it-notes\/\">post-it notes&hellip;<\/a>]. These CSRs seem to vary a lot from trial to trial, but many, like this one, don&#8217;t have the raw data included. In this case, that data was listed as Appendices, but they weren&#8217;t released along with the narrative in 2004. And nobody much noticed for 8 years!&nbsp; The CSR, like the published paper, is only an authored proxy for the data itself. In fact, in the case of <\/font><strong><font color=\"#01717c\">Study 329<\/font><\/strong><font color=\"#200020\">, the CSR is as flawed as the paper itself, if not even more so. Data Opacity reigned.<\/font><\/p>\n<p align=\"justify\" class=\"small\">In this case, we are asked to believe that the authors, uninfluenced by any foreknowledge of the results, decided to completely revamp the outcome analysis of this study in its 11th hour, and just happened to pick new variables that just happened to turn out to be the only ones achieving statistical significance. I don&#8217;t personally believe that for a second, but what I believe is hardly the point. Belief has no place in such a high stakes enterprise. <font color=\"#200020\">There are only two things that guarantee the integrity of a Clinical Trial&#8217;s analysis &#8211; the <\/font><strong><font color=\"#990000\">a priori<\/font><\/strong> <font color=\"#200020\">protocol and the blinding itself. Since there&#8217;s no real way to be sure about the blind, the <\/font><strong><font color=\"#990000\">a priori<\/font><\/strong> <font color=\"#200020\">protocol [from <strong><em>before starting the study<\/em><\/strong>]  is and should be sacrosanct.<\/font><\/p>\n<div align=\"justify\" class=\"small\">We don&#8217;t even have to ask why one might want to change variables along the way &#8211; the new ones were significant and the old ones weren&#8217;t. But as we&#8217;ll soon see, that wasn&#8217;t the only change that occurred <em><font color=\"#200020\">along the way<\/font><\/em>&#8230;<font color=\"#200020\"><br \/><\/font><\/div>\n","protected":false},"excerpt":{"rendered":"<p>WARNING AGAIN: Sometimes the devil really is in the details. Here too, we need to get pretty far into the details&#8230; The 528 page Full study report acute is an exhaustive narrative of the entire study including the results in much more detail than the published article &#8211; filled with charts and tables. It was [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[5],"tags":[],"class_list":["post-59367","post","type-post","status-publish","format-standard","hentry","category-opinion"],"_links":{"self":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/59367","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/comments?post=59367"}],"version-history":[{"count":22,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/59367\/revisions"}],"predecessor-version":[{"id":60017,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/59367\/revisions\/60017"}],"wp:attachment":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/media?parent=59367"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/categories?post=59367"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/tags?post=59367"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}