{"id":5220,"date":"2011-03-03T15:46:24","date_gmt":"2011-03-03T20:46:24","guid":{"rendered":"http:\/\/1boringoldman.com\/?p=5220"},"modified":"2011-03-03T15:50:17","modified_gmt":"2011-03-03T20:50:17","slug":"contents-of-an-empty-mind","status":"publish","type":"post","link":"https:\/\/1boringoldman.com\/index.php\/2011\/03\/03\/contents-of-an-empty-mind\/","title":{"rendered":"contents of an empty mind&#8230;&#8230;"},"content":{"rendered":"\n<ul>\n<div align=\"justify\"><em><font color=\"#999966\">Occasionally, not thinking about things allows some room in the mind to think.<\/font><\/em><\/div>\n<\/ul>\n<div align=\"justify\">I&#8217;ve spent weeks reading and rereading these <strong><font color=\"#660099\">Seroquel<\/font><\/strong> Clinical Trials and remained befuddled. The graphs all look almost the same. I even named them after the Companies that make them [<strong><font color=\"#200020\">Clinical Research Organizations<\/font><\/strong>]:<\/div>\n<div align=\"center\"><img loading=\"lazy\" decoding=\"async\" width=\"300\" vspace=\"5\" height=\"308\" border=\"0\" src=\"http:\/\/1boringoldman.com\/images\/cro-1.gif\" \/><\/div>\n<div align=\"justify\">The most obvious problem is that the Placebo subjects improve dramatically in many of the trials. When I go looking for how to explain that, the answers aren&#8217;t very satisfying. Since there&#8217;s a big drop-out rate [50%+], maybe it&#8217;s because the people on placebos that are getting worse drop out. But in the studies with more available data, that&#8217;s not always the case. And that&#8217;s not all of what&#8217;s wrong with these graphs &#8211; the differences generated may be statistically significant, but they often look kind of trivial.<\/div>\n<p align=\"justify\">Earlier [<strong><a target=\"_blank\" href=\"http:\/\/1boringoldman.com\/index.php\/2011\/02\/09\/seroquel-ii-version-20-guessing\"><font color=\"#200020\">seroquel II [version 2.0]: guessing&hellip;<\/font><\/a><\/strong>], there was an example that has stayed with me [<strong><font color=\"#200020\">Trial 0006<\/font><\/strong>]. This was the graph <a href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/8690831\" target=\"_blank\"><u><strong><font color=\"#200020\">published<\/font><\/strong><\/u><\/a> in the study itself, a <strong><font color=\"#200020\">CRO-Chart <\/font><\/strong><em>extraordinare<\/em>:                                    <\/p>\n<p align=\"center\"><img loading=\"lazy\" decoding=\"async\" width=\"384\" height=\"281\" border=\"0\" src=\"http:\/\/1boringoldman.com\/images\/fda-seroquel-9.gif\" \/><\/p>\n<p align=\"justify\">But in the <a href=\"http:\/\/www.accessdata.fda.gov\/drugsatfda_docs\/nda\/97\/020639s000_StatR_P2.pdf\" target=\"_blank\"><u><strong><font color=\"#200020\">F.D.A. approval documents<\/font><\/strong><\/u><\/a>, they had the  information from before the LOCF [Last Observation Brought Forward]  correction was applied, so we got closer to what they actually observed &#8211;  the mean scores of the non-drop-outs (BPRS [PSS][OC]) uncorrected:<\/p>\n<p align=\"center\"><img loading=\"lazy\" decoding=\"async\" width=\"520\" height=\"373\" border=\"0\" src=\"http:\/\/1boringoldman.com\/images\/fda-seroquel-6.gif\" \/><\/p>\n<div align=\"justify\">In the <a href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/8690831\" target=\"_blank\"><u><strong>published paper<\/strong><\/u><\/a>, the authors concluded:<\/div>\n<ul>\n<div align=\"justify\">In this placebo-controlled, multicenter trial,  ICI 204,636 was effective   in the treatment of the positive and  negative symptoms of   schizophrenia. Significant differences (p less or  equal to 0.05) between   treatment groups were identified for the  primary efficacy variables,   the BPRS total score and CGI Severity of  Illness item score, at most   times throughout the trial, with marginal  significance achieved at   endpoint (p = 0.07 for both)&hellip;<\/div>\n<\/ul>\n<div align=\"justify\">Looking at the Observed Cases, I think  that&#8217;s a totally unsupportable conclusion at best. And in the F.D.A. Report,  they give us the the Mean scores of the drop-outs at the time they  dropped out:<\/div>\n<div align=\"center\"><img loading=\"lazy\" decoding=\"async\" width=\"384\" vspace=\"5\" height=\"197\" border=\"0\" src=\"http:\/\/1boringoldman.com\/images\/fda-seroquel-8.gif\" \/><\/div>\n<div align=\"justify\">Even with this additional information, we can&#8217;t reconstruct the primary  data-set. What we can see is that the profiles of the scores of the  drop-outs between the Placebo and Quetiapine are dramatically different [but the number of drop-outs at each week is missing]. Those aren&#8217;t &quot;random drop-outs.&quot;          <\/div>\n<p>                    <\/p>\n<div align=\"justify\"><img loading=\"lazy\" decoding=\"async\" width=\"203\" hspace=\"4\" height=\"204\" border=\"0\" align=\"right\" src=\"http:\/\/1boringoldman.com\/images\/cro-2.gif\" \/>Since I&#8217;d never heard of this Last Observation Carried  Forward business, I wrote a more savvy friend about it. He explained  that it was a way to fill in for missing data. Since there are a lot of  drop-outs in these Clinical Trials, they take the Last Observed value at  the time of drop-out and pretend it&#8217;s the value for the remaining  periods in the study [see the graph on the right]. Obviously the graphs  of the means using <strong>Observed Case<\/strong> Values and <strong>Last Observation Carried Forward<\/strong>  values are remarkably different. In reading about this method, they  talk about two kinds of drop-outs: non-informative [random] and  informative [having to do with the variables measured]. Examples of the  latter would be things like clinical worsening, drug side effects, etc.  The LOCF method is recommended for data with non-informative drop-outs  [which is not necessarily the case in these studies]. What it comes down  to is that this is just a convention in Clinical Trials. There&#8217;s another  more complex model MMRM [Mixed-Effect Model Repeated Measure] that is  increasingly recommended. Here&#8217;s how people at the F.D.A. talk about the  difference between the two methods:<\/div>\n<blockquote>\n<div align=\"center\"> <strong><font color=\"#200020\"><a href=\"http:\/\/www.informaworld.com\/smpp\/content%7Econtent=a908658805%7Edb=all%7Ejumptype=rss\" target=\"_blank\">MMRM vs. LOCF: A Comprehensive Comparison Based on Simulation Study and 25 NDA Datasets<\/a><br \/>                            Journal of Biopharmaceutical Statistics<\/font><\/strong> 2009, 19:227 &#8211; 246<br \/>                            by Ohidul Siddiqui, H. M. James Hung, Robert O&#8217;Neill<\/div>\n<p>                            <\/p>\n<div align=\"center\"><em><strong><font color=\"#200020\">Abstract<\/font><\/strong><\/em><\/div>\n<div align=\"justify\"><sup>In  recent years, the use of the last observation carried forward (LOCF)   approach in imputing missing data in clinical trials has been greatly   criticized, and several likelihood-based modeling approaches are   proposed to analyze such incomplete data. One of the proposed   likelihood-based methods is the Mixed-Effect Model Repeated Measure   (MMRM) model. To compare the performance of LOCF and MMRM approaches in   analyzing incomplete data, two extensive simulation studies are   conducted, and the empirical bias and Type I error rates associated with   estimators and tests of treatment effects under three missing data   paradigms are evaluated. The simulation studies demonstrate that LOCF   analysis can lead to substantial biases in estimators of treatment   effects and can greatly inflate Type I error rates of the statistical   tests, whereas MMRM analysis on the available data leads to estimators   with comparatively small bias, and controls Type I error rates at a   nominal level in the presence of missing completely at random (MCAR) or   missing at random (MAR) and some possibility of missing not at random   (MNAR) data. In a sensitivity analysis of 48 clinical trial datasets   obtained from 25 New Drug Applications (NDA) submissions of neurological   and psychiatric drug products, MMRM analysis appears to be a superior   approach in controlling Type I error rates and minimizing biases, as   compared to LOCF ANCOVA analysis. In the exploratory analyses of the   datasets, no clear evidence of the presence of MNAR missingness is   found.<\/sup><\/div>\n<\/blockquote>\n<div align=\"justify\">I don&#8217;t understand all that I read in this abstract or the paper itself, but I <u>do<\/u> get the main point. The LOCF method is as shaky as it seems like it ought to be, and it produces Type I [false positive] errors and hides bias. In reading all of the published papers of Clinical Trials of <strong><font color=\"#660099\">Seroquel<\/font><\/strong>, I only found <u>one<\/u> that used the MMRM method to analyze the data.          <\/div>\n<ul>\n<div align=\"justify\"><em><font color=\"#999966\">So back to that empty mind thing. While I was watching the waves coming in at the beach last week, I realized something about why I spent 20 years practicing Psychiatry but largely ignored what was going on in the psychopharmacology world. <strong>I was intimidated by their scientific rhetoric.<\/strong> I thought they knew what they were talking about and I didn&#8217;t. So I didn&#8217;t learn how to separate the wheat from the chaff. I avoided the literature and relied on others to tell me about these new drugs [that I rarely used] because I felt the studies were beyond me [and terminally boring]. It didn&#8217;t occur to me that the opacity of the articles might be a deliberate trick. I hadn&#8217;t learned to look at who funded the study, who assisted with medical writing, where all those additional authors worked, to find out what the LOCF method was &#8211; that it&#8217;s fraught with errors. <strong>I assumed a scientific integrity where none existed. I&#8217;ll bet a lot of us did that &#8211; got scared off by the jargon &#8211; and didn&#8217;t apply a critical eye to the journals [or avoided them altogether].<\/strong> That&#8217;s one of the things that popped into my empty mind at the beach.<br \/>          <\/font><\/em><\/div>\n<\/ul>\n<div align=\"justify\">A few posts back, I mentioned a vignette from 2004 [<strong><u><a target=\"_blank\" href=\"http:\/\/1boringoldman.com\/index.php\/2011\/03\/01\/selling-seroquel-vii-indication-sprawl\"><font color=\"#200020\">selling seroquel VII: indication sprawl&hellip;<\/font><\/a><\/u><\/strong>]. Dr. Nassir Ghaemi was asked to be a second author on one of those <strong><font color=\"#200020\">CRO-Chart<\/font><\/strong> articles comparing <strong><font color=\"#660099\">Seroquel<\/font><\/strong> and <strong><font color=\"#000099\">Haldol<\/font><\/strong> in Mania. He insisted on seeing the data from the study and actually in participating in writing the paper. <strong><font color=\"#660033\">AstraZeneca<\/font><\/strong> said &quot;No.&quot; More than that, they felt that if they were forced to show him the data, he&#8217;d have to look at it in the company of their own scientists and statisticians. I had run across references to those emails before in an index, but I hadn&#8217;t read them:<\/div>\n<ul>\n<div align=\"justify\"><em><font color=\"#999966\">Another empty mind at the beach thing&#8230; I&#8217;ve spent a couple months going from study to study, trying to figure out about the drop-out rates, trying to find a primary data set where I could see how all this LOCF business worked, and I can&#8217;t find one &#8211; not one. In my beach empty mind I thought, <strong>&quot;The reason I can&#8217;t find any raw data is not because I&#8217;m an inadequate searcher, it&#8217;s because it&#8217;s not there &#8211; for a reason. They don&#8217;t want me to see it.&quot;<\/strong> Then I remembered the Ghaemi email, and when I got home to my real computer, I looked it up. They didn&#8217;t even want the paper&#8217;s author to see the raw data.<br \/>        <\/font><\/em><\/div>\n<\/ul>\n<div align=\"justify\">I ended my last post with, &quot;<em>Statistical significance in a <strong><font color=\"#200020\">CRO-Chart<\/font><\/strong> is simply not enough. That&rsquo;s my take-home from reading the published versions of these <strong><font color=\"#660099\">Seroquel<\/font><\/strong> Clinical Trials. It has become a &#8216;time-to-market&#8217; racket&hellip;<\/em>&quot; I know that to those of you who have been following this stuff for a long time, that might be something you&#8217;ve known forever. My problem is that I only &quot;sort of&quot; knew it &#8211; or only knew pieces of it. I knew Dr. Nemeroff at Emory was a self-serving opportunist. I knew that <strong><font color=\"#660033\">GSK<\/font><\/strong> had misbehaved with Paxil and <strong><font color=\"#660033\">AstraZeneca<\/font><\/strong> was overselling <strong><font color=\"#660099\">Seroquel<\/font><\/strong>. I didn&#8217;t miss the point that the F.D.A. fined <strong><font color=\"#660033\">Eli Lilly<\/font><\/strong> $1.4 B for something. But somehow, I hadn&#8217;t really clicked into understanding that <strong><font color=\"#300030\">the entire industry-funded Clinical Trials scene in Psychiatry is suspect<\/font><\/strong>. It really is a racket. It feels kind of paranoid to say that [just like it felt like saying that Sub-Prime Mortgages were a racket four or five years ago]. Before I launch an attempt to understand the <strong><font color=\"#200020\">CRO<\/font><\/strong>s, I&#8217;d like to mention three articles I ran across in my wanderings. They are attached [<a target=\"_blank\" href=\"http:\/\/1boringoldman.com\/index.php\/clinical-trials\/\"><u><strong><font color=\"#200020\">abstracts <em>and<\/em> conclusions<\/font><\/strong><\/u><\/a>] for those who haven&#8217;t read them. <\/div>\n<ol>\n<li>\n<div align=\"justify\">The first is <u><font><strong><a href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/19947800\" target=\"_blank\"><font color=\"#200020\">Last-Observation-Carried-Forward Imputation Method in Clinical Efficacy Trials: Review of 352 Antidepressant Studies<\/font><\/a><\/strong><\/font><\/u>. The authors did a literature search of the Clinical Trials for the Antidepressants between 1965 and 2004. They demonstrate how these studies increasingly drifted towards using the LOCF methods without including the ancillary information that allow an accurate assessment of the results. They make it clear what information needs to be included in the articles [and it never is]. These authors already knew what it&#8217;s taken me months and a beach trip to figure out &#8211; in their current form, the Clinical Trials articles are deliberately opaque.<\/div>\n<\/li>\n<li>\n<div align=\"justify\">The next article is <u><a target=\"_blank\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed?term=Why%20Olanzapine%20Beats%20Risperidone%2C\"><strong><font color=\"#200020\">Why   Olanzapine Beats Risperidone, Risperidone Beats Quetiapine, and   Quetiapine Beats Olanzapine: An Exploratory Analysis of Head-to-Head   Comparison Studies of Second-Generation Antipsychotics<\/font><\/strong><\/a><\/u>. In this article, the authors look at the industry supported head-to-head Clinical Trials and point out the obvious &#8211; the sponsor&#8217;s drug wins. But they go further and demonstrate the subtle ways that these studies are manipulated by the study design and writing methods to move the outcomes in the desired directions. It&#8217;s a must-read in full.<\/div>\n<\/li>\n<li>\n<div align=\"justify\">This final article is recent, sent to me by PharmaGossip, <u><a target=\"_blank\" href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/21327723\"><strong><font color=\"#200020\">Those Who Have the Gold Make the Evidence: How the Pharmaceutical Industry Biases the Outcomes of Clinical Trials of Medications<\/font><\/strong><\/a><\/u>. While this is the most cynical article of the bunch, it&#8217;s the closest to the emerging truth:<\/div>\n<ul>\n<div>&quot;<strong><font color=\"#200020\">We can reasonably ask that pharmaceutical companies not break the law in their pursuit of profits but anything beyond that is not realistic. There is no evidence that any measures that have been taken so far have stopped the biasing of clinical research and it&rsquo;s not clear that they have even slowed down the process. What will be needed to curb and ultimately stop the bias is a paradigm change in the relationship between pharmaceutical companies and the conduct and reporting of clinical trials.<\/font><\/strong>&quot;<\/div>\n<\/ul>\n<\/li>\n<\/ol>\n<div align=\"justify\">I think the <strong><font color=\"#200020\">CRO<\/font><\/strong>s are the intelligence behind this whole game, <em>the ghost in the machine<\/em>, but since I&#8217;ve only known they exist for a month or so, I&#8217;ve got some catching up to do. It might require another beach trip&#8230; <\/div>\n","protected":false},"excerpt":{"rendered":"<p>Occasionally, not thinking about things allows some room in the mind to think. I&#8217;ve spent weeks reading and rereading these Seroquel Clinical Trials and remained befuddled. The graphs all look almost the same. I even named them after the Companies that make them [Clinical Research Organizations]: The most obvious problem is that the Placebo subjects [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[2],"tags":[],"class_list":["post-5220","post","type-post","status-publish","format-standard","hentry","category-politics"],"_links":{"self":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/5220","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/comments?post=5220"}],"version-history":[{"count":67,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/5220\/revisions"}],"predecessor-version":[{"id":5293,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/5220\/revisions\/5293"}],"wp:attachment":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/media?parent=5220"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/categories?post=5220"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/tags?post=5220"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}