{"id":70161,"date":"2017-01-24T11:44:24","date_gmt":"2017-01-24T16:44:24","guid":{"rendered":"http:\/\/1boringoldman.com\/?p=70161"},"modified":"2017-01-24T13:09:29","modified_gmt":"2017-01-24T18:09:29","slug":"the-basic-efficacy-table-ii","status":"publish","type":"post","link":"https:\/\/1boringoldman.com\/index.php\/2017\/01\/24\/the-basic-efficacy-table-ii\/","title":{"rendered":"the basic efficacy table II\u2026"},"content":{"rendered":"\n

When Dr. Bernard Carroll comments here, he often uses the term "hand waving<\/font>" when describing some of the tricky maneuvers used in the clinical trial reports to smooth over shaky logic or rationalize absurdities. It’s a great term, I think originating in the world of stage magicians who use exaggerated gesticulations to distract your attention. My wife’s a figure skating fan, and there it’s called "hand dancing<\/font>" – dramatic arm and hand gestures to cover up sloppy skating. After reading so many jury-rigged clinical trial reports, <\/a>I’ve almost come to see the whole narrative as organized around a verbal version of these attempts at artifice, and find myself jotting down the essential pieces in a hastily sketched table on the back of a nearby envelope or piece of scrap-paper. So the basic efficacy table<\/font><\/strong> isn’t just a concept or a proposal. It’s an outgrowth of my experience. I guess the formula goes:<\/p>\n

article narrative – bullshit = basic efficacy table<\/font><\/strong><\/em><\/p>\n

One thing the ghost-writers seem to count on is that most doctors look at the abstract, scan the graphs and tables, and move on. I used to see that as virtue – getting through so much material on a regular basis. In my doctor youth, I could do that. But no longer. If a Clinical Trial report or a review article is there to be read, it’s there to be read closely, pencil and envelope back at the ready. At least that’s true of the industry funded clinical trials of psychiatric drugs that I find myself reading these days.<\/p>\n

<\/p>\n

When I drew this diagram, it’s not how things are. It’s how I wish they would be. Step one is the approval of the study Protocol <\/font>by the Institutional Review Board<\/font>. At that point, by my reckoning, the trial should be registered [on ClinicalTrials.gov<\/font>]. There’s no reason at all that the Protocol couldn’t be published at that point. It has been written down for the IRB. Why not make it a part of the registration process. That would mean that a bona fide copy of the a priori declarations<\/font><\/em> would be available from the outset.<\/p>\n

This Fall, Dr. Carroll, Dr. John Noble, and I had been working on Dr. Carroll’s proposal<\/a> built around the scheme in that graphic when two things happened. The NIH\/FDA\/ClinicalTrials.gov<\/font><\/em> issued their own new plan [what to do? the final rule?…<\/a>], and then there was a Presidential election. The latter has had me off my game since. But the fog is clearing a bit, so I want to get back to the former. This is from the summary table in Director Zarin’s paper outlining the changes:<\/div>\n
\n
adapted from Table 1A<\/strong> in<\/font><\/em> <\/p>\n
Trial Reporting in ClinicalTrials.gov — The Final Rule<\/a><\/div>\n
by Deborah A. Zarin, Tony Tse, Rebecca J. Williams, and Sarah Carr<\/div>\n
New England Journal of Medicine<\/font><\/strong>. 2016 375[20]:1998-2004. <\/div>\n
[full text on-line<\/a>]<\/div>\n<\/div>\n<\/blockquote>\n\n\n\n\n\n\n\n\n\n\n
Registration<\/font><\/strong><\/td>\n<\/tr>\n
When does information need to be submitted to or posted on ClinicalTrials.gov?<\/font><\/sup><\/strong><\/td>\n<\/tr>\n
 <\/sup><\/td>\nSubmission<\/u><\/font>: Within 21 days after enrollment of the first trial participant<\/font><\/strong><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nPosting<\/font><\/u>:<\/strong> Generally, within 30 days after submission.<\/font><\/strong> For ACTs of unapproved or uncleared devices, no earlier than FDA approval or clearance and not later than 30 days after FDA approval or clearance (i.e., “delayed posting”), unless a responsible party authorizes posting of submitted information prior to FDA approval or clearance<\/em><\/sup><\/td>\n<\/tr>\n
What information?<\/font><\/strong><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nDescriptive information about the trial<\/u><\/font>:<\/strong> e.g., brief title, study design, primary outcome measure information,<\/font><\/strong> studies an FDA-regulated device product, device product not approved or cleared by the FDA, post prior to FDA approval or clearance, and study completion date<\/em><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nRecruitment information<\/u><\/font>:<\/strong> e.g., eligibility criteria, overall recruitment status,<\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nLocation and contact information<\/font><\/u>:<\/strong> e.g., name of sponsor, facility information<\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nAdministrative data<\/font><\/u>:<\/strong> e.g., secondary ID, human-subjects protection review board status<\/sup><\/td>\n<\/tr>\n<\/table>\n

<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Results information reporting<\/font><\/strong><\/td>\n<\/tr>\n
When does information need to be submitted to or posted on ClinicalTrials.gov?<\/font><\/sup><\/strong><\/td>\n<\/tr>\n
 <\/sup><\/td>\nSubmission<\/u><\/font>:<\/strong><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\n <\/sup><\/td>\nStandard deadline: <\/font><\/strong><\/u>Within 12 months after the date of final data collection for the prespecified primary outcome measures (primary completion date)<\/font><\/strong><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\n <\/sup><\/td>\nDelayed submission with certification:<\/font><\/strong><\/u> May be delayed for up to 2 additional years (i.e., up to 3 years total after the primary completion date) for trials certified to be undergoing commercial product development for initial FDA marketing approval or clearance or approval or clearance for a new use<\/font><\/strong><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\n <\/sup><\/td>\nS<\/font><\/strong>u<\/u>bmitting partial results:<\/font><\/strong><\/u> Deadlines are established for submitting results information for a secondary outcome measure or additional adverse information that has not been collected by the primary completion date<\/sup><\/em><\/td>\n<\/tr>\n
 <\/sup><\/td>\n <\/sup><\/td>\nExtension request:<\/font><\/strong><\/u> After receiving and reviewing requests, NIH may extend deadlines for “good cause”<\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nPosting<\/font><\/u>:<\/strong> Within 30 days after submission<\/font><\/strong><\/sup><\/td>\n<\/tr>\n
What information?<\/font><\/strong><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nParticipant flow<\/u><\/font>:<\/strong> Information about the progress of participants through the trial by treatment group, including the number who started and completed the trial<\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nDemographic and baseline characteristics<\/u><\/font>:<\/strong> Demographic and baseline characteristics collected by treatment group or comparison group and for the entire population of participants in the trial, including age, sex and gender, race or ethnicity, and other measures that were assessed at baseline and are used in the analysis of the primary outcome measures<\/em><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nOutcomes and statistical analyses<\/font><\/u>:<\/strong> Outcomes and statistical analyses for each primary and secondary outcome measure by treatment group or comparison group, including results of scientifically appropriate statistical analyses performed on these outcomes, if any.<\/font><\/strong><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nAdverse event information:<\/font><\/u><\/strong> Tables of all anticipated and unanticipated serious adverse events and other adverse events that exceed a 5% frequency threshold within any group, including time frame (or specific period over which adverse event information was collected), adverse-event reporting description (if the adverse-event information collected in the clinical trial is collected on the basis of a different definition of adverse event or serious adverse event from that used in the final rule), collection approach (used for adverse events during the study: systematic or nonsystematic), table with the number and frequency of deaths due to any cause by treatment group or comparison group<\/em><\/sup><\/td>\n<\/tr>\n
 <\/sup><\/td>\nProtocol and statistical analysis<\/font><\/u>:<\/strong> Protocol and statistical analysis plan to be submitted at time of results information reporting (may optionally be submitted earlier)<\/font><\/strong><\/sup><\/em><\/td>\n<\/tr>\n
 <\/sup><\/td>\nAdministrative data<\/font><\/u>:<\/strong> Administrative information, including a point of contact to obtain more information about the posted summary results information<\/sup><\/td>\n<\/tr>\n<\/table>\n

First off, anything they do is a step forward. They’ve had the machinery available for two decades, and have done little with it. So they’re finally requiring registration for all the studies, and they pledge to keep up with it. An excellent start. <\/p>\n

Initial Registration<\/font><\/strong><\/em>:  They say that they want the submission of the trial registration within three weeks of the initial subject’s enrollment and posting on-line within the 30 days after submission. Of course I’d prefer our "before the study starts" timing, but within the first two months will do. The point is to get it registered before they can look at the results and modify the Protocol – and two months is early enough for me. As for what’s to be posted, they don’t require posting the whole Protocol. That’s a disappointment. I’d prefer anchoring the outcome parameter at the beginning. But at least they do require declaration of the Primary Outcome Variables with registration.<\/font><\/em><\/p>\n

Posting the Results<\/font><\/strong><\/em>:  This has traditionally been the most ignored requirement. They say: "Outcomes and statistical analyses for each primary and secondary outcome measure by treatment group or comparison group, including results of scientifically appropriate statistical analyses performed on these outcomes, if any<\/font><\/em>" and add in "Protocol and statistical analysis plan to be submitted at time of results information reporting (may optionally be submitted earlier).<\/font><\/em>" And I say A+!<\/font><\/em><\/strong> With that information, I could fill out my entire basic efficacy table<\/font><\/strong>, The only thing they left out was the Effect Size and there would be ample information to do that calculation.<\/p>\n

And for timing on the Results? I’d have to say "barely passing," if that<\/font><\/em><\/strong>. "Within 12 months after the date of final data collection for the prespecified primary outcome measures (primary completion date)<\/font><\/em>" and "May be delayed for up to 2 additional years (i.e., up to 3 years total after the primary completion date) for trials certified to be undergoing commercial product development for initial FDA marketing approval or clearance or approval or clearance for a new use.<\/font><\/em>" That’s a disappointment, and I can’t see any reason for it. The results are just what they are – they’re the results of the prespecified variables analyzed in the prespecified way. Who needs time for that? But I’ll have to admit that if they were to actually to follow these standards, the improvement would still be dramatic, probably satisfy most of us. With new drugs or new indications, they’d still be early in the drug’s patent life. <\/p>\n

The loss here for me has to do with the publication of the Clinical Trial in a Peer-Reviewed Journal<\/font><\/em>. I think the editors<\/font>, peer-reviewers<\/font>, and those of us who read these articles<\/font> have the right to know …<\/div>\n
    <\/p>\n
  • \n
    the a priori<\/font><\/em> declared primary and secondary outcome variables <\/div>\n<\/li>\n
  • \n
    the prospectively defined statistical analysis plan <\/div>\n<\/li>\n
  • \n
    the values, variance, and effect size for those specific parameters <\/div>\n<\/li>\n

    <\/span><\/ul>\n

    … at the time we read the article. These postponed postings may well allow publication in journals prior to the filled out Results Database on ClinicalTrials.gov. So there it is as well as I know how to present it. What do you think?<\/div>\n","protected":false},"excerpt":{"rendered":"

    When Dr. Bernard Carroll comments here, he often uses the term "hand waving" when describing some of the tricky maneuvers used in the clinical trial reports to smooth over shaky logic or rationalize absurdities. It’s a great term, I think originating in the world of stage magicians who use exaggerated gesticulations to distract your attention. […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[5],"tags":[],"class_list":["post-70161","post","type-post","status-publish","format-standard","hentry","category-opinion"],"_links":{"self":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/70161","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/comments?post=70161"}],"version-history":[{"count":26,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/70161\/revisions"}],"predecessor-version":[{"id":70210,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/70161\/revisions\/70210"}],"wp:attachment":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/media?parent=70161"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/categories?post=70161"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/tags?post=70161"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}