What is the clinical significance?<\/font><\/strong><\/p>\nI asked COMPare to elaborate on clinical impact. Beyond the technical violations of best practices in trial reporting, did the team discover any misreporting that concerned them from a clinical perspective? For COMPare, this question was out of scope. “On clinical impact, we’ve deliberately not examined that. We set out to assess whether outcomes are correctly reported in journals listed as endorsing CONSORT,” the team told me, referring to the well established guidelines for proper reporting of randomised trials. “We deliberately steered away from any value judgments on why someone reported something other than their prespecified outcomes, because we needed our letters to be factual, unambiguous, and all comparable between trialists and journals.” <\/p>\n
But clinical and value judgments were central to journal editors’ defence. “COMPare objects to the format in which the data are communicated, but COMPare is silent about whether they dispute the key clinical message of the article,” NEJM<\/em> editor Jeffrey Drazen said through a spokesperson. Annals<\/em> echoed this sentiment: “COMPare’s methodology suggests a lack of recognition of the importance of clinical judgment when assessing pre-stated outcomes”…<\/p>\nAdhering to the original plan<\/font><\/strong><\/p>\nThe challenge of establishing outcome “switching” begins with determining trialists’ prespecified outcomes. But which document—protocol, statistical analysis plan, registry entry, or some combination of the above—details trialists’ true intentions? COMPare’s methods prioritise protocols over registry entries, a practice that troubles Elizabeth Loder, head of research at The BMJ<\/em>, which requires reporting according to the registry entry. “I see a worrying trend away from trial registries to protocols … People seem to think that as long as you’ve registered the trial and given notice of its existence, the details don’t matter so much.” <\/p>\nAnnals<\/em> said that sometimes editors are faced with “a choice between an incomplete or non-updated registry record, with a reliable date stamp, and a more detailed and updated protocol document.” Such situations deeply trouble Deborah Zarin, a long time advocate of trial registration and director of ClinicalTrials.gov, who believes that trial registration is the foundation of the trial reporting system… <\/p>\nEven with trialists’ last testament established, complexity remains. Having herself conducted similar analyses that compare outcomes reported across different sources, Zarin noted that a key challenge is disagreement in the community over how detailed outcome prespecification must be. “From my perspective, prespecification sets the foundation for a statistical analysis; if there is no firmly prespecified outcome measure, then it’s unclear to me what any reported P value means,” she said, noting that ClinicalTrials.gov now requires investigators to delineate “the specific metric and the time point[s]” for all outcome measures. <\/p>\n
Yet some trialists do not sufficiently define outcomes until after the data are collected [but not unmasked] and they finalise the statistical analysis plan, a situation the FDA apparently accepts. The FDA told
The BMJ<\/em> that it “does not require the statistical analysis plan to be in place prior to initiation of the trial.” Annals<\/em> stated that “prespecification is certainly important, but holding blindly to this prespecification may not be good science. Sometimes it becomes clear to informed clinicians and scientists who understand the area that what was prespecified is not going to provide patients and clinicians with the information they need”…<\/div>\n<\/blockquote>\nIf you have made it this far down the page, I’d like you to notice something. We started with Goldacre et al’s data which is quite clear. In the overwhelming majority of cases, the prespecified Outcome Variables have been changed. But at this point, you’re confused – probably would have difficulty even saying succinctly what the exact topic is. Peter Doshi, a clear thinker, seems to be going from person to person who are saying things that add to the confusion rather than clarify. It’s not his fault. Or yours. Or mine. That’s what happens with this topic. The fog rolls in.<\/font> That is probably the most important question here,<\/font> "<\/font><\/strong>Why do these discussions of clinical trial analysis always get lost in the fog and seem to fade away?<\/font><\/strong>"<\/font><\/strong><\/em><\/div>\nThat question has an answer. As long as the confusion remains, one can do whatever makes the outcome look most like you want it to look. And you can justify your choice with the kind of discussion that has Peter’s eyes crossing, continuing to play a frenzied match of ping pong until the end of days. So if you keep from being definite about the outcomes, you can pick the one that fits your fancy, and if challenged – bring on the fog<\/em>.<\/font> It’s confusing on purpose…<\/font><\/em><\/strong><\/p>\nto be continued…<\/font><\/em><\/div>\n","protected":false},"excerpt":{"rendered":"Is this trial misreported? Truth seeking in the burgeoning age of trial transparency by Peter Doshi, associate editor British Medical Journal. 2016 355:i5543 Peter Doshi has, as usual, done a great job of clearly laying out an issue and exploring all of its facets. It’s certainly one I haven’t been able to stop talking about […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[5],"tags":[],"class_list":["post-68860","post","type-post","status-publish","format-standard","hentry","category-opinion"],"_links":{"self":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/68860","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/comments?post=68860"}],"version-history":[{"count":23,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/68860\/revisions"}],"predecessor-version":[{"id":68883,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/posts\/68860\/revisions\/68883"}],"wp:attachment":[{"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/media?parent=68860"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/categories?post=68860"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/1boringoldman.com\/index.php\/wp-json\/wp\/v2\/tags?post=68860"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}