the frenzied match of ping pong: step one…

Posted on Saturday 29 October 2016

So do I think that editors and researchers sit around thinking, "We’ve got to keep them from nailing down the process to insure that there’s some official copy of the a priori Primary and Secondary Outcome variables publicly available"? I kind of doubt it. I expect there are some people inside the walls of Pharma whose understanding of this is more intentional, who consciously know that if they don’t keep this question always open and unresolved, they will lose an extremely valuable tool to market their drugs. But editors? I expect they think the reformers are just being too picky or are anti-pharmacology types or something like that. Maybe they feel like the lady outside the gala whose mink coat has just been sprayed with red paint by some unwashed hippie activist. Or maybe they don’t buy the scientific explanation for preregistration [see de Groot]. I doubt that it occurs to them that there was some savvy ghost·organizer in the background carefully orchestrating what Outcomes got presented in the article they published. But the net result is the same – the issue of the a priori Primary and Secondary Outcome variables just never gets resolved…

So here we are with the powerful publication editors, an astute investigative editor [Peter Doshi], and a preeminent clinical trial researcher [Ben Goldacre] engaged in a tête-à-tête about Clinical Trial preregistration, a priori variables, and "Outcome Switching" [that’s occurring after the fact, disconnected from the specific trial reports]. The clinicians who read all of those trials in their journals likely have no idea that this debate is taking place, and even more to the point, they didn’t and don’t know that the paper they read had the "Outcome Switched" [a selective reporting that may well have painted a picture of the results through rose colored glasses]. The discussion in Is this trial misreported? Truth seeking in the burgeoning age of trial transparency by the editors is focused on COMPare’s methodology, whether the prespecifieded Outcome Variables are the best choice, or whether Goldacre’s team made an "accurate, complete and nuanced evaluation of the published trial reports."

We have to wonder "Why are they talking about that?" Only the BMJ editor seems to have responded to the cogent question, "Did you know you were publishing an article where the Outcome Parameters had been switched?" The editors of the Annals of Internal Medicine and the New England Journal of Medicine either denied the facts or said that it didn’t make any difference – essentially ignoring the question. And Doshi’s article doesn’t address the Sponsors who submitted the articles or ask them the pertinent questions: "Why did you switch the Outcomes?" and "Why didn’t you tell us that you switched the Outcomes?"

the first step

Ben Goldacre’s COMPare team and the BMJ‘s Peter Doshi have done yeoman’s service here. First, the COMPare group made it clear that "Outcome Switching" is not just a thing of the past [as in Paxil Study 329, CIT-MD-18, Paxil Study 352, etc]. It’s happening right now, and frequently. And by querying the journal editors, Doshi got the same kind of defensive or attacking response as years ago.
“COMPare objects to the format in which the data are communicated, but COMPare is silent about whether they dispute the key clinical message of the article,” NEJM editor Jeffrey Drazen said through a spokesperson.
Annals echoed this sentiment: “COMPare’s methodology suggests a lack of recognition of the importance of clinical judgment when assessing pre-stated outcomes.”
That’s a disappointing response to say the least. The COMPare team didn’t even call it "Outcome Switching" if they said they were doing it and why in the article. They only catalogued instances where they changed the outcomes and didn’t tell us they were doing it. That’s a point the editors don’t seem to quite grasp. If either the authors or the editors were using clinical judgement to make their decisions, that’s their prerogative. But they need to say that. When I first read the COMPare study, I was impressed that they could even locate the a priori declarations. Frequently, I’ve either had a hard time finding them or actually failed in the endeavor. From the Compare FAQ:
"Our gold standard for finding pre-specified outcomes is a trial protocol that pre-dates trial commencement, as this is where CONSORT states outcomes should be pre-specified. However this is often not available, in which case, as a second best, we get the pre-specified outcomes from the trial registry entry that pre-dates the trial commencement. Where the registry entry has been modified since the trial began, we access the archived versions, and take the pre-specified outcomes from the last registry entry before the trial began."
So I guess I would say that the first step in evaluating an RCT is to see if the outcomes reported are the same ones that were prespecified in the a priori protocol. That’s the standard. If they’ve been changed, and the change isn’t addressed anywhere in the published article, you’re likely not reading a scientific document. You’re reading an advertisement…
    Peter C. Dwyer, LCSW-C
    October 29, 2016 | 11:15 PM

    “So do I think that editors and researchers sit around thinking, “We’ve got to keep them from nailing down the process to insure that there’s some official copy of the a priori Primary and Secondary Outcome variables publicly available”? ”

    I think it’s entirely likely that some, but not all, editors DO sit around thinking exactly that. But even if I’m wrong about that, here’s the real point:

    ‘It is difficult to get a man to understand something, when his salary depends on his not understanding it.” Upton Sinclair

    Journal editors know their journals depend on 40 or so pages of PhARMA ads per issue; they know of PhARMA’s grants to researchers and medical schools; ditto that big shot KOL’s, consultants and heavy hitters throughout psychiatry get big bucks from PhARMA; that modern medicine is based on a healthy dose of charade clinical trials being published in “peer reviewed journals”; that the APA convention couldn’t exist without PhARMA funding; that PhARMA funding largely sustains Astroturf organizations like CHADD and NAMI; and that to admit the real influence of any or all of the above would draw fierce push back from peers, professional organizations and PhARMA.

    So they don’t need to be Snidely Whiplash conspirators to stonewall, offer lame rationalizations, not “see” what is clear to any disinterested person. They just need to have their salaries, positions, professional prestige and “guild interest” (as Whitaker calls it) dependent on not understanding what Dr. Nardo and others clearly demonstrate here.

    These are smart people, doing what smart people do when the foundations of their salaries (or beliefs) are exposed. They search for rationalizations, and their intelligence allows them to put passable lipstick on their pig. But it’s still a pig.

Sorry, the comment form is closed at this time.