Whatever you’re reading right now [including this blog], you might just put a bookmark in it and read this paper. Besides it being written by luminaries [see scathing indictments… and the hope diamond…], it’s an encyclopedic proposal that deserves everyone’s attention:
by Marcus R. Munafo, Brian A. Nosek, Dorothy V. M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware, and John P. A. loannidisNature: Human Behavior. Published 10 January 2017. Open.
Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.
[abbreviated and reformatted from the paper]
From my perspective, there’s nothing more important in Medicine right now than reclaiming the academic medical literature from its captivity by the paramedical industries and others who are called stakeholders. But the problem in academic science is bigger than just Medicine. In the other fields, it goes under the name, The Reproducibility Crisis.
This paper is too important to whip off a blog post. So I’m going to let it sit for a bit before commenting, and picking out the specific recommendations that have to do with my corner of the world – Randomized Clinical Trials of medications – specifically the medications used in psychiatry.
Full of sound and fury, signifying nothing…
Cue the end of “Escape from LA” when Snake Plissken wipes out electricity from the world, and then ends with “welcome to the human race”…
You really want to see things honestly and sincerely corrected, well, it’s time to start from scratch, the corruption is so pervasive, you have to sacrifice the process and start content from the beginning.
Cue how we found Thorazine and Lithium as a start?
But, I hope your next blog post is both illuminating and therapeutic, for you and the readers at large.
Joel H
Thank you once again for all of your work on this. I have found the amount of information exhausting to read and to contemplate.
I remember the professional journals coming through the mail at my childhood house. I was especially enamored of JAMA because of it’s front cover art work.
I would try to read them as the years went by and always came up short. Of course, I was the preschool child who throw a fit when I discovered that wearing eye glasses was not the path to reading ability.
Eventually, during graduate school, I was able both to read and understand. My hard very hard earned course work in stats and research methodology was extremely helpful. I was able to get into a Phd program with my research knowledge from the graduate level.
I enjoyed the journals and also got my own subscription to HCP. However, as the years went on I found the articles and studies unhelpful. I think I am beginning to understand why. I was never sue if it was my lack of knowledge or the studies themselves.
Later on, on the advice of an ID attending, I researched a family relative’s medical condition. It was helpful to have the background and some of the research articles and studies did make sense and I found it very helpful.
I was never anti -science and always had the greatest respect for many of my professional. colleagues. I also enjoyed working with the Phd/MD students
.
The change you have documented is what I experienced but did not have the background or understanding to comprehend. Thank you.
The article itself argues for better scientists and doesn’t dwell on appearance of conflict of interest or private funding.
Unless the reproducibility crisis is greater in pharma sponsored research than government sponsored research (I’m open to that argument if there is data supporting it), then we ought to focus on lousy medical research as being the result of lousy academics in too many research institutions publishing in too many marginal journals. I can think of research in child development as a prime example of a field tainted by the unconscious biases and psychological wounds of the researchers.
My early impression from my observations was that zealotry and ideology among the principals were far more serious obstacles to good research than funding sources.
The meta study that would be key would be reproducibility of psychological vs. psychiatric research. I suspect both are very bad but that the psychological is slightly worse. And I’m a big supporter of good psychological research, though I think most of the great experiments were done a half century ago.
I found this article (discussed by 1boringoldman previously) of relevance:
http://post.queensu.ca/~sismondo/ghosts.pdf
Especially the paragraph at the top of p193 regarding visible experts. I would argue that the needs of Pharma corporate science disproportionately favored visible experts in academia in the worst sense of the word.
Not sure I could point to objective data backing up that subjective impression.
But fundamentally my focus is more on some of our academics, particularly the thought leader class, than Pharma.
I would suggest that the hollowing out of what should be “high quality” work and journals has been more of a problem for clinicians than the proliferation of crappy Frontiers N! journals.
Perhaps they are all problems.
I would not agree that Pharma has been a non-factor in the nurturing of academic emperors with little clothing. But I certainly wouldn’t argue for any one thing being the only factor, or all bad. Pharma included.
Almost all the late 20th century research on recovered memories was flat out horrible, biased, corrupt or invalid without any financial assistance from Pfizer. It never occurred to the researchers (or, evidently, the publishers) that the power of suggestion (https://en.wikipedia.org/wiki/Kee_MacFarlane) or a reverse Hawthorne effect would influence the results, although any psychological researcher or anyone who has taken freshman Psych 101 should know that.
When you compare that research to studies like Asch conformity in the 50s, you see a real decline in intellectual rigor and a drift toward emotional fanaticism.
I certainly wouldn’t argue for any one thing being the only factor, or all bad. Pharma included.