1 Boring Old Man

A level playing field

The Lancet Psychiatry. 2014 1[6]:403.

At The Lancet Psychiatry we aim to publish research that illuminates and changes clinical practice. Changing practice requires a high standard of evidence, although existing practice does not always have a solid scientific base. Management of mental health often seems intuitive, so many interventions have been developed and rolled out on the basis of good intentions rather than good science [eg, post-traumatic stress disorder counselling after natural disasters]. We do not deny the role of clinical expertise and the art of the individual psychiatrist, but we believe that studies aimed at altering the status quo should be rigorous in planning, execution, and communication.

The acknowledged gold standard in terms of research is the randomised controlled trial [RCT], a method that should be applied to all types of intervention, wherever possible. In psychiatry, there are also good pragmatic studies of intervention outcomes where RCTs are not feasible, the most informative being those that combine cross-sectional and longitudinal observations. Excellent work is done using registries, particularly where health-based registries can be linked with others, such as those held by the educational and justice systems. For each type of study, there are recognised guidelines that list what should be measured and reported [compiled by the EQUATOR network]. The best-known is the Consolidated Standards of Reporting Trials [CONSORT] statement. CONSORT is itself evidence-based, and for each item there is an explanation of key methodological issues and the importance of reporting that item. Many medical journals now include CONSORT in their instructions to authors, but the slow rate of progress can be seen by looking at the search results for any meta-analysis — too frequently, studies have to be excluded because insufficient information is available.

Some claim that psychiatry, especially psychotherapy, is not suited to such rigorous approaches. Psychotherapist and author Darian Leader has stated that “the criteria for the evaluation of therapies has moved to a very narrow view of evidence, based on the medical model of randomised-controlled trials … with a control group, and so on. You can’t do that with therapy, because the whole point of therapy involves the beliefs the person has initially about their treatment or therapeutic experience. So you can’t randomly assign someone to a therapist.”

The Lancet Psychiatry admires this focus on the individual, but believes a strong evidence base is both possible and necessary. In the interests of all those who entrust their lives and well being to mental health professionals, it is time to level the playing field. All interventions should be assessed to the same standards of evidence, from psychopharmacology and psychotherapies, to brain stimulation technologies and new approaches such as video games and apps. Trials should be registered in a publicly available database; the protocol should be available, and the methods and results reported must match the protocol to avoid publication bias. Most important, sufficient information should be provided to enable replication of the study. The basis of scientific research is validation and refutation. For compounds, this means chemical composition, formulation, and dosing schedules. For psychological therapies, it means details of how many sessions, the length of sessions, availability of a manual where a specific therapy was given, details about the therapist, and evaluation of differences between specific therapists and sites. For training interventions, it means access to the training material, and details of the trainers, sessions, etc. This approach also applies to models of care: if something has the potential to be useful, it must be replicable. Circumstances will differ and where people are involved there will always be considerable variation, but providing relevant detail will enable other researchers to test the data and perhaps to explain different outcomes. Where trials are not possible, data on outcomes — including those associated with general well being and function rather than specific symptoms — are necessary. One essential set of observations is that of adverse events. Again, there are people who claim that the different nature of medicine and psychotherapy means that, although it is relevant to collect adverse event data for the latter, to do so in the same manner for the former is the equivalent of comparing apples with oranges. However, The Lancet Psychiatry believes that if an intervention has the potential to confer benefit, it also has the potential to cause harm.

This rigour is needed not just to satisfy journal editors but to convince researchers, clinicians, patients, and ultimately governments. If we want mental health services to be free at point of access, we must demonstrate that they work.

Carrier reimbursed psychotherapy didn’t disappear. It was taken over by other mental health specialties with controlled frequency, duration, and negotiated rates of payment. A psychiatrist colleague recently suggested to me that a physician [like me] shouldn’t be a psychotherapist based on a different argument – economics – saying that it costs too much to train a physician and the physician psychotherapist would therefore not be cost effective [the latter part implied].

The acknowledged gold standard in terms of research is the randomised controlled trial [RCT], a method that should be applied to all types of intervention, wherever possible.

The most obvious objection is the extent to which RCTs can be corrupted, manipulated, jury-rigged, etc. The scientific misbehavior in the RCTs of psychiatric drugs is staggering. I could never have imagined anything like it could even happen. So I spend my days involved in trying to insure that the kind of perverse science we’ve seen in psychiatric and other drug research never happens again. At the moment, access to the raw data from trials with independent analysis seems the best approach to what has gone before.

But even without the corruption, there’s more to be said about RCTs. I think they are essential as a beginning take on cataloging the adverse effects of drugs [psychiatric and otherwise], but only a beginning. Many adverse effects come with chronic use, so ongoing reporting is an essential ingredient in any accurate understanding of drug toxicity [for example, David Healy’s Rxisk site].

One might think that cleaning up  the corruption, the distortion, the exaggeration might make RCTs the preferred standard in efficacy studies as well, but I’m not sure that’s totally right either. RCTs can be too sensitive – detecting small effects that are not clinically relevant even if they are statistically significant. They’re limited by their time frame, the outcome instruments, the subject recruitment and evaluation processes, and then there’s the placebo effect. The results are often hard to replicate, thus the increasing reliance on meta-analysis of multiple studies in evaluating overall efficacy.