The Black Box Warning added to antidepressant labeling in 2004 by the FDA created something of a cottage industry attempting to either discredit it or  reverse it. Those efforts are well documented throughout this and many other blogs. And there have been multiple attempts to say that the net effect has been an increase in suicidality resulting from the fall in prescribing attributed to untreated depression in children and adolescents. This claim has been based on data from large databases from several sources.

This most recent article in the British Medical Journal [Changes in antidepressant use by young people and suicidal behavior after FDA warnings and media coverage: quasi-experimental study] relies primarily on the Mental Health Research Network [3], data from multiple large health plans [not for profit HMOs]. But there are other datasets involved in the analysis: PHARMetrics Patient Centric Database [1], BC Ministry of Health, and the Nationwide Inpatient Sample [2] – the latter two chosen because they include complete ICD-9 E-Codes [codes for external injury]. PHARMetrics is a commercial enterprise used for marketing intelligence [I’ve extracted the descriptions of these datasets from the referenced papers at the end of this post with links to their abstracts and full texts when available. I also added some charts from the reported ages.].

While I’m still grumbling about the Lu et al papers [3], my main agenda in this post is something else – the use of these large databases in research papers. The story begins with a problem – the commercial databases don’t have usable E-Codes, the ICD-9CM codes that directly address suicide attempts. In their Letter to the Editor [3], Lu et al prove that the database they are using doesn’t have these E-Codes with any consistency across sites. So they bring up Patrick et al [2] who did a study trying to deal with the missing E-Codes by locating other parameters that could be used as a proxy for suicide attempts. They tried a number of combinations and came up with an algorithm involving psychiatric diagnosis and injury to the lower arm or asphyxiation or overdose with psychotropic medications using two government databases that had required E-Codes as their gold standard.

"Patrick et al. developed and tested algorithms for identifying hospitalizations for deliberate self-harm in a population aged 10 years and over. This study used the US National Inpatient Sample data and data from British Columbia; both data sources had E-code completeness rates above 85 %. The gold standard for deliberate self-harm was defined as hospitalizations with a diagnosis of E950-958. Patrick et al. found that an algorithm combining diagnoses for psychiatric disorders [including depression] and injury/poisoning can produce a positive predictive value as high as 87.8% for identifying hospitalizations for deliberate self-harm [with specificity of 99.4% and sensitivity of 57.3%]. In the context of our longitudinal study on the impact of Food and Drug Administration warnings on antidepressant use and subsequent suicidality in youth, using Patrick’s algorithm may introduce ascertainment bias because rates of depression diagnosis declined subsequent substantially after the warnings."

Because previous studies showed that rates of depression diagnosis changed after the warnings and that outpatient claims are often incomplete for mental health conditions such as depression, to avoid introducing selection bias, we did not limit our cohorts to those with a coded diagnosis of depression.

Non-fatal poisoning by psychotropic drugs [predominantly tranquilizers] has a positive predictive value of 79.7% for suicide attempts [sensitivity was 38.3% and specificity was 99.3%], outperforming other types of injuries or poisonings.

Even though the intervals vary, just looking at the age ranges [at the end of this post], it’s apparent that these are different datasets. Further, they were created for different reasons, from different sources, and their ontogeny can’t be vetted, at least by me. All those things heighten the need to reproduce the findings of others in their own database, something not very hard to do [and obvious]. I would suggest that this study is uninterpretable, and shouldn’t have been published in a mainstream journal as is, but rather sent back suggesting resubmission when they’d done their necessary homework. My broader point is that population studies are hard enough to interpret when done thoroughly. Making the assumption that the results from any given dataset can be applied to any other dataset freely is untenable [also obvious]. A simple rule for such studies:

all databases are not created equal…