In May 2021, Cochrane published a network meta-analysis of depression pills for children and adolescents with depression. As the first author is Sarah Hetrick, co-ordinating editor in the Cochrane Mental Disorders group that published the review, one would expect the review to be better than average for Cochrane reviews in depression.

The abstract was seriously misleading

By reading the abstract, I realised that it would be a waste of time to read all the 225 pages in the review. The abstract illustrates a remarkable lack of common sense and overreliance on what the drug companies have chosen to publish, even though we know that such information is unreliable.

Image by Jernej Furman on Flickr.

“Major depressive disorders have a significant impact on children and adolescents, including on educational and vocational outcomes, interpersonal relationships, and physical and mental health and well-being. There is an association between major depressive disorder and suicidal ideation, suicide attempts, and suicide. Antidepressant medication is used in moderate to severe depression; there is now a range of newer generations of these medications.”

This is the whole background section in the abstract. The focus is on the disorder and the need to treat it with pills, warning that there is an association between depression and suicidality. It would have been more appropriate to mention that there is an “association” (actually more than this, as it is a causal relationship) between the use of depression pills and suicidal ideation, suicide attempts, and suicide.

“Newer generation”, “second generation”, and “third generation” of drugs is industry jargon which aims at giving readers the impression that they are better than older drugs. Such marketing terms that have no biochemical relevance should not be used in Cochrane reviews.

“…Proportions of suicide-related outcomes were low for most included studies and 95% confidence intervals were wide for all comparisons. The evidence is very uncertain about the effects of mirtazapine (OR 0.50, 95% CI 0.03, 8.04), duloxetine (OR 1.15, 95% CI 0.72, 1.82), vilazodone (OR 1.01, 95% CI 0.68, 1.48), desvenlafaxine (OR 0.94, 95% CI 0.59, 1.52), citalopram (OR 1.72, 95% CI 0.76, 3.87) or vortioxetine (OR 1.58, 95% CI 0.29, 8.60) on suicide-related outcomes compared with placebo.”

This information is highly misleading. The evidence is not “very uncertain.” We have known for almost 20 years that these drugs, as a class, increase the suicide risk in children and adolescents, which is why drug regulators warn about using them. The Cochrane authors miss the forest by looking at one tree at a time, and it gets worse:

“There is low certainty evidence that escitalopram may “at least slightly” reduce odds of suicide-related outcomes compared with placebo (OR 0.89, 95% CI 0.43, 1.84).”

Firstly, a drug cannot be better than itself. The active ingredient in escitalopram is the same as in citalopram, which is a stereoisomer. Stereoisomers consist of two halves, which are mirror images of each other, but only one of them is active. When the patent runs out, the company may patent the active half, a trick called evergreening, or “me-again.” Our patent laws are really weird since they allow this, which merely benefits the company for no societal gain. Second, none of the drugs reduce odds of suicide-related outcomes compared with placebo; they increase the odds. Third, the confidence interval goes from 0.43 to 1.84. This is not “low certainty evidence”, it is no evidence at all! (A confidence interval for an odds ratio that includes 1 is, by definition, not statistically significant.) At this point, I would like to know if any of the authors are on Lundbeck payroll, since that company sells both drugs.

“There is low certainty evidence that fluoxetine (OR 1.27, 95% CI 0.87, 1.86), paroxetine (OR 1.81, 95% CI 0.85, 3.86), sertraline (OR 3.03, 95% CI 0.60, 15.22), and venlafaxine (OR 13.84, 95% CI 1.79, 106.90) may “at least slightly” increase odds of suicide-related outcomes compared with placebo.”

This nonsense is outright dangerous. There is high certainty evidence that depression pills increase the risk of suicide, and “at least slightly” downgrades this harm to the extreme. The pills kill children and adolescents by driving them into suicide. And as they do not have clinically relevant effects, they should not be used at all.

“There is moderate certainty evidence that venlafaxine probably results in an “at least slightly” increased odds of suicide-related outcomes compared with desvenlafaxine (OR 0.07, 95% CI 0.01, 0.56).”

The problem with this nonsense is the same as with escitalopram and citalopram, as desvenlafaxine is also a “me again” product. Think about it. What is the likelihood that a drug can be better than itself, with a very small odds ratio and a confidence interval that goes from 0.01 to 0.56, far below unity? Extremely close to zero, but Cochrane did not bother but preferred to provide support to the industry’s cheating with the evidence.

Cipriani’s seriously misleading network meta-analysis in Lancet

This Cochrane review is just as disgraceful as Cipriani’s network meta-analysis of depression pills in adults, which was published in Lancet on 7 April 2018. The Cochrane authors should have learned from the devastating criticisms that were raised against this review, but they didn’t. In 2018, I published the article “Rewarding the companies that cheated the most in antidepressant trials,” in which I noted that Cipriani’s meta-analytic exercise was academic, with no clinical value, and that they drowned the many biases in the trials in statistics that were so complicated that it was impossible to know what all this led to.

In Cipriani’s case as well, the abstract was revealing. The authors claimed that in head-to-head trials, agomelatine, escitalopram, and vortioxetine were more effective than other antidepressants and that the same three drugs were also more tolerable than other antidepressants. This is highly unlikely to be true, and I therefore took a closer look at these three drugs and easily found out that it wasn’t true.

In 2019, Munkholm et al. reanalysed some of the data in Cipriani’s meta-analysis. They found that several methodological limitations were either unrecognised or underestimated; that the effect size was significantly higher in trials with a placebo run-in period and in published trials; and that the outcome data differed from the clinical study reports in 12 (63%) of the 19 trials they examined. When Munkholm et al.’s paper had been accepted for publication in BMJ Open, the editor wrote to Cipriani to ask him to respond. It is unbelievable that Cipriani did not find it necessary to defend his research.

The Cipriani network meta-analysis got colossal media attention, despite the fact that its estimate of the effect of depression pills was virtually the same as in previous meta-analyses—the pills were barely better than placebo. There were also highly critical letters in Lancet on 22 September 2018, which are listed below the PubMed record for Cipriani’s paper. Being a co-ordinating editor in the Cochrane Mental Disorders group, Hetrick has no excuse for not paying attention to the flaws in Cipriani’s review and for publishing another totally unreliable network meta-analysis of depression pills. This is really depressing, but I won’t take a pill for it.

Which data did the Cochrane authors include?

Despite the report’s 225 pages, it is not clear what Hetrick et al. did in their Cochrane review. I have access to clinical study reports from European drug regulators and I therefore know that Graham Emslie omitted two suicide attempts among 48 children on fluoxetine (there were none on placebo) from the publication of his trial in children and adolescents in 1997.

I tried to check if the two suicide attempts had been included in the Cochrane meta-analysis, but it proved impossible, as there are no data for suicide-related outcomes for individual drugs or studies (see page 59 in the review). The two suicide attempts are included in Lilly’s unpublished clinical study report X065,8 and I therefore searched on X065 in the Cochrane review but found the information confusing. On page 70, references to Emslie’s study are listed and the heading is this one:

“Emslie 1997 {published data only}

On the next page, there is this text:

“Medicines and Healthcare products Regulatory Agency (MHRA).
Fluoxetine Study 1 ID# X065. (accessed
20 June 2004).”

If Hetrick et al. only used published data, it would make little sense to mention Lilly’s internal report. On page 103 there are “Notes” for this study, which say:

“Additional data were sought and supplied by the authors. Data in the MA for child, adolescent and total
populations taken from paper publication and these additional data
Child and adolescent data from author. MHRA # X065
MHRA contacted for additional data some of which was provided”

This is gobbledegook. Some text seems to be missing, and I can make no sense out of it. I have a simple question: Are the two suicide attempts included, yes or no? But I get no answer.

Cochrane is beholden to industry

The above shows that Cochrane is beholden to industry and is too uncritical. On 23 April 2021, Professor Ken Stein, Director of the Evidence Synthesis Programme at the UK National Institute for Health Research, spoke at a webinar for about half an hour about the work in the UK Cochrane groups and their future funding. His criticism of Cochrane was much the same as mine when I was a Cochrane Governing Board member, and he emphasized that Cochrane authors should be iconoclastic, which we were when we founded the Cochrane Collaboration in 1993. Stein also spoke about the importance of scientific integrity and said that this was a point raised by people in the Collaboration to ensure that “garbage does not go into the reviews; otherwise, your reviews will be garbage.”

It is highly unusual for a top funder to say that the recipient of the funding must ensure that garbage does not go into the research. This suggests that Stein is aware of the statement by Cochrane editor and author Tom Jefferson in the article, “Cochrane – a sinking ship”: “If your review is made up of studies which are biased and in some cases are ghost written or the studies are cherry picked and you don’t take that into account in your review, then it’s garbage in and garbage out … with a nice little Cochrane logo on it.” Stein was present at the Cochrane colloquium in Edinburgh in September 2018 where I was expelled after a show trial because of my criticism of psychiatric drugs and he knew very well what had happened. Just after my expulsion, BMJ’s editor-in-chief, Fiona Godlee, wrote that Cochrane should be committed to holding industry and academia to account, and that my expulsion from Cochrane reflects “a deep seated difference of opinion about how close to industry is too close.”

Cochrane’s CEO Mark Wilson, who ensured I was expelled, had seen the “writing on the wall,” which Stein said had been present for 8 years, which is exactly the period when Wilson, a journalist, ruled the organisation and destroyed it. I have been unable to find a resignation letter or anything else that could elucidate the circumstances around Wilson’s sudden departure from Cochrane in the middle of a month, five days before Stein’s webinar.

Stein indicated that there would be a major cut in funding in 2022. I find it likely that this is the beginning of the end for Cochrane as we know it, as 21 Cochrane review groups out of 52 worldwide are based in the UK. This move is deserved. Cochrane has degenerated into a highly ineffective behemoth, a cook-book exercise with plenty of nonsense, gobbledegook, and a lack of common sense, filling hundreds of pages for a single review and yet lacking essential information. Not even a co-ordinating editor is able to lift herself up over this mess, but propagates it.

Wilson did not have much sense for science and was preoccupied with branding. He introduced the motto “Trusted evidence” for Cochrane reviews. When it comes to psychiatric drugs, the Cochrane motto should be “Distrust the so-called evidence.”


Mad in America hosts blogs by a diverse group of writers. These posts are designed to serve as a public forum for a discussion—broadly speaking—of psychiatry and its treatments. The opinions expressed are the writers’ own.

Source link


Please enter your comment!
Please enter your name here