Effects of Zoloft on Childhood Anxiety Incredible, Indeed

A quote from a physician describing the results of study published this week in the New England Journal of Medicine on the effects of Zoloft (sertraline) or cognitive behavioral therapy (CBT) or a combination of the two on childhood anxiety disorders, describing the results of the study as “incredible”, lead me to take a look at the study myself to see if this claim was in fact, er, credible, or more of the same fare we have been dished out regarding the use of SSRIs in children, such as the infamous Study 329 of paroxetine (Paxil) in the treatment of depression in teenagers.

The current study looked at 488 children between the ages of 7 and 17 years who had the diagnosis of separation disorder, social phobia, and/or generalized anxiety disorder (GAD). They received 14 sessions of cognitive behavioral therapy, sertraline (up to 200 mg per day), a combination, or placebo for 12 weeks. The authors reported a score of “much improved” or greater on the Clinical Global Impression-Improvement Scale in 81% of kids treated with combination drug/therapy, 60% for CBT alone, 55% for sertraline, and 24% for placebo.

However, using a categorical outcome like “improvement” can be misleading. Take the example of the infamous Study 329 which pointed to the outcome of a 50% or greater improvement on the Hamilton Depression Scale as evidence for benefit of paroxetine in the treatment of childhood depression. The original primary outcome of the 329 Study was improvement in depression as measured by the Hamilton Depression Scale. The authors later “changed their minds” about what they should focus on, a fact that came out later. It is important to define a primary outcome in advance, otherwise there is a tendancy to fish around for a positive result, which may lead to something that is just a fluke being interpreted as due to something real.

In the sertraline/CBT study, the authors (as far as we know) had “improvement” as their primary outcome. However improvement can be misleading. Let’s focus on the sertraline treated group alone, since CBT has no side effects and I am fine with people using CBT, and since the authors did not report a statistically better outcome of the combination therapy compared to CBT alone (although the press releases trumpet, incorrectly, that the combination is better) Say the primary goal is to run a mile in 10 minutes (or whatever, I say as I sit on the couch). If out of 100 people running, people wearing green shirts do it, on average, in 9 minutes 45 seconds, and people wearing red shirts do it in 10 minutes 15 seconds, you could have a result where 60% of the green shirts make the goal versus 25% of the red shirts, which sounds like a big deal, even though there is only a 5% difference in their times.

So lets look at these studies. In the case of Study 329, 66% of kids treated with Paxil (paroxetine) were “much improved” or better as measured by the Clinical Global Impression Scale (CGI) [criteria used in the Zoloft study] versus 48% of those treated with placebo, which they reported as statistically significant. Not bad, you say, however they did not find a significant change in their primary outcome, and to report the study as positive is a violation of the rules of clinical trials, as pointed out in a subsequent letter to the editor. In fact, if you look at the actual data, the Ham D score went from a baseline of 19.0 in both groups, to 8.2 in the paroxetine group and 9.9 in the placebo group, a paltry 3% difference in a 56 point scale which was not reported as significantly different because it was not, well, different.

So now let’s turn to the “incredible” results of this week’s study of sertraline (Zoloft) in kids. Although there was a difference in “responders” based on much improved on the CGI of 60% versus 24% for placebo, when you look at the actual data, the Pediatric Anxiety Scale, a 30 point scale, went from 18.8 at baseline to 9.8 in the zoloft group, and from 19.6 to 12.6 in the placebo group, a difference of 9%, again, not reported as statistically significant because it was not, in fact, very different. In fact only CBT (not combination) was better than placebo on the anxiety scale. My clinical methodology experts tell me that a study is pretty weak if it only shows a positive outcome on a single categorical (yes/no) outcome and not on the continuous (multi item) scale. And the combination group had no comparison group. Just, here is your psychotherapy (66% get better), and now open up your mouth and let me give you a yummy blue pill that Mommy says is going to make you better (80% get better).

OK, dictionary time.

incredible Pronunciation [in-kred-uh-buhl] adjective

1. so extraordinary as to seem impossible; incredible speed.

2. not credible; hard to believe; unbelievable: The plot of the book is incredible.

[Origin: 1375-1425; late ME incredibilis]

Related forms incredibility, incredibleness, noun

incredibly, adverb

Incredible, indeed.

[originally posted on Oct 31 2008]

COMMENTS
Philip wrote on Furious Seasons on November 3, 2008

Philip wrote about this study on his post which triggered a string of comments including ‘Bremner sucks cuz he does research on Paxil’. Seriously though folks I have already gone over this in detail on my site here before (see also here and other sites, as well.

John Grohol wrote on Psych Central on November 3, 2008

Bremner’s off-the-cuff analysis wrongly suggests the current researchers had no specific outcome objectives

.
He also states that I did not focus on the combination CBT and zoloft group, which had an 80% response rate, which was statistically significant and in his opinion clinically significant. You can read the rest of his post here.

As I will write on his site:

I did not mean to imply that there was no specific outcome. By saying they used ‘much improved’ I understand that you could take it that way, but I very much know that much improved corresponds to a 2 on the CGI [very much improved is a 1, so subjects had either a 1 or 2 post treatment] which is a validated scale used throughout clinical trials, by myself and others. And I have no reason to doubt that it was chosen a priori as the primary outcome measure, although I guess we can wait for the lawyers and their experts to go digging through the emails and files as they always inevitably seem to do, to attempt to prove a change in primary outcome [these days with clinicaltrials.gov primary outcomes are registered in advance]. However, my criticism was of using the CGI as the primary outcome in isolation, and that what I consider the more relevant Pediatric Anxiety Scale did not show a significant change (time by treatment interaction) for zoloft OR combination compared to placebo. Also, that the combination group had no comparison (i.e. CBT plus placebo) and the combination group knew they were getting zoloft. Having categorical (yes/no, ie. CGI) and not continous (anxiety scale) outcomes makes it, not necessarily a negative trial, but not strong either, and certainly not “incredible” as described in the press.

Read More and comment (1+ comments) –