What Me Worry? Result No Good, Change Outcome

There has been a recent movement to get some greater accountability in research these days. One of the things that has happened is the development of clinicaltrials.gov. That system involves logging on an registering your clinical trial and saying what it is you are actually hypothesizing your result will be. In the lingo the result is called the primary outcome, which is what you think the result will be.

To make this simple, let’s say that you think a drug will make your face turned blue. In this case the “primary outcome” would be “make your face turn turn blue.” So what if your face turned red? Maybe you would like that your drug actually “did” something and you decided that was better than nothing. So after the fact you *changed* the primary outcome and said that you actually had wanted the drug to turn your face red? Pretty dishonest, huh? Well it actually happens all the time in clinical research, especially for drug trials where pharma companies have billions riding on the outcomes of these trials. 

So lets turn to the evidence. Guys like the investigators in the Paxil 329 study, which I have written about previously here, were deciding that the primary outcome they were studying should change after they found that it didn’t turn out to be significant, and they should come up with something else. In the case of the 329 study there was a rather arrogant letter on the behalf of the lead author along the lines of “I know best and I changed the outcome because I wanted to and I am an expert and leader in my field.”

Well now a lot of investigators have piled on to registering their trials and some government and funding agencies are requiring it. That is why it is interesting that a recent study has actually looked at the relationship between the registry and what people actually reported in the literature. What they found is that 46% of trials were properly registered. However of those, 31% had changed their primary outcome. In those studies in which it could be evaluated they favored results that were statistically significant over those that were not in 83% of cases.

In other words, 1/3 are blatantly lying and cheating. And that is in the ones that have registered. What about the ones that never did or in era before clinicaltrials.gov? Can we trust the data collected by all those drug company trials? If not what does that mean