Searching for truth in published research

One of the mysteries of the scientific method to us non-scientists is how an "oops, we got it all wrong, folks" realization is arrived at. Think of it as a screw up's eureka moment.

It often results in a paper or papers being retracted, but traditionally with little deep explanation of why the originals were deemed fundamentally wrong. Until now.

The website Retraction Watch is run by Ivan Oransky, the executive editor of Reuters Health, and Adam Marcus, managing editor of Anesthesiology News. It not only finds retractions but goes to the people involved and gets them to explain what was so screwed up about the research that it had to be entirely recanted.

Some are explanations of personal pathos. Shane Mayack, a post doctoral student who was dismissed from her position after a paper in Nature magazine describing a technique for rejuvenating blood-forming stem cells in older mice, issued a deeply personal mea culpa after her work was retracted:

"So, what went wrong? The answer to that question begins with the fact that errors, not fabrications, were made in assembling figures for these manuscripts. I am likely the one who made these errors.  In any case, I believe that since I signed the manuscript submission forms — as all authors are required to do — I hold a responsibility to the accuracy of the manuscript contents. I believe these errors occurred due to mistakes made in data retrieval that were a cause of a poor, but not a unique, data management and archiving system,she wrote.

Other times editors say they just published without looking at what the paper really was saying.  See, for example, a paper published in a mathematics journal which seemed to imply that evolution was incompatible with the Second Law of Thermodynamics — and in so doing became fodder for a creationist's website.

The big question

All of which leads me to autism and vaccination and a different kind of retraction question. Why aren't some papers retracted?

The background is the huge amount of publicity about the discredited research of British gastroenterologist Andrew Wakefield. In 1998, Dr. Wakefield and others published in the journal The Lancet research on 12 children who seemed to have gotten autism following a measles/mumps/rubella vaccination.

After British journalist Brian Deer looked into the matter, all kinds of problem with the data were laid out — not the least of which being that the children in the study were sent to Wakefield from an English lawyer seeking to sue vaccine manufacturers. Not to mention Wakefield was paid considerable sums of money by the lawyer to testify at trials. Deer also found that the parents' supposed recollection of the dates of the vaccination and the onset of symptoms were skewed to establish a cause/effect relationship which didn't really exist.

One result of the investigation is that Wakefield lost his medical licence, and another was that The Lancet and two other journals withdrew the papers which reported what was now seen as fraudulent data and conclusions.

But I had another question after hearing Deer give a presentation about his investigation in Toronto recently. Were there other papers that should be withdrawn as well but hadn't been?

Searching for the truth

It wasn't an idle speculation, as the reality is that the internet is full of people saying that there had been documentation of Wakefield's research by findings. But they often refer to other papers in which he was author or co-author.

So I e-mailed editors of 11 different journals who my researches indicated had published research co-authored by Wakefield and who cited the now retracted papers, and I asked if they had looked into possible fraud in them. Four journals responded that it was a good question and they were now looking into the articles in question to see what to do.

Two journals responded that they had looked at the articles I cited and decided not to retract them. Some reasons were more than a little obscure. Editor Nicholas Talley said that an article published in Alimentary Pharmacology and Therapeutics was a "review article."

'I do think that based on the way the internet is pushing scientific information it is time for evolution to be pushed — by scientific publishers among others.'

"Review articles are opinion pieces and do not contain any original research," he wrote. "Authors in review articles are free to express their opinions based on published research ... a review article whether peer reviewed or not cannot be used to support truth in science. Retracting an opinion piece based on new evidence however is not standard practice. This would set an unfortunate precedent and potentially may curtail scientists from being prepared to provide their written opinions on data which remains vitally important for uncovering the truth. The journal has therefore decided not to retract the review article."

I guess. But I would be more comfortable with that if 93 other authors hadn't cited the piece in Google Scholar, if three of the authors hadn't been on The Lancet piece, and if there was some marking in the article indicating that two of its citations had been retracted. But to be fair the article didn't really make a clear vaccine/autism argument.

The guilt-by-Wakefield association question was raised by Paul Moayyedi of McMaster University, who is co-editor-in-chief of The American Journal of Gastroenterology. It retracted one article but not another.

"The important thing to emphasize is that the paper we withdrew relies on subjective assessments which are very open to manipulation (as Brian Deer's article emphasizes)," he wrote. "The assays reported in Torrente et al. in (2004) 99, 598-605 are more objective and not open to the same subjective interpretation.  This does not exclude fraud but it would be a different fraud than that committed by Dr. Wakefield and there is no evidence that these authors have done the type of data manipulation that would be needed before or since we published the paper.  We could, of course, simply retract all papers that were ever co-authored by anyone associated with Dr. Wakefield and indeed any article that quoted his work.  Such draconian approaches smack of a McCarthy style witch hunt that I do not think is in the best interest of progress in science," he said.

I wrote him back saying that did make sense — but didn't it also make sense, given the scandals surrounding Wakefield's frauds, for the journal to do something that hadn't been done in the past: Explain why an article hadn't been retracted.

"I know the "non-retraction explanation" is not how things were done in the past, but it seems to me the web has radically changed the world of scientific correction and communication. And in this regard I am absolutely sure I am not the only person to wonder about the validity of Wakefield's other co-authored articles," I emailed.

Moayyedi didn't disagree. "I think the scientific community needs to work together to have consistent approaches to detecting fraud and dealing with it fairly (hence no witch hunts) but rigorously ... currently this is progressing at a glacial pace.  This is frustrating, but at least part of the reason for this is the way the system has evolved."

Well, maybe, but also I do think that based on the way the internet is pushing scientific information it is time for evolution to be pushed — by scientific publishers among others.

Prime example

The strangest response to my queries came from L.H. Huntoon, editor-in-chief of the Journal of American Physicians and Surgeons. In 2000 the Journal published an article which absolutely linked itself to the fraudulent vaccine/autism hypothesis. It began, "A possible association between MMR vaccination and autistic encephalopathic regression (AE) in previously developmentally normal children has been reported." It then went on to make exactly those arguments The Lancet paper subsequently retracted.

In response to my query, Huntoon wrote: "The Journal evaluates articles based on scientific content, not on any organization's opinion about prior, very tenuously connected work by the fifth co-author. We are not aware of any problems with the observations reported in our case series. It does seem doubtful that any scientist of integrity will be eager to try to replicate them, out of fear for his career should he succeed in doing so."

Wow, I thought, and went back to check.

It turned out that one of the authors was also on The Lancet papers. Two of the parents whose data was being reported in the paper "are seeking compensation under the National Vaccine Injury Compensation Program in the U.S." One of the other authors provided testimony to people trying similar suits in England. It has been cited in 24 other papers on Google Scholar.  There are 1,704 references to it on the internet in general.

Wow indeed. I think if there ever was a paper that should explain why it hasn't been withdrawn it is this one. If there is ever a reason for a journal to be super-honest and open about its inaction it is this one. Without doing this, it is not just out of tune with 21st century scientific communication, in many ways it just isn't scientific.