Correction to climate change study highlights flaws in peer-review process

"Extraordinary claims require extraordinary evidence."

Scientists and journalists alike were reminded of that oft-quoted phrase by famed cosmologist Carl Sagan when authors of a study published in Nature admitted this week that they needed to issue a correction. The study was widely covered by outlets including The New York Times, The Washington Post, CNN, BBC and Scientific American.

While the correction, which has yet to appear, may provide fodder for climate change skeptics, many in the scientific community are praising the authors for their quick action after recognizing their error. And some believe this is a reminder that there are inherent flaws in journal publication.

The study was led by Ralph Keeling, a professor in the geosciences research division at the Scripps Institution of Oceanography.

Using a novel method that measures oxygen and carbon dioxide in the air from around the world, the researchers concluded Earth's oceans are absorbing 60 per cent more heat than estimates by the UN-backed Intergovernmental Panel on Climate Change (IPCC).

I accept responsibility for these oversights.- Ralph Keeling, Scripps Institution of Oceanography

Soon after it was published on Nov. 1, independent researcher Nic Lewis wrote there was a "major problem" with the study. While Lewis acknowledged that its method was novel and "certainly worthy of publication," he found errors in the calculations that, he concluded, underestimated the uncertainty of the findings.

Three days later, the authors acknowledged the error, said they were redoing the calculations and preparing a correction to be published in Nature. The new calculations are expected to bring the oceans' heat absorption rate more in line with IPCC estimates. 

"These problems do not invalidate the methodology or the new insights into ocean biogeochemistry on which it is based, but they do influence the mean rate of warming we infer, and more importantly, the uncertainties of that calculation," Keeling said in a response published on the website Real Climate.

"I accept responsibility for these oversights, because it was my role to ensure that details of the measurements were correctly understood and taken up by co-authors."

Nature says it's looking into the matter "carefully."

"We take all concerns related to papers we have published very seriously and will issue an update once further information is available," the weekly journal said in a statement to CBC News. 

Imperfect system

Ivan Oransky, a health journalism professor at New York University and co-founder of Retraction Watch, a website that tracks errors in science journals, believes that, while the error was unfortunate for the authors, part of the issue lies in the amount of faith the public — and journalists — put in the peer-review process.

"Science is done by human beings," said Oransky. "When we start to think about peer review as a magical process that, somehow you put in something that may have some issues, but somehow the peer reviewers find all the problems, and then we should trust everything that's peer reviewed, we're setting ourselves up for failure."

He still believes in the process, but says it's not the catch-all that scientists and journalists believe it is.

"We need to stop thinking… that just because something is peer-reviewed means that it's a Good Housekeeping seal of approval," Oransky said.

An important thing to consider, says Keeling, is that peer reviewers aren't necessarily equipped to carefully pore over calculations. They may not even be experts in that field, particularly if the paper spans different disciplines, as was the case with this one. 

"It didn't get seen by every audience that should have seen it, so perhaps we should have sent it to some of these other audiences for review," Keeling told CBC News.

"That would perhaps be a lesson that I take from this: that particularly in a paper like this, it should have been circulated among other colleagues even informally. That would've been helpful."

With division running deep between climatologists and those who believe they are simply being alarmist, there's the danger that other studies will be dismissed as a result of this or any other study where an error has been made.

But Oransky says that it's hot-button topics — like climate change, vaccines or GMOs — that are usually scrutinized far more closely than other topics. This, he says, could be a good thing, but if errors are present, it doesn't invalidate the research or the issue.

"What happens now is, 'See? The climate change researchers made a mistake.' So you have this logical fallacy that everything that they've ever done is flawed and we can't trust any of it," Oransky said.

"Well, that's not true either. We don't live in a binary world, and we do ourselves a disservice when we act like it."