Web journals may turn peer-review system on its head

Scientists frustrated by the iron grip that academic journals hold over their research can now pursue another path to fame by taking their research straight to the public online.

Instead of having a group of hand-picked scholars review research in secret before publication, a growing number of internet-based journals are publishing studies with little or no scrutiny by the authors' peers. It's then up to rank-and-file researchers to debate the value of the work in cyberspace.

The web journals are threatening to turn on its head the traditional peer-review system that for decades has been the established way to pick apart research before it's made public.

In November, the San Francisco-based nonprofit Public Library of Science will launch its first open peer-reviewed journal called PLoS ONE, focusing on science and medicine. Like its sister publications, it will make research articles available for free online by charging authors to publish.

But unlike articles in other PLoS journals that undergo rigorous peer review, manuscripts in PLoS ONE are posted for the world to dissect after an editor gives them just a cursory look.

"If we publish a vast number of papers, some of which are mediocre and some of which are stellar, Nobel Prize-winning work, I will be happy," said Chris Surridge, the journal's managing editor.

It's too early to tell how useful this open airing will be. Some open peer-reviewed journals launched in the past year haven't been big draws. Still, there appears to be enough interest that even some mainstream journals like the prestigious British publication Nature are experimenting.

Democratizing the peer-review process raises sticky questions. Not all studies are useful and flooding the web with essentially unfiltered research could create a deluge of junk science. There's also the potential for online abuse as rogue researchers could unfairly ridicule a rival's work.

Supporters point out that rushing research to the public could accelerate scientific discovery, while online critiques may help detect mistakes or fraud more quickly.

The open peer review movement stems from dissatisfaction with the status quo, which gives reviewers great power and can cause long publication delays. In traditional peer review, an editor sends a manuscript to two or three experts— referees who are unpaid and not publicly named, yet they hold tremendous sway.

Careers can be at stake. In the cutthroat world of research, publishing establishes a pedigree, which can help scientists gain tenure at a university or obtain lucrative federal grants.

Researchers whose work appears in traditional journals are often more highly regarded. That attitude appears to be slowly changing. In 2002, the reclusive Russian mathematician Grigori Perelman created a buzz when he bypassed the peer-review system and posted a landmark paper to the online repository, arXiv. Perelman later won the Fields Medal this year for his contribution to the Poincaré conjecture, one of mathematics' oldest and most puzzling problems.

Editors of traditional, subscription-based journals say the peer-review system weeds out sloppy science. The traditional process isn't designed to detect fraud (referees rarely look at a researcher's raw data), and prestigious journals have unwittingly published bogus work. Last year, for example, Science retracted papers on embryonic stem cell research by a South Korean cloning scientist who admitted falsifying his results.

Work submitted to PLoS ONE, for instance, is debated after publication by colleagues who rate the research based on quality, originality and other factors. Commenters cannot alter the paper, which becomes part of the public record and is archived in databases. If there is disagreement, authors can respond to comments. To prevent abuse, the site is monitored for inflammatory language and the postings can't be anonymous.

"The fact that you get published in PLoS ONE isn't going to tell you whether it's a brilliant paper. What it's going to say is that this is something worth being in the scientific literature, but you need to look at it more closely," Surridge said.

Another open peer-reviewed journal, Philica, launched earlier this yearwith a more radical approach.

Authors are responsible for uploading their research to the website at no cost and without any peer review. Comments are anonymous, but users whose identities have not been verified by site administrators are flagged with a question mark next to their comments. The journal, still in the trial stage, has published about 35 papers so far. About a third still need to be critiqued.

Philica co-founder and University of Bath psychology professor Ian Walker said the system discourages authors from publishing fake studies because others can rat them out.

"Imagine if somebody puts up absolute garbage, you will have plenty of reviews that will say, 'This is terrible, terrible, terrible,'" he said.

Academics are eyeing the open peer-review experiment with interest.

Andrew Odlyzko, a mathematician who heads the University of Minnesota's Digital Technology Center, is encouraged by the growing number of online journals. Whether they will work— he's not sure. Some researchers might only post unhelpful one-liners for fear of reprisal. Granting anonymity may boost participation, but could lead to "malicious postings from cracks," Odlyzko said.

Even some mainstream journals are toying with a tame form of open peer review. This summer, Nature allowed authors whose papers were selected for traditional peer review to have their manuscripts judged by the public at the same time. Editors weigh both sides when deciding whether to publish a paper, and rejected research can be submitted elsewhere.

Linda Miller, the journal's U.S. executive editor, said she was encouraged by the participation. More than 60 papers have been posted on Nature's site for open peer review as of mid-September including one that has been accepted for publication. Several others are on the path to being published.

Miller said Nature's experimentation with the internet is just another way the journal is trying to reach out to the public. Two of its specialized journals on neuroscience and genetics already offer a blog-like forum for researchers to post their thoughts on published articles, though they have attracted little attention, she said.

"If we don't serve the community well, we will become irrelevant," she said.