Tuesday, May 17, 2011

Peer review is flawed but the best we've got

Source: http://www.guardian.co.uk/commentisfree/2009/sep/12/bad-science-peer-review-goldacre

Peer review is flawed but the best we've got

This week the peer review system has been in the newspapers, after a survey of scientists suggested it had some problems. That is barely news. Peer review – where articles submitted to an academic journal are reviewed by other scientists from the same field for an opinion on their quality – has always been recognised as problematic. It is time-consuming, it could be open to corruption, and it cannot prevent fraud, plagiarism, or duplicate publication, although in a more obvious case it might. The problem with peer review is, it's hard to find anything better.

Here is one example of a failing alternative. This month, after a concerted campaign by academics aggregating around websites such asAidstruth.org, academic publishers Elsevier have withdrawn two papers from a journal called Medical Hypotheses. This journal is a rarity: it does not have peer review, and instead, submissions are approved for publication by its one editor.

Articles from Medical Hypotheses have appeared in this column quite a lot. They carried one almost surreally crass paper in which two Italian doctors argued "mongoloid" really was an appropriate term for people with Down's syndrome after all, because they share many characteristics with oriental populations (including: sitting cross-legged; eating small amounts of lots of types of food with MSG in it; and an enjoyment of handicrafts). You might also remember two pieces discussing the benefits and side-effects of masturbation as a treatment for nasal congestion.

The papers withdrawn this month step into a new domain of foolishness. Both were from the community who characterise themselves as "Aids dissidents", and one was co-authored by their figureheads, Peter Duesberg and David Rasnick.

To say a peer reviewer might have spotted the flaws in their paper – which had already been rejected by the Journal of Aids – is an understatement. My favourite part is the whole page they devote to arguing that there cannot be lots of people dying of Aids in South Africa because the population of the country has grown in the past few years.

We might expect anyone to spot such poor reasoning but they also misrepresent landmark papers from the literature on Aids research. Rasnick and Duesberg discuss antiretroviral drugs that have side-effects but which have stopped Aids being a death sentence, and attack the notion their benefits outweigh the toxicity: "contrary to these claims", they say, "hundreds of American and British researchers jointly published a collaborative analysis in The Lancet in 2006, concluding treatment of Aids patients with anti-viral drugs has 'not translated into a decrease in mortality'."

That is a simple, flat, unambiguous misrepresentation of the Lancet paper to which they refer.

What does this tell us about peer review? The editor of Medical Hypotheses, Bruce Charlton, has repeatedly argued – very reasonably – that the academic world benefits from having journals with different editorial models, that peer review can censor provocative ideas, and that scientists should be free to pontificate in their internal professional literature.

But there are blogs where Aids dissidents, or anyone, can pontificate wildly and to their colleagues: from journals we expect a little more
.

No comments: