A couple of articles have come to our attention that bring to the fore some of the thorny issues surrounding open access and peer review. The first article Critics Say Sting on Open-Access Journals Misses Larger Point was written by Paul Basken, senior reporter for the Chronicle of Higher Education, and focuses on the elaborate Science magazine exposé authored by independent journalist John Bohannon that has upset so many proponents of open access.
In an apparent effort to measure the quality control exercised by open access journals in accepting submissions, Mr. Bohannon, fabricated “a cancer-research article and painstakingly tracked the responses to it from more than 300 journals.” According to Mr. Bohannon more than half, some 157 agreed to publish it. Mr. Bohannon’s disturbing verdict: “The data from this sting operation reveal the contours of an emerging Wild West in academic publishing.”
Open access proponents are crying foul. They question Mr. Bohannon’s methods saying the sample size was too small and included journals of known unreliable quality. They also note that prominent open-access journals like those at PLoS, Hindawi, and BioMed Central all rejected the paper. Some critics even went further accusing Science, where Mr. Bohannon’s article appeared, of bias because subscription journals like Science were not included in the sting.
Regardless, the results set off alarm bells. Some journals accepted the bogus paper without bothering to evaluate it and a number actually put the article through their peer review process and still accepted it. In fact, some published the article even after Mr. Bohannon asked them to withdraw it.
The second article we wanted to bring to your attention is Puzzling Peer Reviews by Carl Straumsheim that was posted on the Inside HigherED website. This article describes Mr. Bohannon’s “sting” in more detail and clearly notes that “culprit” in Mr. Bohannon’s troubling report is “a lack of a rigorous peer review process.” However, Mr. Straumsheim’s post doesn’t stop there. He expands his examination of OA journals and peer review to portray a diverse, evolving, and often confusing process.
Drawing his examples from digital humanities publishing, Mr Straunsheim discusses three instances including one where two authors thought that that the journal in question used a post-publication review model only to learn that “its editors will request revisions or expanded versions where needed” and that “about one-third of the journal’s content undergoes extensive revisions.” In the second example “a collective of administrative staff, faculty members and graduate students” rotates editing duties with one member of the collective freely admitting “we’re a young journal, so we haven’t really solidified our processes yet.” However, the journal emphasizes author support maintaining open communication between contributors and reviewers and “consistent updates” provided as to where submissions are in the process. The third is an experimental site called DHThis “that substitutes the formal review process for a crowdsourced alternative” that allows readers to decide what submissions get promoted.
Both of these articles offer thought provoking perspectives on an important issue. Mr. Bohannon points out real issues with peer review in a number of OA journals. But is it fair to paint all OA journals with the same brush? Are his critics right to say that he picked on the weakest links in the OA chain? Is this a problem limited to OA journals? If he offered his paper to a similar sampling of subscription journals would the acceptance rates been similar? Is the problem faulty methods of peer review in general?
Are the examples Mr Straunsheim cites at the vanguard of new forms of peer review more in tune with needs of today’s scholarly community? Or are they a flash in the pan that will be unable to provide the rigorous review necessary to quality scholarship? Is post publication review and allowing readers to decide a valid way to access quality? Or are they a poor substitute for traditional peer review?
As always, we’d love to know what you think! Please comment! Thanks!