Column Editor: Rick Anderson (Associate Dean for Collections & Scholarly Communication, Marriott Library, University of Utah; Phone: 801-721-1687)
If you want to start a scholarly-communication bar fight, bring up the issue of predatory journal publishing. It doesn’t matter what you say on the topic, someone in the group will not only be offended by it, but may even take a swing at you.
Of course, it’s possible that you’ve never heard of predatory publishing, and you’re now thanking your lucky stars that you’ve managed to avoid this particular corner of the Scholcomm Culture Wars. Allow me to ruin that for you.
The term “predatory publishing” was coined by librarian Jeffrey Beall. Back in 2012 he noticed a growing phenomenon: journal publishers who, instead of charging for access to their publications, made their journals available on an open access (OA) basis and charged authors a fee (an “article processing charge,” or APC) for the privilege of placing their articles in their journals. Usually these publishers promised the usual editorial rigor, and almost invariably advertised their journals as “peer-reviewed,” while also claiming that their publications had high impact factors, prestigious editorial boards, etc. But in the case of the publishers Beall was noticing, most or all of these claims seemed to be false; in reality, they had no impact factors at all (or low ones), their editorial boards were populated significantly by people who had no idea their names were listed, there was little or no peer review, and the journals seemed willing to publish any article submitted, as long as the author paid the APC. Beall started keeping a list of these journals and publishers, and thus ignited a controversy that continues even though the list itself has been shut down. (It persists in archived form,1 but is no longer being actively managed.2)
It’s important to pause here and note that there’s no reason why the APC model can’t be implemented responsibly, and in fact many reputable publishers do so — there are lots of quality APC-funded OA journals out there, including PLOS ONE, Nature Communications, Scientific Reports, and Heliyon, among others, and in fact the majority of legitimate articles published on an OA basis are funded by APCs.3 When it comes to scientific and scholarly quality, however, the problem with this publishing model is that it involves an unavoidable conflict of interest: if a journal makes its revenue by accepting and publishing articles (rather than by selling access to articles), it has a financial incentive to accept and publish as much as possible and a financial disincentive to spend much time and money checking to see whether the articles being accepted actually represent solid scholarship. Again, there are lots of journals that effectively manage this conflict of interest — but the conflict is still there, and its most extreme and uncontrolled manifestation is the predatory publisher.
Whether or not predatory publishing is actually a big problem turns out to be a controversial issue in itself. People who are skeptical about OA (in particular Beall himself, who has publicly positioned himself as being downright opposed to it4) tend to use the predatory publishing issue as a stick with which to hit the OA movement. For their part, OA advocates can get quite defensive when the issue arises, often insisting that the problem isn’t even worth discussing5 — that it’s limited to a bunch of fringe actors who aren’t fooling anyone with their clumsy imitation “journals,” and therefore isn’t doing any actual damage to the integrity of scholarly communication.
How to settle this dispute? Well, one way is to test the hypothesis that no one is fooled by predatory journals and that these journals aren’t doing any damage to scholarly communication.
And this is where the “sting” comes in.
Possibly the first and certainly the most controversial sting operation aimed at predatory publishers was described in an article titled “Who’s Afraid of Peer Review?,” which was published in Science6 in October 2013. The article described an experiment undertaken by its author, science journalist John Bohannon. Bohannon had written a purportedly scientific paper that, as he described it, would have been spotted as nonsense by any reviewer “with more than a high-school knowledge of chemistry and the ability to understand a basic data plot.” He invented a fake identity, and a fake institutional affiliation to go along with it. He then submitted the paper to just over 300 journals: 183 from the Directory of Open Access Journals (DOAJ), plus 137 identified by Beall’s List as “potential, possible, or probable” predators — the set included 16 journals that were listed in both places—and waited to see how many of them accepted it. More than half of them did so.
The reaction to Bohannon’s article was swift and, in some cases, savage. Milder reactions included those from SPARC7 (“the journals… were not selected in an appropriately randomized way”) and OASPA8 (using the exact same language, interestingly); nastier ones came from the Directory of Open Access Journals (which accused Bohannon of racism before deleting the accusation from its web page) and from Björn Brembs9 (“the outcome of this stunt is entirely meaningless”), among many others.
Not everyone thought Bohannon’s sting was such a waste of time, however, and soon enough others got into the game. A staff writer from the Ottawa Citizen cobbled together a random assortment of plagiarized passages from previously-published papers on geology and hematology, threw in some graphs taken from a paper about Mars, and submitted the resulting mess to 18 suspicious-looking science journals; all but two accepted it.10 (Even after he explained that he had submitted nonsense to them, one of the journals offered to make a few tweaks and publish it anyway.) John McCool, a science editor who received an unsolicited invitation to contribute to a suspicious-looking title called Urology & Nephrology Open Access Journal, responded by submitting an article about a nonexistent urological disorder, using a false name derived from a TV character.11 The article was accepted “for further peer review” within hours, and accepted for publication in three days (subject to payment of an $800 APC, of course).
It’s worth pointing out, however, that while all three of these sting operations illustrated the willingness of predatory journals to publish any nonsense that an APC-paying author wants to submit, the existence of scam journals like these was never in question; none of these stings demonstrated that predatory journals were fooling readers, or having any real impact on the world of scholarship or (still less) the broader public conversation about science.
For evidence of that, we had to wait for Bohannon’s next sting: the infamous Chocolate Makes You Lose Weight experiment.12 For this one, Bohannon and his collaborators put together a clinical study of the impact on weight loss of eating one chocolate bar per day. I’ll let you read the details of how Bohannon and his crew put together a lousy study design and successfully p-hacked the resulting data,13 but suffice it to say that any competent (or interested) editor would probably have seen the problems immediately, and if he or she didn’t see them right off, a diligent peer reviewer would have. Bohannon submitted the resulting paper to 20 journals that he suspected of being predators, and within 24 hours had an acceptance letter from International Archives of Medicine.14 Two weeks later the article was published, without any editorial intervention whatsoever (according to Bohannon, “not a single word was changed”).
But the fact that Bohannon was able to get a deeply flawed study published quickly by a questionable journal isn’t the interesting part of the story; the interesting part is what came next, when the article went viral.
Once the article was published, Bohannon composed a press release and started soliciting media coverage. He succeeded quickly and on an international scale: breathless stories on Bohannon’s research findings were published in Shape magazine, Bild, the Times of India, Express, international editions of the Huffington Post, and other outlets before the hoax was revealed. Even today, you can see the study still being cited as authoritative on diet-and-nutrition websites like Dr. Murray.com,15 Fat Loss for Women,16 and Ready Set Health17 — and both the Times of India article18 and the Express article19 are still online, with no indication of the fact that the study on which they’re based was a hoax — despite, in both cases, reader comments alerting them to the fact.
The fact that Bohannon’s second hoax was amplified and disseminated by the popular rather than the scientific press is exactly the point: what this demonstrated was that we can’t assume the impact of predatory publishing practices will always be contained to narrow niches of specialist science. Publish fraudulent “science” in a scam journal on the right topic, and (with the help of a credulous and ratings-hungry popular media) you may be able to mislead millions of people into making poor or even disastrous choices. As troubling as predatory publishing is in its implications for the integrity of formally-published science, it is perhaps even more so in its capability to shape public understanding of science.
But Bohannon’s chocolate sting left still unresolved another question, one that had been posed by defensive OA advocates ever since his first one: given that his experiment (like others since) only targeted OA publishers, how do we know that those publishers are more predatory than toll-access publishers? How can we be certain that subscription journals would have fared any better when offered nonsense in the guise of science?
A partial answer to that question comes now, in the form of yet another sting operation — this one undertaken by a group led by Dr. Katarzyna Pisanski, a research fellow at the University of Sussex.20 Pisanski and her colleagues created a fictitious author named Dr. Anna O. Szust (oszust is Polish for “a fraud”), gave her a made-up CV consisting of nonexistent degrees and a make-believe publishing history, and created an online profile that showed her to be, in Pisanski’s words, “dismally inadequate for a role as editor.” Dr. Szust had no indexed citations, no publications in academic journals, and no editorial experience at all. Anyone who tried to locate the informal and non-peer-reviewed publications listed on her CV would not have been able to do so, because they didn’t exist.
Pisanski et al. submitted Szust’s name to 360 journals, asking that she be considered as a candidate for their editorial boards. Of those journals, 120 — a mix of toll-access and OA titles — were listed in Journal Citation Reports (JCR), 120 were listed in the Directory of Open Access Journals (DOAJ), and 120 were drawn from Beall’s list of “potential, possible, or probable” predators.
The result? Of the JCR titles, 60% ignored the application and 40% rejected it; none accepted it. Of the DOAJ titles, 7% accepted Szust’s application, 38% rejected it, and 55% did not respond. As for the suspected predators, only 13% rejected her application; just over half failed to respond, while fully one-third of them accepted Szust as an editor outright — four of them appointing her editor in chief. (A choice quote from one acceptance notification: “It’s our pleasure to add your name as our editor in chief for this journal with no responsibilities.”) There are additional fascinating details about the responses Szust got from the questionable journal publishers — Pisanski’s report in the journal Nature makes for fascinating, if disturbing, reading.
What should we make of all this? A few possible conclusions suggest themselves:
- For all of its weaknesses and problems of administration, the evidence of this latest sting suggests that Beall’s list was a reasonably reliable source of information about predators.
- The DOAJ, even after its recent tightening of criteria, still certifies as “high quality, peer reviewed Open Access research journals” a significant number of journals that do not seem to fit those criteria.
- Journals that are indexed in JCR do seem to be markedly more careful about whom they accept into editorial positions than (unsurprisingly) Beall’s List titles, and (more surprisingly) even than DOAJ-certified OA journals.
I’m sure we can expect to see more stings and exposés of varying types as time goes on.
One of the great advantages of the current scholarly-communication ecosystem is the degree to which barriers to entry have been lowered, and to which it’s now possible for scholars, scientists, and organizations that could not have done so in the past to make their work freely and easily available to the public. Of course, that strength is also a weakness, as the growing phenomenon of predatory publishing makes clear. But another aspect of the ecosystem’s strength is the multitude of ways in which it makes it possible to shine a light on predators and other bad actors. Let’s keep doing so. Eternal vigilance is, as they say, the price of a reliable scholarly record.