A Brief Introduction on Peer Review: New Possibilities

by Sean Scanlan

updated October 2014 (Originally published June 2013)

“Peer review is the evaluation of work by one or more people of similar competence to the producers of the work (peers). It constitutes a form of self-regulation by qualified members of a profession within the relevant field. Peer review methods are employed to maintain standards of quality, improve performance, and provide credibility. In academia peer review is often used to determine an academic paper's suitability for publication.” 

Even if it goes by a different name, most people—especially those reading NANO—understand that peer review takes place across many different disciplines and is performed in many different ways. The Wikipedia definition above, woven together by a number of anonymous peer reviewers, helps reinforce the primacy and the concrete value of this mode of information processing to our globalized need to share and build together.

From the classroom, to the lab, to the studio, to industry, and to the academy, evaluating work is part of everyday life. Whether we call it critiquing, refereeing, or peer reviewing, the goals are the same: to make the object under review better, to verify that its claims are not false, or at the very least, to ascertain that the object has some merit. That said, peer review has a special and fraught role in academia, one that tethers its mysterious mechanisms to promotion and tenure, to success and failure. Yet the problem of peer review is not with its purported goals; the problem is how to implement peer review efficiently, usefully, and fairly. How should we do it? This special issue of nano tackles some of the newer, more digital, means by which peer review is being done and provides insight into what we might do better.

Jenna Pack Sheffield’s note "Open Peer Review: Collective Intelligence as a Framework for Theorizing Approaches to Peer Review in the Humanities" discusses the definition of “open peer review” and looks at various ways it’s been used to highlight the advantages and disadvantages of the process. Working toward these goals, she first makes use of Pierre Levy’s theory of collective intelligence, or “universally distributed knowledge,” to reframe peer review. The idea here is that digital, widely dispersed, review can foster enriched interactions between the members of a vast readership, and thus improve coordination. Pack Sheffield goes on to explore non-traditional, or new, peer review models that seek a sense of balance between open and closed review, models in which “anonymous readers, identifiable readers, and ultimately the editors all have a say in the publication decision.” While only hinting at the thorny issues of promotion, tenure, and the incentive systems required to change traditional, blind peer review to open peer review, Pack Sheffield does state that a balanced method will have a beneficial effect for all parties involved; and, of central importance, a hybrid system will benefit the author in terms of increased—and more timely—feedback. The balanced approach of leveraging the best of blind peer review in tandem with the broad, fast system of digital, open peer review is an alluring model, one that should add needed fuel to the conversation of how new knowledge should be reviewed, published, and maintained.

Theories on peer review abound. History of science scholar Mario Biagioli asked, a decade ago, “[w]hy do we tend to perceive peer review as either good or bad, helpful or obstructive, but not as one of the fundamental conditions of [the] possibility of academic knowledge and the construction of value?” (11). But where Biagioli sees the possibility for a Foucauldian analog between the disciplinary functions of prisons and academic peer review, newer scholars, such as Pack Sheffield and Aaron Barlow, see a more positive set of alternatives borne of hybrid, open processes. As Bonnie Wheeler, past president of the Council of Editors of Learned Journals (CELJ) has said, “[a]cademic editors think about peer review constantly” (313). And we can add to editors a long list: department chairs, new faculty, graduate students—the whole spectrum of educators, writers, and artists think about the concept of production and review; it remains to be seen which processes will gain acceptance and which will fade.

This issue of NANO includes interviews with three editors who share their ideas about the shape of current peer review problems and what the future might look like for academics, tenure review boards, and publishers.

First, I conducted an email interview with Masoud Yazdani, the editor of Intellect Books, an independent academic publisher in the fields of creative practice and popular culture, whose aim is to publish scholarly books and journals that provide a vital space for widening critical debate in new and emerging subjects. Before beginning his press, Yazdani was a professor of digital media and so he is particularly interested in creating business models that help scholars in emerging fields, scholars who might not have a wide range of peer-reviewed publication options. In addition, he reflects on how peer review is the “life blood” of academic publishing, on the sustainability of Open Access publishing, and the contentious issue of submission fees. Read the full interview here.

Second, Aaron Barlow, my colleague at New York City College of Technology, agreed to share his views with me on the problems of traditional peer review. Barlow is Faculty Editor of Academe, the magazine of the American Association of University Professors (AAUP). According to Barlow, his own views on business-as-usual peer review and academic publishing are becoming more radical the more he studies its issues. His key concepts are openness and change. Fear has the potential to hold back young scholars, but, according to Barlow, the winds have changed so much that people performing tenure review would have to strongly argue against a candidate who breaks new ground in open publishing. Barlow is hopeful about the future and encourages scholars to seek newer publishing formats. Read the full interview here.

Lastly, NANO assistant editor Rebecca Devers interviews Martha J. Cutter, the former editor of MELUS, about the complexities of processing, reviewing, and publishing a journal that receives in over 300 submissions each year. Cutter is a firm believer in the value of blind peer review for both the reviews and the authors. And she shares her views on the merits and concerns of open access and on four ways that journals are under assault.

Works Cited

Biagioli, Mario. “From Book Censorship to Academic Peer Review.” Emergences 12.1 (2002): 11-45. Print.

Wheeler, Bonnie. “The Ontology of the Scholarly Journal and the Place of Peer Review.” Journal of Scholarly Publishing 42.3 (April 2011): 307-322. Print.

Image

“Wikipedia Logo.” Wikipedia. Wikimedia Foundation, Inc. n.d. Screenshot. 18 June 2013.