Have the Necessary Expertise

Do not review manuscripts beyond your scope of competency. Journals are more likely to publish papers with serious errors or deficiencies if their manuscripts have not gone through rigorous review by reviewers with the necessary expertise. Such problems may even occur in American Psychologist, the flagship of the American Psychological Association. Here are two cases: one involving the critical positivity ratio (Brown, Sokal, & Friedman, 2013, 2014) and another related to the existential issue of meaning in life (Brown & Wong, 2015). How these critical reviews ever got published are a story worth telling, because they could have been buried without vigorous protest against review bias.

Show Some Respect

Reviewers should show some respect by at least reading over each manuscript. Some manuscripts represent the cultivation of several years of research and deserve a couple of hours of reviewers’ time. I have personally seen reviewers rejecting manuscripts simply by reading the first paragraph. One of my own papers, the Death Attitude Profile (Gesser, Wong, & Reker, 1987-1988) was rejected because “there are already too many instruments measuring death anxiety,” even though this was the first paper providing a comprehensive measure of three kinds of death acceptance. Clearly, neither the reviewer nor the editor had taken time to read my paper. That is why I urge all my mentees to summarize what each manuscript is all about, thus indicating that they have read and understood the content of the manuscript.

Be Fair to All Manuscripts

It is difficult to be fair in your treatment of all manuscripts, because we all have our likes and dislikes, and no one is above biases. In order to guard against any review bias, one should openly acknowledge and suspend one’s bias for or against certain research at the beginning of a review. Such “bracketing” practice should improve objectivity in judgement. In this regard, one should also refrain from reviewing a manuscript, if there is a clear conflict of interest which may jeopardize one’s critical judgement, such as being a collaborator in some prior research.

Secondly, one should apply the same criterion to evaluate all papers for the higher purpose of safeguarding the integrity of science. One should especially guard against “halo effects;” one should not let the reputation of the author or the prestige of the author’s affiliation dull one’s ability for critical thinking and vigorous assessment.

Be Constructive

It is very difficult to be constructive, especially when the editor instructs reviewers to be very strict because the journal has a very high rejection rate, sometimes as high as 80%. Under such circumstances, reviewers automatically focus on “fault finding” in order to support a recommendation of “rejection.” Such negativity bias is not a healthy practice, because it tends to favor the established elite researchers and discriminate against beginners who do not have a track record in publications. I propose that a reviewer should do the honest job of taking a constructive approach to review and let the editor tackle the problem of maintaining a high rejection rate.

Being constructive means that one looks for both the strengths and weaknesses of each manuscript. Even in the case of recommending rejection because of fatal flaws, provide logical and empirical justification for the criticism so that the authors can improve their research. From this perspective, it takes much less time to accept a manuscript than to reject one, as illustrated in this review, “How to Measure Existential Meaning” (Wong, 2017).

Be Inclusive and Balanced

Most authors suffer from the bias of “tribalism.” This practice is revealed in several ways. The most common form of tribalism is to ignore the findings or theories that may question their pet views. The second common practice is that within any domain of research, they only cite the work of their circle of associates or friends, and ignore “outsiders,” even when the most cited work comes from the latter.

Such intentional citation amnesia is neither scholarly nor ethical, because it ignores important publications that are germane to the topic under investigation. A knowledgeable reviewer should be able to call out such omissions, so that the manuscript can become more inclusive and balanced in its literature review and discussion.

Do Not be Afraid of Reprisal

Many reviewers, especially young researchers, do not want to offend “big names” because of fear of reprisal. A renowned psychologist once said to me, “I will make sure that this guy will never have another publication,” after bitterly complaining about a paper critical of his theory. I myself have suffered the consequence of daring to write a critical review of a book authored by a prominent psychologist. Over the years, I have witnessed editors who refused to publish any paper critical of an influential theory or author.

The blind review system provides some cover to a reviewer’s anonymity, but not the editor’s identity. In order to protect the integrity of the peer review, both editors and reviewers need to have the courage to do the right thing for the sake of justice and science, even if it involves the possibility of reprisal.

Conclusion

Ideally, a good reviewer serves two functions. Firstly, one serves the professional function of screening out what is not up to standard and improving what is acceptable. Secondly, one serves the humanistic function of contributing to fairness or justice in the distribution of research funding and publication space.

Being an established author does not automatically make one a good editor or peer reviewer; training is needed to develop competent and objective reviewers. I am so pleased the Publons Academy has taken up the challenge to train master reviewers.