RCR Casebook: Peer Review

Claire is a widely recognized epidemiologist who has served on the review board of several journals. She takes her role of reviewer seriously and expects editors to uphold the highest standards about the research that gets published.

Claire continues to review articles nearly every month for a highly esteemed science journal whose long-time editor is retiring. Claire finds the new editor a trifle flashy, and at first assumes that he is probably just trying to modernize the journal and up its visibility in the marketplace. After all, journals are businesses that seek to have a positive cash flow.

However, Claire soon notices other problems. Since the appointment of the new editor, several articles have been published despite her recommendations for rejection. Moreover, they are published without any appropriate revision. The conceptual framework is often poorly developed. The methodology is not properly presented and explained. For example, authors fail to report effect sizes and other important indicators regarding the power of the study. Worse, in the abstract, discussion, and conclusion sections, they make claims about their findings that do not seem to be supported by the actual methodology and results.

Claire knows that these problems do not arise because of space limitations. While the journal receives many submissions and can accept very few, the journal has a policy of including supplementary documents online which contain details that cannot be included in the printed article due to space limitations. The papers fail to include methodological detail so that readers can figure out how a study was done and replicate it.

The final straw is the way the editor seems to be pandering to the public.  Many of the poorest papers published are also papers with considerable press value—papers that address issues that have been of great concern lately. The few modifications to these poor submissions are designed to hype the spectacular findings that the authors claim. It appears that the editor is seeking to build a reputation for reporting exciting results, even if they are not valid.

From a scientific and public health perspective, this strikes Claire as the worst possible kind of error for an editor to make. Not only is the science poor, but it is also likely to harm the public health since a journal can influence policy if policy makers do not recognize the poor quality of the articles. In any event, the reputation of science and especially of that journal will eventually suffer. Claire is very frustrated and is trying to decide whether the editor has gone too far in revamping the journal and putting his own stamp on it.

What should Claire do?

Discussion Questions for the Facilitator

• Are there any generally recognized standards for editors of journals to follow?
• With whom do you think Claire, a long-time reviewer for the journal, should consult?
• Should editors share with peer reviewers the letters of acceptance and letters of rejection that are sent to authors?
• Could protesting the editorial style have either a negative or positive impact on Claire’s reputation?
• What is the role of the journal’s editorial board?
• Do you think editors should have the final say regarding which articles are accepted? Or should such decisions be made by a majority vote of peer reviewers? *