Roles and Responsibilities for Peer Reviewers of International Journals

Abstract

There is a noticeable paucity of recently published research on the roles and responsibilities of peer reviewers for international journals. Concurrently, the pool of these peer reviewers is decreasing. Using a narrative research method developed by the author, this study questioned these roles and responsibilities through the author’s assessment in reviewing for five publishing houses July–December 2022, in comparison with two recent studies regarding peer review, and the guidelines of the five publishing houses. What should be most important in peer review is found discrepant among the author, those judging peer review in these publications, and the five publishing houses. Furthermore, efforts to increase the pool of peer reviewers are identified as ineffective because they focus on the reviewer qua reviewer, rather than on their primary role as researchers. To improve consistency, authors have regularly called for peer review training. Yet, this advice neglects to recognize the efforts of journals in making their particular requirements for peer review clear, comprehensive and readily accessible. Consequently, rather than peer reviewers being trained and rewarded as peer reviewers, journals are advised to make peer review a requirement for research publication, and their guidelines necessary reading and advice to follow for peer reviewers.

1. Introduction

Peer reviewers of international journals engage in a process of detailed examination by which authors’ submitted manuscripts are scrutinized, improved, and evaluated by these academics for the authors’ ability to satisfy the receiving journal’s editors and guidelines [1]. Since the mid 1960s, peer review has been deemed essential for rigorous academic publications [2].

Although previously considered ambiguous and recognized as a concept open to change [3], international journals are defined as follows: the language of publication is one employed by researchers in a number of countries, the editorial board and pool of reviewers include members from a variety of countries, and the journal maintains comprehensive, clear and public standards for publication—practices that result in greater visibility and higher citation rates regarding international journals [4]. Globalized patterns are increasingly supporting publication in international journals [5]. That peer reviewers of international journals be qualified, unbiased and available is the foundation of legitimacy regarding the publication of the research conducted [2,6]. As such, it is important that peer reviewers—in volunteering their time to maintain a high standard for research publication—be able to assume each of the roles inherent to the activity, and they willingly and competently accept the concomitant responsibilities.

The definition of the role of the peer review mirrors the definition provided by the Open Education Sociology Dictionary—a role is a position represented by a set of socially defined attributes and expectations determining appropriate behavior for an individual based on their status in relation to the group [6]. In contrast, “responsibilities”, with respect to specific roles, according to the Online Oxford English Dictionary are the state or fact of being in charge of or of having a duty towards a person or thing [7]. In other words, the responsibilities of peer reviewers represent a subset of their roles, developed and understood in relation to them.

Although peer review is ubiquitous and fundamental, there has been a paucity of recently published research clarifying the roles and responsibilities of peer reviewers. This is despite the number of articles regarding issues related to peer review being found, in one estimate, to have advanced from 50 published in 1974 to approximately 120 published in 2014 [8]. In another article, the mention of peer review in publications was considered to have doubled between 2005 in 2015. Yet, examining the article’s “Figure 1, Number of publications about peer review”, an even greater increase is evident, from approximately 120 articles to over 400 [9]. On the other hand, in contrast to considering how the roles and responsibilities of peer reviewers could be defined, what has been published has instead lamented that the journals have been insufficiently diligent in defining peer reviewers’ roles and responsibilities [10].

After receiving increasing scrutiny in the last few years [9,11], the publishing houses of journals have become responsive to the past criticism [12] regarding the obscurity of the responsibilities they expect of peer reviewers. The various journals associated with their publishing houses now have a separate, detailed page on each of their websites devoted to the guidelines for reviewers (examples of these websites are cited below, in Section 2.2. Methods). Nevertheless, there are researchers who continue to criticize journals for not being clear in their roles and responsibilities, basing their conclusion on research conducted before these changes were made to improve the transparency of the roles and responsibilities of peer reviewers [1]. That these criticisms continue, in spite of the evidence that publishing houses are now making their guidelines obvious to peer reviewers and to authors, may be due in part to the fact that few peer reviewers are themselves active researchers [13]. As such, these continuing assertions regarding problems with the roles and responsibilities of peer reviewers are at best questionable.

Some authors have pondered the necessity of peer review [14]. Yet, researchers continue to recognize its value—as was made clear in the 2018 Publons report that found 98% of respondents considered peer review to be either important (31.2%) or extremely important (66.8%) [15] (p. 9). Still, regardless of the improvements to the guidelines that have been provided to reviewers by journals, with the number of international peer-reviewed journals increasing [2,13], the pool of qualified peers reviewers is diminishing [16]. The previous solutions that have been adopted to increase the pool of peer reviewers have concentrated on rewards to them with respect to their role as peer reviewers [17]. This has not had the type of success necessary to increase the pool of effective reviewers who can attend to the guidelines [18].

Following from this understanding of the context of peer review, the aims of this study are to delineate the roles and responsibilities of peer reviewers in such a way as to increase the number of peer reviewers for maintaining international standards of journal publication. This study is valuable because, with respect to work specifically in the sciences and social sciences, it is important that research on the roles and responsibilities of peer reviewers of international journals be current—from within the last five years [19]. This is a period in which there has been little research on this topic. In commencing this study, the range of the roles and responsibilities of peer reviewers will be investigated in relation to narrative research undertaken to determine the author’s own standards as an active researcher and frequent peer reviewer for various international journals of five distinct publishing houses. The results will be compared with two publications evaluating the roles and responsibilities of peer reviewers based on the views of authors and/or editors regarding peer review [10,20], and with those of the five publishing houses. The major findings of this comparison are that what should be most important in peer review is found to vary among this author, those judging peer review in these two publications, and the five publishing houses for which this author reviewed in the latter half of 2022. As well, current efforts to increase the pool of peer reviewers are found to be ineffective because they focus on the reviewer qua reviewer, rather than the primary role of peer reviewers as researchers. To improve consistency, in contrast to the regular call by authors for peer review training (see Section 4.2. Ranking of Responsibilities of Peer Reviewers for citations regarding a number of publications devoted to this call), it is advised that journals make peer review a requirement for research publication, and their guidelines necessary reading and advice for peer reviewers to follow. This approach would also serve to increase the proportion of peer reviewers who are active researchers, improving the expertise of reviewers. These recommendations are offered for improving peer review with regard to both mitigating weaknesses in the process and increasing the pool of qualified peer reviewers—recognized a decade ago as the fundamental issues with respect to peer review [21].

2. Materials and Methods

Regarding the materials and methods to be used in investigating the roles and responsibilities of peer reviewers, it would seem reasonable that in order for a researcher to make a fair assessment of the roles and responsibilities expected of peer reviewers of international journals, the researcher be both a frequent peer reviewer and one who has current research experience to judge the expectations. In meeting these proposed requirements, the author of this article has undertaken this examination after having reviewed eighty-six manuscripts for fourteen different journals from 1 July 2022 to 31 December 2022, while concurrently having published one book chapter and four articles in international peer reviewed journals, including three MDPI journals (Challenges, Histories and COVID) with two other preprints under review by an MDPI journal (Challenges) and a Wiley journal (Theoria: A Journal of Social and Political Thought), over the same period. The journals for which the reviews were conducted are associated with one of the following five international publishing houses: Springer (specifically BMC [22], a division of Springer Nature), Dove Press, Frontiers, MDPI, and Wiley.

Table 1. Total number of articles reviewed by the author 1 July–31 December 2022 in relation to the journals that sent papers for review and their publishing houses listed in order of when during this period the initial review was completed for each journal.

#	Journal	Dove	Frontiers	MDPI	Springer	Wiley
11	BMC Medical Education				11
20	International Journal of Environmental Research and Public Health			20
3	Challenges			3
6	BMC Public Health				6
1	Risk Management and Healthcare Policy	1
2	Scientific Reports			2
2	Hygiene			2
27	Frontiers in Psychology		27
1	Theoria: A Journal of Social and Political Thought					1
3	Frontiers in Education		3
6	Frontiers in Psychiatry		6
1	Frontiers in Public Health		1
2	Scientia Pharmaceutica			2
1	Nursing Reports			1
86	Totals	1	37	30	17	1

represents the breakdown of the eighty-six articles reviewed by the author during this period by journal name in relation to their publishing house. It will be the guidelines of these five publishing houses that will be compared and contrasted for this evaluation of international journals, as these are the publishing houses with which the author is familiar. The guidelines for reviewers for each of the separate publishing houses are found as part of the publishing houses’ websites: Dove [23], Frontiers [24], MDPI [25], Springer [26], and Wiley [27].

Table 1 reveals that the expertise of the author in reviewing manuscripts during this period has been almost exclusively with respect to biomedical and social science research, with only one review in the humanities. On the other hand, two of the author’s own submissions to journals over the period in question were to two humanities journals, meaning that the author is familiar with differences between publishing in the sciences and the humanities (see Section 4.4. Limitations for an explication of this difference).

2.1. Materials

In assessing the expectations of peer reviewers for each of these five publishing houses, there are four materials that will be used: the data reported in each of

Table 2. Questions self-asked of the author as a peer reviewer using the author-developed narrative research method, dividing those questions posed into fulfilling the function of a reviewer and those concerning what the reviewer should undertake in reviewing the submission.

Fulfilling the Function of Reviewer	Reviewing the Submission
What expertise is needed for the review?	Who are the authors to the reviewer?
	When should the review be completed?
	Who should conduct the review?
	How confidential can the reviewer be?
How should the review be undertaken?	What guidelines are available?
	What process should be followed?
How should the content be examined?	How should the submission be read?
	How clear is the writing?
	Where are the references from?
	What is the likelihood of plagiarism?
What evaluates the submission?	How suitable is the front matter?
	What is the research question?
	What method was used?
	How clear are the tables and figures?
	What are the limitations?
	How are the conclusions supported?
	When were the references published?
	How original is the work?
	Where was research support obtained?
	How reproducible is the work?
	How suitable is the work for the journal?
How should the report be written?	What summarizes the argument?
	What are the strengths and weaknesses?
	What comments/edits are required?
	What will improve the submission?
	What is the personal bias of the reviewer?
What is helpful advice for the editors?	Why might there be ethical concerns?
	Why is additional expertise needed?
	Why accept, reject or revise?

,

Table 3. Roles and corresponding responsibilities of reviewers for international peer-reviewed journals based on the answers to narrative research questions self-posed to the author as a reviewer.

Roles	Responsibilities
Ensure suitability as a reviewer	Declare any conflict of interest
	Determine if deadline can be met
	Establish expertise to do the review
	Maintain confidentiality
Use the journal guidelines for reviewers	Read the guidelines
	Follow the guidelines
Examine content of submission	Read the submission carefully
	Note clarity of writing
	Check references for accuracy
	Identify possible plagiarism
Evaluate submission	Establish title/abstract/keywords suitability
	Determine validity of question
	Assess correctness of method
	Gauge clarity of tables and figures
	Identify if limitations are provided
	Judge if conclusions are supported
	Establish adequacy of references
	Assess the originality of the work
	Consider if there are ethical concerns
	Determine if work is reproducible
	Consider suitability for journal’s audience
Write the report	Summarize the argument presented
	State the strengths and weaknesses
	Detail comments of suggested edits
	Help improve written presentation
	Comment without personal bias
Provide advice to editors	Indicate ethical concerns to editor
	Advise on further expertise required
	Accept, reject or call for revisions

,

Table 4. Data extracted from Table 1 and Table 2 of the 2019 publication, A Scoping Review on the Roles and Tasks of Peer Reviewers in the Manuscript Review Process in Biomedical Journals [10], in relation to ranking the 30 most common replies with respect to the number of replies by 209 authors of articles on the peer review process corresponding to the responsibilities of peer reviewers.

Rank	Responsibilities of Peer Reviewers According to Authors	#
1	Determine if interpretation is supported by the data	92
2	Provide constructive criticism	87
3	Improve manuscript	84
4	Timeliness: meet journal deadline	81
5	Recommendations on publication (e.g., no/minor/major revisions, reject)	74
6	Be expert in the subject area/matter/field and/or be familiar with/trained in research methods and statistics	70
7	Determine validity/quality/technical merit/rigor	69
8	Declare/avoid potential or actual conflict of interest	66
9	Assess adequacy of methods in general	65
10	Evaluate study design	56
11	Assess originality	55
11	Consider general ethical aspects and report on any specific ethical concerns (including manipulation of data, plagiarism, duplicate publication, inappropriate treatment of animal or human subjects)	55
12	Determine clarity of tables	54
12	Assess novelty	54
13	Consider adequacy of discussion (general)	53
14	Comment on interest to journal readership/relevance for journal scope	52
14	Appropriateness and accuracy of references	52
14	Maintain confidentiality of the manuscript, avoiding disclosure/discussion with others	52
15	Comment on clarity of study purpose and hypothesis	50
16	Assess importance/significance	48
17	Consider adequacy of results (general)	46
18	Comment upon relevance to practice/science (clinical relevance)	45
19	Comment upon contribution to the field	42
19	Assess data analysis (methods and tests)	42
19	Consider use of statistics	42
20	Be polite/courteous/respectful in the communication with authors	41
20	Determine overall readability	41
21	Assess presentation (general)	40
21	Advise editors on the merits of manuscripts	40
22	Be familiar with journal’s mission, review process, review criteria, guidelines (i.e., both author and reviewer guidelines) and forms prior to starting the review	39
22	Be fair: evaluate manuscript in a fair manner	39
23	Evaluate adequacy of introduction (in general)	37
23	Assess coherence/clarity and logical flow of the text	37
24	Be objective: objectively judge all aspects of the manuscript	36
24	Consider one’s time availability prior to accepting review request	36
25	Highlight whether the current literature is covered	35
25	Prior to accepting review request, determine whether the manuscript is within one’s area of expertise (only review manuscripts in one’s own field of expertise)	35
25	Be thorough/comprehensive/detailed/accurate	35
26	Assess sampling strategy	34
27	Consider clarity and validity of statistical methods	33
27	Determine how data were collected/reproducibility of methods	33
28	Be unbiased in their assessment: peer reviewers should have an unbiased attitude towards an author’s gender, previous work, institution and nationality	32
28	Provide confidential comments to editor	32
29	Identify strengths and weaknesses	31
30	Assess grammar and spelling	30

and

Table 5. Results of 446 biomedical editors’ and authors’ rated importance of 20 items assessing the quality of peer review reports using a 1–5 Likert scale conducted in 2020 in relation to the similar comparable rankings (C#) provided by the 209 authors on peer review in the 2019 study [20].

Rank	Responsibilities of Peer Reviewers According to Authors and Editors	C#
1	Knowledgeability	6
2	Methodological quality	–
3	Fairness	22
4	Constructiveness	2
5	Strengths and weaknesses (methods)	9
6	Interpretation results	1
7	Objectivity	24
8	Clarity	–
9	Strengths and weaknesses (general)	29
10	Statistical methods	19
11	Support by evidence	–
12	Detail/thoroughness	–
13	Tone	20
14	Applicability and external validity	19
15	Timeliness	4
16	Relevance	18
17	Structure of reviewer’s comments	–
18	Originality	11
19	Presentation and organization	21
20	Adherence to reporting guideline	22

to follow, constructed by the author for the purpose of this study. Table 2 represents the questions the author self-posed using a narrative research method (developed over the last ten years) in relation to what might be the roles and responsibilities of reviewers. Table 3 represents the answers the author provided to those questions divided into the roles and responsibilities of reviewers, while Table 4 is the result of investigating, selecting from, and combining the content of two tables from a 2019 publication on peer review specifically regarding biomedicine [10] using a systematic literature review to glean the information [28]. In comparison, Table 5 is the result of biomedical editors- and authors-rated importance of 20 items assessing the quality of peer review reports using a 1–5 Likert scale from a 2020 study [20]. No research was conducted on human subjects for this study; as a result, an ethical review of the research was not required.

2.1.1. Narrative Research Questions Posed Regarding Function and Process of Reviewing

Table 2 has been constructed regarding questions self-posed to the author with respect to the function and process of reviewing for the purpose of determining the author’s understanding of the roles and responsibilities of reviewers. Question-asking is a form of narrative research, a qualitative research method that extracts and analyzes stories (including scripts, texts, visual images) to understand people, cultures, and societies [29]. In this case the narrative research is in relation to the type of script the author uses for conducting reviews for the purpose of understanding the culture of reviewing.

The author has developed a particular method of narrative research [30,31,32,33] that depends on posing six different types of questions in a certain order: “when”, “where”, “who”, “what”, “how”, and “why”. This order is in relation to considering what is most objective to what is increasingly subjective as a response. In this regard, the first concern in determining the roles and responsibilities of peer reviewers regards questions related to “when”. These include: When should the review be completed? When were the references published? Next, the questions addressed involve “where”: Where are the references from? Where was research support obtained? After these questions are addressed, next comes “who” questions: Who are the authors to the reviewer? Who should conduct the review? Following these questions, “what” questions are posed: What expertise is needed for the review? What evaluates the submission? What is helpful advice for the editors? What guidelines are available? What process should be followed? What is the likelihood of plagiarism? What is the research question? What method was used? What are the limitations? What summarizes the argument? What are the strengths and weaknesses? What comments/edits are required? What will improve the submission? What is the personal bias of the reviewer? These questions represent aspects that are evident. The “how” and “why” questions are those requiring an increased level of analysis for their response. “How” questions include: How should the review be undertaken? How should the content be examined? How should the report be written? How confidential can the reviewer be? How should the submission be read? How clear is the writing? How suitable is the front matter? How clear are the tables and figures? How are the conclusions supported? How original is the work? How reproducible is the work? How suitable is the work for the journal? Lastly are the “why” questions: Why might there be ethical concerns? Why is additional expertise needed? Why accept, reject or revise?

With these questions, the most objective ones starting with “when” are those that can be answered by an initial examination of the submission. As the questions grow increasingly subjective, moving through each of the six types, a greater amount of time and thought must be committed by the reviewer in providing the answers. Yet, with respect to what is the relevant manner of ordering the answers to these questions for the purpose of this analysis of the roles and responsibilities of peer reviewers, time and effort in conducting the review is not the relevant issue. This is because the more important time-related concern is with regard to the order of the stages undertaken in conducting the review itself. It is in taking these stages into consideration that the questions posed to the author with respect to the functions and processes of peer review are those ordered to separate the functions of peer review from the process of undertaking any review. Table 2 has been constructed to situate each of the questions that were self-posed to the author regarding the functions and process of reviewing. These divide the review process into six different functions represented by questions concerned with (1) establishing the expertise of the reviewer before the review is begun, (2) considering how the review should be undertaken, (3) the manner in which the submission should be examined, (4) the process of evaluation, (5) writing the report and (6) the type of advice that it is helpful to provide to the editors. Each of the corresponding process-related questions is then listed in relation to one of these six functions.

Table 2 identifies that certain questions are asked at specific times during the review process. “Who” questions are asked at the start of the process, “what” questions define how the review should be undertaken and how the review should be written, while “why” questions are asked exclusively at the end of the process in relation to the information the author is expected to provide to the editors. “Where” and “how” questions, as well as additional “what” questions, in contrast, are intermingled and asked during more than one part of the peer review process.

2.1.2. Author’s View on the Roles and Responsibilities in Conducting Peer Reviews

Based on the questions posed using this narrative research method, the questions of Table 2 are answered in Table 3. These answers represent the second material for this study and will be used to evaluate the roles and responsibilities of peer reviewers with respect to the two articles concerning the roles and responsibilities of peers reviewers and each of the five publishing houses of international journals for which the author has conducted reviews over the six month period between 1 July 2022 and 31 December 2022.

The material of Table 3, in answer to the questions posed in Table 2, is based on the author’s own experiences as a frequent peer reviewer, and incorporates and refines previous research findings on tips for reviewers [34], the dos and don’ts of peer review [35], as well as the Committee on Publication Ethics (COPE) ethical guidelines for peer review [36]. Developed in this way, the material recognizes six roles of reviewers and, for each role, there are corresponding responsibilities in relation to the order in which each role and responsibility becomes evident when a reviewer is in the process of conducting a review. The various roles of being a reviewer during the process of conducting the review can be considered in connection with the differing relationships the reviewer has to the journal, editors and authors [37]; the responsibilities concern specific tasks associated with the role [38], where a “task” is defined in the Online Oxford English Dictionary as a piece of work undertaken as a duty [39].

Regarding the details of Table 3, the initial role of the reviewer concerns the reviewer’s self-assessment of whether a positive response can be provided to the editors regarding the reviewer’s ability to accept the invitation to review. The responsibilities associated with this role are in regard to four separate aspects: ethical, temporal, disciplinary and personal—i.e., whether the reviewer has a conflict of interest, can meet the deadline, has the expertise to conduct the review effectively, and can guarantee the prudence to refrain from discussing the content of the review.

The second role concerns the reviewer’s relationship to the journal, that is, the willingness of the reviewer to provide the necessary time and concentration to study the guidelines of the journal with respect to peer review. Once the guideless are read, the responsibility of the reviewer is to agree to follow those guidelines when conducting the review.

In the third role, the reviewer acts as an agent for the journal in examining the authors’ manuscript to assess whether it meets the minimum requirements for a submission. This assessment is done via reading through the manuscript carefully. The reviewer notes the clarity of the writing. Depending on the difficulty in deciphering the paper, an unclear presentation may be reason for rejection. However, the authors’ limited knowledge of the language employed by the journal does not represent a reason in itself for rejection [40]. The type of problem with clarity that would point to obvious rejection is if the arguments presented did not follow from one another or from the data presented. References that are cited and listed inaccurately would also point to rejection if the references do not state what the authors’ claim. A necessary point to establish quickly is if the reviewer identifies plagiarism in the submission. A finding of plagiarism must be brought to the attention of the editor expeditiously as this represents grounds for immediate rejection of the manuscript. Regarding this last reason for immediate rejection, this author reports never having come across a manuscript that was plagiarized in any review personally conducted; however, this is known to occur [41].

Following from the third role, the fourth role of reviewers is to judge the adequacy of the submission—a role that bridges the intent of the authors with the expectations of the journal. This bridging is done in relation to the order of the manuscript’s different sections. The reviewer can begin by checking the front matter to establish the suitability of the title, abstract, and the keywords. The relevance of the title is normally evident from the abstract, and each of the keywords are found in the abstract and listed in relation to the order in which they there appear. Reading through the introduction, the reviewer checks if it ends with a valid question based on the points made, and on the research presented in the introduction. The reviewer then reads through the method to assess its correctness, both with respect to how it is applied and that it is the appropriate method, given the research question. Any tables or figures included are gauged for their clarity of presentation and information provided. The discussion includes identified limitations and conclusions supported by the evidence provided. In finalizing reading through the paper, the reviewer needs to be cognizant of establishing the adequacy of references—especially whether the work cited in scientific publications has been published within the last five years, as references older than this (unless the work is seminal in the field or is supporting a historical point) are likely out of date. Once the submission has been examined fully, the reviewer then has the responsibility of assessing the originality of the work, considering if there are ethical concerns and determining if the work is reproducible. If any one of these three is lacking, this would involve the reviewer calling for a rejection of the article, as originality, meeting ethical standards, and reproducibility are the foundation of research suitable for publication in international peer-reviewed journals. Although the importance of originality may be questioned, editors of journals see this as a key concern [42]. With studies intended to determine the reproducibility of a previous study, the originality of the paper would be judged in relation to the authors’ assessment of the results from a current standpoint in relation to when the initial research results were published.

At this point in the review process, the reviewer assumes the fifth role—writing the report. This is a role that involves the relationship between the reviewer and the authors, and includes responsibilities undertaken in the following order: the reviewer summarizes the argument presented in one paragraph then states the paper’s strengths and weaknesses in a second paragraph. After this, detailed comments and suggested edits can be provided line by line, if line numbers are available. Otherwise, these comments and suggested edits can be referred to page by page. However, the structure of the journal review and submission system may dictate how the review is to be written. Furthermore, in the case of some multidisciplinary open access journals, the requirements for review are different in that they may not be confined to those of the biomedical and social sciences. The purpose of these comments and suggestions is to improve the paper, not to criticize nor embarrass the authors. The reviewer must be sure to maintain an objectively helpful manner in making comments, and that they are made without personal bias.

The final role of the reviewer corresponds to the relationship of the reviewer to the editor, and involves providing advice directly to the editor, some of which remains unseen by the authors. Those points that are private between the reviewer and the editor regard two things. The first is whether the reviewer has ethical concerns about the submission. These can range from a lack of informed consent obtained by the authors in conducting the research, to an inappropriate relationship recognized by the reviewer between the funding agency and the authors, or to improper self-citation by the authors to references that are irrelevant to the submission (although a rare occurrence, this author has come across improper self-citation as a reviewer). Advice provided to the editors by the reviewer on whether further expertise is required to give a balanced review is also unseen by the authors. This type of advice generally involves whether the reviewer is deficient in some area, such as not being a statistician (this author’s own deficiency as a reviewer). The advice to the editors that is seen by the authors is the final decision of the reviewer to accept, reject or call for revisions of the submission. This decision will be made in relation to the results of each of the previous responsibilities conducted by the reviewer. The first time a submission is assessed, it is unlikely to be accepted outright. Common problems are that the references chosen are either out of date for the current claims made by the authors, certain claims are not referenced, and/or there are no, or insufficient, limitations noted by the authors. In each case, the call will be for revision. Rejection is the assessment if there are major problems with the submission that cannot be fixed as a result of revision. Some major problems in this regard are plagiarism, the inappropriate use of a research method, the conclusions not following from the results, or the authors not following ethical protocol for a research study involving human subjects. In this author’s estimation as a reviewer, it is advisable to provide sufficient information to authors so that a paper might be revised even if there are a significant number of issues, rather than moving hastily to call for a rejection.

2.1.3. Ranking 209 Authors Views Regarding Peer Reviewers’ Roles and Responsibilities

Table 4 represents the third material to be used in assessing the roles and responsibilities of peer reviewers for international journals. It has been constructed by the author based on the results of a 2019 study of 209 authors of publications on the roles and tasks of peer reviewers [10] focused on biomedicine. Table 4 combines the data from two separate tables in relation to (1) the roles and (2) the tasks of peer reviewers, as viewed by various authors. Although these two tables from the 2019 publication are differentiated by their authors as roles and tasks, in examining the content of the tables, each relates rather to the responsibilities of a peer reviewer, as roles in the case of this 2019 article are merely more general responsibilities and tasks are more specific. In this respect, the 2019 publication does not actually refer to the roles of peer reviewers, as have been defined above in Section 1. Introduction. What has been extracted from those two tables from the 2019 publication is the thirty most common replies of these 209 authors. Table 4 lists the number of authors who mentioned a particular responsibility, providing the rank of that responsibility as a result.

2.1.4. Ranking of Views of the 20 Items Assessing the Quality of Peer Review Reports

A comparison between the 2019 publication concerning the views of 209 authors on what should be the responsibilities and tasks of peer review and a 2020 study of 446 biomedical editors and authors with respect to whether they endorse the study’s authors’ predesigned proposed definition of what makes a quality peer review report would appear, on the surface, to be relatively uncomplicated. However, there are differences between what was investigated in each study that must be taken into consideration. The differences include: (1) the first study in 2019 permitted authors to state what they believed were the primary roles and tasks of peer reviewers, while the second study only offered a choice among the paper’s authors’ predesigned criteria; (2) the first study concentrates on how to determine the suitability of the submission through peer review, while the second focuses on what makes a good peer review report; (3) the first study is specific to authors of articles on peer review, while the second is composed of authors who have experienced peer review and editors of journals, and (4) the 2019 study is detailed in the specific roles and tasks the authors provided, whereas the authors and editors of the 2020 study gave their rating to, for the most part, vague terms provided by the study’s authors. As a result of these variances, pronounced differences regarding what are considered the roles and responsibilities of peer reviewers in making this comparison can be expected.

Table 5—the fourth material for this study—is constructed from the publicly available web application mentioned in the qualitative results section of the 2020 paper (available at https://shiny-eio.upc.edu/MIRoR/ReportQuality/ (accessed on 26 April 2023) under the “item averages” tab) [20]. There, the twenty items provided to the participating authors and editors are ranked in relation to the responses received. It is notable that all items were rated highly, with a mean Likert score ranging from 3.38 to 4.60. Since all were rated highly, what is most interesting is that the ranking of the items is entirely different from the ranking provided in the 2019 publication with which they are compared. The contrast between the left and the right columns of Table 5 is extensive—there is not one ranking between the two that is similar. Furthermore, there are five responsibilities considered by the authors of the 2020 study to be necessary that do not appear in the ranking of the 2019 publication: “Methodological quality”, “Clarity”, “Support by evidence”, “Detail/Thoroughness”, and “Structure of reviewer’s comments”. However, the reason they are not represented in the 2019 listing from Table 4 is that these items are focused on evaluating the reviewer’s report, rather than the submission being reviewed. As such, methodological quality is not related to the submission under review—it is in regards to the method the reviewer uses to review the paper. Similarly, clarity is not regarding the clarity of the paper. Instead, it is the clarity of the comments of the reviewer, in that they are easy to read. Following from this, support by evidence does not mean that the reviewer is to note if the conclusion is supported by evidence. What it relates to is whether the reviewer has evidence for the critique being offered to the submission’s authors. Detail/thoroughness does not concern the detail and thoroughness of the submitted paper. It is, in contrast, the detail and thoroughness of the review itself. It is in examining these five responsibilities of reviewers that were assumed by the 2020 paper (see that paper’s Table 1: “The 20 items to assess peer-review (PR) report quality included in the survey” [20]) that it becomes clear the roles and responsibilities being investigated by the 2019 and 2020 paper are not readily comparable.

2.2. Methods

To examine the responsibilities of peer reviewers of the five publishing houses for which the author has reviewed in the last six months, the responsibilities itemized in Table 3 will be listed in

Table 6. Responsibilities of peers reviewers, grouped by roles set out in Table 3, and whether the responsibilities are identified by five publishing houses, by the order in which they occur in the review process, ranked in relation to the number of times authors on peer review recognize the responsibilities of peer reviewers as per Table 4, and concerning how important authors and editors believe certain predesigned attributes regarding peer review are as responsibilities in relation to Table 5.

Ranks	Responsibility of Peer Reviewers	Dove	Frontiers	MDPI	Springer	Wiley
8,—	Declare any conflict of interest	Yes	Yes	Yes	Yes	Yes
24,15	Determine if deadline can be met	Yes	Yes	Yes	Yes	Yes
25,1	Establish expertise to do the review		Yes	Yes	Yes	Yes
14,—	Maintain confidentiality	Yes	Yes	Yes		Yes
22,—	Read the guidelines	Yes	Yes	Yes	Yes
—,20	Follow the guidelines		Yes		Yes
—,—	Read the submission carefully	Yes		Yes	Yes	Yes
20,8	Note clarity of writing	Yes	Yes	Yes	Yes	Yes
14,—	Check references for accuracy		Yes			Yes
11,—	Identify possible plagiarism	Yes	Yes	Yes	Yes	Yes
—,—	Establish title/abstract/keywords suitability					Yes
15,—	Determine validity of question		Yes	Yes	Yes	Yes
9,10	Assess correctness of method	Yes	Yes	Yes	Yes	Yes
12,—	Gauge clarity of tables and figures	Yes	Yes	Yes		Yes
17,6	Judge if conclusions are supported	Yes	Yes	Yes	Yes	Yes
25,—	Establish adequacy of references		Yes	Yes		Yes
11,18	Assess the originality of the work	Yes	Yes	Yes	Yes	Yes
11,—	Consider if there are ethical concerns	Yes	Yes	Yes	Yes	Yes
—,—	Identify if limitations are provided				Yes
27,—	Determine if work is reproducible		Yes	Yes		Yes
14,16	Consider suitability for journal’s audience		Yes	Yes
—,—	Summarize the argument presented			Yes	Yes
29,5,9	State the strengths and weaknesses	Yes	Yes	Yes	Yes	Yes
25,—	Detail comments of suggested edits	Yes	Yes	Yes	Yes	Yes
3,4	Help improve written presentation	Yes	Yes	Yes	Yes	Yes
28,3,7	Comment without personal bias	Yes		Yes	Yes	Yes
28,—	Indicate ethical concerns to editor		Yes	Yes	Yes	Yes
—,—	Advise on further expertise required		Yes		Yes
5,—	Accept, reject or call for revisions	Yes	Yes	Yes	Yes	Yes

to follow, as they appear in Table 3, representing the order of these responsibilities in the manner they arise during the review process. The horizonal lines dividing the table group the responsibilities in relation to the six different roles of peer reviewers noted in Table 3. Whether each of the publishing houses clearly states a particular responsibility for reviewers in their website guidelines for reviewers will be indicted by a “Yes”. If the responsibility is not clearly stated, the space is left blank, as it would be misleading to conclude that the answer should be “No” merely because it is not definitely stated. The point is only to determine if the responsibility is clearly indicated, not whether it is assumed by the publishing house that authors will understand the importance of the unstated responsibility. In this regard, the publishing house may expect or prefer a particular responsibility, but not state it directly. The rank order established in Table 4 for the 2019 article then represents the first rank number in the left column of Table 6. The second rank number in that column corresponds to the rank in comparison to the results of Table 5.

The purpose of this method is to both compare the five publishing houses on their stated responsibilities of peer reviewers, and to do so in comparison with the roles and responsibilities arrived at by the author through narrative research (see the results of Table 2 and Table 3). Additionally, these responsibilities are compared to the rank in importance of the particular responsibility to that which was identified in the 2019 study of the 209 authors of articles on the peer review process, and in the 2020 study of the 446 authors and editors who gave their rankings for the roles and responsibilities of peer reviewers predesigned by the study’s authors. The point is to compare both the five publishing houses regarding responsibilities of peer reviewers and to do so with respect to what authors on peer review and authors and editors considering peer review think should be the importance of a particular responsibility. The publishing companies’ information is obtained from reading the guidelines on their respective webpages and looking for statements with the same words as the particular responsibility being searched for [23,24,25,26,27].

3. Results

The results to be presented will be with respect to Table 6 and be divided into three parts. The first will be regarding the ways in which the guidelines for peer reviewers of the five publishing houses explicitly state a particular responsibility for peer reviewers that this author has deemed important. The second part will be comparing the rank of the responsibilities with respect to the views of the 209 authors on the responsibilities of peer reviewers. The third part will contrast the rank of the responsibilities identified by the 446 authors and editors concerning what is important in evaluating peer reviewers. This will be done in order to see both how what the author deems important compares with these authors and/or editors, and which of the responsibilities expected of peer reviewers by the five publishing houses is given clear mention by them.

Generally, concerning both articles in comparison with the author’s specified responsibilities and those mentioned on the websites of the five publishing houses, there are five items on the author’s list that are not found mentioned in either the 2019 or the 2020 article: “Read the submission carefully”, “Establish title/abstract/keywords suitability”, “Identify if limitations are provided”, “Summarize the argument presented”, and “Advise on further expertise required”. What is interesting to note concerning these five is that the first three are the only responsibilities that are mentioned by one publishing house alone, while the last two are mentioned by only two publishing houses. There are no other cases of only one publishing house mentioning a particular responsibility on its website, and only one other case where two publishing houses alone mention the particular responsibility—“Consider suitability for journal’s audience”. In this case, both the 2019 and the 2020 articles’ respondents included this in their list of responsibilities. However, in neither case was the ranking within the top ten mentions.

3.1. Comparing Responsibilities of Peer Reviewers for the Five Publishing Houses

In comparing the “Yes” results of the five publishing houses of journals with which this author has reviewed in the final six months of 2022, it is evident that most of the responsibilities the author considers important are mentioned specifically in the responsibilities of each publishing house’s guidelines for peer reviewers. However, none of the publishing houses mentions all of the responsibilities; yet, there is no responsibility that is mentioned by none of the publishing houses.

There are a total of 29 individual responsibilities listed in Table 6. Of those, the number each of the publishing houses mentions explicitly is as follows: Dove—17, Frontiers—24, MDPI—24, Springer—22, Wiley—23. There were only 12 responsibilities that were mentioned by every publishing house, representing 41% of all of the responsibilities the author considers important regarding the responsibilities of peer reviewers. Of these unanimous mentions, the two roles from Table 3 most likely to have unanimous mentions among them are “Evaluate the submission” and “Write the report”. There were eight responsibilities mentioned by four of the publishing houses, two indicated by three of them, five stated explicitly by two of them, and two that were clearly recognized by only one of the publishing houses. It may be because the author’s particular views on what is important in the responsibilities of peer reviewers most correspond with those of Frontiers and MDPI that this author had been asked by journals associated with these two publishing houses to review substantially more articles in the latter half of 2022 than the other three publishing houses (see Table 1).

Regarding Table 6, how the guidelines on peer review of the five journals compare the 29 responsibilities recognized by this author is that there are 12 in which all publishing houses are in full agreement: “Declare any conflict of interest”, “Determine if deadline can be met”, “Note clarity of writing”, “Identify possible plagiarism”, “Assess correctness of method”, “Judge if conclusions are supported”, “Assess the originality of the work”, “Consider if there are ethical concerns”, “State the strengths and weaknesses”, “Detail comments of suggested edits”, “Help improve written presentation”, and “Accept, reject or call for revisions”. There are eight in which four of the five publishing houses mention a particular responsibility listed in the table: “Establish expertise to do the review”, “Maintain confidentiality”, “Read the guidelines”, “Read the submission carefully”, “Determine validity of question”, “Gauge clarity of tables and figures”, “Comment without personal bias”, and “Indicate ethical concerns to editor”. Two of the five publishing houses mention three of the responsibilities: “Establish adequacy of references” and “Determine if work is reproducible”. Five responsibilities are recognized by two publishing houses: “Follow the guidelines”, “Check references for accuracy”, “Consider suitability for journal’s audience”, “Summarize the argument presented”, and “Advise on further expertise required”. There are two instances where only one of the five publishing houses recognizes a particular responsibility listed on the table as part of their instructions for peer reviewers on their website: “Establish title/abstract/keywords suitability”, and “Identify if limitations are provided”.

3.2. Comparing the 2019 Rank of Responsibilities in Relation to the Five Publishing Houses

Before comparing the mentions of responsibilities by the five publishing houses with respect to the rank they are seen to have in amalgamating the views of 209 authors on peer review, it should be noted again that these 209 authors (see Table 4) showed low agreement among themselves. Although there were 209 authors, the highest number of those who agreed on the importance of any particular responsibility was 92. In other words, at best, less than half of the total number agreed. This means there is little concurrence in what responsibilities authors on peer review think should be included in the guidelines for reviewers. Furthermore, it is interesting to note also that six of the responsibilities thought important by this author were not mentioned by any of the 209 authors in the top 30 rankings.

Of the 11 unanimous mentions of responsibilities among the five publishing houses, their ranks from highest to lowest in comparison with the views of the 209 authors are as follows: 3, 5, 8, 9, 11 (with three ranking as 11), 17, 20, 24, 25 and 29. The responsibilities that were not unanimous, and which ranked somewhere in between those that were unanimous, included the following by rank and the number of publishing houses that mentioned the particular ranked responsibility: 12—4, 14—4, 14—2, 14—2, 15—4, 22—4, 25—4, 25—3, 27—3, 28—4 and 28—4 (two of the responsibilities had the same rank and same number of publishing houses that mentioned them).

3.3. Comparing the 2020 Rank of Responsibilities in Relation to the Five Publishing Houses

In investigating the relationship between (1) the ranking of responsibilities provided by the authors and editors surveyed regarding twenty predesigned estimations by the study’s authors of the responsibilities of peer reviewers, and (2) the responsibilities as ranked by this author as a frequent peer reviewer, it is evident that there is little that is clear or consistent. In most respects, the responsibilities put forward in the 2020 publication provided a less detailed description of the responsibility compared with what this author has provided. In contrast, for two cases—“State the strengths and weaknesses”, and “Comment without personal bias”—the 2020 publication made a greater distinction than this author. It is for this reason that both of these responsibilities listed on Table 6 represent two responsibilities from Table 5.

Unlike the rankings of the 2019 article, in which the majority of the responsibilities of peer reviewers thought important by those authors coincided with this author’s list, only 11 of the 29 responsibilities were thought important enough by the authors of the 2020 publication to ask authors and editors for their agreement, even though there were twenty items on these authors’ list. Seven of these responsibilities that corresponded with those of this author were ones that all of the five publishing houses included as part of the responsibilities of reviewers on their websites: “Determine if deadline can be met”, “Note clarity of writing”, “Assess correctness of method”, “Judge if conclusions are supported”, “Assess the originality of the work”, “State the strengths and weaknesses” and “Help improve written presentation”. Two of these responsibilities were recognized as important by four of the publishing houses: “Establish expertise to do the review” and “Comment without personal bias”. Two of the responsibilities are considered important by two of the publishing houses: “Follow the guidelines” and “Consider suitability for journal’s audience”. There is no publishing house that lists on their website all 12 of the responsibilities thought important for peer review by the 2020 article’s authors; however, both Frontiers and MDPI mention 11. MDPI does not highlight that peer reviewers should follow the guidelines, and Frontiers does not state that peer reviewers should “Comment without personal bias”.

4. Discussion

With respect to the results, this discussion will be divided into three parts. The first will be a comparison of the guidelines of the five different publishing houses. The second will concern the ranking of the responsibilities of peer reviewers with respect to the views of the 209 authors who have written on peer review, and the third will be regarding the 446 authors and editors who provided their rankings of twenty responsibilities of peer reviewers submitted to them. The limitations of this examination will then follow.

4.1. Guidelines of the Five Publishing Houses

Of the five publishing houses representing journals for which the author has reviewed articles in the last half of 2022, Dove is the least explicit in its guidelines for reviewers, while Frontiers and MDPI are both the most specific. How the latter two differ is that Frontiers is clear in telling reviewers they should follow the guidelines, while MDPI does not provide this advice directly. Instead, MDPI focuses on advising reviewers to read the submission carefully, which Frontiers does not. Nevertheless, Frontiers, unlike MDPI, states that the references should be checked for accuracy. In this regard, the way that Frontiers and MDPI differ seems to be with respect to their focus—Frontiers details a systematic approach to reviewing in all respects, while MDPI seems to want reviewers to attend foremost to assessing the value of the content of the manuscript. This hypothesis regarding the differing focus of these two publishing houses with respect to peer review is further supported by Frontiers providing no explicit responsibility to summarize the paper when writing the review, nor in reminding reviewers to comment without personal bias. Further justification is provided for concluding that MDPI is less concerned with reviewers taking an entirely systematic approach to reviewing in that it does not directly state that reviewers should let editors know when additional expertise is required to fully evaluate the submission.

It is only with respect to two of the responsibilities of peer reviewers thought important by this author that neither Frontiers nor MDPI provide explicit mention in their guidelines to peer reviewers: “Establish title/abstract/keywords suitability”, and “Identify if limitations are provided”. That reviewers should establish the suitability of the different parts of the front matter is evident in that there are instances when the title does not conform to the content of the submission, the abstract is poorly written, and/or the keywords are either not mentioned in the abstract or they are identified out of order (all keywords should be found in the abstract and listed in the order in which they there appear). These are problems the reviewer can bring to the attention of the authors during the review process, and the importance of doing so has been recognized by one author in 2017 [43]. If they are left for the editors to improve then the authors are not provided with an explanation regarding why the front matter is required to be changed—they are merely advised to make the changes by the copy editor. Still, the front matter will be attended to by the editors if the reviewer does not bring the proposed changes to the attention of the author. The importance of this responsibility differs from that of missing limitations. If the reviewer does not mention the need for a limitations section, the editors will not necessarily expect that the authors include one. This problem is a greater concern than that of the front matter deficiencies. Without a limitations section, the authors have not provided a balanced approach to the assessment of their research. This is a major flaw of any submission. It is for this reason that—although it is helpful if reviewers identify the need to correct the front matter of a paper—it is imperative that they request a limitations section if none is provided. This lack of providing direct instructions to reviewers in the guidelines of both the Frontiers and MDPI publishing houses concerning the need for a limitations section is an important omission that requires amendment. Of the five publishing houses, only Springer unequivocally states in its guidelines that reviewers should determine if a limitations section is provided.

Regarding the responsibility that reviewers should “Establish title/abstract/keywords suitability”, there is only one publishing house that states directly that reviewers must be mindful of examining these aspects of the front matter—Wiley. The Wiley publishing house in many respects is the most particular in the responsibilities it expects through its guidelines for reviewers. Why it falls a little short of Frontiers and MDPI in this regard is that, unlike these two publishing houses, it neither pointedly tells reviewers to read the guidelines nor to follow them. Perhaps this publishing house, which is very detailed regarding responsibilities in other ways, considers the need to read the guidelines and follow them to be self-evident. In contrast—from this author’s experience in reading reviews conducted by other reviewers of the same paper the author has been charged with reviewing (a privilege only available once the reviewer has submitted their own contribution to the review process)—many reviewers do not follow the guidelines, and this appears to be because they have not read them. Reviews based on evidence suggesting a lack of familiarity with the evaluating journal’s guidelines generally are neither helpful to the authors who have submitted the manuscript nor to the editors to whom the review is intended to provide guidance. In most other respects, the guidelines provided by Wiley take into consideration the necessary responsibilities as recognized by this author. What is missing is pointedly telling reviewers to: “Identify if limitations are provided”, “Consider suitability for journal’s audience”, “Summarize the argument presented” and “Advise on further expertise required”. In this regard, the Wiley publishing house appears to recognize the submission as an independent entity specific to one journal, rather than a contribution to the larger world of academic publishing.

Although, as mentioned, Springer is the only publishing house that clearly states in its guidelines that the reviewers should check for the limitations section, in other regards, this publishing house, and Dove, are both more relaxed about the requirements for the review process they specify in their guidelines. For Springer, this flexibility with respect to what is required by the reviewer extends primarily to the supporting details of the submission. Checking references for accuracy and adequacy is not distinctly stated, nor is gauging the quality of the tables and figures. That the adequacy of the references is not necessarily checked means that there is a potential for work to be published with dated references, leading to claims that cannot be currently supported. It is notable to mention that although Springer and Dove do not indicate checking the accuracy of references, only MDPI explicitly states references should be from the last five years—Wiley, in contrast, specifies the last ten years. This need for current research is important to the sciences, but generally does not extend to research published in the humanities where older publications continue to have research relevance [44]. This lack of focus on the details of the review process extends to there being no direct comment on the need for reviewers to determine if the study is reproducible. Dove, in being more informal in the guidelines to reviewers in general, does not direct reviewers to follow the guidelines, advise the editors on the ethical concerns of the paper, or indicate the reviewer’s own limitations as a reviewer. It is the only publishing house that does not ask that reviewers “Determine validity of question”. Reviewers are instead expected to provide a more general account of the strengths and weaknesses of the paper from a disciplinary perspective.

4.2. Ranking of Responsibilities of Peer Reviewers

In comparing the results of Table 4 with those of Table 6, it is immediately evident that the rankings 209 authors of papers on peer review gave to specific responsibilities considered necessary for peer reviewers differ substantially from the responsibilities this author considers important, and which are also found important by the five publishing houses. As pointed out previously, the 209 authors had little agreement among themselves. There is no responsibility that even half of the authors agreed is necessary. The largest agreement was among 92 authors. What is interesting with respect to Table 6 is that the two tasks that the 209 authors thought most important, “Determine if interpretation is supported by the data” and “Provide constructive criticism”, are not part of Table 6. The reason these points are not included is that they are both too broad and too vague. It is unclear in what way reviewers are to determine if the interpretation is supported by the data. Rather than provide this ambiguous statement, the data of Table 6 instead break down, into a number of different steps, how this task might be completed throughout the process of conducting the review. This breakdown includes each of the responsibilities under the role of “Evaluate Submission”, as laid out in Table 3. Similarly, “Provide constructive criticism” is too obscure a task for reviews to know what is wanted and what should be avoided. In this respect, Table 3, under the role of “Write the report”, outlines exactly what the responsibilities should be for the reviewer in this regard. When reviewers concentrate on their roles in evaluating the submission and writing the report, as detailed in Table 3, the need for the extraneous responsibilities that have first and second place in the rankings of Table 4 is eliminated.

Nevertheless, there is important agreement among the rankings of the 209 authors and the five publishing houses if the rankings on Table 6 that appear in the top 11 of Table 4 are examined, as they are the few for which all the publishing houses explicitly include the responsibility as part of their guidelines. That said, only the ranks of 3, 5, 8, 9 and 11 appear. The others between 1 and 11 from Table 4 are not found on Table 6. For 4, 6, 7 and 10, the reason they do not appear has more to do with how the authors (of the 2019 article in which the results were found) have made particular distinctions. Regarding the responsibility ranked as 4, “Timeliness: meet journal deadline”, this is very similar to “Determine if deadline can be met”, which ranks as item 24. As the publishing houses refer to the need for authors to determine if the deadline can be met, this coincides with the responsibility set out by this author in Table 3, rather than the idea of timeliness specified by the authors of the 2019 paper. However, they likely represent the same point, and should not be differentiated as they are in the 2019 paper. The responsibility ranked as 6 in Table 4, “Be expert in the subject area/matter/field and/or be familiar with/trained in research methods and statistics”, similarly is very close to “Establish expertise to do the review”, which ranks 25 in Table 4. Given that the point ranked sixth conflates a number of issues together and would have been better broken down into separate points by the 2019 authors of the study—yet it contains the idea of the reviewer establishing their expertise to do the review—the responsibility of establishing expertise is one considered important in both Table 4 and Table 6. Again, “Determine validity/quality/technical merit/rigor”, which represents rank 7 on Table 4, contains more than one task and, therefore, would have needed to be broken into more than one point by the 2019 authors. With more than one task as part of rank 7, it is difficult to tell how this task differs from “Assess adequacy of methods in general”, which ranks as 9. Finally, “Evaluate study design”, ranking as 10 in Table 4, is too general to be a helpful instruction to peer reviewers. As with the item ranking 1 discussed above, responsibility 10 is too broad, and reviewers are better informed to undertake each of the responsibilities in the role of “Evaluate Submission” from Table 3.

There are six responsibilities listed on Table 6 that have no corresponding rankings from Table 4. These are, “Follow the guidelines”, “Read the submission carefully”, “Establish title/abstract/keywords suitability”, “Identify if limitations are provided”, “Summarize the argument presented”, and “Advise on further expertise required”. Generally, these responsibilities initially laid out in Table 3 are not ones expressly stated in the guidelines and are found in fewer of the five publishing houses, with only one or two of these publishing houses directly advising reviewers to undertake these responsibilities. The exception is “Read the submission carefully”—four of the five publishing houses make a point of telling this to their peer reviewers in their guidelines. That none of the 209 authors on peer review thought it important to mention that the guidelines for peer review of the particular journal should be followed, and that few of the publishing houses include this advice in their guidelines, perpetuates the idea voiced by authors over the last fifteen years [1,4,10,12,13,19,21,23,45,46,47,48,49] that peer reviewers need training (an idea that is not supported by most journal editors [27]). This is in contrast to expecting that they merely are obligated to read the guidelines to know what is important in conducting a peer review. Similar to the responsibility of “Establish title/abstract/keywords suitability”, neglecting to provide “Advise on further expertise required” prolongs the process of peer review. For this reason, even though neither the 209 authors nor most of the publishing houses exactly state these as responsibilities of peer reviewers (in that they do not explicitly state them), there is insufficient guidance on including these ways of reducing the time between submission and decision. The neglect of most publishing houses in this study to decidedly tell reviewers in their guidelines that they should “Summarize the argument presented” means that the authors of the submission and the editors cannot be sure that the reviewer has understood the content of the manuscript—potentially leading to additional confusion and wasted time in the review process. This is why it is problematic that this responsibility did not receive a ranking in the study of 209 authors. Yet, by far the biggest omission by both those 209 authors in the 2019 study and the various publishing houses is regarding “Identify if limitations are provided”. Only Springer specifies that reviewers need to ensure that there is a limitations section to the submission under review. That none of the 209 authors considered this important and four of the publishing houses do not explicitly include this information in their guidelines means there is a substantial amount of research published that may not have been scrutinized to ensure limitations are covered in the discussion.

4.3. Ranking of Responsibilities by Authors and Editors

The results of the 446 biomedical editors and authors rated the importance of 20 items assessing the quality of peer review reports using a 1–5 Likert scale conducted in 2020, in comparison with the ranking provided by the 209 authors on peer review in the 2019 study, are evidenced in Table 5. What is clear is that not only is there a difference in how the responsibilities are rated regarding the two studies, but also, 5 of the 20 responsibilities recognized by the authors of the 2020 study were not part of the top 30 responsibilities provided by authors on peer review in 2019. It should be noted that this may in large part be due to the 2020 paper studying how authors and editors thought peer reviewers’ reports should be judged, rather than what their responsibilities are with respect to peer review. As well, unlike the 2019 publication, where the authors of articles on peer review presented their own views on what are the responsibilities of peer reviewers, the 2020 study presented authors and editors with twenty responsibilities that the study’s authors thought important, and asked the participants to rate their level of importance. That these responsibilities might not have been those chosen by the authors and editors who participated in the study is recognized by a number of them providing their own ideas regarding what should be the responsibilities (suggestions by the participants are noted in by the authors of the 2020 paper in their online supplementary file 5 [20]).

What is interesting to note about the responsibilities provided to these authors and editors to judge in the 2020 study is that all except one of the top ten responsibilities are part of those first recognized by this author in Table 3. The one that is not is that which is ranked second: “Methodological quality”. It is not included because methodological quality with respect to the 2020 study regards the methodological quality of the review done by the reviewer, not the method used by the author of the submission under review. That there was so much agreement between the authors’ list of responsibilities for peer reviewers developed through the narrative research process of self-questioning detailed in Table 2 and the group of responsibilities developed by the authors of the 2020 publication is particularly noteworthy, as these two understandings of peer review were developed for different reasons. This author’s purpose was to establish the roles and responsibilities of peer reviewers in the process of reviewing a submission. The 2020 article, in contrast, was created as a means of rating peer reviewers’ reports with respect to their responsibilities. On the other hand, a judgement had to be made by this author to establish when there was a commonality. This is because, in a number of cases, the responsibilities listed in the 2020 paper were vague and could include the author’s more detailed responsibility merely for this reason. An example is the highest ranking item on the 2020 list, “Knowledgeability” (see Table 5). This was translated on Table 6 as, “Establish expertise to do the review”. While it is clear that one cannot be an expert and lack knowledge, to have “knowledgeability” could mean a number of things besides expertise in the area to be reviewed. Similarly, there were two items on the author’s list that were more inclusive than those provided by the 2020 publication: “State the strengths and weaknesses”, and “Comment without personal bias” (see Table 6). “State the strengths and weaknesses” in the 2020 article related to both “Strengths and weaknesses (methods)” and “Strengths and weaknesses (general)” as seen in Table 5. As well, “Comment without personal bias” has been interpreted to include both “Fairness” and “Objectivity”, as is evident from Table 5.

4.4. Limitations

The primary limitation of this study is that the material of Table 2 and Table 3 is based on the author’s own perception as a frequent reviewer of various international journals. Although this author’s view is informed by narrative research [29,30,31,32,33] and a number of sources [34,35,36], it still represents the summation of only one reviewer. Why this is a problem is that reviewers, by the nature of their task, work alone. As such, they do not necessarily communicate their process of reviewing to others and receive feedback on the strengths and weaknesses of their established process. That said, when reviewers finish a review, they are given the opportunity by the journal to read the reviews of the other reviewers who have peer-reviewed the same manuscript. Usually, this means the reviewer is permitted to read two additional reviews. It is as a result of always taking the opportunity to compare reviews that this author can consider the material created for this study (in Table 2 and Table 3) represents a meaningful and useful summation of the roles and responsibilities of reviewers.

Another limitation of this work is that research related to the views of authors on peer reviews concerning their roles and responsibilities was pertaining to one study conducted in 2019 of 209 authors of articles on peer review and another on the responsibilities of a peer reviewer in writing their report as considered by 446 authors and editors. This reliance on only these two sources for data was especially problematic because, in the first study, the authors of that paper differentiated the responsibilities and tasks of peer reviewers and did not consider the roles (although they believed they did in recognizing their distinction between roles and tasks). As such, by here evaluating the tasks and roles voiced by these authors as each being subsumed under responsibilities for the purpose of this evaluation, this author may have extended the context of responsibilities further than the 209 authors would have assumed. Furthermore, without the 2019 paper actually identifying the roles of reviewers (as defined in this paper’s Introduction), it remains unclear how the 209 authors conceptualized the roles of reviewers, and this could not be assessed. Further research in this regard could go back to the individual articles of the 209 authors and specifically look for what each has said about the roles of peer reviewers in contrast to responsibilities (or tasks). Regarding the second study conducted in 2020, the authors considered the responsibilities of peer reviewers from a different perspective than was required for this particular analysis, given that the authors and editors were rating peer reviewers’ reports rather than considering what should be their roles and responsibilities when judging a submission. Therefore, a significant degree of interpretation was needed to identify in what way these studies could be compared, and the author may have erred in identifying in what way this 2020 paper could be compared with the 2019 publication and with the author’s identified roles and responsibilities of peer reviewers.

An additional limitation is that a comparison of the advice to reviewers offered by each of the five publishing houses on their websites has been conducted with respect to the list of roles and responsibilities developed by the author through narrative research. This is in contrast to listing all the points made on the five websites as advice to reviewers. In this regard, for the purpose of this study, the author’s created roles and responsibilities listed in Table 3 are the standard for comparison. As such, this research is not concerned with the extent of the roles and responsibilities for reviewers described by the five publishing houses. Rather, at issue is the creation of a rubric to compare the five publishing houses’ views on the roles and responsibilities of the peer reviewers with the articles from 2019 and 2020. On the other hand, that a table was not included and comparison undertaken of all of the advice that the five publishing houses provide to peer reviewers remains a limitation of this work

One aspect of the peer review process that was not considered was whether it makes a difference to the roles and responsibilities of peer reviewers for the journal to have a blind review, or if the journal expects open identities and/or open reports [15] (p. 48). This possibility was not considered because, with respect to the journals for which this author reviews, most have the option of choosing whether the reviewer would prefer an open review or does not want to make their identity known. In this author’s own experience, the transparency of the review has been irrelevant to the roles and responsibilities the reviewer should assume—a view that has been supported by research that, however, also notes there may be a difference in how scholars in the humanities and social sciences view open journals in comparison with the sciences [50]. As such, some of the 209 reviewers and the 446 authors and editors who participated in the 2019 study may not have held the same view regarding whether open review changes the roles and responsibilities of peer reviewers. That their view on this matter is unknown is a limitation.

Following from this, another limitation of this work is that the roles and responsibilities that have been suggested do not readily extend to research published in the humanities, as the citation-based research widely used in the physical, biomedical and social sciences as measures for research performance is opposed in the humanities [51]. There has been no recent research on the roles and responsibilities specific to peer reviewers in the humanities, yet humanities research differs in important ways from that of the biomedical and social sciences, with quality assessment in the humanities remaining highly controversial [49]. Furthermore, unlike in the sciences, plagiarism persists as a primary concern of peer review in humanities research, and is an increasing cause for retraction among scholarly publications [52]. As well, in the humanities, publishing in books is more common, with international journals having a less prominent role and the citing of older literature remaining important [53]—this in contrast to the demand for cited research to be published in the last five years in the sciences. This study, therefore, can only be considered with respect to that which was examined, i.e., the roles and responsibilities of peer reviewers within the biomedical sciences. As such, it must be recognized that there is a complexity of disciplinary differences that impact the peer review process, and these differences may necessarily require differences in the roles and responsibilities of peer reviewers in international journals. Furthermore, it is notable that peer review in mathematics differs from that of either the sciences or the humanities. Generally, only one peer reviewer is used, and these reviewers do not check the submission line by line; yet, peer review in mathematics has been considered the most reliable, in spite of this less detailed process [54].

As a further limitation regarding this complexity, regardless of the expertise this author may have in assessing the roles and responsibilities of peer preview, this contribution remains that of a single researcher and, given the broad nature of disciplines, research, publication and peer review, this work can only represent one aspect of the discussion. International organizations, such as COPE, are best placed to consider interdisciplinary concerns for the development of roles and responsibilities of peer review. Thus, although this paper can contribute to that work, it does not claim to provide a definitive list of the roles and responsibilities of peer review.

A final limitation of this study is the assumption that when authors receive responses from peer reviewers, they make use of the advice received. This presupposition may itself be questionable. In a recent study [55], it was found that 80% of authors who receive rejections from one journal publish their work in another journal without necessarily taking the advice into consideration of the reviewers who rejected their submission before submitting their paper for publication elsewhere. It is considered that this may result from the authors believing that the peer reviews they received in rejection are particular to the journal, and not a reflection of the quality of their work. If authors are not treating peer reviews as worthy of consideration when their work is rejected, then this may also be a reason for the decrease in the pool of peer reviewers that has not been considered.

5. Conclusions

Maintaining the quality of peer review, especially when the number of peer reviewers in comparison to submissions is decreasing, is necessary if the peer review process is to remain effective. Various authors since 2007, as noted in Section 4.2. Ranking of Responsibilities of Peer Reviewers, have considered that the best way to do this is to require all potential peer reviewers to undertake peer review training. Although this suggestion may establish consistency in the requirements for peer review, if the curriculum for peer review were standardized—and following it demanded—it neglects to recognize the efforts of journals in making their requirements for peer review readily accessible, transparent, extensive, detailed, and particular regarding their journal’s needs. As such, rather than necessitating that peer reviewers be trained, journals need only make their guidelines required reading for peer reviewers, and advise peer reviewers to follow them when conducting the review.

Nevertheless, although there is much to be commended about the guidelines for reviewers offered by each of the five publishing houses examined, in some cases, these guidelines still do not highlight a few key components of peer review in the sciences. The first, the importance of author’s identifying research from the last five years, has not been stressed sufficiently to peer reviewers by most publishing houses. Furthermore, peer reviewers have not been asked to focus on the vital importance of the limitation section to submissions. In this regard, peer reviewers have scrutinized submissions without effective attention being given to requiring that authors assess the weaknesses of their research. In following a recommendation to publishing houses that they include a focus on these important aspects, the current value of international publications and their veracity will be increased.

With respect to successfully increasing the pool of reviewers, recognition that does not focus on rewards in relation to the role of peer reviewer—a role that is not the primary focus of academics—is the recommended solution. This is because the more substantial concern of scholars is the publication of their own research, so increasing the pool should be based on promoting academic scholarship, not professionalizing the role of reviewer. Although one author has argued that there should be no obligation to review papers, as this demand may produce ill-conceived reports that are “misleading and useless” [40], this present author suggests, in contrast, that each corresponding author should review at least one paper in their field before their accepted publication goes to press. Although this idea is in opposition to that of the abovementioned author, it is not a new idea. Furthermore, it has even been proposed that authors of a submission should review three other papers to compensate for the three reviews they have received in regard to their submission during the review process [12,56]. However, such a stringent requirement, although it would significantly increase the number of peer reviewers available, would at the same time increase the time to decision to a degree that likely would be detrimental to prompt publication in international peer reviewed journals. Since the actual purpose of peer review is to determine suitability for publication in a timely manner, rather than to calculate a fair distribution of hours devoted to peer reviewing among reviewers, it would seem reasonable that the publication of research not be slowed down unnecessarily by expecting all authors with an accepted manuscript to then each review what would normally be three other manuscripts to match the number of peer reviews they received in the acceptance of their own submission. As a growing amount of scientific research is conducted in large teams [57], there may be a very large number of authors for a particular publication. If, before an accepted paper were published, every one of these authors was expected to conduct a peer review, the focus of the role of the journal would change from publishing to administering peer reviews. Therefore, the suggestion is to have only the corresponding author of an accepted publication conduct a peer review before publication, to keep the administration of these peer reviews manageable. This is a suggestion supported by recent research published in this journal [58]. In this way, the pool of qualified peer reviewers would necessarily be increased, while maintaining the aim of expeditious publication.

In this tenth anniversary issue of Publications, it is hoped that this focus on the roles and responsibilities of peer reviewers for international journals will exemplify the importance the peer review process continues to have for international journal publication, and this effort presented will be identified as an effective means for making these roles and responsibilities evident, while at the same time increasing the pool of qualified peer reviewers.