Roles and Responsibilities for Peer Reviewers of International Journals
Abstract
:1. Introduction
2. Materials and Methods
2.1. Materials
2.1.1. Narrative Research Questions Posed Regarding Function and Process of Reviewing
2.1.2. Author’s View on the Roles and Responsibilities in Conducting Peer Reviews
2.1.3. Ranking 209 Authors Views Regarding Peer Reviewers’ Roles and Responsibilities
2.1.4. Ranking of Views of the 20 Items Assessing the Quality of Peer Review Reports
2.2. Methods
3. Results
3.1. Comparing Responsibilities of Peer Reviewers for the Five Publishing Houses
3.2. Comparing the 2019 Rank of Responsibilities in Relation to the Five Publishing Houses
3.3. Comparing the 2020 Rank of Responsibilities in Relation to the Five Publishing Houses
4. Discussion
4.1. Guidelines of the Five Publishing Houses
4.2. Ranking of Responsibilities of Peer Reviewers
4.3. Ranking of Responsibilities by Authors and Editors
4.4. Limitations
5. Conclusions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Hoffman, A.J. A modest proposal to the peer review process: A collaborative and interdisciplinary approach in the assessment of scholarly communication. Res. Ethics 2022, 18, 84–91. [Google Scholar] [CrossRef]
- Fyfe, A.; Squazzoni, F.; Torny, D.; Dondio, P. Managing the growth of peer review at the Royal Society journals, 1865–1965. Sci. Technol. Human Values 2020, 45, 405–429. [Google Scholar] [CrossRef] [Green Version]
- Buela-Casal, G.; Perakakis, P.; Taylor, M.; Checa, P. Measuring internationality: Reflections and perspectives on academic journals. Scientometrics 2006, 67, 45–65. [Google Scholar] [CrossRef]
- Tennant, J.P.; Ross-Hellauer, T. The limitations to our understanding of peer review. Res. Integr. Peer Rev. 2020, 5, 6. [Google Scholar] [CrossRef] [PubMed]
- Witt, S. Global Trends in Knowledge Production and the Evolving Peer Review Process. JLIS.It 2023, 14, 1–9. [Google Scholar] [CrossRef]
- Bell, K. (Ed.) Role. Open Education Sociology Dictionary. 2013. Available online: https://https://sociologydictionary.org/role/ (accessed on 19 February 2023).
- Responsibility, n. OED Online Oxford University Press. 2022. Available online: https://www.oed.com/view/Entry/163862 (accessed on 19 February 2023).
- Willis, M. Why do peer reviewers decline to review manuscripts? A study of reviewer invitation responses. Learn. Publ. 2016, 29, 5–7. [Google Scholar] [CrossRef]
- Grimaldo, F.; Marušić, A.; Squazzoni, F. Fragments of peer review: A quantitative analysis of the literature (1969–2015). PLoS ONE 2018, 13, e0193148. [Google Scholar] [CrossRef] [Green Version]
- Glonti, K.; Cauchi, D.; Cobo, E.; Boutron, I.; Moher, D.; Hren, D. A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals. BMC Med. 2019, 17, 118. [Google Scholar] [CrossRef] [Green Version]
- Brezis, E.S.; Birukou, A. Arbitrariness in the peer review process. Scientometrics 2020, 123, 393–411. [Google Scholar] [CrossRef] [Green Version]
- Grainger, D.W. Peer review as professional responsibility: A quality control system only as good as the participants. Biomaterials 2007, 28, 5199–5203. [Google Scholar] [CrossRef]
- Pomponi, F.; D’Amico, B.; Rye, T. Who Is (Likely) Peer-Reviewing Your Papers? A Partial Insight into the World’s Top Reviewers. Publications 2019, 7, 15. [Google Scholar] [CrossRef] [Green Version]
- Blockeel, C.; Drakopoulos, P.; Polyzos, N.P.; Tournaye, H.; García-Velasco, J.A. Review the ‘peer review’. Reprod. Biomed. Online 2017, 35, 747–749. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Publons|Clarivate Analytics. Global State of Peer Review. 2018. Available online: https://publons.com/static/Publons-Global-State-Of-Peer-Review-2018.pdf (accessed on 9 February 2023).
- Fox, C.W.; Albert, A.Y.; Vines, T.H. Recruitment of reviewers is becoming harder at some journals: A test of the influence of reviewer fatigue at six journals in ecology and evolution. Res. Integr. Peer Rev. 2017, 2, 3. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Warne, V. Rewarding reviewers–sense or sensibility? A Wiley study explained. Learn. Publ. 2016, 29, 41–50. [Google Scholar] [CrossRef] [Green Version]
- Zaharie, M.A.; Seeber, M. Are non-monetary rewards effective in attracting peer reviewers? A natural experiment. Scientometrics 2018, 117, 1587–1609. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Paul, J.; Lim, W.M.; O’Cass, A.; Hao, A.W.; Bresciani, S. Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). Int. J. Consum. Stud. 2021, 45, O1–O16. [Google Scholar] [CrossRef]
- Superchi, C.; Hren, D.; Blanco, D.; Rius, R.; Recchioni, A.; Boutron, I.; González, J.A. Development of ARCADIA: A tool for assessing the quality of peer-review reports in biomedical research. BMJ Open 2020, 10, e035604. [Google Scholar] [CrossRef]
- Hope, A.A.; Munro, C.L. Criticism and Judgment: A Critical Look at Scientific Peer Review. Am. J. Crit. Care 2019, 28, 242–245. [Google Scholar] [CrossRef] [Green Version]
- BMC, Research in Progress. Available online: https://www.biomedcentral.com (accessed on 26 April 2023).
- Writing a Peer Review. Available online: https://www.dovepress.com/peer-review-guidelines/writing-peer-reviews (accessed on 21 January 2023).
- Peer Review. Available online: https://www.frontiersin.org/about/peer-review (accessed on 21 January 2023).
- Guidelines for Reviewers. Available online: https://www.mdpi.com/reviewers#_bookmark1 (accessed on 21 January 2023).
- Advancing Peer Review at BMC. Available online: https://www.biomedcentral.com/about/advancing-peer-review (accessed on 21 January 2023).
- Step by Step Guide for Reviewing a Manuscript. Available online: https://authorservices.wiley.com/Reviewers/journal-reviewers/how-to-perform-a-peer-review/step-by-step-guide-to-reviewing-a-manuscript.html (accessed on 21 January 2023).
- Snyder, H. Literature review as a research methodology: An overview and guidelines. J. Bus. Res. 2019, 104, 333–339. [Google Scholar] [CrossRef]
- Wolgemuth, J.R.; Agosto, V. Narrative Research. In The Blackwell Encyclopedia of Sociology; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2019; pp. 1–3. [Google Scholar] [CrossRef]
- Nash, C. Report on Digital Literacy in Academic Meetings during the 2020 COVID-19 Lockdown. Challenges 2020, 11, 20. [Google Scholar] [CrossRef]
- Nash, C. Online Meeting Challenges in a Research Group Resulting from COVID-19 Limitations. Challenges 2021, 12, 29. [Google Scholar] [CrossRef]
- Nash, C. Enhancing Hopeful Resilience Regarding Depression and Anxiety with a Narrative Method of Ordering Memory Effective in Researchers Experiencing Burnout. Challenges 2022, 13, 28. [Google Scholar] [CrossRef]
- Nash, C. A Framework for European Thought on Psychology, Education, and Health Based on Foucault’s The Order of Things. Histories 2022, 2, 222–240. [Google Scholar] [CrossRef]
- Azer, S.A.; Ramani, S.; Peterson, R. Becoming a peer reviewer to medical education journals. Med. Teach. 2012, 34, 698–704. [Google Scholar] [CrossRef] [PubMed]
- Rees, L.; Flynn, J. Supportive peer review: Why and how to constructively review a paper. Pediatr. Nephrol. 2022, 37, 1701–1703. [Google Scholar] [CrossRef] [PubMed]
- COPE Council. COPE Ethical Guidelines for Peer Reviewers—English; COPE Council: New York, NY, USA, 2017. [Google Scholar] [CrossRef]
- Glonti, K.; Hren, D. Editors’ perspectives on the peer-review process in biomedical journals: Protocol for a qualitative study. BMJ Open 2018, 8, e020568. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Glonti, K.; Boutron, I.; Moher, D.; Hren, D. Journal editors’ perspectives on the roles and tasks of peer reviewers in biomedical journals: A qualitative study. BMJ Open 2019, 9, e033421. [Google Scholar] [CrossRef]
- Task, N. OED Online Oxford University Press. 2022. Available online: www.oed.com/view/Entry/198017 (accessed on 20 February 2023).
- Buchanan, N.T.; Perez, M.; Prinstein, M.J.; Thurston, I.B. Upending racism in psychological science: Strategies to change how science is conducted, reported, reviewed, and disseminated. Am. Psychol. 2021, 76, 1097–1112. [Google Scholar] [CrossRef]
- Else, H.; Van Noorden, R. The fight against fake-paper factories that churn out sham science. Nature 2021, 591, 516–519. [Google Scholar] [CrossRef]
- Warner, T.A. How to write an effective peer-review report: An editor’s perspective. Int. J. Remote Sens. 2019, 40, 4871–4875. [Google Scholar] [CrossRef]
- Pytynia, K.B. Why Participate in Peer Review as a Journal Manuscript Reviewer: What’s in It for You? Otolaryngol. Neck Surg. 2017, 156, 976–977. [Google Scholar] [CrossRef]
- Chávez-García, M. Strategies for Publishing in the Humanities: A Senior Professor Advises Junior Scholars. J. Sch. Publ. 2017, 48, 199–220. [Google Scholar] [CrossRef]
- Drvenica, I.; Bravo, G.; Vejmelka, L.; Dekanski, A.; Nedić, O. Peer Review of Reviewers: The Author’s Perspective. Publications 2019, 7, 1. [Google Scholar] [CrossRef] [Green Version]
- Teplitskiy, M.; Acuna, D.; Elamrani-Raoult, A.; Körding, K.; Evans, J. The sociology of scientific validity: How professional networks shape judgement in peer review. Res. Policy 2018, 47, 1825–1841. [Google Scholar] [CrossRef] [Green Version]
- Barroga, E. Innovative Strategies for Peer Review. J. Korean Med Sci. 2020, 35, e138. [Google Scholar] [CrossRef]
- Moher, D.; Altman, D.G. Four proposals to help improve the medical research literature. PLoS Med. 2015, 12, e1001864. [Google Scholar] [CrossRef] [Green Version]
- Cintas, P. Peer review: From recognition to improved practices. FEMS Microbio. Lett. 2016, 363, fnw115. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Bravo, G.; Grimaldo, F.; López-Iñesta, E.; Mehmani, B.; Squazzoni, F. The effect of publishing peer review reports on referee behavior in five scholarly journals. Nat. Commun. 2019, 10, 322. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ochsner, M.; Hug, S.E.; Daniel, H.-D. (Eds.) Research Assessment in the Humanities: Introduction. In Research Assessment in the Humanities: Towards Criteria and Procedures; Springer Open: Berlin/Heidelberg, Germany, 2016; pp. 1–12. [Google Scholar] [CrossRef]
- Halevi, G. Why articles in arts and humanities are being retracted? Pub. Res. Quart. 2020, 36, 55–62. [Google Scholar] [CrossRef]
- Aksnes, D.W.; Langfeldt, L.; Wouters, P. Citations, citation indicators, and research quality: An overview of basic concepts and theories. Sage Open 2019, 9, 1–17. [Google Scholar] [CrossRef] [Green Version]
- Andersen, L.E. On the nature and role of peer review in mathematics. Account. Res. 2017, 24, 177–192. [Google Scholar] [CrossRef] [PubMed]
- Crijns, T.J.; Ottenhoff, J.S.; Ring, D. The effect of peer review on the improvement of rejected manuscripts. Account. Res. 2021, 28, 517–527. [Google Scholar] [CrossRef] [PubMed]
- Schultz, D.M. How to Be a More Effective Reviewer. Mon. Weather. Rev. 2022, 150, 1201–1205. [Google Scholar] [CrossRef]
- Wu, L.; Wang, D.; Evans, J.A. Large teams develop and small teams disrupt science and technology. Nature 2019, 566, 378–382. [Google Scholar] [CrossRef]
- Raoult, V. How Many Papers Should Scientists Be Reviewing? An Analysis Using Verified Peer Review Reports. Publications 2020, 8, 4. [Google Scholar] [CrossRef] [Green Version]
# | Journal | Dove | Frontiers | MDPI | Springer | Wiley |
---|---|---|---|---|---|---|
11 | BMC Medical Education | 11 | ||||
20 | International Journal of Environmental Research and Public Health | 20 | ||||
3 | Challenges | 3 | ||||
6 | BMC Public Health | 6 | ||||
1 | Risk Management and Healthcare Policy | 1 | ||||
2 | Scientific Reports | 2 | ||||
2 | Hygiene | 2 | ||||
27 | Frontiers in Psychology | 27 | ||||
1 | Theoria: A Journal of Social and Political Thought | 1 | ||||
3 | Frontiers in Education | 3 | ||||
6 | Frontiers in Psychiatry | 6 | ||||
1 | Frontiers in Public Health | 1 | ||||
2 | Scientia Pharmaceutica | 2 | ||||
1 | Nursing Reports | 1 | ||||
86 | Totals | 1 | 37 | 30 | 17 | 1 |
Fulfilling the Function of Reviewer | Reviewing the Submission |
---|---|
What expertise is needed for the review? | Who are the authors to the reviewer? |
When should the review be completed? | |
Who should conduct the review? | |
How confidential can the reviewer be? | |
How should the review be undertaken? | What guidelines are available? |
What process should be followed? | |
How should the content be examined? | How should the submission be read? |
How clear is the writing? | |
Where are the references from? | |
What is the likelihood of plagiarism? | |
What evaluates the submission? | How suitable is the front matter? |
What is the research question? | |
What method was used? | |
How clear are the tables and figures? | |
What are the limitations? | |
How are the conclusions supported? | |
When were the references published? | |
How original is the work? | |
Where was research support obtained? | |
How reproducible is the work? | |
How suitable is the work for the journal? | |
How should the report be written? | What summarizes the argument? |
What are the strengths and weaknesses? | |
What comments/edits are required? | |
What will improve the submission? | |
What is the personal bias of the reviewer? | |
What is helpful advice for the editors? | Why might there be ethical concerns? |
Why is additional expertise needed? | |
Why accept, reject or revise? |
Roles | Responsibilities |
---|---|
Ensure suitability as a reviewer | Declare any conflict of interest |
Determine if deadline can be met | |
Establish expertise to do the review | |
Maintain confidentiality | |
Use the journal guidelines for reviewers | Read the guidelines |
Follow the guidelines | |
Examine content of submission | Read the submission carefully |
Note clarity of writing | |
Check references for accuracy | |
Identify possible plagiarism | |
Evaluate submission | Establish title/abstract/keywords suitability |
Determine validity of question | |
Assess correctness of method | |
Gauge clarity of tables and figures | |
Identify if limitations are provided | |
Judge if conclusions are supported | |
Establish adequacy of references | |
Assess the originality of the work | |
Consider if there are ethical concerns | |
Determine if work is reproducible | |
Consider suitability for journal’s audience | |
Write the report | Summarize the argument presented |
State the strengths and weaknesses | |
Detail comments of suggested edits | |
Help improve written presentation | |
Comment without personal bias | |
Provide advice to editors | Indicate ethical concerns to editor |
Advise on further expertise required | |
Accept, reject or call for revisions |
Rank | Responsibilities of Peer Reviewers According to Authors | # |
---|---|---|
1 | Determine if interpretation is supported by the data | 92 |
2 | Provide constructive criticism | 87 |
3 | Improve manuscript | 84 |
4 | Timeliness: meet journal deadline | 81 |
5 | Recommendations on publication (e.g., no/minor/major revisions, reject) | 74 |
6 | Be expert in the subject area/matter/field and/or be familiar with/trained in research methods and statistics | 70 |
7 | Determine validity/quality/technical merit/rigor | 69 |
8 | Declare/avoid potential or actual conflict of interest | 66 |
9 | Assess adequacy of methods in general | 65 |
10 | Evaluate study design | 56 |
11 | Assess originality | 55 |
11 | Consider general ethical aspects and report on any specific ethical concerns (including manipulation of data, plagiarism, duplicate publication, inappropriate treatment of animal or human subjects) | 55 |
12 | Determine clarity of tables | 54 |
12 | Assess novelty | 54 |
13 | Consider adequacy of discussion (general) | 53 |
14 | Comment on interest to journal readership/relevance for journal scope | 52 |
14 | Appropriateness and accuracy of references | 52 |
14 | Maintain confidentiality of the manuscript, avoiding disclosure/discussion with others | 52 |
15 | Comment on clarity of study purpose and hypothesis | 50 |
16 | Assess importance/significance | 48 |
17 | Consider adequacy of results (general) | 46 |
18 | Comment upon relevance to practice/science (clinical relevance) | 45 |
19 | Comment upon contribution to the field | 42 |
19 | Assess data analysis (methods and tests) | 42 |
19 | Consider use of statistics | 42 |
20 | Be polite/courteous/respectful in the communication with authors | 41 |
20 | Determine overall readability | 41 |
21 | Assess presentation (general) | 40 |
21 | Advise editors on the merits of manuscripts | 40 |
22 | Be familiar with journal’s mission, review process, review criteria, guidelines (i.e., both author and reviewer guidelines) and forms prior to starting the review | 39 |
22 | Be fair: evaluate manuscript in a fair manner | 39 |
23 | Evaluate adequacy of introduction (in general) | 37 |
23 | Assess coherence/clarity and logical flow of the text | 37 |
24 | Be objective: objectively judge all aspects of the manuscript | 36 |
24 | Consider one’s time availability prior to accepting review request | 36 |
25 | Highlight whether the current literature is covered | 35 |
25 | Prior to accepting review request, determine whether the manuscript is within one’s area of expertise (only review manuscripts in one’s own field of expertise) | 35 |
25 | Be thorough/comprehensive/detailed/accurate | 35 |
26 | Assess sampling strategy | 34 |
27 | Consider clarity and validity of statistical methods | 33 |
27 | Determine how data were collected/reproducibility of methods | 33 |
28 | Be unbiased in their assessment: peer reviewers should have an unbiased attitude towards an author’s gender, previous work, institution and nationality | 32 |
28 | Provide confidential comments to editor | 32 |
29 | Identify strengths and weaknesses | 31 |
30 | Assess grammar and spelling | 30 |
Rank | Responsibilities of Peer Reviewers According to Authors and Editors | C# |
---|---|---|
1 | Knowledgeability | 6 |
2 | Methodological quality | – |
3 | Fairness | 22 |
4 | Constructiveness | 2 |
5 | Strengths and weaknesses (methods) | 9 |
6 | Interpretation results | 1 |
7 | Objectivity | 24 |
8 | Clarity | – |
9 | Strengths and weaknesses (general) | 29 |
10 | Statistical methods | 19 |
11 | Support by evidence | – |
12 | Detail/thoroughness | – |
13 | Tone | 20 |
14 | Applicability and external validity | 19 |
15 | Timeliness | 4 |
16 | Relevance | 18 |
17 | Structure of reviewer’s comments | – |
18 | Originality | 11 |
19 | Presentation and organization | 21 |
20 | Adherence to reporting guideline | 22 |
Ranks | Responsibility of Peer Reviewers | Dove | Frontiers | MDPI | Springer | Wiley |
---|---|---|---|---|---|---|
8,— | Declare any conflict of interest | Yes | Yes | Yes | Yes | Yes |
24,15 | Determine if deadline can be met | Yes | Yes | Yes | Yes | Yes |
25,1 | Establish expertise to do the review | Yes | Yes | Yes | Yes | |
14,— | Maintain confidentiality | Yes | Yes | Yes | Yes | |
22,— | Read the guidelines | Yes | Yes | Yes | Yes | |
—,20 | Follow the guidelines | Yes | Yes | |||
—,— | Read the submission carefully | Yes | Yes | Yes | Yes | |
20,8 | Note clarity of writing | Yes | Yes | Yes | Yes | Yes |
14,— | Check references for accuracy | Yes | Yes | |||
11,— | Identify possible plagiarism | Yes | Yes | Yes | Yes | Yes |
—,— | Establish title/abstract/keywords suitability | Yes | ||||
15,— | Determine validity of question | Yes | Yes | Yes | Yes | |
9,10 | Assess correctness of method | Yes | Yes | Yes | Yes | Yes |
12,— | Gauge clarity of tables and figures | Yes | Yes | Yes | Yes | |
17,6 | Judge if conclusions are supported | Yes | Yes | Yes | Yes | Yes |
25,— | Establish adequacy of references | Yes | Yes | Yes | ||
11,18 | Assess the originality of the work | Yes | Yes | Yes | Yes | Yes |
11,— | Consider if there are ethical concerns | Yes | Yes | Yes | Yes | Yes |
—,— | Identify if limitations are provided | Yes | ||||
27,— | Determine if work is reproducible | Yes | Yes | Yes | ||
14,16 | Consider suitability for journal’s audience | Yes | Yes | |||
—,— | Summarize the argument presented | Yes | Yes | |||
29,5,9 | State the strengths and weaknesses | Yes | Yes | Yes | Yes | Yes |
25,— | Detail comments of suggested edits | Yes | Yes | Yes | Yes | Yes |
3,4 | Help improve written presentation | Yes | Yes | Yes | Yes | Yes |
28,3,7 | Comment without personal bias | Yes | Yes | Yes | Yes | |
28,— | Indicate ethical concerns to editor | Yes | Yes | Yes | Yes | |
—,— | Advise on further expertise required | Yes | Yes | |||
5,— | Accept, reject or call for revisions | Yes | Yes | Yes | Yes | Yes |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nash, C. Roles and Responsibilities for Peer Reviewers of International Journals. Publications 2023, 11, 32. https://doi.org/10.3390/publications11020032
Nash C. Roles and Responsibilities for Peer Reviewers of International Journals. Publications. 2023; 11(2):32. https://doi.org/10.3390/publications11020032
Chicago/Turabian StyleNash, Carol. 2023. "Roles and Responsibilities for Peer Reviewers of International Journals" Publications 11, no. 2: 32. https://doi.org/10.3390/publications11020032