Reviewing an Paper for Tourism and Hospitality—A Peep behind the Curtain

Abstract

Academic journals rely crucially upon the willingness of academics to review papers and for them to provide high-quality review reports. The purpose of this Editorial is to assist reviewers for Tourism and Hospitality in providing review reports that will be helpful to the journal editors in making decisions about whether to accept, reject or require revisions to papers that are submitted to the journal. It provides reviewers with a ‘peep behind the curtain’ of what can sometimes seem a rather opaque paper review process, with the aim of clarifying the purpose and format of the review process, and thereby to gain a greater appreciation of what it required of them. Several tips for writing high-quality review reports are also presented.

1. Introduction

Academic reviewers have played a vital role in the production of knowledge for the last 350 years, when the practice of academic review is believed to have first begun [1]. Society needs knowledge it can trust. The role of the reviewer is to ensure that the knowledge that academics produce is reliable. This means that whoever uses the knowledge—be they politicians, policy makers, planners, business managers, leaders of social organisations, or individual citizens—can be confident they are making decisions based on solid, dependable evidence.

In order to fully grasp the value of academic review, it might be helpful to consider a situation in which peer review was absent or has failed. Unfortunately, there have been all too many cases of this reported in the world’s media of late. To take one example, in 2021 it was reported that ‘scammers’ had been using the Guest Editor system to push through the publication of around 300 poor-quality papers—some containing little more than “gibberish” [2] (p. 361)—in reputable, peer-reviewed journals. Two things can be concluded from this: first, that the review process must have been weak or even absent for such papers to be accepted for publication; and second, that decision-makers would be ill-advised to apply the ‘knowledge’ being reported in the papers concerned. Unfortunately, the scam had been operating for some time before it was detected and the various papers were withdrawn, so it is likely that decision-makers had already been using them. Although some of those decisions may have been minor or inconsequential, others are likely to have been very significant, perhaps even matters of life or death. The refereeing process had failed to distinguish trustworthy knowledge from trash, and this almost certainly had important, tangible impacts upon real people. Academic research therefore relies on competent and diligent peer-reviewers for its outputs to be valuable. Without excellent reviewers conducting excellent reviews, decision-makers would be forced to put blind faith in the veracity of the knowledge they are seeking to harness.

Journal editors could, of course, make decisions on submissions themselves, but that would carry with it a number of risks. First, there is the very real possibility that none of the journal editors have sufficient expertise in the particular topic to review the paper thoroughly themselves—or indeed, the time available to do so. Second, there is the issue of potential personal bias, or accusations of such, should the journal’s editors be the sole people responsible for accepting or rejecting submissions. They could collude so as to effectively block the publication of papers that took positions with which they did not agree or were submitted by authors they for some reason did not like. This could be particularly detrimental to the career prospects of the authors of such papers. In most cases, therefore, journal editors play only a coordinating role in handling submissions, surrendering the responsibility to review the papers to others. This dilution of power is considered vital in the case of the publication of academic work, where personal proclivities could easily come into play in deciding what should be published and what should not.

2. How the Peer-Review Process Works

The peer-review process used by Tourism and Hospitality is of a fairly standard form, although it does have some features that are particular to MDPI (see Figure 1). These have been introduced mainly to speed up the system—which in its traditional form has been widely criticised for being too slow and unnecessarily cumbersome [3]—without loss of integrity. This section outlines the process, allowing readers a ‘peep behind the curtain’ to allow them to gain insights into how their review report is commissioned and how it will be used. In the case of Tourism and Hospitality, the Academic Editor who handles the paper may be the Editor-in-Chief, a member of the Editorial Board or, in the case of special editions, a Guest Editor. On receiving a new submission, the Academic Editor will perform an initial check of the paper to determine whether it is in scope and has the basic potential to be published. Those that are out of scope may be referred to other journals at this stage, with the author’s consent, or returned to the author with an explanation that the paper does not fit well with the subject coverage of the journal. Papers may also be returned to the author at this stage because they fail to demonstrate a proper academic approach or are not in a fit state to be reviewed. Sometimes the Academic Editor will recommend revisions that would need to be undertaken for the paper to fit into the journal’s scope or achieve the minimum academic standard required. Papers are also checked at this stage for evidence of academic misconduct, such as plagiarism or the use of artificial intelligence text creation.

Once it has been decided that the paper is appropriate for review, the Academic Editor selects candidate reviewers for the paper, usually with the help of a Managing Editor or other member of the Editorial Office staff, who then invites them. Reviewers should be peers of the authors, and a such are expected to:

• “… hold a Ph.D. (or MD for medical fields), preferably with postdoctoral experience;
• Be an active researcher;
• Possess official and recognised affiliation (University or Research Institute) relevant experience and have a proven publication record in the field of the submitted paper (Scopus, ORCID); [… and …]
• Have the necessary expertise to judge the scientific quality of the manuscript” [4] (p. 1).

It should also be the case the reviewers should not have published regularly with the author(s) on the subject of the paper.

In conducting their review, the reviewer should be willing to:

• Provide quality review reports and remain responsive throughout the peer review process;
• Maintain standards of professionalism and ethics.

The latter are covered in the ethical guidelines for reviewers by the Committee on Publication Ethics (COPE) [5], which has been adopted by MDPI. These guidelines cover matters of professional responsibility, competing interests and bias, timeliness, confidentiality, and dealing with suspicions of ethical violation. Regarding timeliness, reviewers are usually asked to submit their reports within two weeks or accepting the invitation.

Reviews for Tourism and Hospitality are undertaken on a single-blind basis, meaning that although the reviewers are aware of the identity of the authors, authors are not informed of the names of the reviewers. Reviewers may, however, elect for their names to be appended to the final accepted paper and for their review reports to be published alongside. This is known as ‘open review’ and is encouraged by Tourism and Hospitality [6] because it provides evidence that the review process has been conducted to a proper standard. Open review is not, however, mandatory in Tourism and Hospitality at the present time.

Decisions on papers are made on the basis of the recommendations of at least two reviewers. Experience suggests that reviewers will not always submit their reviews within the allocated time, so if an invited reviewer does not respond within a few days, a third or even a fourth reviewer may be recruited to ensure a timely decision can be made. Reviewers may recommend acceptance, minor revision, major revision, or rejection. In the case of minor or major revisions, the authors will be asked to revise their paper according to the recommendations made by the reviewers and submit it to the journal within a specified time, along with a ‘responses to reviewers’ document in which they explain what revisions they have made and provide suitable rebuttals to recommendations they have not addressed.

A decision on the final disposition of papers that have been resubmitted following revisions—whether major or minor—will then be made. The decision at this point will be either to accept or reject the paper. If a reviewer has recommended minor revisions, the paper will not usually be sent to them for final approval, although they are welcome to ask to review the resubmitted paper if they so wish. If a reviewer has recommended major revisions, they will normally be asked to re-review the revised paper. the author’s ‘responses to reviewers’ document is provided to help them do this.

Accepted papers are then edited internally for formatting according to house style and for the use of English. Papers that have many language errors may first need to be returned to the author for further correction, however, particularly when the intended meaning is unclear. Accepted papers are then published promptly (usually within a few days) online on a creative commons license. Copyright is retained by the authors. As such they are free to be downloaded and read by anyone without the payment of any fee.

This academic publication system relies upon the quality of peer reviews. If reviews are done poorly, the Academic Editor will not be able to be confident that the paper is of the standard required for publication and will usually have to ask for additional reviewers to comment on the paper. This will inevitably result in delays in the review process

3. How to Conduct a Review

The review report for Tourism and Hospitality comprises two parts: a series of closed questions and an open format report. Both are important.

Table 1. Closed responses in a reviewer report.

General view (answers may be: high, average, low)	Originality Contribution to scholarship Quality of structure and clarity Logical coherence, strength of argument, academic soundness Engagement with sources, as well as recent scholarship Overall merit
Specific questions (answers may be: yes, can be improved, must be improved, not applicable)	Is the content sufficiently described and contextualised with respect to previous and present theoretical background and empirical research (if available) on the topic? Are all the cited references relevant to the research? Are the research design, questions, hypotheses and methods clearly stated? Are the arguments and discussions of findings coherent, balanced and compelling? For empirical research, are the results clearly presented? Is the article adequately referenced? Are the conclusions thoroughly supported by the results presented in the article or referenced in the secondary literature?
English language and style (choose one)	Extensive editing of English language and style required Moderate English changes required English language and style are fine/minor spell check required I don’t feel qualified to judge about the English language and style
Recommendation (choose one)	Accept in present form Accept after minor revision Reconsider after major revision Reject

lists the closed questions, which are intended to provide an overview of the reviewer’s opinion on the suitability of the paper for publication in Tourism and Hospitality.

It is important that the overall recommendation matches the rest of the review. If, for example, the reviewer has noted low originality and that the results need to be more clearly presented, an overall recommendation of ‘accept in present form’ or even ‘accept after minor revision’ would be unlikely to be appropriate.

Table 2. Guidance on making overall recommendations.

Recommendation	Guidance to Reviewers
Accept in present form	The only notable shortcomings of the paper are with the usage of the English language and/or the style (including the formatting in house style). The paper has achieved an acceptable standard in every respect in terms of its content. This would normally be indicated by the paper being considered ‘high’ or ‘average’ in terms of the paper’s merits, including its overall merit, and a response of ‘yes’ to the quality-check responses below that. Few papers submitted to any journal can be expected to be without shortcomings in their content, so ‘accept in its present form’ should be expected to be an unusual recommendation for papers that have not been revised. Choosing this option should be a rarity and requires full justification in the written part of the review.
Accept subject to minor revisions	The paper is considered to be ‘low’ in at least some respects in terms of its various merits, although not necessarily its overall merit. It may also have at least some aspects that ‘can be improved’. There may even be a small number of aspects that ‘must be improved’. Such revisions should usually be of the form that can easily verified by the Academic Editor. Only limited judgment should be required on their part to determine whether the paper can be accepted or must now be rejected. Note that, in this case, reviewers will only get to review the revised paper if they request this. If the recommendations are such that judgment is needed to verify their adequacy, then a ‘reconsider after major revisions’ recommendation would be more appropriate.
Reconsider after major revisions	The paper will usually have few or no areas where its merit is considered ‘high’ and many where it is considered ‘average’ and some where it is ‘low’. Overall merit is usually rated ‘low’ but may, in some cases, be considered ‘average’, depending on the reviewer’s opinion on the various aspects of the paper’s content, which will mostly be rated as ‘can be improved’ or ‘must be improved’. This recommendation signals to the Academic Editor that the reviewer considers the paper to have the potential to be published but final judgement must be reserved until those revisions have been made and the paper reviewed again. Reviewers who make a ‘reconsider after major revisions’ recommendation will usually be asked to review the paper again once the authors have revised and resubmitted it.
Reject	The paper has many merits that are considered ‘low’, although there may be some that are ‘average’ and perhaps even one that is ‘high’. Overall merit will usually be considered ‘low’. The reviewer will consider most aspects ‘must be improved’ but there may be some that ‘can be improved’. There may even be some aspects of the paper content that are acceptable and do not need to be improved. Making a recommendation rejection does not necessarily mean that the review believes that the paper has no prospect of ever being accepted, merely that this is unlikely in its present state. The purpose of this recommendation is therefore to signal that the paper is not ready for publication and that the authors could, if they wish to, re-work the paper to present it as a new submission in the future.

provides further guidance on what is expected for each of the possible overall recommendations that can be made.

Often the most difficult decision for most reviewers is between recommending ‘accept after minor revisions’ and ‘reconsider after major revisions’. As noted in the table, for Tourism and Hospitality, it is not standard practice to send it back to the reviewer if only minor revisions have been recommended. As such, it is worth reviewers remembering that if they want to make a final judgment on the paper, after having seen the revisions that the authors have made, a recommendation of ‘reconsider after major revisions’ would be most appropriate.

The guidance in Table 2 is not intended to be mechanical. This would unduly restrict reviewers in exercising their judgment. Reviewers should, however, fully justify their recommendation in the second part of the review. This is particularly the case with a recommendation to accept the paper in its current form. The Academic Editor will need to be convinced that the reviewer has thoroughly considered the paper, in every respect, and has found no aspect of the work that can be improved or must be improved.

Indeed, while reviewers’ answers to the closed questions are very important, they are not usually sufficient for the Academic Editor to make confident decisions on the paper. It is vital, therefore, that reviewers provide a written report, which makes up the second part of the review. A review of around 500 words will normally suffice, although there is no technical limit and longer reviews are welcomed. Of course, the technical content of the review is infinitely more important than its length, and there is no point in trying to lengthen a review by adding pointless content to it.

The purpose of the written report is to take a more detailed critical review of the paper than is possible using the closed responses alone. As such, reviewers should be prepared to spend some time crafting their written report. Reviewers are advised to take the position of a ‘critical friend’ in doing so. A critical friend will point out flaws and weaknesses in the paper in such a way as to build up and encourage the authors.

Above all, a paper review needs to be constructive. It is quite possible to write a constructive review without being cruel [7]. The authors have probably spent months, perhaps even years, undertaking the research and writing the paper. Reviewers should imagine they are receiving the review report and should ask themselves whether they would feel supported and encouraged by it. The best thing to do is to maintain a respectful tone, avoiding personal comments about the authors, particularly those they may find rude or degrading. Such comments do not contribute anything to the academic process and are unlikely to influence the final decision made by the Academic Editor.

Chong and Mason [7] also note that while reviewers are chosen because they are experts in a particular subject area and/or research methodology, the most important personal quality for a reviewer to have—and to display in their report—is humility. If the reviewer does not have the skills to judge a particular aspect of a paper, then it is best not to try to do so. Instead, please note this for the editor’s attention. The editor can then ensure that this is covered by the other reviewer(s) assigned to the paper.

This principle has particular importance when the reviewer notes that one or more of their own papers could usefully be referred to in the literature review. Some reviewers do this in order to attempt to increase the citations for their papers. It is therefore best to reserve this practice for when there is an exceptional case for the authors including those papers.

4. How to Structure the Review and What to Include

It is vital, therefore, to focus on the content of the paper when conducting a review. With regard to the report structure, some general guidance is available on the MDPI [8] website. It is important to bear in mind that there is no standard structure for a review report. A suitable structure might be to start with a short paragraph stating what their overall recommendation is and briefly justifying why they have made it.

The opening paragraph can then be followed by a list of bullet points. It is helpful if these are numbered, so that the authors can refer to the comments by number in their ‘responses to reviewers’ submission. It is best to start with more significant and/or general comments and to move on to the more minor and/or specific ones. Some reviewers divide the list into two in order to make it clear.

The list of significant/general comments should usually justify the responses the reviewer made in the first section of the review, where closed questions were used to give an overall assessment of the paper. They will normally be the merits that received a ‘low’ rating, or features that ‘must be improved’. As such, these comments will often relate to the reasons why the reviewer is recommending that the paper should be reconsidered following major revision or is to be rejected. Each point should expand on one of those responses, but it is not necessary to respond to each one unless there is value in doing so. The important thing is to ensure that the authors will be able to understand clearly what the weakness or shortcoming is and how they can revise their paper to address it. It is best therefore to use full sentences in this part of the report.

The list of minor/specific weaknesses or shortcomings tends to comprise those aspects that the reviewer considers will improve the paper if they can be appropriately addressed. As such, they tend to relate to merits that were given an ‘average’ rating or features that ‘can be improved’ in the first part of the review report. They are, therefore, appropriate whether the reviewer is recommending acceptance after minor revisions, reconsideration after major revisions, or outright rejection. Again, it is best to write in full sentences. It is also good practice to refer to the page and line number of the submission where the issue presents itself, so that both the authors and editor can locate the part of the paper in question.

The report can then end with a summary if desired. When the reviewer is recommending that the paper be rejected, this is often an encouraging comment along the lines of the paper showing promise but not being ready for publications at the present time.

Written comments are vital to explain why and how the paper needs to be revised either for acceptance or further consideration; however, they are even more important for papers where the reviewer is recommending rejection. Indeed, the other reviewer(s) may be more enthusiastic as to the prospects of the paper to be accepted and the authors will undoubtedly find the reviewer’s comments useful for revising the paper, if the Academic Editor decides that is to be the outcome of the review process. The authors will also find constructive comments helpful even if the reviewer has recommended their paper be rejected, as they are likely to want to revise it for submission elsewhere. With final rejection rates often as high as 95% for some journals [3], authors must expect few of their papers to be accepted for publication in the first journal they try.

It is also very important that the reviewer writes a substantial and meaningful commentary for papers where the reviewer is recommending acceptance. This will be for the benefit of the editor, who will appreciate this when it comes to making an overall decision on the paper. With multiple reviewers for each paper, recommendations are very often split, sometimes diametrically so (for example, one reviewer recommending acceptance in current form and the other outright rejection). At this point, the editor will have to make a carefully balanced judgment. There can often be a fine line involved in these judgments, with arguments on both sides, so the editor will be very grateful of a good explanation for the accept recommendation.

Reviewers are welcome to submit a version of the paper with track changes or mark-ups, but this is not obligatory in Tourism and Hospitality. Reviewers are also very welcome to write a separate or supplementary review report for the attention of the Academic Editor. This may be particularly useful when the author wishes to say something confidential that they do not wish the authors to see. Again, this is not obligatory. Most reviewers provide a single review report that can be read both by the Academic Editor and the author(s).

Another consideration is whether the paper is a new submission or is a resubmission following revisions. In the former case, the review report can often focus on the general/major issues, particularly when a recommendation of ‘reconsider after major revisions’ is made. Any specific/minor issues for when the paper can then be picked up when the revised paper is submitted.

It is important to remember that there are several types of paper in Tourism and Hospitality.

Table 3. Types of paper in Tourism and Hospitality.

Type	Purpose and Style	Review Process
Article	These are original research manuscripts. The work should report scientifically sound experiments and provide a substantial amount of new information. The article should include the most recent and relevant references in the field. The structure should include abstract, keywords, introduction, materials and methods, results, discussion, and conclusions (optional), with a suggested minimum word count of 4000 words. Please refer to the journal webpage for specific instructions and templates.A wide range of methodologies and methods are acceptable, including quantitative, qualitative, and a variety of mixed methods. Such papers are not purely discursive: they collect and analyse data, using either an inductive or deductive framework, to generate and/or test theory. The data may be numerical, textual, visual, or material. A wide variety of methods of analysis are acceptable, including statistics, hermeneutics, semiotics, content analysis, etc. Not all of the hypotheses in an empirical paper need to be supported, but a sufficient number of them should be supported for the conclusions of the paper to be considered to be making a significant contribution to what is now known, as opposed to what is still not known.	Single-blind peer review confirmed by the Academic Editor
Review paper	Review papers map out and analyse the literature within a specified field relevant to the purpose and scope of the journal concerned. Such fields tend to be well-demarcated, although this is not necessarily the case, and often represent emerging areas of interest. The reviews may be systematic or selective: no review can claim to be comprehensive of the field as a whole. To be published in this journal, review papers should comprise more than a simple bibliometric analysis and categorisation of papers, or the production of rankings of topics, authors, keywords, etc. Review papers should take a critical approach to the subject and use the associated literature it to comment upon the theoretical and/or applied development of the chosen field. Systematic reviews should follow the PRISMA guidelines.	Single-blind peer review confirmed by the Academic Editor
Technical note	Technical notes are abridged empirical research papers, usually intended to announce the initial findings from a research project. Such findings should, however, be self-contained and robust in themselves. Technical notes should not be seen as a poor alternative to empirical papers that are not considered ‘worthy’ of full paper status. They should include all the elements of empirical papers and be judged according to the same standards of research conduct. It should describe important modifications or unique applications for the described method. Technical notes can also be used for describing a new software tool or computational method. The structure should include an Abstract, Keywords, Introduction, Materials and Methods, Results, Discussion and Conclusions, with a suggested minimum word count of 3500 words.	Single-blind peer review confirmed by the Academic Editor
Perspective paper	These papers are intended to contribute to policymaking and practice dealing with a specific issue that fits within the purpose and scope of the journal. They are not usually empirically based but rather attempt to highlight a new or neglected issue within the field relevant to theory, policymaking, or practice. Such papers could present concepts, specific regulations, plans, strategies, protocols, and/or policy/practice guidance. Perspective papers should not be simply descriptive but should make a critical analysis of a concept, issue, problem, or impact.	Single-blind peer review confirmed by the Academic Editor
Comment paper (plus rejoinder and reply)	Comment papers are written in response to a paper that has recently been published in the journal. They are intended to identify one or more aspects of that paper that raise significant concerns in terms of the quality or safety of its conclusions. Such concerns could relate to the paper’s motivation, theoretical background, relationship to previous research, methodology and methods, results, conclusions, implications, and/or limitations. The comment paper should identify the concerns, demonstrate their implications, and suggest a way forward to ensure that such practices do not reoccur. The author(s) of the original paper will be given a right of reply (called a ‘rejoinder’); if they choose to do so, the author(s) of the comment may make a final summing up (called a ‘reply’). Comments, rejoinders, and replies should not include ad hominem remarks.	Consideration by the Academic Editor

Source: MDPI [8].

provides a summary. Reviewers are therefore advised to familiarise themselves with the different types of paper, and the expectations of such, and to tailor their review accordingly. It would not be appropriate, for example, to criticise a comment paper for not including a methodology section, as would be necessary in a full paper.

Finally, as Chong and Mason [7] note, it is important for reviewers to focus on what the paper is, rather than what they believe it could have been. Reviewer reports that claim that the paper should have focused on a different topic or used a different methodology to pursue the research are rarely helpful to the Academic Editor. Their task is to consider the paper, as submitted, and authors deserve this to be the case.