Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Dear authors,

I really enjoyed reading your manuscript and congratulate you on the importance of the topic you have chosen. To improve the text, I have included a set of observations below:

The title is clear and objective, reflecting the focus and type of literature review.

The abstract provides an overview of the review.

Line 85: For what purpose? What are its main findings?

Line 111: The three research questions seem to justify the relevance of conducting an integrative literature review

Line 130: Why was this theoretical framework chosen? How does it facilitate understanding of the phenomenon under review?

Line 140: It is unclear in the manuscript how the theoretical framework helped operationalise the concepts of individual and workplace-related factors.

Line 155: Was a health sciences librarian consulted to ensure the thoroughness and accuracy of the search?

Line 165: Why was the grey literature not searched? Could the omission of grey literature threaten the comprehensiveness and rigour of the literature review? Could this constitute a publication bias that should be highlighted in the limitations?

Line 169: Please review the word “Screening”

Line 175: Including the PRISMA checklist as supplementary material would strengthen the credibility of the literature review

Line 228: Please cite the studies that did not report cutoff values

Good luck with your research!

Author Response

Response to Reviewer 1 Comments
Thank you very much for taking the time to review this manuscript. We hope that our paper has been improved greatly by implementing your constructive advice and suggested modifications.

Reviewer: 1

Comments 1: The title is clear and objective, reflecting the focus and type of literature review.

Response 1: Thank you very much for your positive evaluation of the title. In response to a comment from Reviewer 2 regarding the inappropriateness of the term "before-after comparison," we have revised the title and the corresponding expressions throughout the manuscript.

Comments 2: The abstract provides an overview of the review.

Response 2: Thank you very much for your encouraging comment stating that the abstract effectively provides an overview of our study.

Comments 3: Line 85: For what purpose? What are its main findings?

Response 3: Thank you for pointing that out. We have revised Line 88 to provide a clearer context for readers by clearly stating the purpose of the study you mentioned and summarizing the main findings.

Comments 4: Line 111: The three research questions seem to justify the relevance of conducting an integrative literature review

Response 4: Thank you very much for recognizing the relevance and importance of our research questions within the context of an integrative literature review.

Comments 5: Line 130: Why was this theoretical framework chosen? How does it facilitate understanding of the phenomenon under review?

Response 5: Thank you for your question. In lines 98–103, we stated that the JD-R model is also effective for examining burnout among nursing managers. In response to your comment, we have added a further explanation in line 143 explaining why the JD-R model was chosen and how it helps to understand the phenomenon it targets.

Comments 6: Line 140: It is unclear in the manuscript how the theoretical framework helped operationalise the concepts of individual and workplace-related factors.

Response 6: Thank you for this important comment. We agree that the role of the theoretical framework in operationalizing workplace-related and individual factors required further clarification. In this review, the revised JD–R model was used not only as a conceptual framework, but also as an analytical framework to guide the extraction, organization, and interpretation of burnout-associated factors identified in the included studies. Specifically, workplace-related factors were operationalized through the domains of job demands (e.g., workload, staffing shortages, role ambiguity) and job resources (e.g., organizational support, communication, managerial discretion), while individual factors were operationalized through the domain of personal resources (e.g., resilience, leadership capacity, psychological capital, and sense of purpose).
Following data extraction, burnout-associated factors reported in each study were categorized into these three domains to facilitate structured comparison across studies and between the pre-pandemic and pandemic-era literature. We have revised Section 2.2 to clarify how the revised JD–R model informed the operationalization and synthesis of burnout-related factors throughout the review.

Comments 7: Line 155: Was a health sciences librarian consulted to ensure the thoroughness and accuracy of the search?

Response 7: Thank you for bringing this to our attention. Yes, to ensure the comprehensiveness and accuracy of our searches, guidance was provided by the Medical Library staff of our university. We have added this information to Line 170. 

Comments 8: Line 165: Why was the grey literature not searched? Could the omission of grey literature threaten the comprehensiveness and rigour of the literature review? Could this constitute a publication bias that should be highlighted in the limitations?

Response 8: Thank you for your feedback. To ensure a certain standard of methodological quality, we decided to focus only on peer-reviewed academic journals for this integrative review. However, we recognize that this may introduce publication bias. We have added this point to Line 456.

Comments 9: Line 169: Please review the word “Screening”

Response 9: Thank you for finding this typo.
 

Comments 10: Line 175: Including the PRISMA checklist as supplementary material would strengthen the credibility of the literature review

Response 10: Thank you for your suggestion. We had previously referred to the “PRISMA Guidelines” in Line 201, but this was an error—it should have been the “PRISMA Flowchart”—and we have corrected it. There is currently no PRISMA extension specifically for integrative reviews. We are sorry for the confusion.
 

Comments 11: Line 228: Please cite the studies that did not report cutoff values

Response 11: Thank you for your suggestion. We have cited the study that did not report a cutoff value on line 262.

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript addresses an important topic. Burnout among nurse managers is highly relevant to workforce sustainability, and the attempt to organize the literature using the Job Demands-Resources framework is potentially useful. The inclusion of quantitative, qualitative, and mixed-methods studies could also be a strength if handled rigorously.

At the same time, there are substantial conceptual, methodological, and reporting issues that limit confidence in the review and in its main conclusions.

First, the central comparison of factors before and after the COVID-19 pandemic is not convincingly supported by the review design. The pre-pandemic period is effectively restricted to a very short window, whereas the post-2020 evidence spans several years. This produces a structurally unbalanced comparison. More importantly, the studies being compared differ widely in country, setting, burnout instrument, cutoff values, managerial role definitions, and design. As a result, differences between the pre- and post-pandemic groups of studies cannot be interpreted as if they reflect temporal change attributable to the pandemic itself. Relatedly, the manuscript repeatedly uses “post-pandemic” to refer to studies conducted after March 2020, which is conceptually inaccurate. These are pandemic-era studies rather than truly post-pandemic studies.

Second, conformity with systematic review reporting standards is incomplete. No protocol registration or accessible pre-specified protocol is reported. The review question and eligibility criteria are not framed as clearly as would be expected under a PICOS or PECO style approach. In fact, the current PECO formulation is conceptually strained because COVID-19 is not a study-level exposure in most included papers, and the comparator of pre- versus post-pandemic conditions is usually imposed by the review rather than being built into the primary studies. Full reproducible search strategies are not provided in sufficient detail, the date on which each search was conducted is not clearly stated, and additional search procedures such as hand-searching or citation tracking are not described.

Third, the selection and extraction procedures are not sufficiently robust for a review of this kind. Screening was performed by one reviewer, and duplicate independent screening is not reported. Data extraction also does not appear to have been conducted independently by more than one reviewer. These features increase the risk of selection error and subjective bias. This is particularly important here because several inclusion decisions appear borderline.

Fourth, the included evidence base is not fully consistent with the stated population and aims. The review is framed as focusing on nurse managers, yet several included studies use mixed samples, broader healthcare professional groups, or comparisons between managers and staff. In some cases, burnout among nurse managers is not the clear primary focus. This weakens the coherence of the review and makes the synthesis harder to interpret.

Fifth, the quality appraisal approach is not sufficiently rigorous. The two-point framework adapted from the integrative review methodology is too limited to assess the risk of bias of such a heterogeneous evidence base. Sample size and response rate are not enough to judge the rigor of quantitative studies, and the criteria for qualitative and mixed-methods studies are also underdeveloped. In addition, studies rated as low quality are retained and said to receive less interpretive weight, but the manuscript does not explain clearly how that weighting was implemented in the synthesis. No formal assessment of certainty or confidence in the body of evidence is provided.

Sixth, the synthesis itself is often too assertive given the heterogeneity of the studies. The mapping of findings into job demands, job resources, and personal resources is reasonable in principle, but the coding procedure is not described in enough detail to judge reliability. It is also not clear how ambiguous or overlapping factors were handled. Because the studies use different burnout instruments, different subscales, different thresholds, and different reporting formats, statements about prevalence levels, common patterns, or consistent shifts across periods should be much more cautious.

Seventh, the third review question on effective interventions in crisis situations is underdeveloped and not well matched to the search strategy. The search does not appear to have been designed specifically to identify interventions, and only a small number of heterogeneous interventional studies were included. Some of these are also appraised as low quality. Under these conditions, the review cannot support strong claims about which interventions are effective in crisis situations.

Finally, the manuscript would benefit from substantial editorial revision. There are inconsistencies in terminology, dates, and framing across sections. The review might make a clearer contribution if it focused more modestly on synthesizing recent evidence on burnout-associated factors among nurse managers, rather than implying a strong before-versus-after comparison that the current design cannot adequately sustain.

Overall, the topic is important and the paper has potential, but the current review methods and reporting do not yet provide a sufficiently rigorous basis for the manuscript’s main comparative claims.

Author Response

Response to Reviewer 2 Comments
Thank you very much for taking the time to review this manuscript. We hope that our paper has been improved greatly by implementing your constructive advice and suggested modifications.

Reviewer: 2

Comments 1: First, the central comparison of factors before and after the COVID-19 pandemic is not convincingly supported by the review design. The pre-pandemic period is effectively restricted to a very short window, whereas the post-2020 evidence spans several years. This produces a structurally unbalanced comparison. More importantly, the studies being compared differ widely in country, setting, burnout instrument, cutoff values, managerial role definitions, and design. As a result, differences between the pre- and post-pandemic groups of studies cannot be interpreted as if they reflect temporal change attributable to the pandemic itself. Relatedly, the manuscript repeatedly uses “post-pandemic” to refer to studies conducted after March 2020, which is conceptually inaccurate. These are pandemic-era studies rather than truly post-pandemic studies.

Response 1: Thank you for your valuable feedback. As suggested in Line 107, we have revised the text to express the focus of the comparison not as a clear change over time, but as “observed changes in the focus of the study.” Additionally, throughout the entire paper, we have changed “post-pandemic” to “pandemic-era” and “after pandemic” to “during pandemic.”

Comments 2: Second, conformity with systematic review reporting standards is incomplete. No protocol registration or accessible pre-specified protocol is reported. The review question and eligibility criteria are not framed as clearly as would be expected under a PICOS or PECO style approach. In fact, the current PECO formulation is conceptually strained because COVID-19 is not a study-level exposure in most included papers, and the comparator of pre- versus post-pandemic conditions is usually imposed by the review rather than being built into the primary studies. Full reproducible search strategies are not provided in sufficient detail, the date on which each search was conducted is not clearly stated, and additional search procedures such as hand-searching or citation tracking are not described.

Response 2: Thank you for these important comments. We agree that several methodological and reporting expectations outlined in your comment are commonly associated with systematic reviews. However, the present study was conducted as an integrative review rather than a systematic review, and the methodology was therefore designed to align with the broader and more interpretive nature of integrative review approaches. As noted, the classification of studies into pre-pandemic and pandemic-era phases was conducted at the review level based on data collection periods, rather than representing a direct exposure comparison within the primary studies themselves.
We also appreciate your observations regarding reporting transparency. We have corrected the wording in line 201 from “PRISMA guidelines” to “PRISMA flowchart,” as the latter more accurately reflects the procedure used in this review. In addition, we have revised the Methods section to clarify the dates on which the database searches were conducted (line 175) and have explicitly stated that manual searching and citation tracking were not performed (line 177). We have also further clarified the review aims and eligibility criteria within the manuscript to improve transparency and reproducibility.

Comments 3: Third, the selection and extraction procedures are not sufficiently robust for a review of this kind. Screening was performed by one reviewer, and duplicate independent screening is not reported. Data extraction also does not appear to have been conducted independently by more than one reviewer. These features increase the risk of selection error and subjective bias. This is particularly important here because several inclusion decisions appear borderline.

Response 3: 
Thank you for your comments regarding the selection and data extraction procedures. We acknowledge that duplicate independent screening and extraction are commonly recommended in systematic reviews to reduce the risk of selection bias and subjective interpretation. However, the present study was conducted as an integrative review, which permits greater methodological flexibility in order to synthesize evidence from heterogeneous study designs, including quantitative, qualitative, and mixed-methods research. Within integrative review methodology, screening and extraction conducted primarily by a single reviewer are considered acceptable when supported by predefined eligibility criteria, transparent procedures, and collaborative review processes.
Nevertheless, we do recognize that screening by a single reviewer has limitations, and we have acknowledged this point in the Limitations section. To enhance methodological rigor and minimize potential bias, the screening and categorization processes followed predefined eligibility criteria and a structured review protocol. In addition, studies with ambiguous or borderline eligibility, as well as difficult extraction or categorization decisions, were reviewed collectively during meetings involving multiple researchers until consensus was reached among all participants. We have added this clarification to Line 198 to improve transparency regarding the review process.

Comments 4: Fourth, the included evidence base is not fully consistent with the stated population and aims. The review is framed as focusing on nurse managers, yet several included studies use mixed samples, broader healthcare professional groups, or comparisons between managers and staff. In some cases, burnout among nurse managers is not the clear primary focus. This weakens the coherence of the review and makes the synthesis harder to interpret.

Response 4: Thank you for your feedback. When pooled samples were used, we ensured that the analysis focused solely on data specific to nursing managers. We excluded studies in which nursing managers were not the primary focus. Additionally, we discussed among the researchers whether to exclude samples where the decision was unclear. We have added a note regarding this discussion to Line 198.

Comments 5: Fifth, the quality appraisal approach is not sufficiently rigorous. The two-point framework adapted from the integrative review methodology is too limited to assess the risk of bias of such a heterogeneous evidence base. Sample size and response rate are not enough to judge the rigor of quantitative studies, and the criteria for qualitative and mixed-methods studies are also underdeveloped. In addition, studies rated as low quality are retained and said to receive less interpretive weight, but the manuscript does not explain clearly how that weighting was implemented in the synthesis. No formal assessment of certainty or confidence in the body of evidence is provided.

Response 5: 
Thank you for this thoughtful comment. We agree that the methodological quality assessment used in this review is less detailed than formal risk-of-bias tools commonly applied in systematic reviews and meta-analyses. However, the present study was conducted as an integrative review, which aimed to synthesize evidence derived from heterogeneous study designs, including quantitative, qualitative, and mixed-methods research. In this context, we adopted a simplified quality appraisal framework based on integrative review methodology to allow consistent evaluation across diverse forms of evidence.
We acknowledge that assessment criteria based primarily on factors such as sample size, response rate, and overall methodological clarity have limitations and may not capture all dimensions of rigor or risk of bias within individual study designs. We have therefore added this point to the Limitations section (Line 459) to improve transparency regarding the methodological constraints of the review.
To strengthen the interpretive rigor of the synthesis, only burnout-related factors extracted from studies assessed as high quality were included in the comparative analysis presented in Table 4. Although this procedure was previously described in Section 3.2, we have now clarified this point further in the Methods section (Line 218) to make the weighting process more explicit. Because the purpose of this integrative review was exploratory and interpretive rather than to establish pooled effect estimates or certainty of evidence, formal certainty assessment frameworks were not applied. Nevertheless, we agree that more comprehensive appraisal approaches may strengthen future systematic reviews in this area.

Comments 6: Sixth, the synthesis itself is often too assertive given the heterogeneity of the studies. The mapping of findings into job demands, job resources, and personal resources is reasonable in principle, but the coding procedure is not described in enough detail to judge reliability. It is also not clear how ambiguous or overlapping factors were handled. Because the studies use different burnout instruments, different subscales, different thresholds, and different reporting formats, statements about prevalence levels, common patterns, or consistent shifts across periods should be much more cautious.

Response 6: Thank you for this important comment. We agree that the substantial heterogeneity among the included studies—including differences in burnout instruments, subscales, cutoff values, study populations, and reporting formats—necessitates cautious interpretation of the findings. The purpose of this integrative review was not to establish definitive prevalence estimates or causal temporal changes, but rather to provide an exploratory synthesis of burnout-related factors reported in the literature and to examine how the focus of research differed before and during the COVID-19 pandemic.
In response to your comment, we have revised the manuscript throughout to adopt more cautious and interpretive language regarding burnout prevalence, common patterns, and temporal differences. In particular, we clarified that the identified differences reflect shifts in reported research focus and emphasized factors within the literature, rather than definitive changes in burnout itself across time periods. We also revised statements that could be interpreted as causal or overly generalizable given the heterogeneity of the included evidence.
Regarding the JD–R categorization process, burnout-associated factors were classified through collaborative discussion among members of the research team, including researchers with prior experience using the JD–R model in burnout research. Factors considered conceptually ambiguous, overlapping, or difficult to classify were reviewed repeatedly during research meetings, and final categorization decisions were made only after unanimous agreement had been reached among all participants. We have added this clarification to the Methods section (Line 222).
In addition, we further expanded the Limitations section (Line 467) to more clearly acknowledge the challenges associated with heterogeneity in burnout measures, role definitions, cutoff values, and reporting approaches across studies. 

Comments 7: Seventh, the third review question on effective interventions in crisis situations is underdeveloped and not well matched to the search strategy. The search does not appear to have been designed specifically to identify interventions, and only a small number of heterogeneous interventional studies were included. Some of these are also appraised as low quality. Under these conditions, the review cannot support strong claims about which interventions are effective in crisis situations.

Response 7: Thank you for your feedback. We agree that the evidence regarding interventions was limited. To maintain our focus on the factors contributing to burnout, we have removed the third research question from the main objectives. Discussions regarding interventions in crisis situations are now limited to the section titled “4.3. Practical implications for preparing for future crisis situations.”

Comments 8: Finally, the manuscript would benefit from substantial editorial revision. There are inconsistencies in terminology, dates, and framing across sections. The review might make a clearer contribution if it focused more modestly on synthesizing recent evidence on burnout-associated factors among nurse managers, rather than implying a strong before-versus-after comparison that the current design cannot adequately sustain.

Response 8: Thank you for your feedback. We have revised the terminology, dates, and arguments in each section. Furthermore, we have taken your point—that it is difficult to claim a robust “before-and-after comparison” with the current research design—seriously and have revised the text throughout to use more cautious and accurate language. Accordingly, we have revised the title to “Shifts in Research Focus on Factors Associated with Burnout Among Nursing Managers Before and After the COVID-19 Pandemic: A Systematic Review.”

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Thanks for addressing all my comments.