Rethinking the Unthinkable: A Delphi Study on Remote Work during COVID-19 Pandemic

Round 1

Reviewer 1 Report

Have you studied tha Italian Act num. 81/2017? That is the reason of the expression "smart working". 

Maybe you can analyze this law and then reconsider the background and interest of the study.

Author Response

Thank you for this suggestion. Certainly, we are familiar with the law you are referring to. However, our study suggests that this law may not adequately encompass the complexity of the changes that have occurred since the Covid-19 pandemic. These changes have compelled institutions and organizations to reconsider the concept of "smart working."

Reviewer 2 Report

- The question reserach should be clarified and mentionned already in the introduction.

- The litterature review is okay per se, but it is only considered a "premise" (line 141), i.e. only being part of a general context of telework. The litterature does not rely to a clear question research and, thus, it does not help to problematize the paper.

- I have doubts about the relevance of your methodology. Why do you consider HR managers and directors as experts? Are they involved in any activity that makes them exeperts, like HR associations; or have they involved in any study of telework?

- The Delphy methodology lacks of a concrete definition of the opportunities, feasibility and obstacles to be overcome. You should use the litterature review to define such aspects.

- The analysis that you propose (section 5) doesn't let you know what do the HR think concretely in general (opinions) and in relation to their experiences. In short, it lacks concreteness. 

- The sub-section Qualitative resultats (section 5.1) is interesting but too short. It is the only one with concrete citations, i.e. with substance. It could be further developped and may become the core of your paper.

- Your quantitative analysis (section 5.2) is based only on word associations, without any knowledge whatsoever of HR's working context. I don't see its usefulness and value. The analysis you propose on this basis seem to be weak. Moreover, you don't consider for the analysis the missings associations, i.e. what we could expect accordingly to the litterature but does not appear.

- The discussion (section 6) is interesting, but i don't see on what strong empirical empirical evidences you enumerate your findings.

- The sample composition (what does it represent) should also be considered in the limits of the study (section 7).

Author Response

Q - The question research should be clarified and mentioned already in the introduction.

1. Thank you very much for this suggestion. In the revised version of paper, we added the purpose of the study already in the introduction (page 2, line 46-50).

1. The literature review is okay per se, but it is only considered a "premise" (line 141), i.e. only being part of a general context of telework. The literature does not rely to a clear question research and, thus, it does not help to problematize the paper.
2. We appreciate your suggestion and acknowledge the importance of problematizing the paper and clearly defining the research question. In light of your comment, we expanded the literature review section to provide a more comprehensive overview of the relevant literature and its connection to our research question. By doing so, we aim to have strengthen the foundation of our study, highlighted the gaps in the existing literature, and clearly articulated the research problem that our paper seeks to address. 

1. I have doubts about the relevance of your methodology. Why do you consider HR managers and directors as experts? Are they involved in any activity that makes them experts, like HR associations; or have they involved in any study of telework?
2. We understand your doubts about the relevance of the methodology. In our study, we consider HR managers and directors as experts based on their professional experience and expertise in the field of human resources management. While they may not have been directly involved in studies on telework, their role and responsibilities within organizations often involve managing and implementing telework policies and practices. Additionally, HR managers and directors are typically responsible for overseeing various aspects of employee management, including remote work arrangements. Their insights and perspectives on telework can provide valuable information about organizational practices, challenges, and potential areas for improvement. Indeed, it is necessary to specify that all the experts involved in the research had implemented, either since the pandemic or prior to it, agile working arrangements within their respective organizations.
3. The Delphi methodology lacks a concrete definition of the opportunities, feasibility and obstacles to be overcome. You should use the literature review to define such aspects.
4. Thank you very much for this suggestion. We have utilized the literature review to define and explore the opportunities, feasibility, and obstacles related to remote work. The literature review has allowed us to identify and analyze relevant studies, theories, and empirical evidence that shed light on the opportunities that arise from remote work adoption. Indeed, during our literature review, we observed a gap in the existing studies and qualitative analyses that provide a deep understanding of the topic at hand. Recognizing this limitation, we deemed it necessary to employ a qualitative research methodology, specifically the Delphi interview method. By doing so, we aim to fill this gap and gain a more in-depth understanding of the research topic.

The Delphi interview method is well-suited for exploring complex and multifaceted subjects, such as the implications of remote work in human resource management. Through this qualitative approach, we can tap into the expertise and insights of a panel of experts in the field. The iterative nature of the Delphi process allows for the collection and synthesis of expert opinions over multiple rounds, enabling a comprehensive exploration of the research topic.

By utilizing the Delphi interview method, we can delve deeper into the intricacies, nuances, and practical implications of managing remote teams and the associated challenges and opportunities. This qualitative approach will provide rich and valuable insights that complement the existing literature and contribute to a more comprehensive understanding of the research area.

5. The analysis that you propose (section 5) doesn't let you know what do the HR think concretely in general (opinions) and in relation to their experiences. In short, it lacks concreteness. The sub-section Qualitative results (section 5.1) is interesting but too short. It is the only one with concrete citations, i.e. with substance. It could be further developed and may become the core of your paper.
6. Thank you for your feedback. We understand your suggestion and we have better emphasized the qualitative results of our study. 
7. Your quantitative analysis (section 5.2) is based only on word associations, without any knowledge whatsoever of HR's working context. I don't see its usefulness and value. The analysis you propose on this basis seem to be weak. Moreover, you don't consider for the analysis the missings associations, i.e. what we could expect accordingly to the litterature but does not appear.
8. Thank you very much. While the word association analysis provides a quantitative approach to explore relationships and patterns among concepts, we acknowledge that it may have limitations in capturing the specific nuances and complexities of the HR working context. We agree that a purely quantitative analysis based solely on word associations may not provide a comprehensive understanding of the HR working context and its intricacies. For this reason and in accordance with the literature (Lancia, 2004; Cortini and Tria, 2014), we adopted a triangulation of method for data analysis, matching both this quantitative analysis with a qualitative thematic analysis (Section 5.1). The analysis of occurrences and co-occurrences was conducted based on the thematic analysis itself, with the purpose of quantifying the emerged results and assessing their statistical relevance (using the Cosine Coefficient).

1. The discussion (section 6) is interesting, but i don't see on what strong empirical empirical evidences you enumerate your findings.
2. Thank you very much for this suggestion, we revised the text accordingly
3. The sample composition (what does it represent) should also be considered in the limits of the study (section 7).
4. Thank you very much. We appreciate your valuable feedback, and we have included a thorough discussion of the sample composition and its limitations in section 7 of our paper.

Author Response File: Author Response.pdf

Reviewer 3 Report

Thank you for giving me the opportunity of reading the work “Rethinking the Unthinkable: A Delphi Study on Remote Work 2 during COVID-19 Pandemic”, it has been a very big pleasure to collaborate reviewing this manuscript.

The manuscript is potentially relevant, and it might have a high interest for readers of this journal.

However, some aspects should be addressed before publication.

Introduction

This section briefly introduces the main topic and theoretical arguments. However, I didn’t find here a clear definition of the reasons why this study is needed and what contributions it might bring to remote working comprehension. In particular, I’ll appreciate it if the authors would make even more evident the theoretical, practical, and/or methodological gaps this study wants to fill. Also, specific referments should be provided about the innovative theoretical/methodological improvements this research would bring to previous literature and knowledge about remote working’s implications after Covid-19 lockdowns.

Literature review

This section is well-written and gives an interesting overview of what previous research has done about remote working implications. However, at the end of this section, it lacks a clear definition of the research questions your exploratory study wants to address. It might be very helpful if you condense the previous literature you quoted in this section into specific research question/s. 

Aim of the study

In my opinion, to facilitate the reader's comprehension, this section should be shifted and integrated into the introduction section. In any case, as it is written, it provides a too general definition of the study’s purposes, and it requires a more precise specification about why and how this study wants to contribute to remote working’s implications comprehension.

Material and Methods

Participants and procedures

By reading this paragraph, it has not been evident to me why you decided to adopt a Delphi interview method and how this method might help in answering this study’s research questions. Please, provide more details in support of your methodological choice.

Results

On page 5 you describe a “linguistic premise” about the conceptual overlap between “smart working” and “remote working”. From the theoretical point of view, these two concepts have substantial differences in that they might be neglected in the interviewed managers’ slang. Could the fuzzy, mixed use of the two terms affect your Delphi study results? If this possibility is not negligible, how can you limit or manage the influence of this “semantic” overlap? Please provide some information in this section or in Limits section if you can’t manage this issue.

Discussion

This section summarizes and effectively presents the study’s findings. However, in this section you didn’t provide clear referments about how these results answer or are linked to the study’s research questions. Without this information, it’s not clear to me the added value of this study results in consideration of the knowledge coming from previous literature. Some further explanations are needed.

Theoretical and practical Implications

In this section, it doesn’t seem clear to me how this study’s findings are linked to the theoretical and practical implications you mentioned here. Please provide these links or, if they are already provided, make them more evident in the text.

Author Response

Thank you for giving me the opportunity of reading the work “Rethinking the Unthinkable: A Delphi Study on Remote Work 2 during COVID-19 Pandemic”, it has been a very big pleasure to collaborate reviewing this manuscript.

The manuscript is potentially relevant, and it might have a high interest for readers of this journal.

However, some aspects should be addressed before publication.

Introduction

1. This section briefly introduces the main topic and theoretical arguments. However, I didn’t find here a clear definition of the reasons why this study is needed and what contributions it might bring to remote working comprehension. In particular, I’ll appreciate it if the authors would make even more evident the theoretical, practical, and/or methodological gaps this study wants to fill. Also, specific referments should be provided about the innovative theoretical/methodological improvements this research would bring to previous literature and knowledge about remote working’s implications after Covid-19 lockdowns.
2. Thank you very much for this suggestion. In the revised version of our paper, we better explained the existing gaps and better specified how this study contribute to the existing literature on remote working in the post-Covid-19 lockdown era. You can see these revisions at lines 190-222.By addressing these gaps and leveraging the Delphi interview method, we aim to provide a comprehensive understanding of the implications of remote working in the post-Covid-19 era. We believe that our study's contributions lie in its theoretical advancements, practical insights, and methodological innovation, all of which contribute to the broader literature and knowledge about remote working's implications after Covid-19 lockdowns. Once again, we appreciate your feedback and we have made these points more explicit in the introduction section of our paper.

Literature review

This section is well-written and gives an interesting overview of what previous research has done about remote working implications. However, at the end of this section, it lacks a clear definition of the research questions your exploratory study wants to address. It might be very helpful if you condense the previous literature you quoted in this section into specific research question/s. 

1. Thank you very much. In the revised version of the manuscript, we rewrite this section, better specify positive and negative implications of remote work arrangements and our research purpose.  

Aim of the study

In my opinion, to facilitate the reader's comprehension, this section should be shifted and integrated into the introduction section. In any case, as it is written, it provides a too general definition of the study’s purposes, and it requires a more precise specification about why and how this study wants to contribute to remote working’s implications comprehension.

1. Thank you very much. We shifted this section in the last subsection of literature review, better specify the gaps (theoretical, practical, and methodological) that that have prompted us to undertake this study.

Material and Methods

Participants and procedures

By reading this paragraph, it has not been evident to me why you decided to adopt a Delphi interview method and how this method might help in answering this study’s research questions. Please, provide more details in support of your methodological choice.

219. Thank you very much. We provided more details in support of our methodological choice, starting from better explaining of the methodological gap we have found in precedent studies, as you can read at lines 211-219.

Results

On page 5 you describe a “linguistic premise” about the conceptual overlap between “smart working” and “remote working”. From the theoretical point of view, these two concepts have substantial differences in that they might be neglected in the interviewed managers’ slang. Could the fuzzy, mixed use of the two terms affect your Delphi study results? If this possibility is not negligible, how can you limit or manage the influence of this “semantic” overlap? Please provide some information in this section or in Limits section if you can’t manage this issue.

1. Thank you very much for this valuable insight. Starting from the second round of the Delphi study, we "controlled" this variable by explicitly addressing the participants' attention to the specificities of remote working and inviting them to reflect on how well these practices aligned with those currently used in their organizations. This approach aimed to mitigate the potential influence of the semantic overlap between "smart working" and "remote working" and ensure that participants focused on the distinct aspects of remote work. We have also made sure to specify this in the revised version of the paper. Thank you again. 

Discussion

1. This section summarizes and effectively presents the study’s findings. However, in this section you didn’t provide clear referments about how these results answer or are linked to the study’s research questions. Without this information, it’s not clear to me the added value of this study results in consideration of the knowledge coming from previous literature. Some further explanations are needed.
2. Thank you for your comment. We modified the discussion section to better clarify the connection between the discussions and the results. With this regard, we pursued three strategies: 1) we linked more clearly the discussion themes to the best and worst practices themes emergeing in the final list produced by our partiicpants; 2) we underlined eventual links with the analysis of th ethemes emerging from the Delphi rounds and the co-occurrence analysis; 3) we underlinend eventual transversal connection between the practices emerging in the list and their implications. 

Theoretical and practical Implications

In this section, it doesn’t seem clear to me how this study’s findings are linked to the theoretical and practical implications you mentioned here. Please provide these links or, if they are already provided, make them more evident in the text.

1. Thank you, we revised the text accordingly.

Reviewer 4 Report

General: This is a very interesting topic namely the barriers, implications and efficacy in the light of remote work after the covid pandamic. The paper however, needs more structure to improve the readability. I am not an English experts, but I did see some language errors, so I would suggest a check on this. 

Abstract: 

- Please ad when the study was conducted, since the Covid19 period was a 2.5 year period is seems relevant to mention when this study was conducted

- the abstract states that the study will assess the point of view of HR managers, the implications for workers and the efficacy of remote working; it should be clear from the abstract what questions are asked to gain insight into these three topics, and also the results need to be structured in this way. 

- From what kind of organisations are the HR managers (i.e. the branches should be mentioned)

Intro:

- Line 46-47: you mention 'opposing views', but it is not clear which opposing views, the views mentioned in the previous sentence are not necessarily opposing

- line 53-55: reference is missing here

- In the intro you provide a nice overview of what is already known regarding remote work; it is not clear what the added value of your study is, why is it important to add this study, conducted among HR experts specifically? This need to be cleared up in the introduction/aims.

Materials and methods:

- In the abstract it is stated that there are 14 experts, in paragraph 4.1 you state 19 experts. 

- it should be clear at what kind of organisations the hr experts work, and you also need to reflect on this in the discussion. Because i can imagine that views differ per branche.

- I would move the last section in paragraph 4.1, about advantages and limations, to the discussion section. Here you should only underline what you did and why.

- I am not sure if it is because of the guidelines of the journal, but I would describe the outcomes before the analyses, because that helps the reader place the analyses in context. 

- it is not clear how you defined consensus, is it 80% of agreement?

- It is not clear how the quantitative analyses are conducted

- Substantial information is missing, or not explained well, namely: questions asked per round, scales, how participants were recruited, how the questions were asked (paper, online, interviews), etc. 

Results:

- The results would benefit form a structure, related to the three themes in the abstract. See earlier comments on this. 

Discussion:

- The discussion also needs the previous mentioned structure, linked to the three topics as mentioned in the abstract. The abstract mentions these three topics, but in the rest of the paper this structure is missing. 

Author Response

Abstract: 

- Please ad when the study was conducted, since the Covid19 period was a 2.5 year period is seems relevant to mention when this study was conducted

1. Thank you very much for this suggestion. We better specified when conducted our study. 

- the abstract states that the study will assess the point of view of HR managers, the implications for workers and the efficacy of remote working; it should be clear from the abstract what questions are asked to gain insight into these three topics, and also the results need to be structured in this way. 

1. Thank you very much. We rewrite part of the abstract accordingly, specifying that our purpose is to explore the point of view of HR managers of Italian SME, along with the examination of  both the advantages and risks of remote work, and provides a shared list of best and worst practices related to remote work adoption.

- From what kind of organisations are the HR managers (i.e. the branches should be mentioned)

1. thank you very much. In the revised version, we provided more information about our experts.

Intro:

- Line 46-47: you mention 'opposing views', but it is not clear which opposing views, the views mentioned in the previous sentence are not necessarily opposing

1. Thank you. We corrected this sentence.  

- line 53-55: reference is missing here

1. Thank you for this alert. We added citations here. 

- In the intro you provide a nice overview of what is already known regarding remote work; it is not clear what the added value of your study is, why is it important to add this study, conducted among HR experts specifically? This need to be cleared up in the introduction/aims.

1. Thank you very much. We added a new subsection to specify the existing gaps (theoretical, practical, and methodological) and the added value of our study. 

Materials and methods:

- In the abstract it is stated that there are 14 experts, in paragraph 4.1 you state 19 experts. 

19. Thank you. We corrected this error. The correct number of experts is 19. 

- it should be clear at what kind of organisations the hr experts work, and you also need to reflect on this in the discussion. Because i can imagine that views differ per branche.

1. Thank you very much. Unfortunately, we didn’t conduct a different analysis for branches. However, we added in the manuscript that the experts are HR managers from Italian SME, specifying what this means in application terms. Moreover, we add this lack in the limits of our studies. Thank you again. 

- I would move the last section in paragraph 4.1, about advantages and limations, to the discussion section. Here you should only underline what you did and why.

1. Thank you very much for this suggestion. We moved this section accordingly. 

- I am not sure if it is because of the guidelines of the journal, but I would describe the outcomes before the analyses, because that helps the reader place the analyses in context. 

1. Thank you. We better specify the outcome. 

- it is not clear how you defined consensus, is it 80% of agreement?

274. Thank you. We better defined this aspect at line 274. 

- It is not clear how the quantitative analyses are conducted

1. We conducted a content analysis with the software T-Lab, which mapped the contents of the interviews. Starting from line 394, we exposed all the procedure in details. 

- Substantial information is missing, or not explained well, namely: questions asked per round, scales, how participants were recruited, how the questions were asked (paper, online, interviews), etc. 

Thank you very much. We better described participants and procedures. 

Results:

The results would benefit form a structure, related to the three themes in the abstract. See earlier comments on this. 

Thank you very much, we revised the results section accordingly.

Discussion:

- The discussion also needs the previous mentioned structure, linked to the three topics as mentioned in the abstract. The abstract mentions these three topics, but in the rest of the paper this structure is missing. 

1. Thank you very much for this suggestion. We revised the abstract to better align it with the content of paper. 

Round 2

Reviewer 2 Report

Thanks for your replies and for improving the manuscript. 

Author Response

Thanks for your support in improving our manuscript. 

Reviewer 3 Report

I think the authors did a valuable effort in improving the manuscript and they effectively addressed all the revision points.

Author Response

Thank you for your support in improving our manuscript. 

Reviewer 4 Report

Thank you for improving this paper and for your response to my feedback.

Author Response

Thank you for your support in improving our manuscript. We believe that the revised version following your suggestions has superior quality and scientific rigor. 

Thank you again!