Collateral Damage from Offensive Cyber Operations—A Systematic Literature Review
Round 1
Reviewer 1 Report
This is a strong literature review worthy of publication. If possible I would have liked to see more economic evaluation from cyber-physical attacks. Specifically calling out urban critical infrastructure could be helpful in this regard
The table at the end of the paper seems out of place. Relevant figures should be framed in the early part of the document. Further, insights should be compiled where the total is greater than the sum of its parts. Currently, the paper is strong as a summary of literature but more can be done to synthesize across the articles read. It recommend a minor augmentation of the discussion to reflect this.
The paper's intent is to ascertain how the academic literature across disciplines defines cyber collateral damage and where the gaps remain in this line of research. There has been poor coverage on unanticipated effects overall so this was a good study to try to look at this.
While the topic is not entirely novel, the aggregation of the literature is useful because it is a multidisciplinary field of study that collateral damage will fall under for cyber effects. As an engineer, I would generally draw from certain studies that a social scientist studying cyber effects would not. This incorporates all sides into the equation and focuses on how cyber has impacted everyday people.
This paper adds considerable value in working across disciplines and pulling otherwise scattered work under a unified framework for thinking. It is a strong paper for teeing up a research agenda as a starting point for a larger research campaign.
There could have been greater focus on a methodological framework for how the papers were selected and evaluated. Frameworks could help to unify the papers discussed and help future researcher further categorize other work in this area.
The references are appropriate, but I think more could have been included on urban critical infrastructure where civilians are harmed. For example, a paper like this: https://fsi9-prod.s3.us-west-1.amazonaws.com/s3fs-public/cyber_negotiation_a_cyber_risk_management_approach_to_defend_urban_critical_infrastructure_from_cyberattacks.pdf
Author Response
Comment 1: If possible I would have liked to see more economic evaluation from cyber-physical attacks. Specifically calling out urban critical infrastructure could be helpful in this regard
Response: This is an important area which fits well into the review. Examples of critical infrastructure functions have been added to section 4.8. The discussion on the militarization of the cyber domain in section 5.1 has also been expanded to discuss urban critical infrastructure at greater length. The description of the 2017 Wannacry attack in section 5.2 has been clarified in this regard.
Comment 2: The table at the end of the paper seems out of place. Relevant figures should be framed in the early part of the document. Further, insights should be compiled where the total is greater than the sum of its parts. Currently, the paper is strong as a summary of literature but more can be done to synthesize across the articles read. It recommend a minor augmentation of the discussion to reflect this.
Response 2: The table in section 6 has been removed, and the discussion section has been edited and expanded, especially in subsection 5.4 where the possibilities of future research measuring the costs of cyber attacks to local governments has been proposed.
Comment 3: There could have been greater focus on a methodological framework for how the papers were selected and evaluated. Frameworks could help to unify the papers discussed and help future researcher further categorize other work in this area.
Response 3: The framework and methodology for coding papers into themes has been expanded and improved for clarification (section 4.1). Text coding for a sample paper is now visualized in figure 4.
Comment 4: The references are appropriate, but I think more could have been included on urban critical infrastructure where civilians are harmed. For example, a paper like this: https://fsi9-prod.s3.us-west-1.amazonaws.com/s3fs-public/cyber_negotiation_a_cyber_risk_management_approach_to_defend_urban_critical_infrastructure_from_cyberattacks.pdf
Response: Further text and references have been adding supporting the discussion on civilian harm from cyber attacks on urban critical infrastructure, specifically in subsections 5.1 and 5.2.
Reviewer 2 Report
This manuscript provides a comprehensive and well-structured SLR on CCD caused by offensive cyber operations. It makes a timely and relevant contribution to the cybersecurity and policy domain, particularly where there has been a lack of structured analysis on CCD. It also reflects solid methodology and insightful observations.
Here are some critical weakness.
Section 4.1lays out the main thematic areas, which are helpful for organizing the discussion. But it’s not really clear how these themes were developed. Several questions: Were they based on a coding framework? Was there a process for reviewing and agreeing on the categories? And how did you know when you had enough themes, that is, when thematic saturation was reached? In thematic analysis, these are quite standard elements. It is suggested to explain them, even briefly, would help readers trust the structure and see how it was grounded in the data.
Table 2 gives a helpful overview of the six themes and the corresponding discussion is helpful. What’s missing here is a more detailed mapping of the actual papers. It is harder to navigate or dive deeper into individual contributions. I’d suggest indexing the reviewed papers and clearly identifying which ones fall under each thematic area using separate tables. That way, readers can easily zoom in on the papers that are most relevant to their own interests. Such simple index or cross-reference would really improve the usability of the review.
Right now, the manuscript doesn’t really touch on any of the limitations of the review process, which is a bit of a gap in SLR. The authors should add a specific section to address the known limitations like the potential for database bias, filtering constraints, or the fact that identifying themes can involve a degree of subjectivity.
n/a
Author Response
Comment 1: Section 4.1 lays out the main thematic areas, which are helpful for organizing the discussion. But it’s not really clear how these themes were developed. Several questions: Were they based on a coding framework? Was there a process for reviewing and agreeing on the categories? And how did you know when you had enough themes, that is, when thematic saturation was reached? In thematic analysis, these are quite standard elements. It is suggested to explain them, even briefly, would help readers trust the structure and see how it was grounded in the data.
Response: We agree, showing more of this process will definitely improve trust and understanding. Section 4.1 has been expanded and improved to describe the development of the themes. A sample figure (Figure 4) visualizing thematic development for a reviewed paper has also been added.
Comment 2: Table 2 gives a helpful overview of the six themes and the corresponding discussion is helpful. What’s missing here is a more detailed mapping of the actual papers. It is harder to navigate or dive deeper into individual contributions. I’d suggest indexing the reviewed papers and clearly identifying which ones fall under each thematic area using separate tables. That way, readers can easily zoom in on the papers that are most relevant to their own interests. Such simple index or cross-reference would really improve the usability of the review.
Response: Index tables have been added to the subsection for each category, subsections 4.2-4.8, to help readers navigate the text.
Comment 3: Right now, the manuscript doesn’t really touch on any of the limitations of the review process, which is a bit of a gap in SLR. The authors should add a specific section to address the known limitations like the potential for database bias, filtering constraints, or the fact that identifying themes can involve a degree of subjectivity.
Response: The current limitations section (3.5) has been rewritten and expanded to cover these highly relevant constraints and forms of bias and subjectivity.
Round 2
Reviewer 2 Report
All the major concerns have been addressed.
None.
