Risk Assessment for Cyber Resilience of Critical Infrastructures: Methods, Governance, and Standards
Abstract
:1. Introduction
- Identifying and classifying different risk assessment methodologies used for cyber resilience;
- Pointing out and investigating key standards and guidelines used in studies related to cyber resilience;
- Providing an overview of relevant regulations influencing cyber risk assessment for cyber resilience;
- Highlights gaps in current research and practice, suggesting areas for further exploration in cyber resilience strategies.
2. General Background Review
2.1. Security Risk Assessment Standards
2.2. Cyber Resilience
2.3. Risk Assessment for Cyber Resilience
2.4. Literature Limitations and Research Gaps
- Subjectivity and bias: The PIM relies on expert judgment to estimate the probability and impact of risks [79]. Different stakeholders may have varying perceptions, leading to inconsistencies and biases in the scoring. This reliance on subjective opinions can undermine the matrix’s effectiveness [79,80].
- Limited criteria consideration: Typically, there is a need for requirement analysis to select the right criteria in multi-criteria problems [81,82,83]. PIM considers only two factors: probability and impact. Real-world risks are often more complex and influenced by additional criteria such as time sensitivity, interdependencies, and resource availability [84,85]. This oversimplification can lead to inaccurate prioritization.
- Lack of precision: Although accuracy is a hot topic in uncertainty analysis [86,87,88,89], PIM usually categorizes risks into broad, qualitative ranges (e.g., low, medium, high) rather than using precise quantitative values [90]. This lack of granularity can cause ambiguity, especially when risks fall near the boundaries of these categories, hindering effective decision making.
- Limitation in inability to handle interdependencies: Risks are often interconnected, with one risk potentially triggering or amplifying another. On the other hand, the emerging systems and infrastructures are also interconnected. The PIM does not adequately account for these interdependencies, leading to an incomplete understanding of the overall risk landscape [91].
- Static nature challenge: PIM is typically used as a one-time assessment tool. However, risks evolve over time [92,93], and their probability and impact can change. The static nature of PIM limits its ability to dynamically adjust to these changes. Updating the PIM periodically introduces several challenges. First, frequent updates require additional resources, including time, personnel, and funding, which can strain organizational capacity. Second, collecting accurate and up-to-date data to reassess risks can be complex, particularly if external conditions or internal structures change rapidly. Additionally, periodic updates may lead to inconsistencies in results if the assessment criteria or methodologies are not standardized over time. Finally, scheduling regular updates while maintaining accuracy and relevance adds logistical complexity, requiring careful planning to avoid disruptions to regular operations.
3. Materials and Methods
3.1. Research Design
3.2. Data Attributes
- Number of yearly publications by source type: Tracks the volume and source types (e.g., journals, conferences) of publications over time, reflecting trends and shifts in scholarly attention toward resilience enhancement in risk assessment.
- CI sectors addressed: Identifies the focus sectors (e.g., energy, transportation) targeted by each paper, providing insight into which CI areas are prioritized in resilience-related studies.
- Author keywords: Lists the most frequently occurring keywords, indicating prominent themes within the literature.
- Inter-article connectivity via shared topics: Examines the network of articles interconnected by shared topics, highlighting collaborative patterns and thematic linkages across studies.
3.3. Research Questions
3.4. Search Process
3.5. Analysis Criteria
4. Results
4.1. General Synthesis of Results
4.2. Risk Assessment-Related Methods and Models
4.3. Risk Assessment-Related Standards
4.4. Risk Assessment-Related Governance and Lawmaking
5. Discussion
5.1. Risk Assessment Methods for Cyber Resilience
5.2. Risk Assessment Standards for Cyber Resilience
5.3. Risk Assessment Governance for Cyber Resilience
5.4. Research Limitations
6. Conclusions
- The concept of cyber resilience is relatively young in the context of safeguarding critical systems, and its integration with risk assessment is at its starting stages. This is reflected by the fact that more than half of the papers on this topic have been published only in the past two years.
- Studies that address risk assessment for cyber resilience, focus only on a small number of CI sectors, namely, transportation, energy, healthcare, IT, and emergency services.
- The intersection of risk assessment methods and cyber resilience is continuous monitoring. In order to establish and maintain cyber resilience, cyber risks have to be constantly monitored to identify new threats and vulnerabilities and mitigate the corresponding risks.
- Risk assessment methods for cyber resilience can differ based on how they define risk and which sectors they are designed to cover.
- Cyber resilience in risk assessment standards is often addressed through risk management procedures to realize pre- and post-disturbance protection.
- Although there are some well-designed regulations that cover risk assessment for cyber resilience, such as NIS 2 and CRA, CI sectors are still not motivated enough to implement them.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A. Shortlisted Articles
ID | Ref. | Year | Institutions | Country |
---|---|---|---|---|
Qi1 | [129] | 2023 | Pukhov Institute for Modelling in Energy Engineering “IPME” | Ukraine |
Qi15 | [133] | 2021 | Infili Technologies PC, i2CAT Foundation, University of Murcia, ORION Innovations PC, Sfera IT, UBITECH Ubiquitous Solutions, Politecnico di Torino, NEC Laboratories Europe GmbH, inCITES Consulting SA, Hewlett Packard Enterprise, Space Hellas, Stratotiki Sxoli Evelpidon, DBC Europe SA | Greece, Spain, Italy, Slovenia, Germany, Luxembourg, UK |
Qi4 | [128] | 2023 | Centre for Cyber Resilience and Trust (CREST) | Australia |
Qs59 | [117] | 2023 | University of Warwick, Kohat University of Science & Technology, University of the West of England | UK, Pakistan |
Qs63 | [118] | 2022 | Melentiev Energy Systems Institute | Russian Federation |
Qs66 | [134] | 2019 | University of Dar es Salaam, Old Dominion University, University of Illinois at Urbana-Champaign | Tanzania, USA |
Qs77 | [120] | 2023 | Fraunhofer SIT, ATHENE, Technische Hochschule Mittelhessen | Germany |
Qs85 | [109] | 2024 | University of Tras-os-Montes and Alto Douro, University of Aveiro | Portugal |
Qs52 | [68] | 2024 | Hansung University, Myongji University | South Korea |
Qs84 | [104] | 2019 | Athens University of Economics & Business (AUEB) | Greece |
Qs74 | [107] | 2024 | Van Yuzuncu Yıl University, Istanbul Technical University | Turkey |
Qs83 | [106] | 2022 | Istanbul Technical University Maritime Faculty, University of Plymouth | Turkey, UK |
Qs21 | [130] | 2022 | The University of Indonesia, International Islamic University Malaysia | Indonesia, Malaysia |
Qs31 | [99] | 2018 | Thales UK Research Technology and Innovation, Brno University of Technology | UK, Czech Republic |
Qs15 | [121] | 2020 | University of Huddersfield School of Art Design and Architecture, United Nations Office for Disaster Risk Reduction New York | UK, USA |
Qs20 | [114] | 2017 | Tennessee State University, Old Dominion University | USA |
Qs22 | [115] | 2020 | Mississippi State University, US Army Engineer Research Development Center (ERDC) | USA |
Qs41 | [119] | 2024 | Hitachi Energy, Iowa State University of Science and Technology | USA |
Qs48 | [126] | 2024 | University Of Petra, General Administration Of Electronic Training Institute Of Public Administration, King Abdulaziz University | Jordan, Saudi Arabia |
Qs49 | [131] | 2020 | Old Dominion University, U.S. Army Research Laboratory, Crane Division | USA |
Qs5 | [116] | 2021 | Idaho National Laboratory, EnerNex, Xanthus Consulting International | USA |
Qs26 | [110] | 2024 | SPHYNX Technology Solutions AG, City University of London, University General Hospital of Heraklion | Switzerland, UK, Greece |
Qs54 | [39] | 2023 | University of Tras-os-Montes and Alto Douro | Portugal |
Qs55 | [127] | 2021 | Università degli Studi di Genova, Università degli Studi di Napoli Federico II | Italy |
Qs58 | [127] | 2024 | Z&Co, Murdoch University Dubai, Applied Science Private University | UAE, Jordan |
Qs47 | [108] | 2024 | Deakin University, Bangladesh National CERT, King Saud University | Australia, Bangladesh, Saudi Arabia |
Qs60 | [123] | 2024 | Leiden University | Netherlands |
Qs25 | [112] | 2024 | La Trobe University, Cyberoo Pty Ltd, Massey University | Australia, New Zealand |
Qs64 | [113] | 2024 | Technological University | Ireland |
Qs43 | [111] | 2024 | Indiana Tech and Lionfish Cyber Security, Lionfish Cyber Security | USA |
Qs2 | [102] | 2021 | Chalmers University of Technology, Volvo Car Corporation | Sweden |
Qs40 | [101] | 2023 | Queen’s University Belfast | UK |
Qs78 | [125] | 2024 | Riga Technical University | Latvia |
Qs81 | [124] | 2024 | Institute of Public Administration | Saudi Arabia |
Qs23 | [22] | 2016 | Thales UK Limited | UK |
Qs35 | [105] | 2021 | Minia University, Nahda University in Beni Suef | Egypt |
Qs44 | [103] | 2023 | Queen’s University, Institute of Flight Systems | UK, Germany |
Qs86 | [20] | 2023 | PRIME RE Solutions, ESPRIT School of Engineering, ESSEC Business School | Switzerland, Tunisia, France |
Qs9 | [9] | 2024 | The University of Glasgow, University of Surrey, The University of Auckland | UK, New Zealand |
Qw1 | [122] | 2020 | Technical University of Denmark, Man Energy Solutions | Denmark |
References
- Yaacoub, J.P.A.; Salman, O.; Noura, H.N.; Kaaniche, N.; Chehab, A.; Malli, M. Cyber-physical systems security: Limitations, issues and future trends. Microprocess. Microsyst. 2020, 77, 103201. [Google Scholar] [CrossRef] [PubMed]
- De Felice, F.; Baffo, I.; Petrillo, A. Critical Infrastructures Overview: Past, Present and Future. Sustainability 2022, 14, 2233. [Google Scholar] [CrossRef]
- Ardebili, A.A.; Padoano, E.; Longo, A.; Ficarella, A. The Risky-Opportunity Analysis Method (ROAM) to Support Risk-Based Decisions in a Case-Study of Critical Infrastructure Digitization. Risks 2022, 10, 48. [Google Scholar] [CrossRef]
- Setola, R.; Luiijf, E.; Theocharidou, M. Critical Infrastructures, Protection and Resilience; Springer International Publishing: Berlin/Heidelberg, Germany, 2016. [Google Scholar]
- Lazari, A. European Critical Infrastructure Protection; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
- Araujo, M.S.D.; Machado, B.A.S.; Passos, F.U. Resilience in the Context of Cyber Security: A Review of the Fundamental Concepts and Relevance. Appl. Sci. 2024, 14, 2116. [Google Scholar] [CrossRef]
- Ardebili, A.A.; Martella, C.; Martella, A.; Lazari, A.; Longo, A.; Ficarella, A. Smart Critical Infrastructures Security management and governance: Implementation of Cyber Resilience KPIs for Decentralized Energy Asset. In Proceedings of the CEUR Workshop—8th Italian Conference on Cyber Security (ITASEC 2024), Salerno, Italy, 8–12 April 2024; Volume 3731. [Google Scholar]
- Pettersen, S.; Grøtan, T.O. Exploring the grounds for cyber resilience in the hyper-connected oil and gas industry. Saf. Sci. 2024, 171, 106384. [Google Scholar] [CrossRef]
- Alhidaifi, S.M.; Asghar, M.R.; Ansari, I.S. A Survey on Cyber Resilience: Key Strategies, Research Challenges, and Future Directions. ACM Comput. Surv. 2024, 56, 196. [Google Scholar] [CrossRef]
- Sharkov, G. From Cybersecurity to Collaborative Resiliency. In Proceedings of the SafeConfig ’16: 2016 ACM Workshop on Automated Decision Making for Active Cyber Defense, Vienna, Austria, 24 October 2016; pp. 3–9. [Google Scholar] [CrossRef]
- Björck, F.; Henkel, M.; Stirna, J.; Zdravkovic, J. Cyber resilience–fundamentals for a definition. In New Contributions in Information Systems and Technologies; Springer: Berlin/Heidelberg, Germany, 2015; Volume 1, pp. 311–316. [Google Scholar]
- Goldman, H.; McQuaid, R.; Picciotto, J. Cyber resilience for mission assurance. In Proceedings of the 2011 IEEE International Conference on Technologies for Homeland Security (HST), Waltham, MA, USA, 15–17 November 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 236–241. [Google Scholar]
- Petrenko, S. Cyber Resilience; River Publishers: Aalborg, Denmark, 2022. [Google Scholar]
- Mottahedi, A.; Sereshki, F.; Ataei, M.; Nouri Qarahasanlou, A.; Barabadi, A. The resilience of critical infrastructure systems: A systematic literature review. Energies 2021, 14, 1571. [Google Scholar] [CrossRef]
- Osei-Kyei, R.; Tam, V.; Ma, M.; Mashiri, F. Critical review of the threats affecting the building of critical infrastructure resilience. Int. J. Disaster Risk Reduct. 2021, 60, 102316. [Google Scholar] [CrossRef]
- Harrop, W.; Matteson, A. Cyber resilience: A review of critical national infrastructure and cyber security protection measures applied in the UK and USA. J. Bus. Contin. Emerg. Plan. 2014, 7, 149–162. [Google Scholar] [CrossRef]
- Poulin, C.; Kane, M.B. Infrastructure resilience curves: Performance measures and summary metrics. Reliab. Eng. Syst. Saf. 2021, 216, 107926. [Google Scholar] [CrossRef]
- Curt, C.; Tacnet, J.M. Resilience of critical infrastructures: Review and analysis of current approaches. Risk Anal. 2018, 38, 2441–2458. [Google Scholar] [CrossRef] [PubMed]
- Aven, T. The Call for a Shift from Risk to Resilience: What Does it Mean? Risk Anal. 2019, 39, 1196–1203. [Google Scholar] [CrossRef] [PubMed]
- Dacorogna, M.; Debbabi, N.; Kratz, M. Building up cyber resilience by better grasping cyber risk via a new algorithm for modelling heavy-tailed data. Eur. J. Oper. Res. 2023, 311, 708–729. [Google Scholar] [CrossRef]
- Jacobs, B. A Comparative Study of EU and US Regulatory Approaches to Cybersecurity in Space. Air Space Law 2023, 48, 477–492. [Google Scholar] [CrossRef]
- Asgari, H.; Haines, S.; Waller, A. Security Risk Assessment and Risk Treatment for Integrated Modular Communication. In Proceedings of the 2016 11th International Conference on Availability, Reliability and Security (ARES), Salzburg, Austria, 31 August–2 September 2016; pp. 503–509. [Google Scholar] [CrossRef]
- Lalonde, C.; Boiral, O. Managing risks through ISO 31000: A critical analysis. Risk Manag. 2012, 14, 272–300. [Google Scholar] [CrossRef]
- Moeller, R.R. COSO Enterprise Risk Management: Understanding the New Integrated ERM Framework; John Wiley & Sons: Hoboken, NJ, USA, 2007. [Google Scholar]
- Moeller, R.R. COSO Enterprise Risk Management: Establishing Effective Governance, Risk, and Compliance Processes; John Wiley & Sons: Hoboken, NJ, USA, 2011; Volume 560. [Google Scholar]
- Stackpole, C.S. A User’s Manual to the PMBOK Guide; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar]
- Ord, T.; Hillerbrand, R.; Sandberg, A. Probing the improbable: Methodological challenges for risks with low probabilities and high stakes. J. Risk Res. 2010, 13, 191–205. [Google Scholar] [CrossRef]
- Bier, V.M. Challenges to the acceptance of probabilistic risk analysis. Risk Anal. 1999, 19, 703–710. [Google Scholar] [CrossRef]
- Yan, S.; Zhou, X.; Hu, J.; Hanly, S.V. Low probability of detection communication: Opportunities and challenges. IEEE Wirel. Commun. 2019, 26, 19–25. [Google Scholar] [CrossRef]
- Ardebili, A.A.; Longo, A.; Ficarella, A. Digital Twins bonds society with cyber-physical Energy Systems: A literature review. In Proceedings of the 2021 IEEE International Conferences on Internet of Things (iThings) and IEEE Green Computing & Communications (GreenCom) and IEEE Cyber, Physical & Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics (Cybermatics), Melbourne, Australia, 6–8 December 2021; pp. 284–289. [Google Scholar] [CrossRef]
- Kassem, M.A.; Khoiry, M.A.; Hamzah, N. Using probability impact matrix (PIM) in analyzing risk factors affecting the success of oil and gas construction projects in Yemen. Int. J. Energy Sect. Manag. 2020, 14, 527–546. [Google Scholar] [CrossRef]
- Aghazadeh Ardebili, A.; Padoano, E. A literature review of the concepts of resilience and sustainability in group decision-making. Sustainability 2020, 12, 2602. [Google Scholar] [CrossRef]
- Fraccascia, L.; Giannoccaro, I.; Albino, V. Resilience of complex systems: State of the art and directions for future research. Complexity 2018, 2018, 3421529. [Google Scholar] [CrossRef]
- Dvorský, P.; Baštán, O.; Fiedler, P. Complex systems resilience qualification. In Proceedings of the Electrical Engineering, Information and Communication Technologies, EEICT, Brno, Czech Republic, 26 April 2022; pp. 383–387. [Google Scholar]
- Zhai, C. Introduction to Complex System Resilience. In Control and Optimization Methods for Complex System Resilience; Springer Nature: Singapore, 2023; pp. 1–11. [Google Scholar] [CrossRef]
- Werner, M.J.E.; Yamada, A.P.L.; Domingos, E.G.N.; Leite, L.R.; Pereira, C.R. Exploring organizational resilience through key performance indicators. J. Ind. Prod. Eng. 2021, 38, 51–65. [Google Scholar] [CrossRef]
- Windle, G.; Bennett, K.M.; Noyes, J. A methodological review of resilience measurement scales. Health Qual. Life Outcomes 2011, 9, 8. [Google Scholar] [CrossRef] [PubMed]
- Srinivas, J.; Das, A.K.; Kumar, N. Government regulations in cyber security: Framework, standards and recommendations. Future Gener. Comput. Syst. 2019, 92, 178–188. [Google Scholar] [CrossRef]
- Pavão, J.; Bastardo, R.; Carreira, D.; Rocha, N.P. Cyber Resilience, a Survey of Case Studies. Procedia Comput. Sci. 2023, 219, 312–318. [Google Scholar] [CrossRef]
- ISO/TS 31050:2023; Risk Management—Guidelines for Managing an Emerging Risk to Enhance Resilience. International Organization for Standardization: Geneva, Switzerland, 2023.
- Fahrurozi, M.; Tarigan, S.A.; Tanjung, M.A.; Mutijarsa, K. The use of ISO/IEC 27005: 2018 for strengthening information security management (a case study at data and information Center of Ministry of Defence). In Proceedings of the 2020 12th International Conference on Information Technology and Electrical Engineering (ICITEE), Yogyakarta, Indonesia, 6–8 October 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 86–91. [Google Scholar]
- Ţigănoaia, B. Some aspects regarding the information security management system within organizations–adopting the ISO/IEC 27001:2013 standard. Stud. Inform. Control 2015, 24, 201–210. [Google Scholar]
- Hendaryatna, H.; Firmansyah, G.; Tjahjono, B.; Widodo, A.M. Performance Evaluation of Business Continuity Plan in Dealing with Threats and Risks in Cilegon Companies Use ISO 22301:2019 & NIST Sp 800-30 R1 Frameworks Case Study: PT. X. Asian J. Soc. Humanit. 2023, 1, 1159–1174. [Google Scholar]
- Ross, R.S. Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy; Technical Report; National Institute of Standards and Technology: Gaithersburg, MD, USA, 2018. [Google Scholar] [CrossRef]
- Ekimova, E.I.; Silaeva, V.V.; Mikhaylov, Y.I. Quality Assurance of Industrial Enterprise Processes Using Risk Identification Methods. In Proceedings of the 2022 Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus), Saint Petersburg, Russia, 25–28 January 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1650–1653. [Google Scholar]
- Crask, J. Business Continuity Management: A Practical Guide to Organization Resilience and ISO 22301; Kogan Page Publishers: London, UK, 2024. [Google Scholar]
- Mishra, S.; Shukla, V.K. Latest risk management guideline (iso 14971:2019) & environmental aspects of medical device. Int. J. Drug Regul. Aff. 2020, 8, 15–24. [Google Scholar]
- Jun, B.; Kim, J.; Kang, B. A study on the improving ways for effective operation of ISO 37001. J. Korea Soc. Digit. Ind. Inf. Manag. 2020, 16, 73–82. [Google Scholar]
- Prislan, K.; Bernik, I. Risk management with ISO 27000 standards in information security. Inf. Secur. 2010, 58–63. [Google Scholar]
- Co, M.; Coleman, C.L.; Davidson, J.W.; Ghosh, S.; Hiser, J.D.; Knight, J.C.; Nguyen-Tuong, A. A lightweight software control system for cyber awareness and security. In Proceedings of the 2009 2nd International Symposium on Resilient Control Systems, Idaho Falls, ID, USA, 11–13 August 2009; pp. 19–24. [Google Scholar] [CrossRef]
- Stillman, R.; Defiore, C. Computer Security and Networking Protocols: Technical Issues in Military Data Communications Networks. IEEE Trans. Commun. 1980, 28, 1472–1477. [Google Scholar] [CrossRef]
- Dupont, B. The cyber-resilience of financial institutions: Significance and applicability. J. Cybersecur. 2019, 5, tyz013. [Google Scholar] [CrossRef]
- Galinec, D.; Steingartner, W. Combining cybersecurity and cyber defense to achieve cyber resilience. In Proceedings of the 2017 IEEE 14th International Scientific Conference on Informatics, Poprad, Slovakia, 14–16 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 87–93. [Google Scholar]
- Onwubiko, C. Focusing on the Recovery Aspects of Cyber Resilience. In Proceedings of the 2020 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA), Dublin, Ireland, 15–19 June 2020; pp. 1–13. [Google Scholar] [CrossRef]
- Mihalache, S.F.; Pricop, E.; Fattahi, J. Resilience enhancement of cyber-physical systems: A review. In Power Systems Resilience: Modeling, Analysis and Practice; Springer: Cham, Switzerland, 2019; pp. 269–287. [Google Scholar]
- Bodeau, D.; Graubart, R.; Picciotto, J.; McQuaid, R. Cyber Resiliency Engineering Framework; MTR110237; MITRECorporation: Bedford, MA, USA, 2011. [Google Scholar]
- Kott, A.; Linkov, I. To improve cyber resilience, measure it. arXiv 2021, arXiv:2102.09455. [Google Scholar] [CrossRef]
- Bellini, E.; Sargsyan, G.; Kavallieros, D. Cyber-resilience. In Internet of Things, Threats, Landscape, and Countermeasures; CRC Press: Boca Raton, FL, USA, 2021; pp. 291–333. [Google Scholar]
- Haimes, Y.Y.; Crowther, K.; Horowitz, B.M. Homeland security preparedness: Balancing protection with resilience in emergent systems. Syst. Eng. 2008, 11, 287–308. [Google Scholar] [CrossRef]
- Linkov, I.; Kott, A. Fundamental concepts of cyber resilience: Introduction and overview. In Cyber resilience of systems and networks; Springer: Berlin/Heidelberg, Germany, 2019; pp. 1–25. [Google Scholar]
- Tjoa, S.; Gafić, M.; Kieseberg, P. Standards and Best Practices. In Cyber Resilience Fundamentals; Springer: Berlin/Heidelberg, Germany, 2024; pp. 23–35. [Google Scholar]
- Brás, J.C.; Pereira, R.F.; Moro, S.; Bianchi, I.S.; Ribeiro, R. Understanding How Intelligent Process Automation Impacts Business Continuity: Mapping IEEE/2755:2020 and ISO/22301:2019. IEEE Access 2023, 11, 134239–134258. [Google Scholar] [CrossRef]
- Jovanović, A.; Klimek, P.; Renn, O.; Schneider, R.; Øien, K.; Brown, J.; DiGennaro, M.; Liu, Y.; Pfau, V.; Jelić, M.; et al. Assessing resilience of healthcare infrastructure exposed to COVID-19: Emerging risks, resilience indicators, interdependencies and international standards. Environ. Syst. Decis. 2020, 40, 252–286. [Google Scholar] [CrossRef]
- Jovanovic, A.; Renn, O.; Petit, F. Towards more aligned/standardized solutions for indicator-based resilience assessment. Resilience 2017, 2, 111. [Google Scholar]
- Janta, P.; Leeraphun, N.; Thapmanee, K.; Niyomna, P.; Sintuya, H.; Setthapun, W.; Maneechot, P.; Sriprapakhan, P.; Chollacoop, N.; Silva, K. Energy resilience assessment: Incorporating consideration of recoverability and adaptability in risk assessment of energy infrastructure. Energy Sustain. Dev. 2024, 81, 101506. [Google Scholar] [CrossRef]
- Proag, V. The concept of vulnerability and resilience. Procedia Econ. Financ. 2014, 18, 369–376. [Google Scholar] [CrossRef]
- Ciapessoni, E.; Cirio, D.; Pitto, A.; Sforna, M. A risk-based resilience assessment tool to anticipate critical system conditions in case of natural threats. In Proceedings of the 2019 IEEE Milan PowerTech, Milan, Italy, 23–27 June 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1–6. [Google Scholar]
- Park, S.; Park, H. PIER: Cyber-Resilient Risk Assessment Model for Connected and Autonomous Vehicles. Wirel. Netw. 2022, 30, 4591–4605. [Google Scholar] [CrossRef]
- Chua Chow, C.; Sarin, R.K. Known, unknown, and unknowable uncertainties. Theory Decis. 2002, 52, 127–138. [Google Scholar] [CrossRef]
- Dester, W.; Blockley, D. Managing the uncertainty of unknown risks. Civ. Eng. Environ. Syst. 2003, 20, 83–103. [Google Scholar] [CrossRef]
- Baltussen, G.; Van Bekkum, S.; Van Der Grient, B. Unknown unknowns: Uncertainty about risk and stock returns. J. Financ. Quant. Anal. 2018, 53, 1615–1651. [Google Scholar] [CrossRef]
- Park, J.; Seager, T.P.; Rao, P.S.C.; Convertino, M.; Linkov, I. Integrating Risk and Resilience Approaches to Catastrophe Management in Engineering Systems. Risk Anal. 2013, 33, 356–367. [Google Scholar] [CrossRef] [PubMed]
- Ganin, A.A.; Massaro, E.; Gutfraind, A.; Steen, N.; Keisler, J.M.; Kott, A.; Mangoubi, R.; Linkov, I. Operational resilience: Concepts, design and analysis. Sci. Rep. 2016, 6, 19540. [Google Scholar] [CrossRef]
- Al-Turkistani, H.F.; AlFaadhel, A. Cyber Resiliency in the Context of Cloud Computing Through Cyber Risk Assessment. In Proceedings of the 2021 1st International Conference on Artificial Intelligence and Data Analytics (CAIDA), Riyadh, Saudi Arabia, 6–7 April 2021; pp. 73–78. [Google Scholar] [CrossRef]
- Rosato, V.; Pietro, A.D.; Kotzanikolaou, P.; Stergiopoulos, G.; Smedile, G. Integrating Resilience in Time-based Dependency Analysis: A Large-Scale Case Study for Urban Critical Infrastructures. In Issues on Risk Analysis for Critical Infrastructure Protection; Rosato, V., Pietro, A.D., Eds.; IntechOpen: Rijeka, Croatia, 2021; Chapter 5. [Google Scholar] [CrossRef]
- Stergiopoulos, G.; Kotzanikolaou, P.; Theocharidou, M.; Lykou, G.; Gritzalis, D. Time-based critical infrastructure dependency analysis for large-scale and cross-sectoral failures. Int. J. Crit. Infrastruct. Prot. 2016, 12, 46–60. [Google Scholar] [CrossRef]
- Ale, B.; Burnap, P.; Slater, D. On the origin of PCDS—(Probability consequence diagrams). Saf. Sci. 2015, 72, 229–239. [Google Scholar] [CrossRef]
- Tiusanen, R. Qualitative Risk Analysis. In Handbook of Safety Principles; Wiley: Hoboken, NJ, USA, 2017. [Google Scholar] [CrossRef]
- Anjum, R.L.; Rocca, E. Defining and Assessing Risk. Bias about Values and Probability. In Philosophy of Science; Springer: Berlin/Heidelberg, Germany, 2024; pp. 139–159. [Google Scholar]
- Hubbard, D.; Evans, D. Problems with scoring methods and ordinal scales in risk assessment. IBM J. Res. Dev. 2010, 54, 246–255. [Google Scholar] [CrossRef]
- Woolf, H. Assessment criteria: Reflections on current practices. Assess. Eval. High. Educ. 2004, 29, 479–493. [Google Scholar] [CrossRef]
- Weitz, J. Criteria for criteria. Am. Psychol. 1961, 16, 228. [Google Scholar] [CrossRef]
- Wieringa, R.; Maiden, N.; Mead, N.; Rolland, C. Requirements engineering paper classification and evaluation criteria: A proposal and a discussion. Requir. Eng. 2006, 11, 102–107. [Google Scholar] [CrossRef]
- Montmain, J.; Labreuche, C.; Imoussaten, A.; Trousset, F. Multi-criteria improvement of complex systems. Inf. Sci. 2015, 291, 61–84. [Google Scholar] [CrossRef]
- Voronin, A. Multi-Criteria Decision Making for the Management of Complex Systems; IGI Global: Hershey, PA, USA, 2017. [Google Scholar]
- Rowe, W.D. Understanding uncertainty. Risk Anal. 1994, 14, 743–750. [Google Scholar] [CrossRef]
- Rozet, E.; Marini, R.; Ziemons, E.; Boulanger, B.; Hubert, P. Advances in validation, risk and uncertainty assessment of bioanalytical methods. J. Pharm. Biomed. Anal. 2011, 55, 848–858. [Google Scholar] [CrossRef]
- Carot, J.M.; Conchado, A. Detecting measurement biases: Sources of uncertainty, accuracy, and precision of the measurements. In Imaging Biomarkers: Development and Clinical Integration; Springer: Berlin/Heidelberg, Germany, 2017; pp. 101–113. [Google Scholar]
- Perkins, J.; Suh, S. Uncertainty implications of hybrid approach in LCA: Precision versus accuracy. Environ. Sci. Technol. 2019, 53, 3681–3688. [Google Scholar] [CrossRef]
- Maurer, S.D.; Lee, T.W. Accuracy of the situational interview in rating multiple job candidates. J. Bus. Psychol. 2000, 15, 73–96. [Google Scholar] [CrossRef]
- Heracleous, C.; Kolios, P.; Panayiotou, C.G.; Ellinas, G.; Polycarpou, M.M. Hybrid systems modeling for critical infrastructures interdependency analysis. Reliab. Eng. Syst. Saf. 2017, 165, 89–101. [Google Scholar] [CrossRef]
- Kuhn, A.M.; Youngberg, B.J. The need for risk management to evolve to assure a culture of safety. BMJ Qual. Saf. 2002, 11, 158–162. [Google Scholar] [CrossRef]
- Simpson, A.; Murnane, R.; Saito, K.; Phillips, E.; Reid, R.; Himmelfarb, A. Understanding Risk in an Evolving World: A Policy Note; World Bank Publications: Washington, DC, USA, 2014. [Google Scholar]
- Tranfield, D.; Denyer, D.; Smart, P. Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. Br. J. Manag. 2003, 14, 207–222. [Google Scholar] [CrossRef]
- Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
- Lezzi, M.; Lazoi, M.; Corallo, A. Cybersecurity for Industry 4.0 in the current literature: A reference framework. Comput. Ind. 2018, 103, 97–110. [Google Scholar] [CrossRef]
- Ramirez, R.; Choucri, N. Improving Interdisciplinary Communication With Standardized Cyber Security Terminology: A Literature Review. IEEE Access 2016, 4, 2216–2243. [Google Scholar] [CrossRef]
- Hull, D.; Pettifer, S.R.; Kell, D.B. Defrosting the Digital Library: Bibliographic Tools for the next Generation Web. PLoS Comput. Biol. 2008, 4, e1000204. [Google Scholar] [CrossRef]
- Asgari, H.; Haines, S.; Rysavy, O. Identification of Threats and Security Risk Assessments for Recursive Internet Architecture. IEEE Syst. J. 2018, 12, 2437–2448. [Google Scholar] [CrossRef]
- Koay, A.M.Y.; Ko, R.K.L.; Hettema, H.; Radke, K. Machine Learning in Industrial Control System (ICS) Security: Current Landscape, Opportunities and Challenges. J. Intell. Inf. Syst. 2023, 60, 377–405. [Google Scholar] [CrossRef]
- Siddiqui, F.; Khan, R.; Tasdemir, S.Y.; Hui, H.; Sonigara, B.; Sezer, S.; McLaughlin, K. Cybersecurity Engineering: Bridging the Security Gaps in Advanced Automotive Systems and ISO/SAE 21434. In Proceedings of the 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring), Florence, Italy, 20–23 June 2023; pp. 1–6. [Google Scholar] [CrossRef]
- Strandberg, K.; Rosenstatter, T.; Jolak, R.; Nowdehi, N.; Olovsson, T. Resilient Shield: Reinforcing the Resilience of Vehicles Against Security Threats. In Proceedings of the 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), Helsinki, Finland, 25–28 April 2021; pp. 1–7. [Google Scholar] [CrossRef]
- Siddiqui, F.; Ahlbrecht, A.; Khan, R.; Tasdemir, S.Y.; Hui, H.; Sonigara, B.; Sezer, S.; McLaughlin, K.; Zaeske, W.; Durak, U. Cybersecurity Engineering: Bridging the Security Gaps in Avionics Architectures and DO-326A/ED-202A. In Proceedings of the 2023 IEEE/AIAA 42nd Digital Avionics Systems Conference (DASC), Barcelona, Spain, 1–5 October 2023; pp. 1–8. [Google Scholar] [CrossRef]
- Lykou, G.; Anagnostopoulou, A.; Gritzalis, D. Smart Airport Cybersecurity: Threat Mitigation and Cyber Resilience Controls. Sensors 2019, 19, 19. [Google Scholar] [CrossRef]
- Elmarady, A.A.; Rahouma, K. Studying Cybersecurity in Civil Aviation, Including Developing and Applying Aviation Cybersecurity Risk Assessment. IEEE Access 2021, 9, 143997–144016. [Google Scholar] [CrossRef]
- Kayisoglu, G.; Bolat, P.; Tam, K. Evaluating SLIM-based Human Error Probability for ECDIS Cybersecurity in Maritime. J. Navig. 2022, 75, 1364–1388. [Google Scholar] [CrossRef]
- Soner, O.; Kayisoglu, G.; Bolat, P.; Tam, K. Risk Sensitivity Analysis of AIS Cyber Security through Maritime Cyber Regulatory Frameworks. Appl. Ocean Res. 2024, 142, 103855. [Google Scholar] [CrossRef]
- Huda, S.; Islam, M.R.; Abawajy, J.; Kottala, V.N.V.; Ahmad, S. A Cyber Risk Assessment Approach to Federated Identity Management Framework-Based Digital Healthcare System. Sensors 2024, 24, 5282. [Google Scholar] [CrossRef]
- Pavão, J.; Bastardo, R.; Rocha, N.P. Cyber Resilience and Healthcare Information Systems, a Systematic Review. Procedia Comput. Sci. 2024, 239, 149–157. [Google Scholar] [CrossRef]
- Smyrlis, M.; Floros, E.; Basdekis, I.; Prelipcean, D.B.; Sotiropoulos, A.; Debar, H.; Zarras, A.; Spanoudakis, G. RAMA: A Risk Assessment Solution for Healthcare Organizations. Int. J. Inf. Secur. 2024, 23, 1821–1838. [Google Scholar] [CrossRef]
- Jawhar, S.; Kimble, C.E.; Miller, J.R.; Bitar, Z. Enhancing Cyber Resilience with AI-Powered Cyber Insurance Risk Assessment. In Proceedings of the 2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 8–10 January 2024; pp. 0435–0438. [Google Scholar] [CrossRef]
- McIntosh, T.R.; Susnjak, T.; Liu, T.; Watters, P.; Xu, D.; Liu, D.; Nowrozy, R.; Halgamuge, M.N. From COBIT to ISO 42001: Evaluating Cybersecurity Frameworks for Opportunities, Risks, and Regulatory Compliance in Commercializing Large Language Models. Comput. Secur. 2024, 144, 103964. [Google Scholar] [CrossRef]
- Abdi, A.; Bennouri, H.; Keane, A. Cyber Resilience, Risk Management, and Security Challenges in Enterprise-Scale Cloud Systems: Comprehensive Review. In Proceedings of the 2024 13th Mediterranean Conference on Embedded Computing (MECO), Budva, Montenegro, 11–14 June 2024; pp. 1–8. [Google Scholar] [CrossRef]
- Maziku, H.; Shetty, S. Software Defined Networking Enabled Resilience for IEC 61850-Based Substation Communication Systems. In Proceedings of the 2017 International Conference on Computing, Networking and Communications (ICNC), Santa Clara, CA, USA, 26–29 January 2017; pp. 690–694. [Google Scholar] [CrossRef]
- Ibne Hossain, N.U.; Nagahi, M.; Jaradat, R.; Shah, C.; Buchanan, R.; Hamilton, M. Modeling and Assessing Cyber Resilience of Smart Grid Using Bayesian Network-Based Approach: A System of Systems Problem. J. Comput. Des. Eng. 2020, 7, 352–366. [Google Scholar] [CrossRef]
- Culler, M.J.; Morash, S.; Smith, B.; Cleveland, F.; Gentle, J. A Cyber-Resilience Risk Management Architecture for Distributed Wind. In Proceedings of the 2021 Resilience Week (RWS), Salt Lake City, UT, USA, 18–21 October 2021; pp. 1–8. [Google Scholar] [CrossRef]
- Mehmood, A.; Epiphaniou, G.; Maple, C.; Ersotelos, N.; Wiseman, R. A Hybrid Methodology to Assess Cyber Resilience of IoT in Energy Management and Connected Sites. Sensors 2023, 23, 8720. [Google Scholar] [CrossRef]
- Kolosok, I.; Gurina, L. Cyber Resilience Models of Systems for Monitoring and Operational Dispatch Control of Electric Power Systems. IFAC-PapersOnLine 2022, 55, 485–490. [Google Scholar] [CrossRef]
- Khanna, K.; Govindarasu, M. Resiliency-Driven Cyber–Physical Risk Assessment and Investment Planning for Power Substations. IEEE Trans. Control Syst. Technol. 2024, 32, 1743–1754. [Google Scholar] [CrossRef]
- Gkoktsis, G.; Lauer, H.; Jaeger, L. Risk Assessments in Virtual Power Plants with NESCOR Criteria, Practical Application, Advantages and Disadvantages. In Proceedings of the 18th International Conference on Availability, Reliability and Security, ARES ’23, Benevento, Italy, 29 August–1 September 2023; pp. 1–11. [Google Scholar] [CrossRef]
- Panda, A.; Bower, A. Cyber Security and the Disaster Resilience Framework. Int. J. Disaster Resil. Built Environ. 2020, 11, 507–518. [Google Scholar] [CrossRef]
- Sepúlveda Estay, D.A.; Sahay, R.; Barfod, M.B.; Jensen, C.D. A Systematic Review of Cyber-Resilience Assessment Frameworks. Comput. Secur. 2020, 97, 101996. [Google Scholar] [CrossRef]
- Shaffique, M.R. Cyber Resilience Act 2022: A Silver Bullet for Cybersecurity of IoT Devices or a Shot in the Dark? Comput. Law Secur. Rev. 2024, 54, 106009. [Google Scholar] [CrossRef]
- AL-Hawamleh, A. Cyber Resilience Framework: Strengthening Defenses and Enhancing Continuity in Business Security. Int. J. Comput. Digit. Syst. 2024, 15, 1315–1331. [Google Scholar] [CrossRef] [PubMed]
- Bahmanova, A.; Lace, N. Cyber Risks: Systematic Literature Analysis. In Proceedings of the 15th International Multi-Conference on Complexity, Informatics and Cybernetics, Virtual, 26–29 March 2024; pp. 177–184. [Google Scholar] [CrossRef]
- Kanaan, A.; AL-Hawamleh, A.; Alorfi, A.; Aloun, M. Cybersecurity Resilience for Business: A Comprehensive Model for Proactive Defense and Swift Recovery. In Proceedings of the 2024 2nd International Conference on Cyber Resilience (ICCR), Dubai, United Arab Emirates, 26–28 February 2024; IEEE: Piscataway, NJ, USA, 2024; pp. 1–7. [Google Scholar] [CrossRef]
- Itani, D.; Itani, R.; Eltweri, A.A.; Faccia, A.; Wanganoo, L. Enhancing Cybersecurity Through Compliance and Auditing: A Strategic Approach to Resilience. In Proceedings of the 2024 2nd International Conference on Cyber Resilience (ICCR), Dubai, United Arab Emirates, 26–28 February 2024; pp. 1–10. [Google Scholar] [CrossRef]
- Al-Hawawreh, M.; Doss, R. Enhancing Security in Industrial IoT: A Taxonomy-driven Approach to Risk Assessment. In Proceedings of the 2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Exeter, UK, 1–3 November 2023; pp. 434–443. [Google Scholar] [CrossRef]
- Bakalynskyi, O.; Korobeynikov, F. Establishing Goals in the Creation of Cyber-Resilient Systems per NIST. In Proceedings of the 2023 13th International Conference on Dependable Systems, Services and Technologies (DESSERT), Athens, Greece, 13–15 October 2023; pp. 1–4. [Google Scholar] [CrossRef]
- Chandra, N.A.; Ramli, K.; Ratna, A.A.P.; Gunawan, T.S. Information Security Risk Assessment Using Situational Awareness Frameworks and Application Tools. Risks 2022, 10, 165. [Google Scholar] [CrossRef]
- Haque, M.A.; Shetty, S.; Kamhoua, C.A.; Gold, K. Integrating Mission-Centric Impact Assessment to Operational Resiliency in Cyber-Physical Systems. In Proceedings of the GLOBECOM 2020—2020 IEEE Global Communications Conference, Taipei, Taiwan, 7–11 December 2020; pp. 1–7. [Google Scholar] [CrossRef]
- Murino, G.; Ribaudo, M.; Romano, S.; Tacchella, A. OT Cyber Security Frameworks Comparison Tool (CSFCTool). In Proceedings of the Italian Conference on Cybersecurity, Online, 8–9 April 2021. [Google Scholar]
- Mantas, E.; Papadopoulos, D.; Fernández, C.; Ortiz, N.; Compastié, M.; Martínez, A.L.; Pérez, M.G.; Kourtis, A.; Xylouris, G.; Mlakar, I.; et al. Practical Autonomous Cyberhealth for Resilient Micro, Small and Medium-sized Enterprises. In Proceedings of the 2021 IEEE International Mediterranean Conference on Communications and Networking (MeditCom), Athens, Greece, 7–10 September 2021; pp. 500–505. [Google Scholar] [CrossRef]
- Maziku, H.; Shetty, S.; Nicol, D.M. Security Risk Assessment for SDN-enabled Smart Grids. Comput. Commun. 2019, 133, 1–11. [Google Scholar] [CrossRef]
- Pourmadadkar, M.; Lezzi, M.; Corallo, A. Cyber Security for Cyber-Physical Systems in Critical Infrastructures: Bibliometrics Analysis and Future Directions. IEEE Trans. Eng. Manag. 2024, 71, 15405–15421. [Google Scholar] [CrossRef]
- Zhou, Y.; Yu, F.R.; Chen, J.; Kuo, Y. Cyber-Physical-Social Systems: A State-of-the-Art Survey, Challenges and Opportunities. IEEE Commun. Surv. Tutor. 2020, 22, 389–425. [Google Scholar] [CrossRef]
- Shin, J.; Son, H.; Heo, G. Development of a cyber security risk model using Bayesian networks. Reliab. Eng. Syst. Saf. 2015, 134, 208–217. [Google Scholar] [CrossRef]
- Lallie, H.S.; Debattista, K.; Bal, J. A review of attack graph and attack tree visual syntax in cyber security. Comput. Sci. Rev. 2020, 35, 100219. [Google Scholar] [CrossRef]
- Kathayat, D. International Maritime Organization and Cyber Risk Management Framework. Supremo Amic. 2022, 31, 67. [Google Scholar]
- Ramadan, A.I.H.A.; Ardebili, A.A.; Longo, A.; Ficarella, A. Advancing Resilience in Green Energy Systems: Comprehensive Review of AI-based Data-driven Solutions for Security and Safety. In Proceedings of the 2023 IEEE International Conference on Big Data (BigData), Sorento, Italy, 15–18 December 2023; pp. 4002–4010. [Google Scholar] [CrossRef]
- Bank for International Settlements. Bank for International Settlements; Bank for International Settlements: Basel, Switzerland, 2024. [Google Scholar]
- European Parliament and Council. Regulation (EU) 2022/2554 on Digital Operational Resilience for the Financial Sector (DORA). Off. J. Eur. Union 2022, L 333. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32022R2554 (accessed on 3 September 2024).
- World Economic Forum. The Cyber Resilience Index: Advancing Organizational Cyber Resilience. World Economic Forum. 2024. Available online: https://www.weforum.org/whitepapers/the-cyber-resilience-index-advancing-organizational-cyber-resilience (accessed on 3 September 2024).
- Jeimy, J.; Cano, M. FLEXI—A Conceptual Model for Enterprise Cyber Resilience. Procedia Comput. Sci. 2023, 219, 11–19. [Google Scholar] [CrossRef]
- EUR-Lex-02022L2555-20221227-EN-EUR-Lex—eur-lex.europa.eu. Available online: https://eur-lex.europa.eu/eli/dir/2022/2555 (accessed on 26 September 2024).
- Sievers, T. Proposal for a NIS directive 2.0: Companies covered by the extended scope of application and their obligations. Int. Cybersecur. Law Rev. 2021, 2, 223–231. [Google Scholar] [CrossRef]
- Voigt, P.; von dem Bussche, A. Scope of Application of the GDPR. In The EU General Data Protection Regulation (GDPR): A Practical Guide; Voigt, P., von dem Bussche, A., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 9–30. [Google Scholar] [CrossRef]
- Adamsky, F.; Aubigny, M.; Battisti, F.; Carli, M.; Cimorelli, F.; Cruz, T.; Di Giorgio, A.; Foglietta, C.; Galli, A.; Giuseppi, A.; et al. Integrated protection of industrial control systems from cyber-attacks: The ATENA approach. Int. J. Crit. Infrastruct. Prot. 2018, 21, 72–82. [Google Scholar] [CrossRef]
- Chui, K.T.; Gupta, B.B.; Liu, J.; Arya, V.; Nedjah, N.; Almomani, A.; Chaurasia, P. A Survey of Internet of Things and Cyber-Physical Systems: Standards, Algorithms, Applications, Security, Challenges, and Future Directions. Information 2023, 14, 388. [Google Scholar] [CrossRef]
Standard | Title | Description | Ref. |
---|---|---|---|
ISO 31000:2018 | Risk Management Guidelines | Principles and guidelines for effective risk management across all industries. | [23] |
ISO/TS 31050:2023 | Guidelines for managing an emerging risk to enhance resilience | This document complements ISO 31000. | [40] |
ISO 27005:2018 | Information Security Risk Management | Guidelines for managing information security risks within organizations. | [41] |
ISO/IEC 27001:2013 | Information Security Management Systems (ISMSs) | Requirements for establishing and maintaining an information security management system with risk management processes. | [42] |
ISO 22301:2019 | Business Continuity Management Systems (BCMS) | Framework for identifying and managing risks that could disrupt business operations. | [43] |
NIST SP 800-37 | Risk Management Framework (RMF) | Guidelines for applying risk management to federal information systems in the United States. | [44] |
COSO ERM Framework | Enterprise Risk Management | Integration of risk management into organizational strategy and performance. | [24,25] |
IEC 31010:2019 | Risk Management—Risk Assessment Techniques | Guidance on various risk assessment techniques. | [45] |
ISO 22301, ISO 22313, ISO 22316 | Organizational Resilience | Principles for enhancing organizational resilience through risk management. | [46] |
ISO 14971:2019 | Application of Risk Management to Medical Devices | Guidelines for applying risk management throughout the life cycle of medical devices. | [47] |
ISO 37001:2016 | Anti-bribery Management Systems | Techniques for robust internal controls and compliance processes that are crucial for addressing cyber threats exploiting corruption. | [48] |
No. | Research Question | Area of Analysis |
---|---|---|
RQ1 | What are the methodologies used to address cyber resilience through risk assessment? | Identifying risk assessment methodologies that take cyber resilience into account. |
RQ2 | Which standards are considered by studies that focus on risk assessment for cyber resilience? | Exploring the standards, frameworks, guidelines, etc., frequently cited in risk assessment studies that consider cyber resilience. |
RQ3 | What regulations are relevant to risk assessment for cyber resilience? | Investigating relevant regulations; examining how legal frameworks impact risk assessment practices for cyber resilience. |
Query ID | Search in | Query String | QTY |
---|---|---|---|
Qs[#doc.] | Scopus | TITLE-ABS-KEY (“cyber resilien*” AND “risk assessment”) | 95 |
Qw[#doc.] | WoS | TS = (“cyber resilien*” AND “Risk assessment”) | 32 |
Qi[#doc.] | IEEE | “cyber resilien*” AND “risk assessment” | 46 |
Criteria Type | Criterion |
---|---|
Exclusion Criteria | 1. Records that are not written in English. 2. Records that are published as technical opinions, editorials, testimonials, organizational reports, book series, or book chapters. 3. Studies that do not discuss cyber risk assessment or cyber resilience. |
Inclusion Criteria | 1. Studies that propose, perform, or validate the cyber risk assessment methods with a cyber resilience perspective 2. Studies that discuss standards, guidelines, or regulations related to risk assessment for cyber resilience. |
CI Sector | Ref. |
---|---|
Transportation systems sector | [22,68,101,102,103,104,105,106,107] |
Energy sector | [114,115,116,117,118,119,120] |
Healthcare and public health sector | [108,109,110] |
IT sector | [111,112,113] |
Emergency services sector | [121] |
Communication sector | [99] |
Not categorized | [9,20,39,122,123,124,125,126,127,128,129,130,131,132,133,134] |
Ref. | Title | Description | Cat. |
---|---|---|---|
[22] | Probability-Impact Matrix | Method that uses probability and impact to assess risks within the STRIDE categories. | Method |
[116] | Risk Quantification (TVI) | Risk assessment method calculating risk based on threat, vulnerability, and impact factors. Risk = Threat × Vulnerability × Impact. | Method |
[111,140] | AI-Enabled Risk Assessment | Risk identification, probability, and impact assessment supported by AI models like ChatGPT. | Method |
[22] | SecRAM by SESAR SWP16.2 | Security risk assessment methodology specific to SESAR projects. Tailored to network systems using probability-impact assessments, focusing on confidentiality, integrity, and availability (CIA) of primary assets. | Method |
[116] | CCE Scoring Method | Method used for assessing impact in risk management. | Method |
[117] | Risk Assessment: ICS Cyber Killchain | Methodology for assessing impact through cyber kill chain analysis. | Method |
[117] | Cybersecurity Capability Maturity Model (C2M2) | Maturity model for evaluating and improving cybersecurity capabilities. | Model |
[109] | Endsley’s Three-Level Model of Situational Awareness | Model for understanding and developing situational awareness. | Model |
[130] | OCTAVE | Operationally Critical Threat, Asset, and Vulnerability Evaluation. | Method |
[130] | FAIR | Factor Analysis of Information Risk. | Method |
[130] | CRAMM | Central Computer and Telecommunications Agency (CCTA) Risk Analysis and Management Method. | Method |
[117] | CVSS | Common vulnerability scoring system (CVSS); for measuring the severity of vulnerabilities. | Method |
[101] | STRIDE Threat Model | Threat modeling method focusing on six categories: Spoofing, tampering, repudiation, information disclosure, denial of service, and elevation of privilege. | Model |
[9] | PASTA | Threat modeling method used to align technical requirements with business objectives. | Model |
[9] | DREAD | Method for risk assessment that evaluates threats based on five factors: Damage, reproducibility, exploitability, affected users, and discoverability (DREAD). | Method |
[121] | Holistic Risk Assessments | Assessment method considering interconnected and interdependent risks, such as disaster risks, cybersecurity risks, and cascading effects. | Method |
[114] | Security Score Model | Model that quantifies security based on predefined scoring criteria. | Model |
[130] | Direct Testing Method | Risk assessment through audit, exercise, and penetration testing. | Method |
[115] | Five-Phase Resilience Assessment Framework | Framework based on Bayesian network that quantifies resilience contributors in five phases. | M.F. |
[110] | RAMA | Framework for risk analysis using factors like probability, impact, historical data, and contextual relevance. | M.F. |
[105] | ICAO Security Risk Assessment | Methodology for security risk assessment using probability and impact in aviation security. | Method |
[101] | TARA (Threat Analysis and Risk Assessment) | Combination of STRIDE for categorizing threat scenarios and DREAD for quantifying feasibility, likelihood, and impact. | Method |
[101] | System Security Risk Assessment (SSRA) | Method defined by DO-326A/ED202A for security risk assessment in systems. | Method |
[131] | Logical Attack Graph (LAG) | A method for visualizing potential attack paths and assessing their impact on systems. | Method |
[131] | Mission Impact Propagation Graph (MIPG) | Graph-based method for assessing how cyber-attacks propagate through missions. | Method |
[131] | Mission Impact Assessment Graph (MIAG) | Graph-based method for assessing the impact of threats on mission-critical operations. | Method |
[39] | Event Tree Analysis | Systematic approach for analyzing the possible outcomes following an initial event. | Method |
[39] | Physical and Cyber Risk Analysis Tool (PACRAT) | Tool for conducting comprehensive risk analysis that includes both physical and cyber threats. | Tool |
[117] | Threat, Vulnerability, and Risk Analysis (TVRA) | Method for assessing threats, vulnerabilities, and overall risk in systems. | Method |
[117] | i-TRACE Risk Assessment | Framework for risk assessment combining traceability and vulnerability management. | M.F. |
[107] | SOHRA (Shipboard Operation Human Reliability Analysis) | Method for assessing human error probability in shipboard operations. | Method |
[20] | Extreme Value Theory (EVT) | Statistical approach for modeling extreme risk events in cybersecurity. | Model |
[20] | Stochastic Hybrid Model | Hybrid approach combining stochastic methods with traditional risk assessment models. | Model |
[20] | Tail Threshold Detection | Method for identifying the threshold above which risk events are considered extreme. | Method |
[114] | SDN-Based Mitigation | Mitigation of risks using software-defined networking principles. | Method |
[134] | IED Criticality Assessment | Assessment of the criticality of intelligent electronic devices in industrial systems. | Method |
[114] | Cyber Attack Modeling | Modeling and assessing the impact of denial of service attacks. | Method |
[22] | Probability-Impact through the SecRAM | Risk assessment methodology focusing on probability and impact, tailored for network systems. | Method |
[108] | Vulnerability-Impact-Likelihood | A formulaic approach to quantifying risk based on vulnerability, impact, and likelihood. | Method |
[132] | Raising Awareness | Promoting cybersecurity awareness among IT/OT end users. | Activity |
[104] | Human Error Probability Assessment | Assessing the probability of human error in system operations. | Method |
[109] | Fault Tree Analysis | Method for analyzing system failures and identifying potential causes. | Method |
[129] | Risk Management Strategy (RMS) | Strategic approach to managing and mitigating risks in organizations. | Strategy |
[128] | Taxonomy-Driven Risk Assessment | Risk assessment methodology using a structured taxonomy to categorize and evaluate risks. | Method |
[133] | Threat Intelligence (TI) Module | Component for collecting and analyzing threat data to improve risk assessments. | Tool |
Ref. | Name | Description | Cat. |
---|---|---|---|
[102] | ISO 21434: Automotive Cyber Security | Standard focusing on cybersecurity for road vehicles. | Std. |
[116] | Risk Management Architecture: INL Framework | Aligning with the NIST framework for managing cybersecurity risks. | Fwk. |
[116] | IEEE 1547.3 | Guide for cybersecurity of distributed energy resources interconnected with electric power systems. | Std. |
[107,120,122,133] | ISO/IEC 27001 | Information security management standard. | Std. |
[107,109,113,120,122,124,127,133,134] | NIST Cybersecurity Framework (CSF) | Guidelines for improving CI. | Gdl. |
[9] | Cyber Resilience Framework (CRF) | Structured approach for ensuring resilience in information systems. | Fwk. |
[9] | CERT Resilience Management Model (CERTRMM) | Framework for managing operational resilience. | Fwk. |
[9,112] | COBIT | Framework for enterprise IT governance and management. | Fwk. |
[9,128] | OWASP | Open Web Application Security Project framework for software security. | Fwk. |
[121] | Sendai Framework for Disaster Risk Reduction 2015–2030 | Global strategy for reducing disaster risk and building resilience. | Fwk. |
[121] | UN Guidelines on Man-Made and Technological Hazards | Guidelines provided by the UN Office for Disaster Risk Reduction (DRR). | Gdl. |
[130] | ISO 27001:2013 | Updated version of ISO 27001 focusing on information security management. | Std. |
[109,120,122,130,133] | ISO 27005 | Risk management standard for information security. | Std. |
[22] | ISO 31010 | Standard for risk assessment techniques. | Std. |
[117,132] | ISO 31000 | Guidelines for risk management. | Std. |
[22,68,108,118,123,144] | NIST SP 800-30 | Guide for conducting risk assessments. | Gdl. |
[22,129,144] | MITRE | Framework for threat analysis and mitigation. | Fwk. |
[22,133] | ENISA | European Union Agency for cybersecurity recommendations and guidelines. | Gdl. |
[112] | COBIT 2019 | Updated COBIT framework for IT governance and management. | Fwk. |
[112] | ISO 42001:2023 | Standard for governance, risk, and compliance (GRC). | Std. |
[115] | NISTIR: Guidelines for Smart Grid Cybersecurity | Guidelines for cybersecurity of smart grids. | Gdl. |
[111] | NIST AI RMF | NIST AI risk management framework for managing AI risks. | Fwk. |
[22] | DO326A/ED-202A | Guidelines for cybersecurity in aviation. | Gdl. |
[129] | NIST SP 800-160, Volume 2 | Systems security engineering guidelines. | Gdl. |
[134] | IEC 61850 | International standard for communication networks and systems in substations. | Std. |
[39] | ISO/SAE 21434:2021 | Standard for cybersecurity engineering in road vehicles. | Std. |
[117,119] | NERC-CIP | North American Electric Reliability Corporation CI Protection standards for securing bulk power systems. | Std. |
[106] | IMO MSC.428 (Maritime Cyber Risk Management) | Guidelines for integrating cyber risk management into the safety management systems (SMSs) of vessels. | Gdl. |
[106] | IACS E26 and E27 | Cybersecurity guidelines for new vessels, issued by the International Association of Classification Societies. | Gdl. |
[106] | BIMCO Cyber Security Guidelines | Guidelines provided by the Baltic and International Maritime Council for managing cybersecurity on ships. | Gdl. |
[106] | Safety4Sea STCW Convention Guidelines | General guidelines for officers under the STCW Convention, with a focus on cybersecurity. | Gdl. |
[119] | Cybersecurity and Infrastructure Security Agency (CISA) Report | Report by CISA providing insights into cybersecurity threats and defense strategies. | Rpt. |
[110] | ENISA Cybersecurity Skills Report | Report by the European Union Agency for Cybersecurity on developing cybersecurity skills in the EU. | Rpt. |
[127] | Cyberthreat Defense Report (CDR) | Annual report highlighting trends and data on cyber threats and defenses. | Rpt. |
Ref. | Name | Description | Cat. |
---|---|---|---|
[104,109,117,128,132,133] | NIS Directive | Directive focusing on the security of network and information systems in the EU. | Dir. |
[132] | EU Proposal for NIS 2.0 | Proposal for an updated Network and Information Systems Directive. | Dir. |
[104] | European Community EC/216/2008 Regulation | Regulation concerning data security and protection in the European Community. | Reg. |
[107] | Maritime Safety Management System (SMS) Requirements | Requirements for integrating cyber risk management into maritime SMS. | Oth. |
[106] | STCW Convention | International convention outlining general requirements for seafarers, including cybersecurity. | Oth. |
[109] | HIPAA | Health Insurance Portability and Accountability Act governing data security and privacy in the US healthcare sector. | Act |
[121] | EU Cybersecurity Act (CSA) | Regulation to strengthen the security of network and information systems across the EU. | Reg.; Act. |
[102] | UNECE Regulation | United Nations Economic Commission for Europe regulations focusing on vehicle safety and cybersecurity. | Reg. |
[121] | National Risk Register, UK (2016) | Report outlining the key risks faced by the United Kingdom, including cybersecurity threats. | Oth. |
[121] | National Risk Analysis, Norway (2014) | Analysis detailing the primary risks facing Norway, with emphasis on national security. | Oth. |
[121] | Security Strategy for Society, Finland Security Committee | Strategy document that outlines Finland’s approach to security, including cyber resilience. | Oth. |
[121] | National Safety and Security Strategy, The Netherlands | Strategy for enhancing safety and security in the Netherlands, addressing cyber risks. | Oth. |
[99] | European Operational Concept Validation Methodology (EOCVM) | Methodology for validating operational concepts within the EU. | Oth. |
[112] | EU AI Act | Regulation proposed by the European Union to govern artificial intelligence and its use. | Reg. |
[105] | International Civil Aviation Organization (ICAO) Aviation Cybersecurity Strategy | Strategy for managing cybersecurity risks in international civil aviation. | Oth. |
[101,103] | European Union Aviation Safety Agency (EASA) Concept Paper | Concept paper outlining the approach to aviation safety, including cybersecurity. | Oth. |
[131] | US Department of Energy Cybersecurity Frameworks | Frameworks and guidelines for ensuring cybersecurity in the energy sector. | RegF |
[39,68] | UN Regulation No. 155 on Cybersecurity | Regulation focusing on cybersecurity management systems for vehicles. | Reg. |
[132] | Presidential Policy Directive 21 (PPD-21/2013) | US directive focusing on the security and resilience of CI. | Dir. |
[132] | Cybersecurity Enhancement Act of 2014 | US law to promote cybersecurity research, development, and public–private partnerships. | Act |
[132] | Executive Order (EO) 13636 | US executive order aimed at improving CI cybersecurity. | Act |
[109,117,132] | EU Regulation 2016/679 (GDPR) | General Data Protection Regulation governing data protection and privacy in the EU. | Reg. |
[132] | National Framework for Cybersecurity and Data Protection (FNCS&DP) | Framework for ensuring cybersecurity and data protection at the national level. | RegF. |
[127] | Saudi Arabian Monetary Authority (SAMA) Cybersecurity Framework | Cybersecurity framework used in Saudi Arabia for financial institutions. | RegF. |
Category | Methods |
---|---|
Quantitative Methods |
|
Qualitative Methods |
|
Hybrid Methods |
|
Specialized Models |
|
Scope of Application | Name | Description |
---|---|---|
ISO 21434 | Targets automotive cybersecurity, providing guidelines for protecting vehicle systems from cyber threats [102]. | |
IEEE 1547.3 | Guides the cybersecurity of distributed energy resources within electric power systems [116]. | |
System Level | IEC 61850 | Focuses on communication networks and systems in substations, crucial for electric power system resilience [134]. |
DO326A/ED-202A | Provides guidelines for cybersecurity in aviation, protecting aircraft systems from cyber incidents [22]. | |
NISTIR: Guidelines for Smart Grid Cybersecurity | Addresses cybersecurity issues within smart grid environments [115]. | |
ISO/IEC 27001 | An information security management standard covering a broad range of organizational security aspects [120,133]. | |
COBIT | Framework for IT governance and management that aligns IT processes with business goals [112]. | |
Organizational Level | NIST Cybersecurity Framework (CSF) | Offers guidelines for CI resilience by establishing security protocols for organizational use [133]. |
ISO 31000 | General risk management guidelines applicable across organizations for identifying and mitigating risks [117]. | |
MITRE Framework | Supports threat analysis and mitigation, helping organizations to manage and understand their threat landscapes [129]. | |
ENISA Cybersecurity Skills Report | A report by the EU Agency for Cybersecurity aimed at developing essential cybersecurity skills in the EU [110]. | |
Human-Centric Focus | IMO MSC.428 | Integrates cyber risk management into maritime operations, focusing on the roles and responsibilities of personnel onboard vessels [106]. |
Safety4Sea STCW Convention Guidelines | Provides guidelines for officers with a focus on cybersecurity, emphasizing training and skills needed for cyber resilience in maritime environments [106]. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Aghazadeh Ardebili, A.; Lezzi, M.; Pourmadadkar, M. Risk Assessment for Cyber Resilience of Critical Infrastructures: Methods, Governance, and Standards. Appl. Sci. 2024, 14, 11807. https://doi.org/10.3390/app142411807
Aghazadeh Ardebili A, Lezzi M, Pourmadadkar M. Risk Assessment for Cyber Resilience of Critical Infrastructures: Methods, Governance, and Standards. Applied Sciences. 2024; 14(24):11807. https://doi.org/10.3390/app142411807
Chicago/Turabian StyleAghazadeh Ardebili, Ali, Marianna Lezzi, and Mahdad Pourmadadkar. 2024. "Risk Assessment for Cyber Resilience of Critical Infrastructures: Methods, Governance, and Standards" Applied Sciences 14, no. 24: 11807. https://doi.org/10.3390/app142411807
APA StyleAghazadeh Ardebili, A., Lezzi, M., & Pourmadadkar, M. (2024). Risk Assessment for Cyber Resilience of Critical Infrastructures: Methods, Governance, and Standards. Applied Sciences, 14(24), 11807. https://doi.org/10.3390/app142411807