Guarding Our Vital Systems: A Metric for Critical Infrastructure Cyber Resilience
Abstract
1. Introduction
1.1. Research Contributions
1.2. Research Questions
- How can we design and implement an effective constructive resilience model to detect and respond to the impact of cyber-attacks on critical infrastructure?
- How can we forecast and identify more effective preventive measures to prevent failures in vital infrastructure?
- How can a comprehensive, in-depth comparative analysis using various metrics but limited features be conducted to measure and enhance the level of security in critical infrastructure?
- How can this research provide practical and strategic guidance for decision-makers in companies managing critical infrastructure, as well as supporting global efforts to reduce negative impacts and respond to cyber-attacks?
1.3. Implications and Innovations
2. Related Studies
2.1. A Journey Through Prior Works
2.2. Developing a Metric for Critical Infrastructure’s Cyber Resilience
2.2.1. The Essence of a Critical Infrastructure Cyber Resilience Metric
2.2.2. The Key Components of a Critical Infrastructure Cyber Resilience Metric
- Risk Assessment and Vulnerability Assessment [73]: This metric should include a comprehensive risk assessment to identify potential vulnerabilities in critical infrastructure. This involves evaluating potential cyber threats that infrastructure may face, how often they may occur, and their potential impact [74]. In this context, providing vulnerability scores and risk levels may be necessary to help organizations prioritize actions and allocate resources [75]. Additionally, this assessment should also consider factors such as existing security policies, implemented security controls, and the organization’s level of readiness to face cyber threats [76].
- Response and Recovery Capabilities [77]: This metric should reflect the extent to which critical infrastructure can respond to and recover from cyber-attacks. This includes assessing readiness for cyber incidents, including personnel training, emergency plans, and tools and systems supporting recovery [78]. The ability to respond quickly and effectively to attacks and recover efficiently is a key element in enhancing cyber resilience. Additionally, this metric should also consider factors such as response times, the effectiveness of response actions, and recovery process efficiency.
- The Use of Advanced Technology [79]: The use of advanced technology, such as artificial intelligence, data analytics, and early detection systems, should be measured using this metric [80]. The implementation of these technologies can significantly enhance detection and response capabilities in the face of increasingly complex cyber threats. Therefore, evaluating the use of innovative technology is a key component of a metric of cyber resilience. Additionally, this metric should consider the extent to which these technologies have been integrated into the organization’s security system and their effectiveness in detecting and responding to cyber threats [81].
- Collaboration and Information Sharing [82]: The level of collaboration with other stakeholders, both in the public and private sectors, should be a significant assessment factor in this metric [83]. The ability to share information and collaborate in protecting critical infrastructure collectively is a crucial element in ensuring optimal cyber resilience [84,85]. Additionally, this metric should also consider the extent to which organizations have built and maintained effective collaborative relationships with other stakeholders.
- Impact Measurement and Recovery Times [86]: This metric should include measurement of the impact of cyber-attacks on critical infrastructure, including how quickly infrastructure can recover and resume operations after an attack [87,88]. This measurement provides insights into how much attacks affect operations and how long it takes for full recovery. Additionally, this metric should also consider factors such as the financial impact of attacks, the impact on the organization’s reputation, and the impact on customers or users of critical infrastructure.
2.2.3. The Metric Development Process
- Analysis of Cyber Threat Trends and Patterns [89]: Focus on analyzing trends and patterns in cyber threats without having to collect specific data [90]. Identify common characteristics of previous attacks, frequently targeted infrastructure, and relevant performance indicators. By understanding these trends, strategic insights can be developed to enhance security without further data collection [89].
- The Implementation of Security Metrics [91]: Focus on implementing security metrics by utilizing all previously identified information. This involves applying formulas or equations designed to measure security based on identified trends and patterns in cyber threats [92]. Additionally, the metric implementation should consider factors such as the metric’s consistency with organizational goals, the metric’s usability, and the metric’s capability to provide significant insights.
- Contextual Analysis and Metric Assessment [66,93]: At this stage, the focus is on a contextual analysis and metric assessment. This involves exploring real-world situations without relying on testing [45]. The process may include narrative-based assessments related to the effectiveness of the metric in depicting potential cyber threats faced by organizations. Metric evaluation also needs to consider the reliability, consistency, and relevance of the metric to security environmental dynamics.
- Advanced Development [94]: This stage involves reviewing and improving the metric based on testing results and feedback from users. This may involve adjusting formulas or equations, adding or subtracting metric components, or changes in the data collection processes. The goal is to ensure that the metric remains relevant and effective in measuring the cyber resilience of critical infrastructure [71].
2.2.4. Applications of the Metric
- An Evaluation of Current Resilience Levels [95]: Organizations can use this metric to evaluate the current level of cyber resilience and identify areas where improvements are needed [96,97]. This helps organizations identify priorities for improvement actions and plan effective strategies for enhancing cyber resilience.
- Comparison with Standards and Regulations [98]: This metric allows organizations to compare their level of resilience with industry standards or applicable regulations. This helps organizations ensure compliance with existing guidelines and regulations and determine areas where they may need to make improvements.
- Comparison with Other Organizations [103]: This metric allows for comparison with other organizations in the same sector or a similar sector. Through this comparison, organizations can identify the best practices and see how they compare with other organizations.
2.3. Fortifying Resilience in Critical Infrastructure
2.3.1. Resilience vs. Reliability: Differences and Interconnection
2.3.2. Addressing Diverse Challenges
2.3.3. Fault Tolerance Mechanisms: The Heart of Critical Infrastructure’s Resilience
2.3.4. Categories of Resilience Solutions for Critical Infrastructure
3. Method
3.1. Charting Resilience Metrics
3.1.1. The Important of Process Evaluation Models in Resilience Measurement
3.1.2. Levels of Process Competence in the InfraGuard Cyber Resilience Framework
- Level 5—Security Process Enhancements: This is the highest level in the framework, where security processes have been fully optimized. At this stage, security processes not only run smoothly but are continually improved and perfected based on feedback and learning from previous experiences. Organizations at this level have achieved the highest maturity level in security processes, and each aspect is optimized for full effectiveness. Organizations at this level are leaders in cybersecurity practices.
- Level 4—A Consistent Security Process: At this level, security processes run with consistency and predictability. The processes yield consistent results and meet established quality standards. Consistency is key here, meaning organizations can rely on security processes to deliver predictable results without much variation or uncertainty. Organizations at this level have achieved a very high level of resilience in maintaining cybersecurity.
- Level 3—A Solidified Security Process: At this level, security processes have become robust and established. The processes have proven effective in practice and have become an integral part of daily operations. This indicates that organizations have successfully built a strong foundation for their security, and these processes are considered mature practices in their operations. Organizations at this level have achieved a high level of resilience in maintaining their critical infrastructure.
- Level 2—A Supervised Security Process: At this level, security processes are closely monitored to ensure that all activities proceed according to the planned and established standards. Supervision is a crucial component here, and organizations ensure that security processes unfold as expected, although there may still be room for improvement. Organizations at this level are striving to enhance their resilience and planning the necessary steps to reach a higher level.
- Level 1—An Executed Security Process: At this level, security processes are being executed. Basic security measures have been implemented, and the processes are running according to the basic plan. This is an initial step indicating that the organization has taken basic actions to protect its infrastructure. While work still remains, the first step towards resilience has been taken.
- Level 0—An Unfinished Security Process: This is the lowest level in the framework, where the security processes are unfinished. Some aspects of the processes may not have been implemented or may not be functioning well. This indicates that significant work is needed to achieve a worthy level of resilience.
3.1.3. The Social Dimension in Resilience Measurements
3.1.4. Integrating Process and Social Dimensions into Resilience Measurements
3.1.5. The Importance of a Multidimensional Approach
3.2. The Resilience Spectrum
3.2.1. The Conceptual Structure of the Resilience Spectrum
- Situational Awareness: Situational awareness is crucial in enhancing resilience to cyber threats. Organizations and infrastructure need to actively monitor and detect changes in their operational environment that can affect security. This includes monitoring network traffic, analyzing suspicious activities, and understanding current trends in cyber-attacks.
- Security Assurance: Security assurance involves the actions taken to ensure that systems are secure from threats and attacks. This includes periodic security risk assessments, the implementation of risk mitigation measures, and ensuring strict adherence to security standards.
- Active Defense: The ability to conduct active defense is crucial. Organizations need to have strategies and tools that allow them to detect attacks as early as possible, respond quickly, and even take proactive actions to block or thwart attacks before they damage the system.
- Risk Management: Effective risk management is a vital component of cyber defense. This involves identifying, assessing, and managing the risks associated with cyber-attacks. Risk management can help organizations prioritize risk mitigation and allocate appropriate resources.
- Infrastructure Resilience: The ability of infrastructure to withstand and recover from various threats is key to creating a safe space in the cyber context. This involves planning and implementing strategies to maintain infrastructure operability even in challenging situations.
- Critical Infrastructure Awareness: Awareness of the importance of critical infrastructure is the first step in protecting it and mitigating risks. Organizations need to understand the vulnerabilities in critical infrastructure and how its vulnerabilities can impact national security and the economy.
- Resilience Principles: Resilience principles should guide the design and operation of infrastructure. This includes fundamental principles that guide the design, implementation, and maintenance of infrastructure to remain resilient to threats.
- Infrastructure Safeguarding Policy: Policies and procedures designed to protect infrastructure from physical and cyber threats are key steps in creating a safe space in the cyber context.
- Infrastructure Preparedness: Infrastructure preparedness involves pre-emptive measures to prepare infrastructure for potential threats and attacks. This includes planning, personnel training, and security scenario testing.
- Critical Incident Recovery: Taking steps after a cyber incident to restore normal system operations as quickly as possible is crucial. This involves system recovery, data recovery, and measures to avoid similar incidents in the future.
3.2.2. A Quantitative Scoring Model for Resilience Components
4. Results
Performance Metrics
5. Discussion
5.1. Resilience Grading
- D (Did Not Meet): If a security element or achievement receives a “D” grade, this indicates that the element has not yet reached its goal. A “D” grade indicates that the achievement of this element is in the range of 0% to 20%. This is a concerning point, as it signifies significant weaknesses in that element. Organizations should promptly identify and address these weaknesses to achieve an adequate level of security.
- A (Approaching): When an element or achievement is graded as “A”, this indicates that the element is approaching its goal but still falls within the range of over 20% to 50%. This shows progress, but there is still work required to reach the desired level of security strength. Organizations need to make further improvements to achieve an adequate level of security.
- M (Moderately Met): The “M” grade indicates that an element or achievement has been moderately met, with achievement in the range of more than 50% to 75%. This is a positive sign that the organization has made significant progress in achieving better security. However, there is room for further improvement to reach the optimal level of security.
- W (Well Achieved): The “W” grade signifies that an element or achievement has been well achieved, with achievement in the range of more than 75% to 90%. This is a commendable level of security strength but still allows for minor improvements. Organizations need to monitor this element to maintain a good level of security.
- E (Exceeds Expectations): When an element or achievement receives an “E” grade, this means that the element has not only reached but exceeded expectations, achieving a very high level of security in the range of more than 90% to 100%. This is an outstanding achievement that demonstrates an organization’s ability to maintain security at the highest level. It is essential to continue monitoring and maintaining this very high level of security.
5.2. The Future of Cyber Resilience
5.3. Exploratory Scenarios for the Framework’s Application
- Key Impacted Components: Situational awareness, risk management, active defense;
- Indicative Resilience Level: Very low (Level 1).
- Key Impacted Components: Preparedness, infrastructure resilience, incident recovery;
- Indicative Resilience Level: Developing (Level 2–3).
- Key Impacted Components: Infrastructure preparedness, active defense, response integration;
- Indicative Resilience Level: Strong (Level 3–4).
5.4. Technological Integration and Practical Relevance
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Lawhon, M.; Follmann, A.; Braun, B.; Cornea, N.; Greiner, C.; Guma, P.; Karpouzoglou, T.; Diez, J.R.; Schindler, S.; Schramm, S.; et al. Making heterogeneous infrastructure futures in and beyond the global south. Futures 2023, 154, 103270. [Google Scholar] [CrossRef]
- Kouloufakos, T. Untangling the cyber norm to protect critical infrastructures. Comput. Law Secur. Rev. 2023, 49, 105809. [Google Scholar] [CrossRef]
- Šarūnienė, I.; Martišauskas, L.; Krikštolaitis, R.; Augutis, J.; Setola, R. Risk Assessment of Critical Infrastructures: A Methodology Based on Criticality of Infrastructure Elements. Reliab. Eng. Syst. Saf. 2023, 243, 109797. [Google Scholar] [CrossRef]
- Pursiainen, C.; Kytömaa, E. From European critical infrastructure protection to the resilience of European critical entities: What does it mean? Sustain. Resilient Infrastruct. 2023, 8 (Suppl. S1), 85–101. [Google Scholar] [CrossRef]
- Broto, V.C.; Cortina-Oriol, M.; Durrant, D.; Griggs, S.; Guarneros-Meza, V.; Hayes, G.; Howarth, D.; Isunza-Vera, E.; Wong, M.T.; Zaremberg, G. Infrastructures, processes of insertion and the everyday: Towards a new dialogue in critical policy studies. Crit. Policy Stud. 2022, 16, 121–130. [Google Scholar] [CrossRef]
- Ashley, T.D.; Kwon, R.; Gourisetti, S.N.G.; Katsis, C.; Bonebrake, C.A.; Boyd, P.A. Gamification of Cybersecurity for Workforce Development in Critical Infrastructure. IEEE Access 2022, 10, 112487–112501. [Google Scholar] [CrossRef]
- Dafnos, T. Energy futures and present threats: Critical infrastructure resilience, accumulation, and dispossession. Stud. Political Econ. 2020, 101, 114–134. [Google Scholar] [CrossRef]
- Thekdi, S.A.; Chatterjee, S. Toward adaptive decision support for assessing infrastructure system resilience using hidden performance measures. J. Risk Res. 2019, 22, 1020–1043. [Google Scholar] [CrossRef]
- Abuhasel, K. A Linear Probabilistic Resilience Model for Securing Critical Infrastructure in Industry 5.0. IEEE Access 2023, 11, 80863–80873. [Google Scholar] [CrossRef]
- Barak, I. Critical infrastructure under attack: Lessons from a honeypot. Netw. Secur. 2020, 2020, 16–17. Available online: www.teiss.co.uk/ (accessed on 13 July 2025). [CrossRef]
- Klenk, T. Social services as critical infrastructure—Conceptualising and studying the operational core of the social investment state. Eur. J. Soc. Secur. 2023, 25, 115–138. [Google Scholar] [CrossRef]
- Furlong, K. Geographies of infrastructure III: Infrastructure with Chinese characteristics. Prog. Hum. Geogr. 2022, 46, 915–925. [Google Scholar] [CrossRef]
- Klenk, T.; Reiter, R. Introduction to the Special Issue on social services as critical infrastructure: Taking stock of the promises of the social investment state. Eur. J. Soc. Secur. 2023, 25, 107–114. [Google Scholar] [CrossRef]
- Valinejad, J.; Mili, L. Cyber-Physical-Social Model of Community Resilience by Considering Critical Infrastructure Interdependencies. IEEE Internet Things J. 2023, 10, 17530–17543. [Google Scholar] [CrossRef]
- Codetta-Raiteri, D.; Portinale, L. Decision networks for security risk assessment of critical infrastructures. ACM Trans. Internet Technol. 2018, 18, 29. [Google Scholar] [CrossRef]
- Chochliouros, I.P.; Spiliopoulou, A.S.; Stephanakis, I.M.; Arvanitozisis, D.N.; Sfakianakis, E.; Belesioti, M.; Georgiadou, E.; Mitsopoulou, N. Security and Protection of Critical Infrastructures: A Conceptual and Regulatory Overview for Network and Information Security in the European Framework, also focusing upon the Cloud Perspective. In Proceedings of the 16th International Conference on Engineering Applications of Neural Networks (INNS), Rhodes Island, Greece, 25–28 September 2015; Association for Computing Machinery: New York, NY, USA, 2015. [Google Scholar] [CrossRef]
- Gjesvik, L.; Szulecki, K. Interpreting cyber-energy-security events: Experts, social imaginaries, and policy discourses around the 2016 Ukraine blackout. Eur. Secur. 2023, 32, 104–124. [Google Scholar] [CrossRef]
- Emerson, R.G. Limits to a cyber-threat. Contemp. Politics 2016, 22, 178–196. [Google Scholar] [CrossRef]
- Backman, S. Normal cyber accidents. J. Cyber Policy 2023, 8, 114–130. [Google Scholar] [CrossRef]
- Cobilean, V.; Mavikumbure, H.S.; Mcbride, B.J.; Vaagensmith, B.; Singh, V.K.; Li, R.; Rieger, C.; Manic, M. A Review of Visualization Methods for Cyber-Physical Security: Smart Grid Case Study. IEEE Access 2023, 11, 59788–59803. [Google Scholar] [CrossRef]
- Papadopoulos, L.; Karteris, A.; Soudris, D.; Muñoz-Navarro, E.; Hernandez-Montesinos, J.J.; Paul, S.; Museux, N.; Kuenig, S.; Egger, M.; Schauer, S.; et al. PRAETORIAN: A Framework for the Protection of Critical Infrastructures from advanced Combined Cyber and Physical Threats. In Proceedings of the 18th International Conference on Availability, Reliability and Security (ARES 2023), Benevento, Italy, 29 August–1 September 2023; Association for Computing Machinery: New York, NY, USA, 2023. [Google Scholar] [CrossRef]
- König, S.; Shaaban, A.M.; Hadjina, T.; Gregorc, K.; Kutej, A. Identification and Evaluation of Cyber-Physical Threats on Interdependent Critical Infrastructures. In Proceedings of the 18th International Conference on Availability, Reliability and Security (ARES 2023), Benevento, Italy, 29 August–1 September 2023; Association for Computing Machinery: New York, NY, USA, 2023. [Google Scholar] [CrossRef]
- The International Institute for Strategic Studies. Stuxnet: Targeting Iran’s nuclear programme. Strateg. Comments 2011, 17, 1–3. [Google Scholar] [CrossRef]
- Maher, R. The covert campaign against Iran’s nuclear program: Implications for the theory and practice of counterproliferation. J. Strateg. Stud. 2021, 44, 1014–1040. [Google Scholar] [CrossRef]
- Kostyuk, N.; Zhukov, Y.M. Invisible Digital Front: Can Cyber Attacks Shape Battlefield Events? J. Confl. Resolut. 2019, 63, 317–347. [Google Scholar] [CrossRef]
- Sullivan, J.E.; Kamensky, D. How cyber-attacks in Ukraine show the vulnerability of the U.S. power grid. Electr. J. 2017, 30, 30–35. [Google Scholar] [CrossRef]
- Sikder, M.N.K.; Nguyen, M.B.T.; Elliott, E.D.; Batarseh, F.A. Deep H2O: Cyber attacks detection in water distribution systems using deep learning. J. Water Process Eng. 2023, 52, 103568. [Google Scholar] [CrossRef]
- Sanders, M.C.; Sanders, C.E. A world’s dilemma ‘upon which the sun never sets’—The nuclear waste management strategy (part I): Western European Nation States and the United States of America. Prog. Nucl. Energy 2016, 90, 69–97. [Google Scholar] [CrossRef]
- Mavikumbure, H.S.; Wickramasinghe, C.S.; Marino, D.L.; Cobilean, V.; Manic, M. Anomaly Detection in Critical-Infrastructures using Autoencoders: A Survey. In Proceedings of the IECON 2022—48th Annual Conference of the IEEE Industrial Electronics Society, Brussels, Belgium, 17–20 October 2022; IEEE Computer Society: Washington, DC, USA, 2022. [Google Scholar] [CrossRef]
- Maulana, F.; Fajri, H.; Safitra, M.F.; Lubis, M. Unmasking log4j’s Vulnerability: Protecting Systems against Exploitation through Ethical Hacking and Cyberlaw Perspectives. In Proceedings of the 9th International Conference on Computer and Communication Engineering (ICCCE), Kuala Lumpur, Malaysia, 15–16 August 2023; Institute of Electrical and Electronics Engineers (IEEE): Piscataway, NJ, USA, 2023; pp. 311–316. [Google Scholar] [CrossRef]
- Rahat, R.; Pradhananga, P.; ElZomor, M. A step toward nurturing infrastructure sustainability and rating systems through construction management curricula. Int. J. Sustain. High. Educ. 2023, 24, 1877–1896. [Google Scholar] [CrossRef]
- Osei-Kyei, R.; Almeida, L.M.; Ampratwum, G.; Tam, V. Systematic review of critical infrastructure resilience indicators. Constr. Innov. 2022, 23, 1210–1231. [Google Scholar] [CrossRef]
- Asiedu, R.O.; Manu, P.; Mahamadu, A.-M.; Booth, C.A.; Olomolaiye, P.; Agyekum, K.; Abadi, M. Critical skills for infrastructure procurement: Insights from developing country contexts. J. Eng. Des. Technol. 2021, 21, 1948–1974. [Google Scholar] [CrossRef]
- Koay, K.Y. Perceived risk and digital piracy: A moderated-moderation model. J. Inf. Commun. Ethics Soc. 2023, 21, 521–532. [Google Scholar] [CrossRef]
- Shaked, A.; Tabansky, L.; Reich, Y. Incorporating Systems Thinking into a Cyber Resilience Maturity Model. IEEE Eng. Manag. Rev. 2021, 49, 110–115. [Google Scholar] [CrossRef]
- Yemini, M.; Nedic, A.; Goldsmith, A.; Gil, S. Characterizing Trust and Resilience in Distributed Consensus for Cyberphysical Systems. IEEE Trans. Robot. 2022, 38, 71–91. [Google Scholar] [CrossRef]
- Tasooji, T.K.; Marquez, H.J. A Secure Decentralized Event-Triggered Cooperative Localization in Multi-Robot Systems under Cyber Attack. IEEE Access 2022, 10, 128101–128121. [Google Scholar] [CrossRef]
- Dart, M.; Ahmed, M. CYBER-AIDD: A novel approach to implementing improved cyber security resilience for large Australian healthcare providers using a Unified Modelling Language ontology. Digit. Health 2023, 9, 20552076231191095. [Google Scholar] [CrossRef]
- Staves, A.; Gouglidis, A.; Hutchison, D. An Analysis of Adversary-Centric Security Testing within Information and Operational Technology Environments. Digit. Threat. Res. Pract. 2023, 4, 14. [Google Scholar] [CrossRef]
- Ashley, T.; Gourisetti, S.N.G.; Brown, N.; Bonebrake, C. Aggregate attack surface management for network discovery of operational technology. Comput. Secur. 2022, 123, 102939. [Google Scholar] [CrossRef]
- Sell, M.; Dupuis, M. Designing an Industrial Cybersecurity Program for an Operational Technology Group. In Proceedings of the 24th Annual Conference on Information Technology Education (SIGITE 2023), Marietta, GA, USA, 11–14 October 2023; Association for Computing Machinery, Inc.: New York, NY, USA, 2023; pp. 125–130. [Google Scholar] [CrossRef]
- Chen, Q.; Wang, T.; Legunsen, O.; Li, S.; Xu, T. Understanding and discovering software configuration dependencies in cloud and datacenter systems. In Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020), Virtual, 8–13 November 2020; Association for Computing Machinery, Inc.: New York, NY, USA, 2020; pp. 362–374. [Google Scholar] [CrossRef]
- Xu, X.; Wang, C.; Wang, Z.; Lu, Q.; Zhu, L. Dependency tracking for risk mitigation in machine learning (ML) systems. In Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2022), Pittsburgh, PA, USA, 22–27 May 2022; Association for Computing Machinery (ACM): New York, NY, USA, 2022; pp. 145–146. [Google Scholar] [CrossRef]
- Selim, G.E.I.; Hemdan, E.E.D.; Shehata, A.M.; Fishawy, N.A.E. Anomaly events classification and detection system in critical industrial internet of things infrastructure using machine learning algorithms. Multimed. Tools Appl. 2021, 80, 12619–12640. [Google Scholar] [CrossRef]
- Makrakis, G.M.; Kolias, C.; Kambourakis, G.; Rieger, C.; Benjamin, J. Industrial and Critical Infrastructure Security: Technical Analysis of Real-Life Security Incidents. IEEE Access 2021, 9, 165295–165325. [Google Scholar] [CrossRef]
- Kulugh, V.E.; Mbanaso, U.M.; Chukwudebe, G. Cybersecurity Resilience Maturity Assessment Model for Critical National Information Infrastructure. SN Comput. Sci. 2022, 3, 217. [Google Scholar] [CrossRef]
- Sadaf, M.; Iqbal, Z.; Javed, A.R.; Saba, I.; Krichen, M.; Majeed, S.; Raza, A. Connected and Automated Vehicles: Infrastructure, Applications, Security, Critical Challenges, and Future Aspects. Technologies 2023, 11, 117. [Google Scholar] [CrossRef]
- Saeed, S.; Suayyid, S.A.; Al-Ghamdi, M.S.; Al-Muhaisen, H.; Almuhaideb, A.M. A Systematic Literature Review on Cyber Threat Intelligence for Organizational Cybersecurity Resilience. Sensors 2023, 23, 7273. [Google Scholar] [CrossRef]
- Franchina, L.; Inzerilli, G.; Scatto, E.; Calabrese, A.; Lucariello, A.; Brutti, G.; Roscioli, P. Passive and active training approaches for critical infrastructure protection. Int. J. Disaster Risk Reduct. 2021, 63, 102461. [Google Scholar] [CrossRef]
- Zou, B.; Choobchian, P.; Rozenberg, J. Cyber resilience of autonomous mobility systems: Cyber-attacks and resilience-enhancing strategies. J. Transp. Secur. 2021, 14, 137–155. [Google Scholar] [CrossRef]
- Rao, Y. Discourse as infrastructure: How ‘New Infrastructure’ policies re-infrastructure China. Glob. Media China 2023, 8, 254–270. [Google Scholar] [CrossRef]
- Michalec; Milyaeva, S.; Rashid, A. When the future meets the past: Can safety and cyber security coexist in modern critical infrastructures? Big Data Soc. 2022, 9, 20539517221108369. [Google Scholar] [CrossRef]
- Prokhorenko, V.; Babar, M.A. Architectural resilience in cloud, fog and edge systems: A survey. IEEE Access 2020, 8, 28078–28095. [Google Scholar] [CrossRef]
- Carias, J.F.; Arrizabalaga, S.; Labaka, L.; Hernantes, J. Cyber Resilience Self-Assessment Tool (CR-SAT) for SMEs. IEEE Access 2021, 9, 80741–80762. [Google Scholar] [CrossRef]
- Clark, A.; Zonouz, S. Cyber-physical resilience: Definition and assessment metric. IEEE Trans. Smart Grid 2019, 10, 1671–1684. [Google Scholar] [CrossRef]
- Manuel, D.D.; Carmona-Murillo, J.; Cortes-Polo, D.; Rodriguez-Perez, F.J. CyberTOMP: A Novel Systematic Framework to Manage Asset-Focused Cybersecurity from Tactical and Operational Levels. IEEE Access 2022, 10, 122454–122485. [Google Scholar] [CrossRef]
- Kumar, H.; Alvarez, O.A.; Kumar, S. Experimental Evaluation of Smart Electric Meters’ Resilience Under Cyber Security Attacks. IEEE Access 2023, 11, 55349–55360. [Google Scholar] [CrossRef]
- Simone, F.; Akel, A.J.N.; Di Gravio, G.; Patriarca, R. Thinking in Systems, Sifting Through Simulations: A Way Ahead for Cyber Resilience Assessment. IEEE Access 2023, 11, 11430–11450. [Google Scholar] [CrossRef]
- Trucco, P.; Petrenj, B. Characterisation of resilience metrics in full-scale applications to interdependent infrastructure systems. Reliab. Eng. Syst. Saf. 2023, 235, 109200. [Google Scholar] [CrossRef]
- Liu, B.; Xue, B.; Chen, X. Development of a metric system measuring infrastructure sustainability: Empirical studies of Hong Kong. J. Clean. Prod. 2021, 278, 123904. [Google Scholar] [CrossRef]
- Mounir, M.; Salah, A.; Kamel, A.; Moussa, H. Framework to Measure Agile Software Process Effectiveness in Critical Systems Development. In Proceedings of the 9th International Conference on Software and Information Engineering (ICSIE 2020), Cairo, Egypt, 11–13 November 2020; Association for Computing Machinery: New York, NY, USA, 2020; pp. 25–32. [Google Scholar] [CrossRef]
- Sachse, J. Do metrics matter?: The effect of indicators on scientific search behavior. In Proceedings of the Conference on Human Information Interaction and Retrieval (CHIIR 2019), Glasgow, UK, 10–14 March 2019; Association for Computing Machinery, Inc.: New York, NY, USA, 2019; pp. 417–420. [Google Scholar] [CrossRef]
- Stuart, D. Finding ‘good enough’ metrics for the UK’s Research Excellence Framework. Online Inf. Rev. 2015, 39, 265–269. [Google Scholar] [CrossRef]
- Ma, L. The steering effects of citations and metrics. J. Doc. 2021, 77, 420–431. [Google Scholar] [CrossRef]
- Schmitz, Y.; Vietz, D.; Wanke, E. A note on the complexity of K-metric dimension. Appl. Math. Comput. 2023, 457, 128204. [Google Scholar] [CrossRef]
- Kawamura; Steinberg, F.; Ziegler, M. Complexity theory of (functions on) compact metric spaces. In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS 2016), New York, NY, USA, 5–8 July 2016; Association for Computing Machinery (ACM): New York, NY, USA, 2016; pp. 837–846. [Google Scholar] [CrossRef]
- Jayapal, J.; Kumaraguru, S.; Varadarajan, S. A view similarity-based shape complexity metric to guide part selection for additive manufacturing. Rapid Prototyp. J. 2023, 29, 655–672. [Google Scholar] [CrossRef]
- Laghari, S.U.A.; Manickam, S.; Al-Ani, A.K.; Al-Shareeda, M.A.; Karuppayah, S. ES-SECS/GEM: An efficient security mechanism for SECS/GEM communications. IEEE Access 2023, 11, 31813–31828. [Google Scholar] [CrossRef]
- Gomes, J.; Khan, S.; Svetinovic, D. Fortifying the blockchain: A systematic review and classification of post-quantum consensus solutions for enhanced security and resilience. IEEE Access 2023, 11, 74088–74100. [Google Scholar] [CrossRef]
- Feng, S.; Cetinkaya, A.; Ishii, H.; Tesi, P.; De Persis, C. Networked Control under DoS Attacks: Tradeoffs between Resilience and Data Rate. IEEE Trans. Autom. Contr. 2021, 66, 460–467. [Google Scholar] [CrossRef]
- Thomas, B.S.; Chandar, P.; Hosey, C.; Diaz, F. Mixed Method Development of Evaluation Metrics. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2021), Singapore, 14–18 August 2021; Association for Computing Machinery: New York, NY, USA, 2021; pp. 4070–4071. [Google Scholar] [CrossRef]
- Afolabi, A.O.; Toivanen, P. Improving the design of a recommendation system using evaluation criteria and metrics as a guide. J. Syst. Inf. Technol. 2019, 21, 304–324. [Google Scholar] [CrossRef]
- Nirupama, N. Risk and vulnerability assessment: A comprehensive approach. Int. J. Disaster Resil. Built Environ. 2012, 3, 103–114. [Google Scholar] [CrossRef]
- Li, A.; Yan, W. Research on vulnerability assessment of physical protection system based on combined weighting-two-dimensional cloud model. In Proceedings of the 12th International Conference on Communication and Network Security (ICCNS 2022), Beijing, China, 1–3 December 2022; Association for Computing Machinery: New York, NY, USA, 2022; pp. 85–89. [Google Scholar] [CrossRef]
- Wang, Y.; Wang, W.; Bai, X.; Hu, B.; Ge, G.; Qian, K. RRDD: An ATT&CK-based ICS network security risk assessment method. In Proceedings of the 2nd International Conference on Networks, Communications and Information Technology (CNCIT 2023), Qinghai, China, 16–18 June 2023; Association for Computing Machinery: New York, NY, USA, 2023; pp. 186–192. [Google Scholar] [CrossRef]
- Segovia-Ferreira, M.; Rubio-Hernan, J.; Cavalli, R.; Garcia-Alfaro, J. Switched-based resilient control of cyber-physical systems. IEEE Access 2020, 8, 212194–212208. [Google Scholar] [CrossRef]
- Chowdhury, M.M.H.; Quaddus, M. Supply chain readiness, response and recovery for resilience. Supply Chain Manag. 2016, 21, 709–731. [Google Scholar] [CrossRef]
- Merkus, J.; Helms, R.; Kusters, R. Reference model for generic capabilities in maturity models. In Proceedings of the 12th International Conference on Information Management and Engineering (ICIME 2020), Amsterdam, The Netherlands, 16–18 September 2020; Association for Computing Machinery: New York, NY, USA, 2020; pp. 10–17. [Google Scholar] [CrossRef]
- Ahi, A.; Sinkovics, N.; Shildibekov, Y.; Sinkovics, R.R.; Mehandjiev, N. Advanced technologies and international business: A multidisciplinary analysis of the literature. Int. Bus. Rev. 2022, 31, 101967. [Google Scholar] [CrossRef]
- Capone, P.; Chiarella, P.; Sisto, R. Advanced technologies in genomic toxicology: Current trend and future directions. Curr. Opin. Toxicol. 2023, 37, 100444. [Google Scholar] [CrossRef]
- Mohlin, A. How to facilitate manufacturing industry learning from problems: A review on advanced technology problem-solving. J. Workplace Learn. 2023, 35, 470–489. [Google Scholar] [CrossRef]
- Dezeure, F. Real world information exchange: Challenges and insights. In Proceedings of the 2nd ACM Workshop on Information Sharing and Collaborative Security (WISCS 2015), Denver, CO, USA, 12–16 October 2015; Association for Computing Machinery: New York, NY, USA, 2015; p. 1. [Google Scholar] [CrossRef]
- Li, Y.; Ye, F.; Sheu, C. Social capital, information sharing and performance: Evidence from China. Int. J. Oper. Prod. Manag. 2014, 34, 1440–1462. [Google Scholar] [CrossRef]
- Chernykh; Volodin, D. The principle of international cooperation and sharing of information under international space law: Towards synergy. Space Policy 2023, 67, 101593. [Google Scholar] [CrossRef]
- Evensen, K.B.; Hansen, H. Cooperation and information sharing in institutional food chains. Br. Food J. 2016, 118, 2388–2403. [Google Scholar] [CrossRef]
- Sonnentag, S.; Geurts, S.A.E. Methodological issues in recovery research. Res. Occup. Stress Well Being 2009, 7, 1–36. [Google Scholar] [CrossRef]
- Vo, T.; Christie, C.A. Where Impact Measurement Meets Evaluation: Tensions, Challenges, and Opportunities. Am. J. Eval. 2018, 39, 383–388. [Google Scholar] [CrossRef]
- Wu, Y.M.; Li, Z.; Sun, C.; Wang, Z.B.; Wang, D.S.; Yu, Z. Measurement and control of system resilience recovery by path planning based on improved genetic algorithm. Meas. Control 2021, 54, 1157–1173. [Google Scholar] [CrossRef]
- Staheli, D.; Yu, T.; Crouser, R.J.; Damodaran, S.; Nam, K.; O’GWynn, D.; McKenna, S.; Harrison, L. Visualization evaluation for cyber security: Trends and future directions. In Proceedings of the Visualization for Cyber Security (VizSec 2014), Paris, France, 10 November 2014; Association for Computing Machinery: New York, NY, USA, 2014; pp. 49–56. [Google Scholar] [CrossRef]
- Yosifova, V.; Trifonov, R.; Tasheva, A.; Nakov, O. Trends review of the contemporary security problems in the cyberspace. In Proceedings of the 9th Balkan Conference in Informatics (BCI 2019), Sofia, Bulgaria, 26–28 September 2019; Association for Computing Machinery: New York, NY, USA, 2019. [Google Scholar] [CrossRef]
- Bodeau, D. Cyber Resilience Metrics: Key Observations; The MITRE Corporation: McLean, VA, USA, 2016. [Google Scholar]
- Bodeau, D.J.; Graubart, R.D.; Mcquaid, R.M.; Woodill, J. Cyber Resiliency Metrics, Measures of Effectiveness, and Scoring; The MITRE Corporation: Bedford, MA, USA, 2013. [Google Scholar]
- Linkov; Eisenberg, D.A.; Plourde, K.; Seager, T.P.; Allen, J.; Kott, A. Resilience metrics for cyber systems. Environ. Syst. Decis. 2013, 33, 471–476. [Google Scholar] [CrossRef]
- Benyahya, M.; Collen, A.; Kechagia, S.; Nijdam, N.A. Automated city shuttles: Mapping the key challenges in cybersecurity, privacy and standards to future developments. Comput. Secur. 2022, 122, 102904. [Google Scholar] [CrossRef]
- Fikri, M.; Atrinawati, L.H.; Putra, M.G.L. Cyber Resilience Evaluation Using Cyber Resilience Review Framework at University XYZ. Int. J. Educ. Manag. Innov. 2022, 3, 155–168. [Google Scholar] [CrossRef]
- Segovia, M.; Rubio-Hernan, J.; Cavalli, A.R.; Garcia-Alfaro, J. Cyber-Resilience Evaluation of Cyber-Physical Systems. In Proceedings of the 2020 IEEE 19th International Symposium on Network Computing and Applications (NCA 2020), Cambridge, MA, USA, 24–27 November 2021; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2020. [Google Scholar] [CrossRef]
- Almajali; Viswanathan, A.; Neuman, C. Resilience evaluation of demand response as spinning reserve under cyber-physical threats. Electronics 2017, 6, 2. [Google Scholar] [CrossRef]
- Meng, M.; Liu, L.; Feng, G. Adaptive output regulation of heterogeneous multiagent systems under Markovian switching topologies. IEEE Trans. Cybern. 2018, 48, 2962–2971. [Google Scholar] [CrossRef]
- Doan, X.V.; Shaw, D. Resource allocation when planning for simultaneous disasters. Eur. J. Oper. Res. 2019, 274, 687–709. [Google Scholar] [CrossRef]
- Santhiapillai, F.P.; Ratnayake, R.M.C. Risk-based prioritization method for planning and allocation of resources in public sector. TQM J. 2022, 34, 829–844. [Google Scholar] [CrossRef]
- Bidgoli, M.; Grossklags, J. End user cybercrime reporting: What we know and what we can do to improve it. In Proceedings of the 2016 IEEE International Conference on Cybercrime and Computer Forensic (ICCCF), Vancouver, BC, Canada, 2–14 June 2016; Institute of Electrical and Electronics Engineers (IEEE): Piscataway, NJ, USA, 2016. [Google Scholar] [CrossRef]
- Dang, L.N.; Kahsay, E.T.; James, L.T.N.; Johns, L.J.; Rios, I.E.; Mezuk, B. Research utility and limitations of textual data in the National Violent Death Reporting System: A scoping review and recommendations. Inj. Epidemiol. 2023, 10, 23. [Google Scholar] [CrossRef]
- Sukmana, M.; Meinel, C. E-government and security evaluation tools comparison for Indonesian e-government system. In Proceedings of the 4th International Conference on Information and Network Security (ICINS 2016), Kuala Lumpur, Malaysia, 28–31 December 2016; Association for Computing Machinery: New York, NY, USA, 2016; pp. 96–103. [Google Scholar] [CrossRef]
- Nguyen, T.; Wang, S.; Alhazmi, M.; Nazemi, M.; Estebsari, A.; Dehghanian, P. Electric Power Grid Resilience to Cyber Adversaries: State of the Art. IEEE Access 2020, 8, 87592–87608. [Google Scholar] [CrossRef]
- Ratasich, D.; Khalid, F.; Geissler, F.; Grosu, R.; Shafique, M.; Bartocci, E. A Roadmap Toward the Resilient Internet of Things for Cyber Physical Systems. IEEE Access 2019, 7, 13260–13283. [Google Scholar] [CrossRef]
- Bhusal, N.; Abdelmalak, M.; Kamruzzaman, M.; Benidris, M. Power system resilience: Current practices, challenges, and future directions. IEEE Access 2020, 8, 18064–18086. [Google Scholar] [CrossRef]
- Carías, J.F.; Borges, M.R.S.; Labaka, L.; Arrizabalaga, S.; Hernantes, J. Systematic approach to cyber resilience operationalization in SMEs. IEEE Access 2020, 8, 174200–174221. [Google Scholar] [CrossRef]
- Zhang, J.; Li, L.; Lin, G.; Fang, D.; Tai, Y.; Huang, J. Cyber Resilience in Healthcare Digital Twin on Lung Cancer. IEEE Access 2020, 8, 201900–201913. [Google Scholar] [CrossRef]
- Björck, F.; Henkel, M.; Stirna, J.; Zdravkovic, J. Cyber resilience—Fundamentals for a definition. In New Contributions in Information Systems and Technologies; Advances in Intelligent Systems and Computing; Springer: Cham, Switzerland, 2015; pp. 311–316. [Google Scholar] [CrossRef]
- Vogel, E.; Dyka, Z.; Klann, D.; Langendörfer, P. Resilience in the Cyberworld: Definitions, Features and Models. Future Internet 2021, 13, 293. [Google Scholar] [CrossRef]
- Stoicescu, M.; Fabre, J.C.; Roy, M. Architecting resilient computing systems: A component-based approach for adaptive fault tolerance. J. Syst. Archit. 2017, 73, 6–16. [Google Scholar] [CrossRef]
- Span, M.; Mailloux, L.O.; Mills, R.F.; Young, W. Conceptual systems security requirements analysis: Aerial refueling case study. IEEE Access 2018, 6, 46668–46682. [Google Scholar] [CrossRef]
- Huang, P.; Guo, C.; Zhou, L.; Lorch, J.R.; Dang, Y.; Chintalapati, M.; Yao, R. Gray Failure: The Achilles’ Heel of Cloud-Scale Systems. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS), Whistler, BC, Canada, 7–10 May 2017; IEEE Computer Society: Washington, DC, USA, 2017; pp. 150–155. [Google Scholar] [CrossRef]
- Ligo, K.; Kott, A.; Linkov, I. How to measure cyber-resilience of a system with autonomous agents: Approaches and challenges. IEEE Eng. Manag. Rev. 2021, 49, 89–97. [Google Scholar] [CrossRef]
- Patel; Roy, S.; Baldi, S. Wide-area damping control resilience towards cyber-attacks: A dynamic loop approach. IEEE Trans. Smart Grid 2021, 12, 3438–3447. [Google Scholar] [CrossRef]
- Safitra, M.F.; Lubis, M.; Fakhrurroja, H. Counterattacking cyber threats: A framework for the future of cybersecurity. Sustainability 2023, 15, 13369. [Google Scholar] [CrossRef]
- Bouk, S.H.; Ahmed, S.H.; Hussain, R.; Eun, Y. Named data networking’s intrinsic cyber-resilience for vehicular CPS. IEEE Access 2018, 6, 60570–60585. [Google Scholar] [CrossRef]
- Draǧoicea, M.; Léonard, M.; Ciolofan, S.N.; Militaru, G. Managing data, information, and technology in cyber-physical systems: Public safety as a service and its systems. IEEE Access 2019, 7, 92672–92692. [Google Scholar] [CrossRef]
- Moura, J.; Hutchison, D. Resilience enhancement at edge cloud systems. IEEE Access 2022, 10, 45190–45206. [Google Scholar] [CrossRef]
- Phillips, T.; Marinovici, L.D.; Rieger, C.; Orrell, A. Scalable resilience analysis through power systems co-simulation. IEEE Access 2023, 11, 18205–18214. [Google Scholar] [CrossRef]
- Laghari, S.U.A.; Manickam, S.; Al-Ani, A.K.; Rehman, S.U.; Karuppayah, S. SECS/GEMsec: A mechanism for detection and prevention of cyber-attacks on SECS/GEM communications in Industry 4.0 landscape. IEEE Access 2021, 9, 154380–154394. [Google Scholar] [CrossRef]
- Ziegler, V.; Schneider, P.; Viswanathan, H.; Montag, M.; Kanugovi, S.; Rezaki, A. Security and trust in the 6G era. IEEE Access 2021, 9, 142314–142327. [Google Scholar] [CrossRef]
- ElMarady, A.; Rahouma, K. Studying cybersecurity in civil aviation, including developing and applying aviation cybersecurity risk assessment. IEEE Access 2021, 9, 143997–144016. [Google Scholar] [CrossRef]
- Xin, X.; Keoh, S.L.; Sevegnani, M.; Saerbeck, M.; Khoo, T.P. Adaptive model verification for modularized Industry 4.0 applications. IEEE Access 2022, 10, 125353–125364. [Google Scholar] [CrossRef]
- He, J.; Yuan, Z.; Yang, X.; Huang, W.; Tu, Y.; Li, Y. Reliability modeling and evaluation of urban multi-energy systems: A review of the state of the art and future challenges. IEEE Access 2020, 8, 98887–98909. [Google Scholar] [CrossRef]
- Zheng, Z.; Yang, C.; Yang, W. Resilience analysis of discrete-time networked system in the presence of information disclosure. IEEE Access 2019, 7, 180147–180154. [Google Scholar] [CrossRef]
- Sahoo, S.; Dragicevic, T.; Blaabjerg, F. Multilayer resilience paradigm against cyber attacks in DC microgrids. IEEE Trans. Power Electron. 2021, 36, 2522–2532. [Google Scholar] [CrossRef]
- Pieters, W.; Hadžiosmanović, D.; Dechesne, F. Cyber security as social experiment. In Proceedings of the New Security Paradigms Workshop (NSPW 2014), Victoria, BC, Canada, 15–18 September 2014; Association for Computing Machinery: New York, NY, USA, 2014; pp. 15–24. [Google Scholar] [CrossRef]
- Fu, S.; Yao, Z. Privacy risk estimation of online social networks. In Proceedings of the 2022 International Conference on Networking and Network Applications (NaNA), Urumqi, China, 3–5 December 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 144–151. [Google Scholar] [CrossRef]
- Yüksel, S.; Yüksel, M.E.; Zaim, A.H. An approach for protecting privacy on social networks. In Proceedings of the 5th International Conference on Systems and Networks Communications, Nice, France, 22–27 August 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 154–159. [Google Scholar] [CrossRef]
- Barbeau, M.; Cuppens, F.; Cuppens, N.; Dagnas, R.; Garcia-Alfaro, J. Resilience estimation of cyber-physical systems via quantitative metrics. IEEE Access 2021, 9, 46462–46475. [Google Scholar] [CrossRef]
- Qurashi, J.M.; Jambi, K.M.; Eassa, F.E.; Khemakhem, M.; Alsolami, F.; Basuhail, A.A. Toward attack modeling technique addressing resilience in self-driving car. IEEE Access 2023, 11, 2652–2673. [Google Scholar] [CrossRef]
- Prislan, K.; Mihelič, A.; Bernik, I. A real-world information security performance assessment using a multidimensional socio-technical approach. PLoS ONE 2020, 15, e0238739. [Google Scholar] [CrossRef]
- Asgari, H.; Haines, S.; Rysavy, O. Identification of threats and security risk assessments for recursive internet architecture. IEEE Syst. J. 2018, 12, 2437–2448. [Google Scholar] [CrossRef]
- Amani, M.; Jalili, M. Power grids as complex networks: Resilience and reliability analysis. IEEE Access 2021, 9, 119010–119031. [Google Scholar] [CrossRef]
- Akbarian, F.; Tarneberg, W.; Fitzgerald, E.; Kihl, M. Attack resilient cloud-based control systems for Industry 4.0. IEEE Access 2023, 11, 27865–27882. [Google Scholar] [CrossRef]
- Rieger; Schultz, K.; Carroll, T.; McJunkin, T. Resilient Control Systems—Basis, Benchmarking and Benefit. IEEE Access 2021, 9, 57565–57577. [Google Scholar] [CrossRef]
- Guzman, R.E.P.; Rivera, M.; Wheeler, P.W.; Mirzaeva, G.; Espinosa, E.E.; Rohten, J.A. Microgrid Power Sharing Framework for Software Defined Networking and Cybersecurity Analysis. IEEE Access 2022, 10, 111389–111405. [Google Scholar] [CrossRef]
- Marino, L.; Wickramasinghe, C.S.; Singh, V.K.; Gentle, J.; Rieger, C.; Manic, M. The Virtualized Cyber-Physical Testbed for Machine Learning Anomaly Detection: A Wind Powered Grid Case Study. IEEE Access 2021, 9, 159475–159494. [Google Scholar] [CrossRef]
- Mouelhi, S.; Laarouchi, M.E.; Cancila, D.; Chaouchi, H. Predictive Formal Analysis of Resilience in Cyber-Physical Systems. IEEE Access 2019, 7, 33741–33758. [Google Scholar] [CrossRef]
- Soikkeli, J.; Casale, G.; Munoz-Gonzalez, L.; Lupu, E.C. Redundancy Planning for Cost Efficient Resilience to Cyber Attacks. IEEE Trans. Dependable Secur. Comput. 2023, 20, 1154–1168. [Google Scholar] [CrossRef]
- Borges, F.S.; Laurindo, F.J.B.; Spínola, M.M.; Gonçalves, R.F.; Mattos, C.A. The Strategic Use of Artificial Intelligence in the Digital Era: Systematic Literature Review and Future Research Directions. Int. J. Inf. Manag. 2021, 57, 102225. [Google Scholar] [CrossRef]
- Olowononi, F.O.; Rawat, D.B.; Liu, C. Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS. IEEE Commun. Surv. Tutor. 2021, 23, 524–552. [Google Scholar] [CrossRef]
- Murdoch, S.; Leaver, N. Anonymity vs. Trust in Cyber-Security Collaboration. In Proceedings of the 2nd ACM Workshop on Information Sharing and Collaborative Security (WISCS 2015), Denver, CO, USA, 12 October 2015; pp. 27–29. [Google Scholar] [CrossRef]
- Nykänen, R.; Kärkkäinen, T. Supporting Cyber Resilience with Semantic Wiki. In Proceedings of the 12th International Symposium on Open Collaboration (OpenSym 2016), Berlin, Germany, 17–19 August 2016. [Google Scholar] [CrossRef]
- Hernández-Bejarano, M.; Rodríguez, R.J.; Merseguer, J. A Vision for Improving Business Continuity through Cyber-Resilience Mechanisms and Frameworks. In Proceedings of the 16th Iberian Conference on Information Systems and Technologies (CISTI), Chaves, Portugal, 23–26 June 2021. [Google Scholar]
- Carías, J.F.; Labaka, L.; Sarriegi, J.M.; Hernantes, J. Defining a Cyber Resilience Investment Strategy in an Industrial Internet of Things Context. Sensors 2019, 19, 138. [Google Scholar] [CrossRef]
- Sukiasyan; Badikyan, H.; Pedrosa, T.; Leitao, P. Secure Data Exchange in Industrial Internet of Things. Neurocomputing 2022, 484, 183–195. [Google Scholar] [CrossRef]
- Sánchez-Gordón, M.; Colomo-Palacios, R. Security as Culture: A Systematic Literature Review of DevSecOps. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops (ICSEW 2020), Seoul, Republic of Korea, 27 June–19 July 2020; pp. 266–269. [Google Scholar] [CrossRef]
- Alfawaz, S.; Nelson, K.; Mohannak, K. Information Security Culture: A Behaviour Compliance Conceptual Framework. In Proceedings of the 8th Australasian Information Security Conference (AISC 2010), Perth, Australia, 30 November 2010; pp. 47–55. [Google Scholar]
- Schweitzer, J. Security Awareness. In Proceedings of the Northeast ACM Symposium on Personal Computer Security (PCS), Waltham, MA, USA, 1 September 1986; pp. 13–20. [Google Scholar]
- Zimmer, H.G. Computers and Computations in Algebraic Number Theory. In Proceedings of the 2nd ACM Symposium on Symbolic and Algebraic Manipulation (SYMSAC), Los Angeles, CA, USA, 23–25 March 1971; pp. 172–179. [Google Scholar]
- Alhazmi, H.; Malaiya, Y.K.; Ray, I. Measuring, analyzing and predicting security vulnerabilities in software systems. Comput. Secur. 2007, 26, 219–228. [Google Scholar] [CrossRef]
- Kotenko; Izrailov, K.; Buinevich, M.; Saenko, I.; Shorey, R. Modeling the Development of Energy Network Software, Taking into Account the Detection and Elimination of Vulnerabilities. Energies 2023, 16, 5111. [Google Scholar] [CrossRef]
- Beerman, J.; Berent, D.; Falter, Z.; Bhunia, S. A Review of Colonial Pipeline Ransomware Attack. In Proceedings of the 23rd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW 2023), Bangalore, India, 1–4 May 2023; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2023; pp. 8–15. [Google Scholar] [CrossRef]
- Thiagarajan, K.; Dixit, C.K.; Panneerselvam, M.; Madhuvappan, C.A.; Gadde, S.; Shrote, J.N. Analysis on the Growth of Artificial Intelligence for Application Security in Internet of Things. In Proceedings of the 2nd International Conference on Artificial Intelligence and Smart Energy (ICAIS 2022), Coimbatore, India, 23–25 February 2022; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2022; pp. 6–12. [Google Scholar] [CrossRef]
- Building Cyber Resilience Threats, Enablers and Anticipation. 2021. Available online: www.axa-research.org (accessed on 13 July 2025).
- Nosouhi, M.R.; Shah, S.W.; Pan, L.; Zolotavkin, Y.; Nanda, A.; Gauravaram, P.; Doss, R. Weak-Key Analysis for BIKE Post-Quantum Key Encapsulation Mechanism. IEEE Trans. Inf. Forensics Secur. 2023, 18, 2160–2174. [Google Scholar] [CrossRef]
- Kesswani, N.; Kumar, S. Maintaining cyber security: Implications, cost and returns. In Proceedings of the 2015 ACM SIGMIS Conference on Computers and People Research (SIGMIS-CPR 2015), Newport Beach, CA, USA, 4–6 June 2015; Association for Computing Machinery, Inc.: New York, NY, USA, 2015; pp. 161–164. [Google Scholar] [CrossRef]
- Anang; Gandhi, A.; Sucahyo, Y.G. The Design of Information Security Risk Management: A Case Study Human Resources Information System at XYZ University. In Proceedings of the 4th International Conference on Computer and Informatics Engineering: IT-Based Digital Industrial Innovation for the Welfare of Society (IC2IE 2021), Depok, Indonesia, 14–15 September 2021; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2021; pp. 198–203. [Google Scholar] [CrossRef]
- Mirtsch, M.; Blind, K.; Koch, C.; Dudek, G. Information security management in ICT and non-ICT sector companies: A preventive innovation perspective. Comput. Secur. 2021, 109, 102383. [Google Scholar] [CrossRef]
- Zhang, K.; Li, S. Research on The Integration of Business Intelligence and Innovation and Entrepreneurship Education for Computer Science. In Proceedings of the 5th International Conference on Big Data and Education (ICBDE 2022), Shanghai, China, 26–28 February 2022; Association for Computing Machinery: New York, NY, USA, 2022; pp. 212–216. [Google Scholar] [CrossRef]
- Thinyane, M.; Christine, D. SMART Citizen Cyber Resilience (SC2R) Ontology. In Proceedings of the 13th International Conference on Security of Information and Networks (SIN 2020), Merkez, Turkey, 4–7 November 2020; Association for Computing Machinery: New York, NY, USA, 2020. [Google Scholar] [CrossRef]
- Johnson, F. Democracy, prosperity, citizens and the state. Can. Foreign Policy J. 2002, 10, 23–40. [Google Scholar] [CrossRef]
- Liang, K.; Liu, J.K.; Lu, R.; Wong, D.S. Privacy Concerns for Photo Sharing in Online Social Networks. IEEE Internet Comput. 2015, 19, 58–63. [Google Scholar] [CrossRef]
- Abdullayeva, F. Cyber resilience and cyber security issues of intelligent cloud computing systems. Results Control. Optim. 2023, 12, 100268. [Google Scholar] [CrossRef]
- Ahlan, A.R.; Lubis, M. Information Security Awareness in University: Maintaining Learnability, Performance and Adaptability Through Roles of Responsibility. In Proceedings of the 2011 7th International Conference on Information Assurance and Security (IAS), Melacca, Malaysia, 5–8 December 2011; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2011; pp. 246–250. [Google Scholar] [CrossRef]
- Lubis, A.R.; Fachrizal, F.; Lubis, M.; Tahir, H.M. Wireless Service at Public University: A Survey of Users Perception on Security Aspects. In Proceedings of the 2018 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia, 6–7 March 2018; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2018; pp. 78–83. [Google Scholar] [CrossRef]
- Safitra, M.F.; Lubis, M.; Widjajarto, A. Security Vulnerability Analysis Using Penetration Testing Execution Standard (PTES): Case Study of Government’s Website. In Proceedings of the 2023 6th International Conference On Electronics, Communications and Control Engineering, Fukuoka, Japan, 24–26 March 2023; Association for Computing Machinery: New York, NY, USA, 2023; pp. 139–145. [Google Scholar] [CrossRef]
- Safitra, M.F.; Lubis, M.; Kurniawan, M.T. Cyber resilience: Research opportunities. In Proceedings of the 2023 6th International Conference on Electronics, Communications and Control Engineering, Fukuoka, Japan, 24–26 March 2023; Association for Computing Machinery: New York, NY, USA, 2023; pp. 99–104. [Google Scholar] [CrossRef]
Domain | Component | Indicator | Description | Reference |
---|---|---|---|---|
Cyber as a Shield | Situational Awareness | Threat detection and monitoring capabilities | The organization can proactively observe operational changes and identify potential cyber threats. | [30,95,96,116,117,118] |
Security Assurance | Risk assessments and security controls | Includes routine evaluations and the enforcement of strict standards to ensure system protection. | ||
Active Defense | Rapid response to threats | Involves using tools and strategies to detect and prevent attacks before system damage occurs. | ||
Risk Management | Risk identification and mitigation | A systematic process to assess threats and prioritize mitigation actions. | ||
Cyber as a Space | Infrastructure Resilience | System robustness and recovery capabilities | Infrastructure can sustain operation and recover during or after cyber incidents. | [30,89,90,91,92,93,94,116] |
Critical Infrastructure Awareness | Organizational awareness of vital systems | A deep understanding of the infrastructure’s national significance and associated risks. | ||
Resilience Principles | Resilient design and operational philosophy | Foundational principles for building systems that can withstand disruptions. | ||
Infrastructure Safeguard Policy | Protective policies and procedures | Formal documents and procedures to secure physical and digital infrastructure from threats. | ||
Cyber as a Sword | Infrastructure Preparedness | Pre-emptive readiness and training | The presence of incident response plans, personnel training, and scenario simulations. | [30,111,112,113,114,115,116] |
Critical Incident Recovery | Recovery speed and continuity measures | The ability to restore system functions quickly and efficiently after disruptions. |
Component | Indicator | Measurement Criteria | Description |
---|---|---|---|
Situational Awareness |
| Based on coverage of monitoring and average time to detect anomalies | 0: No monitoring system 1: Manual observation only 2: Partial system monitoring 3: Full system monitored periodically 4: Real-time monitoring 5: Real-time + automated anomaly detection with alerting |
Security Assurance |
| Refer to implemented frameworks (e.g., ISO 27001) and documented controls | 0: No controls or certifications 1: Basic firewall/AV only 2: Partial controls implemented 3: Formal internal policy with controls 4: Certification in progress 5: Full certification (e.g., ISO 27001) and up-to-date controls |
Active Defense |
| Based on system responsiveness and detection accuracy | 0: No response capability 1: A delayed manual response (>72 h) 2: Manual monitoring, reactive response 3: Semi-automated alerts and mitigation 4: Full incident response plan with automation 5: Automated detection and active defense with <2% false positives |
Risk Management |
| Based on risk governance process and follow-up | 0: No risk assessment 1: Ad hoc assessments only 2: Annual risk assessments 3: Quarterly assessments 4: Documented mitigation tracking 5: Continuous risk analysis with >90% risk mitigation execution |
Infrastructure Resilience |
| Based on service continuity and fault tolerance | 0: Unstable system, frequent failures 1: Downtime of >48 h/year 2: Downtime of 24–48 h/year 3: Downtime of 8–24 h/year 4: Downtime of <8 h/year 5: High-availability setup with <1 h/year downtime |
Critical Infrastructure Awareness |
| Based on documentation and prioritization | 0: No asset classification 1: Initial asset list only 2: Incomplete inventory 3: Full classification but outdated 4: An up-to-date list of critical systems 5: Inventory integrated with risk and threat modeling tools |
Resilience Principles |
| Based on architectural design and redundancy coverage | 0: No resilience mechanisms 1: Manual backups only 2: Periodic backups and isolated recovery plans 3: Redundant systems in core infrastructure 4: Partial failover capability 5: Full redundancy and automated failover across systems |
Infrastructure Safeguard Policy |
| Based on comprehensiveness and relevance of official policy documents | 0: No formal policies 1: Single general policy 2: Multiple but outdated policies 3: Up-to-date, role-specific policies 4: Policies reviewed annually 5: Integrated, reviewed biannually and aligned with national/international standards |
Infrastructure Preparedness |
| Based on preparedness programs and regular testing | 0: No training or drills 1: Basic training for some staff 2: Annual training for IT team 3: Annual drills across departments 4: Semi-annual simulations 5: Full organization involved in quarterly simulations with >90% personnel trained |
Critical Incident Recovery |
| Based on recovery performance and SLA compliance | 0: Recovery not defined 1: MTTR of >72 h 2: MTTR of 48–72 h 3: MTTR of 24–48 h 4: MTTR of 4–24 h 5: MTTR of <4 h, 100% SLA compliance |
Score | Interpretation |
---|---|
0–20 | Low Resilience |
21–35 | Developing Resilience |
36–45 | Strong Resilience |
46–50 | Optimized and Adaptive Resilience |
Resilience Grading | ||
---|---|---|
Abbreviation | Description | % Achieved |
D | Did Not Meet | 0–20% achievement |
A | Approaching | >20–50% achievement |
M | Moderately Met | >50–75% achievement |
W | Well Achieved | >75–90% achievement |
E | Exceeds Expectations | >90–100% achievement |
Scenario | Sector | Main Incident | Technical Notes | Key Components | Resilience Level |
---|---|---|---|---|---|
Electrical Grid Disruption | Energy (Power Grid) | SCADA-targeted cyber-attack | Modbus TCP/IP, no encryption, flat network, manual recovery | Situational Awareness, Risk Management, Active Defense | Very Low (Level 1) |
Smart Hospital Ransomware | Healthcare | Ransomware and medical IoT disruption | Weak segmentation, no IR coordination, outdated backups | Preparedness, Resilience, Incident Recovery | Developing (Level 2–3) |
Airport System Sabotage | Transportation | System outage via OT compromise | Legacy PLCs, SOC present, no unified IT-OT drills | Preparedness, Defense, Response Coordination | Strong (Level 3–4) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lubis, M.; Safitra, M.F.; Fakhrurroja, H.; Muttaqin, A.N. Guarding Our Vital Systems: A Metric for Critical Infrastructure Cyber Resilience. Sensors 2025, 25, 4545. https://doi.org/10.3390/s25154545
Lubis M, Safitra MF, Fakhrurroja H, Muttaqin AN. Guarding Our Vital Systems: A Metric for Critical Infrastructure Cyber Resilience. Sensors. 2025; 25(15):4545. https://doi.org/10.3390/s25154545
Chicago/Turabian StyleLubis, Muharman, Muhammad Fakhrul Safitra, Hanif Fakhrurroja, and Alif Noorachmad Muttaqin. 2025. "Guarding Our Vital Systems: A Metric for Critical Infrastructure Cyber Resilience" Sensors 25, no. 15: 4545. https://doi.org/10.3390/s25154545
APA StyleLubis, M., Safitra, M. F., Fakhrurroja, H., & Muttaqin, A. N. (2025). Guarding Our Vital Systems: A Metric for Critical Infrastructure Cyber Resilience. Sensors, 25(15), 4545. https://doi.org/10.3390/s25154545