Cybersecurity and Safety Co-Engineering of Cyberphysical Systems—A Comprehensive Survey

: Safeguarding both safety and cybersecurity is paramount to the smooth and trustworthy operation of contemporary cyber physical systems, many of which support critical functions and services. As safety and security have been known to be interdependent, they need to be jointly considered in such systems. As a result, various approaches have been proposed to address safety and cybersecurity co-engineering in cyber physical systems. This paper provides a comprehensive survey of safety and cybersecurity co-engineering methods, and discusses relevant open issues and research challenges. Despite the extent of the existing literature, several aspects of the subject still remain to be fully addressed.


Introduction
The unification of embedded systems with communication technologies gave rise to "Cyber Physical Systems (CPS)". Such systems are deployed in several application domains, such as automotive, smart manufacturing, and healthcare. Due to the "double nature" of such systems-merging of the cyber and the physical worlds-ensuring both safety and cybersecurity are important prerequisites to their reliable operation.
Thus, the study of potential hazards and threats, and the assessment of safety and cybersecurity risks that potential accidents and cyberattacks pose, is important; it is also, usually, complicated. This is because there exist strong dependencies between the two domains, even though cases where they are independent do also exist. Three types of such dependencies have been identified and studied in Reference [1]: 1. Conditional dependencies: Safe operations may be conditioned by cybersecurity, for example, malicious modifications of sensor data or control programs may prevent safety systems from protecting an installation in accidental conditions. Conversely, safety may be a condition for cybersecurity, for example, when unmanaged catastrophic conditions weaken the security posture of a system or an organization, and lead to opportunistic malicious acts. 2. Reinforcement: Safety and cybersecurity measures can be complementary, for example, event and activity logging may be used both for attack detection and accident anticipation, as well as post-event analysis. 3. Conflict: If safety and cybersecurity are considered separately for the same system, it is possible that conflicting requirements or measures may be identified, for example, a safety requirement for an automatic door shutting system, would be to leave the door open, whereas a security requirement would be to leave the door locked in case of failure.
• Security-informed safety approaches: Approaches that extend the scope of safety engineering by adapting cybersecurity-related techniques. • Safety-informed security approaches: Approaches that extend the scope of security engineering by adapting safety-related techniques. • Combined safety and security approaches: Combined approaches for safety and cybersecurity co-engineering.
In recent research, many proposals for security and safety co-engineering methods have appeared, and some survey articles have reviewed these proposed methods in varied degrees of depth and scopes. Piètre-Cambacédès et al. [3] surveyed the differences and similarities between safety and security aspects focusing on their dependencies per application domain. Kriaa et al. [4] conducted a survey of safety and security methods and analyzed methods for industrial control systems. Various safety and security risk assessment methods, categorized according to their application domain, were reviewed by Chockalingam et al. [5]. Abulamddi [6] surveyed existing methods for safety and security requirements engineering in CPSs. A systematic literature review was conducted by Lisova et al. [7] that focused on already developed and evaluated methods. Lyu et al. [8] provided a short survey, in which five integrated safety and security co-engineering methods were analyzed. Finally, Paul and Rioux [2] provided an extended bibliography of research papers on safety and cybersecurity co-engineering since the early 90s without, however, analyzing them.
In this paper we revisit previous surveys on cybersecurity and safety co-engineering approaches; we report on the results of a systematic literature survey of such approaches that have not been reviewed before; we define a multi-attribute taxonomy and we use it to analyze such approaches; and we discuss pertinent open issues and research challenges. The overall contribution of this paper is a comprehensive discussion on the recent advances in cybersecurity and safety co-engineering. A summary of the contributions of the paper follows: • A comprehensive review of sixty eight methods for cybersecurity and safety co-engineering, of which nine have not been reviewed before. • The development of a multi-attribute taxonomy of cybersecurity and safety co-engineering methods, encompassing inter alia all attributes used in previous surveys. • A discussion on open issues, not fully addressed by existing approaches for cybersecurity and safety co-engineering, that give rise to research challenges.
The remainder of this work is structured as follows: Section 2 reviews related work; it is divided into two subsections, one on previous surveys and another on approaches not previously reviewed. In Section 3 we define a multi-attribute taxonomy of cybersecurity and safety co-engineering approaches and we employ it to analyze the reviewed approaches. In Section 4 we discuss the results of the analysis, and we identify issues not fully addressed by existing approaches that give rise to research challenges. Finally, in Section 5 we summarize our conclusions and we outline our future research plans.

Previous Reviews
There are two types of reviews; narrative and systematic reviews [9]. Narrative reviews aim to identify studies in the literature that describe a specific problem of interest. In such reviews, systematic guidelines regarding the searching method, the identification of research questions, and the screening process are not considered. Thus, such reviews do not provide a comprehensive understanding of the stated problem.
Systematic reviews are methodical approaches to identify, analyze, and criticize the results concerning predefined research questions. Such reviews aim to provide inclusive results regarding a well predefined problem with specific research questions. Specific processes and guidelines exist for conducting a systematic literature review, including on how to formulate the research questions; research the literature; screen and select the results; and analyze and document the conclusions.
A total of six reviews of joint safety and security analysis methods have been identified in the recent literature. Of those five, References [3][4][5][6]8] are narrative and only one [7] is systematic. Even though the total number of methods reviewed therein is 60, surprisingly, the intersection of the set of methods reviewed in Reference [7] and of the set of methods reviewed in all other reviews counts only seven elements.
Piètre-Cambacédès et al. [3] conducted a survey of various safety and security approaches and studied the potential adoption of a safety approach for security analysis and vice versa. Although this work provides insight into safety and security concepts by analyzing the relevant terminology, the methods that are surveyed are not combined approaches but traditional safety and security methods. Specifically, safety standards along with hazard and risk analysis methods on one hand, and vulnerability and threat analysis methodologies on the other have been studied. The surveyed methods have been classified according to type (safety to security or security to safety); according to the approach taken (Architectural concepts, Graphical modeling, structured risk assessment, and Testing); and according to safety characteristics (fault prevention, fault tolerance, fault removal, and fault forecasting).
Kriaa et al. [4] provide a survey of approaches combining safety and security risk analysis for industrial infrastructures. Thirty nine methods were analyzed and grouped considering various criteria: the way each method treats safety and security (unification/integration or harmonization); the lifecycle phase (operational/requirements or design) in which the studied system is; and the type of the risk assessment approach (quantitative/qualitative). The methods are classified into generic and model-based. Methods in the former group describe the lifecycle stages and the sequence of activities in each stage, whereas the latter includes graphical or non-graphical methods, that may be supported by software tools. Furthermore, an overview of the safety and security standards for industrial infrastructures is presented. Finally, by analyzing the safety and security dependencies and interdependencies, the authors concluded that safety and security are complementary and should be treated jointly to improve the risk assessment process.
Chockalingam et al. [5] surveyed several integrated safety and security risk assessment methods and identified their key characteristics. The analysis was performed considering five criteria: First the identified approaches were classified according to the number of the citations that they had received in the scientific literature. The authors argue that the research community started to recognize the importance of the combination of safety and security analyses in 2014 and 2015, with the most prominent methods being the Extended Fault Tree (EFT) and the Extended Component Fault Tree (E-CFT). Additionally, the approaches were grouped according to the steps involved in the risk assessment process. Specifically, the authors identified two types of integrated methods: the sequential integrated safety and security risk assessment method, and the non-sequential method. The third criterion is the stages of the risk management process (risk identification, risk analysis, risk evaluation that the method addresses. Further, the identified approaches are classified according to how the integration is achieved: 1) Combination of a conventional safety assessment method and of a variation of it to assess security; 2) Combination of a conventional security assessment method and of a variation of it to assess safety; 3) Combination of a conventional safety and a conventional security method; and 4) Other -no conventional safety or security assessment method used. Finally, the survey categorized the approaches considering the application and the application domain; four out of seven are methods targeting the transportation domain.
Abulamddi [6] surveyed integrated techniques for requirements engineering in CPSs; eight methods focusing on the requirements engineering phase of the lifecycle were identified and analyzed. The techniques were classified as safety and security requirements or accident analysis.
Lisova et al. [7] conducted a systematic literature review of joint safety and security analysis methods. The search was performed using the keywords ("safety" AND "security" AND "analysis") in four scientific databases (IEEE, ACM, Web of Science, and Springer link). Thirty three methods that analyze safety and security of CPSs early in the development phase have been identified. Five characteristics of these methods were considered: application domain; stage in the system lifecycle; association with relevant standards; existence of validation step; and origin of contribution. Additionally, the identified methods were classified according to the relationship between safety and security in the analysis process (Unified, Parallel), and the overall goal of the analysis (combined safety and security; security informed safety, safety informed security). Additionally, the yearly distribution of the reviewed papers, based on their security and safety focus was studied.
Finally, Lyu et al. [8] surveyed ten methods for safety and security analysis; these included five integrated approaches. The identified approaches were compared considering characteristics such as: quantitative/qualitative, model-based/system-based, top-down/bottom-up analysis, and hierarchical/dynamic analysis. The authors identified the pros and cons of each method and the technical gaps of the interplay of safety and security. Most of the integrated approaches analyzed in this work take a qualitative risk management approach.

Search Method
Additional methods for safety and cybersecurity co-engineering have been identified in the following research databases: ACM Digital Library, Science Direct, Scopus, and IEEE Xplore. The search process was carried out by searching with the groups of keywords; (Safety AND Security AND Cyber-physical systems) and (Safety AND Cybersecurity AND Cyber-physical systems). The initial search returned 1313 results. The selection of the articles to be considered was performed according to the criteria listed below: • The article must be explicitly related to a cybersecurity and safety co-engineering methodology.
• The article must not be included in previously published reviews.
This process resulted in the methods reviewed in the next section.

US 2 [10]
: This is a unified approach that analyzes safety hazards and security threats for CPSs in automotive vehicles, by leveraging a simple quantitative scheme. It aims to analyze safety and security concepts simultaneously, and to obtain consistent safety and security requirements and countermeasures. The elicitation of requirements is based on the ISO 26262 Automotive Safety Integrity Level (ASIL) metric and on the Security Level (SEL) metric, proposed in this work. The analysis is initiated by identifying security threats; in the sequel whether these threats may inflict safety hazards is examined. If so, the ASIL is utilized to identify the corresponding safety and security requirements and countermeasures, otherwise the SEL is used.

STPA and Six
Step Model [11]: It is an integrated approach to analyze safety and security issues and artefacts for autonomous vehicles. It is based on the SAE J3016, SAE J3061, and ISO 26262 standards. By leveraging the Six Step Model (SSM) [12], the authors integrated the System Theoretic Process Analysis (STPA) [13] and the ISO 26262 standard to enrich the SSM hierarchies and, particularly, the lists of functions, failures, and safety countermeasures. The method comprises six steps, similarly to the SSM. In Steps 1 and 2 the functions, structure, and processes of the CPSs in an autonomous vehicle are identified. Steps 3 and 4 pertain to the safety and security analysis of the targeted system. Namely, the Hazard Analysis and Risk Assessment (HARA) as defined in ISO 26262 [14] and STPA methods are followed, to identify hazards, failures, and requirements. The security analysis is based on TARA as defined by the SAE J3061 standard [15]. Finally, in steps 5 and 6 the safety and security countermeasures are identified, by analyzing the functional safety and security requirements, and added to the model. A software too to support the proposed methodology is under development.
FACT [16,17]: Failure-Attack-CounTermeasure (FACT) is a unified graphical approach for safety and security analysis. This approach facilitates the analysis of CPSs and can be used for verification, validation, monitoring, and periodic safety and security assessment. The proposed approach is based on the ISA84 [18] and ISA99 [19] standards. The integration of the two standards is achieved by merging the corresponding lifecycle phases, resulting in a unified lifecycle of fourteen phases. The FACT graph model is developed in phases 1-9 of the unified lifecycle. The graph is constructed by following four distinct steps: (1) Import failure trees; (2) Include safety countermeasures in the graph; (3) Import attack trees; and (4) Include security countermeasures. Further, by leveraging the FACT graph, the security and safety countermeasures can be mapped to the corresponding attacks and faults. This enables the identification of interrelated countermeasures and the analysis of their interdependencies. The proposed approach has been applied to analyze industrial control systems.
CRAF [20]: The Cyber Risk Assessment Framework (CRAF) aims to facilitate the safety and security analysis of a CPS during the whole system lifecycle. The main focus of the framework is to study how a loss of data security could have safety implications. The framework comprises three steps: (1) Communicating a decision; (2) Raising a conflict; and (3) Conflict resolution. CRAF treats safety and security separately, and utilizes traditional security and safety techniques and concepts (e.g., Threat analysis, Hazard analysis). CRAF aims to bridge the gap between safety and security by comparing and consolidating the security and safety data properties.
UFoI-E [21]: The Uncontrolled Flows of information and Energy (UFoI-E) method enables the analysis and representation of the dependencies of CPSs and facilitates their diagrammatic representation for risk analysis. It provides a generic CPS master diagram to distinguish the cyber and physical environments of the system under study. The method integrates the security and safety concepts from physical, control, and computer systems. The dependencies between information and control flows are studied to examine the causes the could provoke harm to humans, assets, or the environment. The method considers these cyber threats in the information domain that could provoke safety hazards in the energy domain. Security aspects are related to deliberate sources of risk while safety aspects are related to unintentional sources of risk. According to the UFoI-E, both an uncontrolled flow of information (security) and an uncontrolled flow of energy (safety) may result in physical harm.
AVES [22]: The Automated Vehicles Safety and Security Analysis Framework (AVES) aims to facilitate the safety and security analysis of autonomous vehicles by leveraging four relationship matrices and a Safety and Cybersecurity Deployment (SCSD) model. The first matrix describes the hazards and the threats of the targeted vehicle along with the associated risks and the pertinent safety and security requirements. The second matrix describes the safety and security countermeasures, and the third analyzes the relationships among the countermeasures. The fourth matrix records and tracks the implementation status of the previously identified countermeasures. Finally, the fifth matrix incorporates the other four matrices into a meta-model to better analyze the system by leveraging various data from different matrices. The method is consistent with relevant safety and security standards, covering the vehicle's development lifecycle. AVES is implemented in eleven stages, that cover the concept; product development; production; operations; and service and decommissioning phases of the vehicle lifecycle, and is able to capture various aspects of the vehicle at different automation levels.
CPS master diagram [23]: The CPS master diagram is a hierarchical three-layer representation of the studied system in different process types. The lower level represents the system's physical layer, that consists of the energy flows and the physical interactions that control the flows. The middle level describes the real time information flows to control, and the third (top) level is the cyber layer which consists of information flows for monitoring and supervision. By leveraging the master diagram, experts from the safety or the security field may apply existing or new assessment approaches to analyze different CPS applications. The framework has been used for preliminary safety and security assessments in the maritime [23] and in the Internet of Things (IoT) [24] domains.
IoT medical devices [25]: This work proposes a method to analyze safety and security issues in IoT medical devices. The method is based on the STPA and an analyst is able to identify and assess accidents due to security threats that violate the functional safety. By leveraging this approach the analyst is able to analyze complex systems and perform a quantitative threat analysis by combining the EFT and the Defense Tree (DT) methods. The approach comprises seven steps: In step 1 the accidents, hazards, and safety constraints are identified; in step 2 the control structure of the system is constructed according to the STPA. In steps 3 and 4 the unsafe control actions are identified, as well as the hazards' causal factors, by employing the EFT and DT methods. In step 5 the probability of occurrence of the fundamental events of the EFT that was developed in the previous step is calculated; the estimate is based on both statistical data and the stakeholders' judgement. Finally, in step 6 the selection of the appropriate countermeasures is performed, by considering the impact of the identified accidents and the probability estimates of step 5. The proposed method has been applied to the case of an insulin pump for diabetic patients.
SARA [26]: SARA (Security Automotive Risk Analysis) provides a framework for facilitating threat modeling and the risk assessment processes for driverless vehicles. Although it is a security risk assessment method, it enables the analysis of safety issues inflicted by security threats. This is achieved by examining the impact on safety of the attack goal, and by estimating the safety severity and controllability metrics. SARA consists of four blocks: (1) Feature definition; (2) Threat specification; (3) Risk assessment; (4) Countermeasures. In the third block (risk assessment) security and safety experts identify attacks and the necessary metrics for the risk estimation, such as severity, observation, controllability, and highest attack likelihood. The proposed method was applied to the case of an autonomous car to present the potential impact of a malicious observer and of damaged road infrastructure on the vehicle.

Analysis
In order to provide a comprehensive description of the current landscape of methods for the joint analysis of safety and security of CPSs, the following list of attributes is used. Several of these attributes have been used in previous reviews or elsewhere. Each attribute is followed by a short description and a reference to the source(s) where it was originally used.

Type of joint analysis (Type):
This attribute may assume either the value "Integrated (I)" or the value "Unified (U)". In the former case the analysis is done in two separate, yet interrelated processes, whereas in the latter the analysis is performed following a single, unified process. The attribute was originally used in References [4,7]. 2. Model type (Model): Describes the model that the analysis is based on. Possible values are "Graphical (G)", "Formal (F)" and "Both graphical and formal (Both)". In graphical methods the analysis is carried out by leveraging graphical models, whilst in formal methods the analysis is based on formulas, equations, and modelling languages. This attribute was originally used in References [4,8].

Standards: The method is informed by and leverages safety/security standards. Possible values
are "Yes (Y)" and "No (N)". This attribute was originally used in Reference [7].

Application domain (Domain):
The application domain(s) where the method is applicable or has been applied. Possible values are "CPS", "IoT", "Automotive (A)", "Control Systems (CS)" or combinations thereof. This attribute was originally used in References [5,7]. 5. Approach: The type of approach followed. Possible values are "Quantitative (QNT)" and "Qualitative (QLT)". This attribute was originally used in References [4,8]. 6. Goal of the analysis (Goal): Describes the overall goal of the analysis and whether the approach aims to ensure safety, security, or both. Possible values are "Security", "Safety" and "Both". This attribute was originally used in Reference [7]. 7. Lifecycle: Describes in which phase of the system lifecycle the method is applied. Possible values are "Requirements (RE)", "Risk Analysis (RA)", "Any phase -Generic (GE)". This attribute was originally used in Reference [4]. 8. Stakeholders: Describes which stakeholders are involved, either by applying it (users) or by giving input (participants); when applying the method. Possible values are "Safety experts (A)", "Security experts (B)", "Developers (C)", "Designers (D)", and "Users or system experts (E)". This attribute was originally used in Reference [27].
The above attributes are depicted in Figure 1. Further, the following characteristics provide additional insight into understanding the operational capacity of each method; these have been inspired by the work in Reference [27]. Each of them, with the exception of Process, may assume the value "Yes (Y)", "No (N)", or "Partially (P)". Process may only assume the values "Yes (Y)" or "No (N)". Examples of such mechanisms are guide-words, checklists and questionnaires. 4. Communication: Is the method offering features to facilitate communication between different stakeholders during its application? Examples of such features are guidelines, diagrams, schematics, and so forth. 5. Conflict resolution (Conflict): Does the method facilitate the identification and study of potential conflicts between safety and security aspects? 6. Software tool (Tool): Does a software tool or toolkit that supports the application of the method exist? Table 1 depicts both the attributes and the characteristics of all methods reviewed in the surveys of Section 2.1 and of those reviewed in Section 2.2.2.

Discussion
The main findings from the analysis of the literature reviewed in the previous section are the following: • A total of sixty eight methods have been reviewed. These span a time period of approximately 20 years, with most having been proposed after 2013, and with a steady increase in the past 5 years. This is an indication of the timeliness of the subject, which can be attributed to the increased proliferation of cyber physical systems and the integration of Information Technology with Operational Technology. • The number of integrated methods (37) is slightly larger than that of unified ones (31). According to Reference [62], approaches that attempt to unify safety and security analysis techniques reduce the developer's understanding of the system being analyzed and prevent a thorough analysis of either property; this leads to an incomplete analysis with subsequent safety and security risks going unobserved. On the other hand, integrated methods extract more rigorous results and facilitate the identification of potential conflicts. • Model-based methods prevail (52 out of 68). Of these, 18 methods employ formal models, 23 methods employ graphical models, and 11 methods employ both formal and graphical models.
Model-based approaches are more practical for modeling a system's components and functionalities for existing and operational systems, by virtue of their qualitative and quantitative capabilities [4].
They are generally able to scale up to complex systems and represent different aspects related to safety and security with different viewpoints and levels of detail. On the other hand, such approaches require the analyst to have a thorough knowledge of the system; engaging all stakeholders in the process my facilitate the fulfillment of this requirement. • Less than half of the reviewed methods (20) are informed by safety and security standards.
Cyberphysical systems often operate in domains and environments highly regulated by safety and security standards. Therefore, they must be engineered to conform to these standards. It follows that safety-security co-engineering methods needs to be informed by standards. This need is more often than not satisfied if the method has been designed for use in a specific application domain. Including a validation phase in the workflow of the method, in which conformance to relevant standards is performed, is a viable alternative that may lead to the development of generic methods informed by standards. A related issue, discussed in the next section, is the need for integrated safety-security standards in several application domains. • Most (45) of the reviewed methods have been used to analyze general CPS architectures and industrial control systems in various application domains, with the transportation domain prevailing. However, the applicability of the generic methods to different application domains is usually not demonstrated. Developing a method applicable to a broad spectrum of domains and at the same time ensuring compliance with relevant standards appears as a very challenging task. • The vast majority of the reviewed methods (66) follow -at least partially-a qualitative approach; only two methods are fully quantitative. This is not surprising, because even though quantitative approaches prevail for safety engineering, the opposite is true for security engineering, where quantitative approaches are very rarely used, as they require the existence of a formal model describing the system under study. Attempting to analyze security, particularly security risk, has been shown quantitatively to be either infeasible or inadvisable in most real-world situations. Hence, a reasonable compromise is to opt for a combination of quantitative and qualitative approaches for safety-security co-engineering. • The number of methods whose goal is to ensure both safety and security (32) is slightly larger than the number of those aiming to ensure safety (30), whilst only six methods have as their primary goal to ensure security. The appropriateness of each of these approaches depends largely on the system's safety/security criticality. When the system under study is safety critical, a method whose goal is to ensure that security will not adversely influence safety is appropriate; the opposite is true when the system is security critical. But if the intention is to also have a secure system beyond the safety relevant security issues, and a safe system beyond the security relevant issues, then neither of these approaches is appropriate. In systems where both safety and security are equally important, an approach aiming to ensure both safety and security would be more appropriate. • The number of reviewed methods that are applied to both the requirements elicitation and risk analysis phases of the system lifecycle (26) is almost equal to that of methods applied to the risk analysis phase (25), and only slightly exceeds that of methods applied to the requirements elicitation phase (17); only fifteen methods are frameworks, hence applicable to any phase of the lifecycle. This nearly uniform distribution reflects the emphasis given into co-engineering safety and security as early as possible in the development lifecycle, while allowing for revisiting the results of the analysis when the system under study has been developed or is even operational. • The application of most of the reviewed methods involves safety and security experts; only few methods require the engagement of developers, designers and systems users. It is important to note that stakeholders, particularly designers and users, may engage with the analysis in two distinct but complementary ways: they provide input to the analysis in the form of domain expert knowledge, and they are the targets of the process of communicating the results; both are equally important. Acknowledging the fact that complex issues such as those of safety and security cannot be effectively analysed by the corresponding experts, it follows that successful methods will seek to involve engaged stakeholders. • Scalability issues have been discussed in the vast majority (61) of the papers proposing methods, in 24 of which these issues have only briefly been considered. Thirty seven methods are scalable, whereas 7 do not scale well. It should be noted that scalability refers to both the ability of the method to handle complex systems and to the level of abstraction at which the system under study is represented. The two are correlated, as high level abstraction allows for more complex systems to be analyzed. The challenge, therefore, is to develop methods that can strike an appropriate balance between those two aspects of scalability, so that the analysis results in an appropriate and practically useful level of detail. • The majority (55) of the reviewed methods provide mechanisms to stimulate creativity among the analysts and other relevant stakeholders. As many methods rely, at least to some extent, on scenario development, creativity is an important characteristic. This is even more so when the application of the method calls for a multi-disciplinary, multi-stakeholder team. • Less than half (28) of the reviewed methods include techniques to communicate their results to the relevant stakeholders. Another 28 methods only briefly address the issue, whilst 10 methods do not address it at all. As already implied above, this characteristic is intertwined with the involvement of stakeholders attribute. • All methods are process-based; the structure and the steps of the process do vary, however.
As pointed out in the sequel, developing a methodology to encompass different process structures is still a challenge. • The majority (49) of the reviewed methods do not address the conflict resolution issue. Sixteen methods do address it, and a further 3 address it only partially. The implications of this central issue is elaborated upon in the sequel. • The majority (41) of the reviewed methods are not supported by any software tool or toolkit. Only 20 methods are fully supported, and another 7 are partially supported by such tools. This has been a rather surprising finding, as the purpose of a safety-security co-engineering method is to be applied in real-world application scenarios. The complexity of such methods requires software support for their usage.
To give a bird's eye view of these findings, and also to facilitate cross-referencing, the above are summarized in Figures 2 and 3. Figure 2 depicts the taxonomy of Figure 1, with the number following each attribute indicates the number of methods having the corresponding attribute. Figure 3 provides the same information on characteristics.  Additionally, a number of issues that have been under-researched have been identified. These are as follows: • Resolving conflicting safety/security results: The problem of conflicting results when studying safety and security jointly has been known for some time. There are two approaches to address this problem: either allow conflicting results to be derived and then resolve these conflicts, or avoid the occurrence of such conflicts by design. Unified safety and security co-engineering methods tend to generate less conflicts than integrated methods do. However, integrated methods tend to allow more comprehensive analyses of both domains. Therefore, integrated methods that would by design prevent the occurrence of conflicting results would address this issue effectively. This could perhaps be achieved if goal oriented integrated methods were developed. Further, the analysis is best performed in the early stages (requirements elicitation phase), as this makes the problem of conflict resolution much easier to solve, and leads to the development of safe-and-secure CPSs by design. • Standard methodology: Despite the sizable extent of the literature on safety and security co-engineering methods, a generic, application-domain-independent methodology, instances of which would be existing methods and those to be developed in the future, is yet to be developed. An example of such a methodology in the security domain is the risk analysis methodology defined in the ISO 27005 standard.
• Validation: Not many of the reviewed papers include information on the validation/evaluation of the method they propose. More research is needed to evaluate the correctness, completeness, effectiveness, efficiency, scalability and so forth, of proposed methods, in a manner that will facilitate comparative assessments. • Safety and security standards: Some standards addressing safety and security for industrial control systems exist. Examples of such standards are ISA99/IEC 6443, IEC 62645, IEC TR63609, ISO 26262 to name a few; cross-references with other standards (e.g., IEC 61508) also exist. However, the applicability of such standards to effectively address both safety and security, particularly in an industry 4.0 context, is still to be firmly established. Hence, a need for revisiting existing standards with an eye towards facilitating their use in industry, by means of reducing ambiguity, arises. Additionally, the adoption of standards specific for industry sectors, along the lines of the practice followed in the nuclear plant domain will guide the development of safe-and-secure by design industrial control systems. • Application domains: As noted before, the transportation domain prevails among application domains addressed by the reviewed methods. Notwithstanding the fact that several methods have been claimed to have been designed to be applicable to any domain, their applicability has not been demonstrated. As several emerging application domains are both safety and security critical (e.g., autonomous vessels, drones), the development of methods addressing specifically systems in such domains remains an issue. • Dynamic character of CPS: CPSs are dynamic by nature. Methods able to model and cope with this characteristic of CPSs are yet to be developed. Existing work on dynamic security and dynamic safety risk assessment can be leveraged to this end. • Model type: Most of the safety analysis approaches are based on formal models. Security techniques on the other hand tend to focus on qualitative analysis. Therefore, an approach able to handle the complexity of CPS by leveraging both graphical models and systematic perspectives would allow the consolidation of advantages of both worlds. • Holistic approach: The human factor in relation with CPSs is often overlooked. In fact, CPSs, particularly safety/security critical ones need to be considered and studied as socio-technical systems. This calls for a holistic approach towards safety and security co-engineering, that would encompass the whole ecosystem into which the CPS under study is expected to operate, and would involve all the relevant stakeholders in the process. To this end, future methods should enjoy previously mentioned attributes such as scalability, communication, and model type, in order to facilitate the analysis of CPSs when both technical and human aspects are considered. Particularly, such methods should be able to handle the complexity (scalability) derived from the human-machine interaction; communicate the results by providing reports and leveraging software tools (communication); and provide graphical models of the system under study (model type) to facilitate the analysis and the validation of the results.

Conclusions
We have revisited previous surveys on cybersecurity and safety co-engineering approaches and performed a systematic literature survey of such approaches. We defined a multi-attribute taxonomy for such approaches and we used this to analyze them. We thus provided a comprehensive discussion on the recent advances in cybersecurity and safety co-engineering. The joint study of safety and security has been a goal of researchers in both fields for more than thirty years. Despite the longevity of the problem and the substantial volume of research results on safety and security co-engineering that has been generated in the past few years, several important issues remain open. Through our review, we identified and discussed such issues and the research challenges that they imply. In the future, among the many possible research challenges in the field, we plan to focus on developing a holistic, integrated, graphical model based, safety and security requirements elicitation co-engineering approach, applicable to the autonomous vessel domain.