# Towards Trustworthy Safety Assessment by Providing Expert and Tool-Based XMECA Techniques

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. Motivation

#### 1.2. State of the Art

- Although FMECA is a well-known technique that has been used in different domains for quite a long time, it is still quite complicated to use due to task dimension, not having a formalized procedure, a huge amount of modifications, etc. Therefore, recent research still provides additional clarifications to FMECA utilization, its peculiarities, etc.
- FMECA is a methodological technique, but its key drawback is semi-formalism and the need for expert support, which is not studied in detail in well-known works;
- To increase the trustworthiness of assessments, experts are needed, but procedures and tools are needed that either improve trustworthiness due to the correct combination of assessments and/or reduce the influence of individual experts by reducing non-formalized operations (tool support). Such an integrated approach requires additional formalization and development.

#### 1.3. Objective and Research Questions

- What approach could be utilized to minimize safety assessment inaccuracy? With what limitations?
- In what way could the generic XMECA technique be applied for safety and security assessment?
- How could the criticality of assumptions usually used to implement FMECA be analyzed?
- What are the impacts of expert approaches and tool support?
- In which manner could FMECA modification (IMECA) be utilized for cybersecurity assessment within XMECA?

#### 1.4. Paper Structure

## 2. Materials and Methods

- a formal description of the shortcomings and the consequences of these shortcomings for the FMECA methodology, which is combined in the form of the XMECA conception, which allows minimizing the risks of erroneous decisions and narrows the area of uncertainty. To accomplish this, we use the EUMECA analysis of XMECA (E—error; U—uncertainty). To evaluate the consequences of possible errors, we use an expert procedure for determining the importance of error and uncertainty factors;
- scenario-oriented integration of expert assessments when using XMECA, considering the complexity of such integration when using verbal, fuzzy, and quantitative assessments. This principle allows various scenarios to achieve the best result when combining expert estimates to maximize the accuracy of estimation. Moreover, the number of operations performed by an expert is being reduced;
- reducing the influence of individual experts and uncertainty factors during the assessment process by minimizing non-automated (manual) operations using improved tools. This principle is a natural addition and support for the first two.

_{E10}= |E

_{1}|/|E

_{0}|, h

_{U10}= |U

_{1}|/|U

_{0}|, h

_{EU10}= (|E

_{1}| + |U

_{1}|)/(|E

_{0}| + |U

_{0}|)

_{E}

_{21}= |E

_{2}|/|E

_{1}|, h

_{U}

_{21}= |U

_{2}|/|U

_{1}|, h

_{EU}

_{21}= (|E

_{2}| + |U

_{2}|)/(|E

_{1}| + |U

_{1}|)

_{1T}and U

_{1T}correspondently and calculate metrics.

_{ET}

_{1}= |E

_{1T}|/|E

_{1}|, h

_{U}

_{T1}= |U

_{1T}|/|U

_{1}|, h

_{EUT}

_{1}= (|E

_{1T}| + |U

_{1T}|)/(|E

_{1}| + |U

_{1}|)

_{EU}= h

_{EU10}× h

_{EUT}

_{1}× h

_{EU}

_{21}

## 3. Results

#### 3.1. XMECA Model

_{i}implies failure cause (failed element);

_{i}is herein taken to mean a set of failure consequences (effects);

_{i}denotes failure probability, which can be preassigned qualitatively with the fuzzy scale (as an example, «low»–«medium»–«high») or quantitatively as a value in range 0–1;

_{i}identifies failure severity, which can also be defined using a fuzzy scale or quantitatively;

_{i}stands for a failure criticality determined as a function of fuzzy variables φ, c

_{i}= φ (p

_{i},s

_{i});

_{i}signifies a set of possible failure modes;

_{i}is the number of considered failure modes of element i; the total number of failure modes is calculated by the following expression:

_{1}+ k

_{2}+ … + k

_{F}

_{i}, m

_{i}, and e

_{i}, and the relation between s

_{i}, p

_{i}, and c

_{i}is shown in Figure 3.

_{1}= k

_{2}= … = k

_{F}= 1; in a general way, F* = K.

- elements f
_{i}(for instance, module components, program operators, process operations, etc.), failures of which are to be considered, that is f_{i}ϵ ΔF, ΔF ϲ M_{F}, where ΔF is a subset of components investigated; M_{F}is a set of components; - failure modes m
_{ij}of element f_{i}, which are to be considered, i.e.,

_{ij}ϵ ΔM

_{i}, ΔM

_{i}ϲ MM

_{i}

_{i}is a set of elements f

_{i}failures investigated; MM

_{i}is a set of all element f

_{i}failures;

- effects e
_{ij}of failure mode m_{ij}of element f_{i}, which are to be considered, i.e.,

_{ij}ϵ ΔE

_{i}, ΔE

_{i}ϲ ME

_{i}

_{ij}of an element f

_{ii}; MEi is a set of all possible effects for a particular failure mode of this element;

- probability p
_{ij}and severity s_{ij}of failure mode m_{ij}of element f_{i}; probability p_{ij}and severity s_{ij}are being adopted according to defined scale on the sets of values MP = {p’h} and MS = {s’_{g}} accordingly; criticality c_{ij}of failure mode m_{ij}of element f_{i}, which could be either explicitly evaluated by an expert using given function φ or assigned by an expert manually on the set of values MC = {c’_{g}}.

#### 3.2. Stages of XMECA Application

#### 3.3. XMECA and Other Assessment Techniques

_{1}HAZOP is a set of techniques based on HAZOP and its modifications (software HAZOP, control HAZOP, etc.).

_{1}= {S, C, …}

_{2}IT is a set of techniques intended for fault/intrusion insertion to verify XMECA or X

_{2}HAZOP assumptions and statements (fault, vulnerability, software fault, etc.).

_{2}= {F, V, SF, …}

_{4}TA is a set of techniques based on FTA and its modifications (FTA, ETA, etc.):

_{3}= {F, E, …}

_{4}BD is a set of techniques based on RBD and its modifications (safety, security, availability, etc.):

_{4}= {R, Saf, Sec, Avail, …}

_{5}= {M, SemiM, …}

_{i}) depends on the input information completeness, requirements to output information, etc.

_{i}= f (I, O.R, …)

#### 3.4. EUMECA Analysis of XMECA

#### 3.4.1. Uncertainty Evaluation Questionnaire

#### 3.4.2. Evaluation in Case of Equal Qualification (Self-Assessment) of Experts

#### Scenario-Based approach

- analysis of divergence types associated with different constituents of the model (1);
- generation of the final version for each divergence;
- preparation of integrated version of XMECA;
- accomplishing analysis of it and provision of eventual safety assessment.

#### Scenario ScC

- generation of a set of elements to be included in FMECA table according to (1):

- generation of sets of failure modes to be considered for all elements f
_{i}ϵ MΔF(ScC):

- generation of sets of failure effects eij of mode m
_{ij}of element f_{i}to be considered:

- evaluation of failure probabilities of mode m
_{ij}of element f_{i}by equation:

_{ij}(ScC) = max {ΔPij(q)}, q = 1, …, Q

- evaluation of failure severities of mode m
_{ij}of element f_{i}by equation:

_{ij}(ScC) = max {ΔSij(q)}, q = 1, …, Q

- evaluation of failure criticalities of mode m
_{ij}of element f_{i}by equation:

_{ij}(ScC) = max {ΔCij(q)}, q = 1, …, Q

#### Scenario ScO

- generation of a set of elements to be included in FMECA table according to (1) using the equation:

- generation of sets of failure modes for all elements f
_{i}ϵ MΔF(ScC) to be considered:

- generation of sets of failure consequences e
_{ij}of mode m_{ij}of element f_{i}to be considered:

- evaluation of probabilities of failure modes m
_{ij}of element f_{i}using equation:

_{ij}(ScO) = min {ΔPij(q)}, q = 1, …, Q

- evaluation of severities of failure modes m
_{ij}of element f_{i}by equation:

_{ij}(ScO) = min {ΔSij(q)}, q = 1, …, Q

- evaluation of failure criticalities of mode m
_{ij}of element f_{i}by equation:

_{ij}(ScO) = min {ΔCij(q)}, q = 1, …, Q

#### Scenario ScW

- generation of set of elements, of which failures are to be included in FMECA table according to (1):

- generation of sets of failure modes for all elements fi ϵ MΔF(ScC), which have to be considered:

- generation of sets of failure consequences e
_{ij}of mode m_{ij}of element f_{i}, which have to be considered:

- evaluation of probabilities of failure modes m
_{ij}of element f_{i}by application of ceiling function to the average:

_{ij}(ScW) = avermax {ΔPij(q)}, q = 1, …, Q

- evaluation of severities of failure modes m
_{ij}of element f_{i}by equation:

_{ij}(ScW) = avermax {ΔSij(q)}, q = 1, …, Q

- evaluation of failure criticalities of mode m
_{ij}of element f_{i}by equation:

_{ij}(ScW) = avermax {ΔCij(q)}, q = 1, …, Q

#### 3.4.3. Evaluation in Case of Different Qualification (Self-Assessment) of Experts

#### Group of Scenarios ScDT

#### Group of Scenarios ScDW

#### 3.5. Case Study. Expert-Based FMECA Assessment of Hardware/Software Module Safety

#### 3.5.1. Results of EUMECA

- in considered cases, higher probability and severity are assigned to hardware-related assumptions;
- by experts’ opinions, the higher risk caused by uncertain assessment of probability in respect to safety overestimation is due to failure mistakenly treated as detected, while, in respect to safety underestimation, it is due to several components used for safety assessment being given too high or excess system levels being considered;
- by experts’ opinions, the higher risk caused by uncertain assessment of severity in respect to safety overestimation is due to not all software faults being considered and hardware and software faults are not considered in respect to possible attacks, while, in respect to safety underestimation, it is due to fact that more than required software faults are considered and more than required hardware faults (physical and project) are considered;

#### 3.5.2. Assumption Modes and Effects Evaluation Example

- different sets of elements: sets of elements provided by different experts can be merged;
- different sets of failure modes: two scenarios of merging are possible: optimistic (intersection of sets) and conservative (union of sets);
- different sets of failure effects: to choose more critical effects, preference relation could be utilized.

#### 3.6. Application of XMECA for Cybersecurity Assessment

#### 3.7. The Tool for XMECA Assessment of Safety and Security

#### 3.7.1. AXMEA Tool

#### 3.7.2. Assessment of Increasing Trustworthiness

- not all components are defined for safety assessment;
- number of components used for safety assessment is given too high;
- not all failure modes are considered;
- excess failure modes are considered;
- failure multiplicity is underestimated;
- failure multiplicity is overestimated;
- multiple faults of different components at one level are not considered;
- multiple faults of different components at different levels are not considered;
- multiple faults of different versions are not considered.

## 4. Discussion

- specifying and excluding traditional assumptions for FMECA and IMECA techniques (first of all, types of faults);
- minimizing errors caused by objective uncertainty of input data, the complexity of systems, and decisions of experts;
- improving part of activities based on automatically executed operations.

## 5. Conclusions

- provision of integration into this platform subsystem for expert assessment and tools developed earlier (IMECA, AXMEA);
- development of automatic vulnerability monitor based on vulnerability data processing from different databases of programs and programmable components;
- improving trustworthiness accuracy assessment by application of considered and new metrics and their calculation considering weights of operations, assumption severity, and so on.

## Author Contributions

## Funding

## Informed Consent Statement

## Acknowledgments

## Conflicts of Interest

## References

- Jiang, Z.; Zhao, T.; Wang, S.; Ren, F. A Novel Risk Assessment and Analysis Method for Correlation in a Complex System Based on Multi-Dimensional Theory. Appl. Sci.
**2020**, 10, 3007. [Google Scholar] [CrossRef] - Sklyar, V. Safety-Critical Certification of FPGA-based Platform against Requirements of U.S. Nuclear Regulatory Commission (NRC): Industrial Case Study. ICTERI. 2016. Available online: http://ceur-ws.org/Vol-1614/paper_32.pdf (accessed on 28 April 2022).
- Kharchenko, V.; Illiashenko, O.; Sklyar, V. Invariant-Based Safety Assessment of FPGA Projects: Conception and Technique. Computers
**2021**, 10, 125. [Google Scholar] [CrossRef] - Hajda, J.; Jakuszewski, R.; Ogonowski, S. Security Challenges in Industry 4.0 PLC Systems. Appl. Sci.
**2021**, 11, 9785. [Google Scholar] [CrossRef] - Takahashi, M.; Anang, Y.; Watanabe, Y. A Safety Analysis Method for Control Software in Coordination with FMEA and FTA. Information
**2021**, 12, 79. [Google Scholar] [CrossRef] - Peeters, J.; Basten, R.; Tinga, T. Improving failure analysis efficiency by combining FTA and FMEA in a recursive manner. Reliab. Eng. Syst. Saf.
**2018**, 172, 36–44. [Google Scholar] [CrossRef][Green Version] - Trivyza, N.L.; Cheliotis, M.; Boulougouris, E.; Theotokatos, G. Safety and Reliability Analysis of an Ammonia-Powered Fuel-Cell System. Safety
**2021**, 7, 80. [Google Scholar] [CrossRef] - Ehrlich, M.; Bröring, A.; Harder, D.; Auhagen-Meyer, T.; Kleen, P.; Wisniewski, L.; Trsek, H.; Jasperneite, J. Alignment of safety and security risk assessments for modular production systems. Elektrotech. Inftech.
**2021**, 138, 454–461. [Google Scholar] [CrossRef] - Wang, Z.; Wang, R.; Deng, W.; Zhao, Y. An Integrated Approach-Based FMECA for Risk Assessment: Application to Offshore Wind Turbine Pitch System. Energies
**2022**, 15, 1858. [Google Scholar] [CrossRef] - IEC/ISO 31010:2019; Risk Management—Risk Assessment Techniques. European Ed. 2.0. International Electrotechnical Commission: Geneva, Switzerland, 2019.
- Babeshko, I.; Leontiiev, K.; Kharchenko, V.; Kovalenko, A.; Brezhniev, E. Application of Assumption Modes and Effects Analysis to XMECA. In Theory and Engineering of Dependable Computer Systems and Networks; DepCoS-RELCOMEX 2021. Advances in Intelligent Systems and Computing; Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J., Eds.; Springer: Cham, Switzerland, 2021; Volume 1389. [Google Scholar] [CrossRef]
- Giardina, M.; Tomarchio, E.; Buffa, P.; Palagonia, M.; Veronese, I.; Cantone, M.C. FMECA Application in Tomotherapy: Comparison between Classic and Fuzzy Methodologies. Environments
**2022**, 9, 50. [Google Scholar] [CrossRef] - Oliveira, J.; Carvalho, G.; Cabral, B.; Bernardino, J. Failure Mode and Effect Analysis for Cyber-Physical Systems. Future Internet
**2020**, 12, 205. [Google Scholar] [CrossRef] - Peyghami, S.; Davari, P.; Firuzabad, M.; Blaabjerg, F. Failure Mode, Effects and Criticality Analysis (FMECA) in Power Electronic based Power Systems. In Proceedings of the 2019 21st European Conference on Power Electronics and Applications (EPE ’19 ECCE Europe), Genova, Italy, 3–5 September 2019; pp. 1–9. [Google Scholar] [CrossRef][Green Version]
- Catelani, M.; Ciani, L.; Galar, D.; Guidi, G.; Matucci, S.; Patrizi, G. FMECA Assessment for Railway Safety-Critical Systems Investigating a New Risk Threshold Method. IEEE Access
**2021**, 9, 86243–86253. [Google Scholar] [CrossRef] - Buja, A.; Manfredi, M.; De Luca, G.; Zampieri, C.; Zanovello, S.; Perkovic, D.; Scotton, F.; Minnicelli, A.; De Polo, A.; Cristofori, V.; et al. Using Failure Mode, Effect and Criticality Analysis to Improve Safety in the COVID Mass Vaccination Campaign. Vaccines
**2021**, 9, 866. [Google Scholar] [CrossRef] [PubMed] - Serafini, A.; Troiano, G.; Franceschini, E.; Calzoni, P.; Nante, N.; Scapellato, C. Use of a systematic risk analysis method (FMECA) to improve quality in a clinical laboratory procedure. Ann. Ig
**2016**, 28, 288–295. [Google Scholar] [CrossRef] [PubMed] - Milioulis, K.; Bolbot, V.; Theotokatos, G. Model-Based Safety Analysis and Design Enhancement of a Marine LNG Fuel Feeding System. J. Mar. Sci. Eng.
**2021**, 9, 69. [Google Scholar] [CrossRef] - Di Nardo, M.; Murino, T.; Osteria, G.; Santillo, L.C. A New Hybrid Dynamic FMECA with Decision-Making Methodology: A Case Study in An Agri-Food Company. Appl. Syst. Innov.
**2022**, 5, 45. [Google Scholar] [CrossRef] - Di Bona, G.; Forcina, A.; Falcone, D.; Silvestri, L. Critical Risks Method (CRM): A New Safety Allocation Approach for a Critical Infrastructure. Sustainability
**2020**, 12, 4949. [Google Scholar] [CrossRef] - Shafiee, M.; Enjema, E.; Kolios, A. An Integrated FTA-FMEA Model for Risk Analysis of Engineering Systems: A Case Study of Subsea Blowout Preventers. Appl. Sci.
**2019**, 9, 1192. [Google Scholar] [CrossRef][Green Version] - Chen, L.; Jiao, J.; Zhao, T. A Novel Hazard Analysis and Risk Assessment Approach for Road Vehicle Functional Safety through Integrating STPA with FMEA. Appl. Sci.
**2020**, 10, 7400. [Google Scholar] [CrossRef] - Bognár, F.; Hegedűs, C. Analysis and Consequences on Some Aggregation Functions of PRISM (Partial Risk Map) Risk Assessment Method. Mathematics
**2022**, 10, 676. [Google Scholar] [CrossRef] - La Fata, C.M.; Giallanza, A.; Micale, R.; La Scalia, G. Improved FMECA for effective risk management decision making by failure modes classification under uncertainty. Eng. Fail. Anal.
**2022**, 135, 106163. [Google Scholar] [CrossRef] - Lee, G.-H.; Akpudo, U.E.; Hur, J.-W. FMECA and MFCC-Based Early Wear Detection in Gear Pumps in Cost-Aware Monitoring Systems. Electronics
**2021**, 10, 2939. [Google Scholar] [CrossRef] - Piumatti, D.; Sini, J.; Borlo, S.; Sonza Reorda, M.; Bojoi, R.; Violante, M. Multilevel Simulation Methodology for FMECA Study Applied to a Complex Cyber-Physical System. Electronics
**2020**, 9, 1736. [Google Scholar] [CrossRef] - Babeshko, E.; Kharchenko, V.; Gorbenko, A. Applying F(I)MEA-technique for SCADA-Based Industrial Control Systems Dependability Assessment and Ensuring. In Proceedings of the 2008 Third International Conference on Dependability of Computer Systems DepCoS-RELCOMEX, Szklarska Poreba, Poland, 26–28 June 2008; pp. 309–315. [Google Scholar] [CrossRef]
- Androulidakis, I.; Kharchenko, V.; Kovalenko, A. IMECA-Based Technique for Security Assessment of Private Communications: Technology and Training. Inf. Secur. Int. J.
**2016**, 35, 99–120. [Google Scholar] [CrossRef][Green Version] - Kharchenko, V. Gap-and-IMECA-Based Assessment of I&C Systems Cyber Security. In Complex Systems and Dependability. Advances in Intelligent and Soft Computing, 170; Kharchenko, V., Andrashov, A., Sklyar, V., Siora, A., Kovalenko, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; 334p. [Google Scholar] [CrossRef]
- Illiashenko, O.; Kharchenko, V.; Chuikov, Y. Safety analysis of FPGA-based systems using XMECA for V-model of life cycle. Radioelectron. Comput. Syst.
**2016**, 80, 141–147. [Google Scholar] - Babeshko, E.; Kharchenko, V.; Leontiiev, K.; Odarushchenko, O.; Strjuk, O. NPP I&C safety assessment by aggregation of formal techniques. In Proceedings of the 2018 26th International Conference on Nuclear Engineering, London, UK, 22–26 July 2018; pp. 21–26. [Google Scholar]
- Lolli, F.; Gamberini, R.; Balugani, E.; Rimini, B.; Mai, F. FMECA-based optimization approaches under an evidential reasoning framework. DEStech Trans. Eng. Technol. Res.
**2017**, 1, 738–743. [Google Scholar] [CrossRef][Green Version] - Ivančan, J.; Lisjak, D. New FMEA Risks Ranking Approach Utilizing Four Fuzzy Logic Systems. Machines
**2021**, 9, 292. [Google Scholar] [CrossRef] - Fabis-Domagala, J.; Domagala, M.; Momeni, H. A Concept of Risk Prioritization in FMEA Analysis for Fluid Power Systems. Energies
**2021**, 14, 6482. [Google Scholar] [CrossRef] - Pikner, H.; Sell, R.; Majak, J.; Karjust, K. Safety System Assessment Case Study of Automated Vehicle Shuttle. Electronics
**2022**, 11, 1162. [Google Scholar] [CrossRef] - Piesik, E.; Sliwinski, M.; Barnert, T. Determining and verifying the safety integrity level of the safety instrumented systems with the uncertainty and security aspects. Reliab. Eng. Syst. Saf.
**2016**, 152, 259–272. [Google Scholar] [CrossRef] - Chin, K.-S.; Wang, Y.-M.; Ka Kwai Poon, G.; Yang, J.-B. Failure mode and effects analysis using a group-based evidential reasoning approach. Comput. Oper. Res.
**2009**, 36, 1768–1779. [Google Scholar] [CrossRef] - Liu, H.-C. FMEA Using Uncertainty Theories and MCDM Methods; Springer Science: Singapore, 2016; p. 219. [Google Scholar]
- Liu, H.-C.; Chen, X.-Q.; Duan, C.-Y.; Wang, Y.-M. Failure mode and effect analysis using multi-criteria decision making methods: A systematic literature review. Comput. Ind. Eng.
**2019**, 135, 881–897. [Google Scholar] [CrossRef] - Liu, H.-C.; Liu, L.; Liu, N. Risk evaluation approaches in failure mode and effects analysis: A literature review. Expert Syst. Appl.
**2013**, 40, 828–838. [Google Scholar] [CrossRef] - Dai, W.; Maropoulos, P.; Cheung, W.; Tang, X. Decision-making in product quality based on failure knowledge. Int. J. Prod. Lifecycle Manag.
**2011**, 5, 143–163. [Google Scholar] [CrossRef] - Lee, Y.-S.; Kim, H.-C.; Cha, J.-M.; Kim, J.-O. A new method for FMECA using expert system and fuzzy theory. In Proceedings of the 2010 9th International Conference on Environment and Electrical Engineering, Prague, Czech Republic, 16–19 May 2010. [Google Scholar]
- Liu, H.-C.; Chen, X.-Q.; You, J.-X.; Li, Z. A New Integrated Approach for Risk Evaluation and Classification With Dynamic Expert Weights. IEEE Trans. Reliab.
**2020**, 70, 163–174. [Google Scholar] [CrossRef] - Colli, M.; Sala, R.; Pirola, F.; Pinto, R.; Cavalieri, S.; Wæhrens, B.V. Implementing a Dynamic FMECA in the Digital Transformation Era; IFAC-PapersOnLine: Berlin, Germany, 2019. [Google Scholar]
- Zhang, P.; Qin, G.; Wang, Y. Risk Assessment System for Oil and Gas Pipelines Laid in One Ditch Based on Quantitative Risk Analysis. Energies
**2019**, 12, 981. [Google Scholar] [CrossRef][Green Version] - Heidary Dahooie, J.; Vanaki, A.S.; Firoozfar, H.R.; Zavadskas, E.K.; Čereška, A. An Extension of the Failure Mode and Effect Analysis with Hesitant Fuzzy Sets to Assess the Occupational Hazards in the Construction Industry. Int. J. Environ. Res. Public Health
**2020**, 17, 1442. [Google Scholar] [CrossRef][Green Version] - Zhou, X.; Tang, Y. Modeling and Fusing the Uncertainty of FMEA Experts Using an Entropy-Like Measure with an Application in Fault Evaluation of Aircraft Turbine Rotor Blades. Entropy
**2018**, 20, 864. [Google Scholar] [CrossRef][Green Version] - Idmessaoud, Y.; Guiochet, J.; Dubois, D. Questionnaire for Estimating Uncertainties in Assurance Cases. 2022. Available online: https://hal.laas.fr/hal-03649068/document (accessed on 28 April 2022).
- Yasko, A.; Babeshko, E.; Kharchenko, V. FMEDA-Based NPP I&C Systems Safety Assessment: Toward to Minimization of Experts’ Decisions Uncertainty. In Proceedings of the 24th International Conference on Nuclear Engineering, Charlotte, NC, USA, 26–30 June 2016. [Google Scholar]
- Yasko, A.; Babeshko, E.; Kharchenko, V. FMEDA and FIT-based safety assessment of NPP I&C systems considering expert uncertainty. In Proceedings of the 2018 26th International Conference on Nuclear Engineering, London, UK, 22–26 July 2018; pp. 231–238. [Google Scholar]
- Leontiiev, K.; Babeshko, I.; Kharchenko, V. Assumption Modes and Effect Analysis of XMECA: Expert based safety assessment. In Proceedings of the 2020 IEEE 11th International Conference on Dependable Systems, Services and Technologies (DESSERT), Kyiv, Ukraine, 14–18 May 2020; pp. 90–94. [Google Scholar]
- Illiashenko, O.; Babeshko, E. Choosing FMECA-Based Techniques and Tools for Safety Analysis of Critical Systems. Inf. Secur. Int. J.
**2012**, 28, 275–285. Available online: http://procon.bg/system/files/28.22_Illiashenko_Babeshko.pdf (accessed on 28 April 2022). [CrossRef][Green Version] - Kharchenko, V.; Illiashenko, O.; Kovalenko, A.; Sklyar, V.; Boyarchuk, A. Security Informed Safety Assessment of NPP I&C Systems: GAP-IMECA Technique. In Proceedings of the 2014 22nd International Conference on Nuclear Engineering, Prague, Czech Republic, 7–11 July 2014; Volume 3. [Google Scholar] [CrossRef]
- Kolisnyk, M. Vulnerability analysis and method of selection of communication protocols for information transfer in Internet of Things systems. Radioelectron. Comput. Syst.
**2021**, 1, 133–149. [Google Scholar] [CrossRef] - Reliability Prediction of Electric Equipment. Department of Defense, Washington DC, USA, Tech. Rep. MIL-HDBK-217F, December 1991. Available online: https://s3vi.ndc.nasa.gov/ssri-kb/static/resources/MIL-HDBK-217F-Notice2.pdf (accessed on 28 April 2022).
- International Electro Technical Commission (Ed.) IEC TR 62380; Reliability Data Handbook—Universal Model for Reliability Prediction of Electronics Components, PCBs and Equipment; IEC: Geneva, Switzerland, 2005. [Google Scholar]
- International Electro Technical Commission (Ed.) IEC 61508; Functional Safety of Electrical/Electronic/Programmable Electronic Safety-Related Systems—Part 1–7; IEC: Geneva, Switzerland, 2010. [Google Scholar]
- Babeshko, E.; Kharchenko, V.; Leoniev, K.; Ruchkov, E. Practical aspects of operating and analytical reliability assessment of FPGA-based I&C systems. Radioelectron. Comput. Syst.
**2020**, 3, 75–83. [Google Scholar] [CrossRef]

Safety Assessment Technique | Type of Techniques/Measure | Expert Support | Reasons of Errors and Uncertainties | Percent of Operations |
---|---|---|---|---|

XMECA | Semi-formal/risk | Selection of critical elements, failure modes, criticality assessment | Task dimension | Over 50% |

Software/hardware fault injection testing | Semi-formal/special metris | Selection of statements (operators and components), error types, criticality assessment | Task dimension and technological complexity | Over 30% |

FTA and RBD | Formal/probability of up or down states | Definition of initial reasons, influence and probabilities of element failures | Task dimension | Over 50% |

Markov and semi-Markov models | Formal/availability function | Definition of states and trasitions, parameters of distribution laws, failure and recovery rates | Task dimension | Over 70% |

Common cause failure (CCF) | Semi-formal/risk of CCF | Definition of diversity types and metrics | Absence of representative statistics, testing complexity | Over 50% |

Assumptions, Limitations | Modes | Effects | Probability | Severity |
---|---|---|---|---|

Expert assessment | Not all components are defined for safety assessment | Safety overestimation | ||

The number of components used for safety assessment is given too high | Safety underestimation | |||

Not all failure modes are considered | Safety overestimation | |||

Excess failure modes are considered | Safety underestimation | |||

Failure criticality (probability, severity) is underestimated | Safety overestimation | |||

Failure criticality (probability, severity) is overestimated | Safety underestimation | |||

Failure mistakenly treated as detected | Safety overestimation | |||

Failure mistakenly treated as undetected | Safety underestimation | |||

Single/multiple faults | Failure multiplicity is underestimated | Safety overestimation | ||

Failure multiplicity is overestimated | Safety underestimation | |||

Multiple faults of different components at one level are not considered | Safety overestimation | |||

Multiple faults of different components at different levels are not considered | Safety overestimation | |||

Multiple faults of different versions are not considered | Safety overestimation | |||

System levels | Not all levels are considered | Safety overestimation | ||

Excess levels are considered | Safety underestimation | |||

Interaction between levels is not considered | Safety overestimation | |||

Excess interaction between levels is considered | Safety underestimation | |||

Types of faults | Not all software faults are considered | Safety overestimation | ||

More than required software faults are considered | Safety underestimation | |||

Not all hardware faults (physical and project) are considered | Safety overestimation | |||

More than required hardware faults (physical and project) are considered | Safety underestimation | |||

Hardware and software faults are not considered considering possible attacks | Safety overestimation |

Divergence | Expression | Explanation |
---|---|---|

definition of different sets of elements in which failures f_{i} are to be considered | set MΔF of sets ΔF(q), q = 1, …, Q, | ΔF(q) is a set of elements in which failures are considered by a q-th expert |

definition of different sets of failure modes m_{ij} of element f_{i} that are to be considered | set MΔMi of sets ΔMi(q), for all q, ΔMi(q) ϲ MMi | ΔMi(q) is a set of element failure modes f_{i} considered by a q-th expert |

definition of different sets of effects e_{ij} of failure mode m_{ij} of element f_{i} that are to be considered | set MΔEi of sets ΔEi(q), for all q, ΔEi(q) ϲ MEi, | ΔEi(q) is a set of failure effects of element f_{i} considered by a q-th expert |

definition of different probabilities of failure modes m_{ij} of element f_{i} | set MΔPij of sets ΔPij(q), for all q, ΔPij(q) ϲ MP | ΔPij(q) is a set of probabilities of failure modes m_{ij} of element f_{i} considered by a q-th expert |

definition of different severities of failure modes m_{ij} of element f_{i} | set MΔSi of sets ΔSi(q), for all q, ΔSi(q) ϲ MS | ΔSi(q) is a set of failure severities of element f_{i} considered by a q-th expert |

obtained different criticalities of failure modes m_{ij} of element f_{i} | set MΔCi of sets ΔCi(q), for all q, ΔCi(q) ϲ MC | criticality is either evaluated explicitly by a q-th expert using specified function φ or is defined by an expert manually (these two cases can be handled separately) |

Assumptions, Limitations | Modes | Effects | Probability | Severity | Risk |
---|---|---|---|---|---|

Expert assessment | Not all components are defined for safety assessment | Safety overestimation | 2.1 | 1.6 | 3.36 |

The number of components used for safety assessment is given too high | Safety underestimation | 2.4 | 2.3 | 5.52 | |

Not all failure modes are considered | Safety overestimation | 1.5 | 1.5 | 2.25 | |

Excess failure modes are considered | Safety underestimation | 2.3 | 2.6 | 5.98 | |

Failure criticality (probability, severity) is underestimated | Safety overestimation | 2 | 1.6 | 3.2 | |

Failure criticality (probability, severity) is overestimated | Safety underestimation | 2.2 | 2.3 | 5.06 | |

Failure mistakenly treated as detected | Safety overestimation | 2.3 | 1.7 | 3.91 | |

Failure mistakenly treated as undetected | Safety underestimation | 2.1 | 2.1 | 4.41 | |

Single/multiple faults | Failure multiplicity is underestimated | Safety overestimation | 1.6 | 1.3 | 2.08 |

Failure multiplicity is overestimated | Safety underestimation | 2 | 2.2 | 4.4 | |

Multiple faults of different components at one level are not considered | Safety overestimation | 1.9 | 1.6 | 3.04 | |

Multiple faults of different components at different levels are not considered | Safety overestimation | 1.8 | 2 | 3.6 | |

Multiple faults of different versions are not considered | Safety overestimation | 1.8 | 2 | 3.6 | |

System levels | Not all levels are considered | Safety overestimation | 2.1 | 1.7 | 3.57 |

Excess levels are considered | Safety underestimation | 2.4 | 2.5 | 6 | |

Interaction between levels is not considered | Safety overestimation | 1.7 | 1.7 | 2.89 | |

Excess interaction between levels is considered | Safety underestimation | 2.3 | 2.5 | 5.75 | |

Types of faults | Not all software faults are considered | Safety overestimation | 1.7 | 1.9 | 3.23 |

More than required software faults are considered | Safety underestimation | 2.2 | 2.7 | 5.94 | |

Not all hardware faults (physical and project) are considered | Safety overestimation | 1.9 | 1.6 | 3.04 | |

More than required hardware faults (physical and project) are considered | Safety underestimation | 2.2 | 2.7 | 5.94 | |

Hardware and software faults are not considered in possible attacks | Safety overestimation | 2 | 1.9 | 3.8 |

Assumption | Mode | Effect |
---|---|---|

Absolute expert credibility | Incomplete analysis | Incorrect assessment |

Expert qualification | Incorrect generation of a set of failure modes | Excess failure modes are chosen |

Not all failure modes chosen | ||

Wrong failure modes chosen | ||

Incorrect generation of a set of failure effects | Overestimation of effect | |

Underestimation of effect | ||

Wrong effect |

Name | Type | Failure Mode | Failure Effect | Failure Probability | Failure Severity |
---|---|---|---|---|---|

D14 | DC-DC converter | No output | No 24 V voltage | 3.7 × 10^{−8} | High |

High output (up to 20%) | Voltage is higher than 24 V | 5.4 × 10^{−9} | High | ||

Low output (up to 20%) | Voltage is lower than 24 V | 5.4 × 10^{−9} | High | ||

Pull high input current | No 24 V voltage | 5.4 × 10^{−9} | High | ||

D17 | Opto-coupler | Open circuit of individual connection | Stuck Off | 6.8 × 10^{−9} | Medium |

Short circuit between any two input connections | Stuck Off | 6.2 × 10^{−9} | Medium | ||

Short circuit between any two output connections | Stuck On | 6.2 × 10^{−9} | High | ||

Short circuit between any two connections of input and output | Isolation Fault | 1.9 × 10^{−10} | High | ||

VD19 | Diode | Short circuit | No effect | 8.4 × 10^{−10} | Medium |

Open circuit | Open input path | 3.6 × 10^{−10} | High | ||

R21 | Resistor | Short circuit | Voltage is lower than 24 V | 9.0 × 10^{−11} | High |

Name | Type | Failure Mode | Failure Effect | Failure Probability | Failure Severity |
---|---|---|---|---|---|

C18 | Capacitor | Short circuit | No 5V voltage | 3.0 × 10^{−10} | High |

Open circuit | No effect | 1.8 × 10^{−10} | Medium | ||

Reduced value up to 0.5× | No effect | 6.0 × 10^{−11} | Low | ||

R21 | Resistor | Short circuit | Voltage is lower than 24 V | 9.0 × 10^{11} | High |

Open circuit | Open input path | 5.4 × 10^{−10} | Medium | ||

Reduced value up to 0.5× | No effect | 1.4 × 10^{−10} | Low | ||

Increased value up to 0.5× | No effect | 1.4 × 10^{−10} | Low | ||

D14 | DC-DC converter | No output | No 24 V voltage | 3.7 × 10^{−08} | High |

Name | Type | Failure Mode | Failure Effect | Failure Probability | Failure Severity |
---|---|---|---|---|---|

FU07 | Fuse | Fail to open | No effect | 5.0 × 10^{−9} | Medium |

Slow to open | No effect | 4.0 × 10^{−9} | Low | ||

Premature open | No 24 V voltage | 1.0 × 10^{−9} | High | ||

C18 | Capacitor | Short circuit | No 5V voltage | 3.0 × 10^{−10} | High |

Open circuit | No effect | 1.8 × 10^{−10} | Medium | ||

Reduced value up to 0.5× | No effect | 6.0 × 10^{−11} | Low | ||

Increased value up to 2× | No effect | 6.0 × 10^{−11} | Low | ||

D14 | DC-DC converter | No output | No 24 V voltage | 3.7 × 10^{−8} | High |

High output (up to 20%) | Voltage is higher than 24 V | 5.4 × 10^{−9} | High | ||

Low output (up to 20%) | Voltage is lower than 24 V | 5.4 × 10^{−9} | High | ||

Pull high input current | No 24 V voltage | 5.4 × 10^{−9} | High |

Name | Type | Failure Mode | Failure Effect | Failure Probability | Failure Severity |
---|---|---|---|---|---|

D14 | DC-DC converter | No output | No 24 V voltage | 3.7 × 10^{−8} | High |

High output (up to 20%) | Voltage is higher than 24 V | 5.4 × 10^{−9} | High | ||

Low output (up to 20%) | Voltage is lower than 24 V | 5.4 × 10^{−9} | High | ||

Pull high input current | No 24 V voltage | 5.4 × 10^{−9} | High | ||

D17 | Opto-coupler | Open circuit of individual connection | Stuck Off | 6.8 × 10^{−9} | Medium |

Short circuit between any two input connections | Stuck Off | 6.2 × 10^{−9} | Medium | ||

Short circuit between any two output connections | Stuck On | 6.2 × 10^{−9} | High | ||

Short circuit between any two connections of input and output | Isolation Fault | 1.9 × 10^{−10} | High | ||

VD19 | Diode | Short circuit | No effect | 8.4 × 10^{−10} | Medium |

Open circuit | Open input path | 3.6 × 10^{−10} | High | ||

C18 | Capacitor | Short circuit | No 5 V voltage | 3.0 × 10^{−10} | High |

Open circuit | No effect | 1.8 × 10^{−10} | Medium | ||

Reduced value up to 0.5× | No effect | 6.0 × 10^{−11} | Low | ||

Increased value up to 2× | No effect | 6.0 × 10^{−11} | Low | ||

R21 | Resistor | Short circuit | Voltage is lower than 24 V | 9.0 × 10^{11} | High |

Open circuit | Open input path | 5.4 × 10^{−10} | Medium | ||

Reduced value up to 0.5× | No effect | 1.4 × 10^{−10} | Low | ||

Increased value up to 0.5× | No effect | 1.4 × 10^{−10} | Low | ||

FU07 | Fuse | Fail to open | No effect | 5.0 × 10^{−9} | Medium |

Slow to open | No effect | 4.0 × 10^{−9} | Low | ||

Premature open | No 24 V voltage | 1.0 × 10^{−9} | High |

Name | Type | Failure Mode | Failure Effect | Failure Probability | Failure Severity |
---|---|---|---|---|---|

R21 | Resistor | Short circuit | Voltage is lower than 24 V | 9.0 × 10^{11} | High |

D14 | DC-DC converter | No output | No 24 V voltage | 3.7 × 10^{−8} | High |

Name | Type | Failure Mode | Failure Effect | Failure Probability | Failure Severity |
---|---|---|---|---|---|

D14 | DC-DC converter | No output | No 24 V voltage | 3.7 × 10^{−8} | High |

High output (up to 20%) | Voltage is higher than 24 V | 5.4 × 10^{−9} | High | ||

Low output (up to 20%) | Voltage is lower than 24 V | 5.4 × 10^{−9} | High | ||

Pull high input current | No 24 V voltage | 5.4 × 10^{−9} | High | ||

C18 | Capacitor | Short circuit | No 5 V voltage | 3.0 × 10^{−10} | High |

Open circuit | No effect | 1.8 × 10^{−10} | Medium | ||

Reduced value up to 0.5× | No effect | 6.0 × 10^{−11} | Low | ||

R21 | Resistor | Short circuit | Voltage is lower than 24 V | 9.0 × 10^{−11} | High |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Babeshko, I.; Illiashenko, O.; Kharchenko, V.; Leontiev, K.
Towards Trustworthy Safety Assessment by Providing Expert and Tool-Based XMECA Techniques. *Mathematics* **2022**, *10*, 2297.
https://doi.org/10.3390/math10132297

**AMA Style**

Babeshko I, Illiashenko O, Kharchenko V, Leontiev K.
Towards Trustworthy Safety Assessment by Providing Expert and Tool-Based XMECA Techniques. *Mathematics*. 2022; 10(13):2297.
https://doi.org/10.3390/math10132297

**Chicago/Turabian Style**

Babeshko, Ievgen, Oleg Illiashenko, Vyacheslav Kharchenko, and Kostiantyn Leontiev.
2022. "Towards Trustworthy Safety Assessment by Providing Expert and Tool-Based XMECA Techniques" *Mathematics* 10, no. 13: 2297.
https://doi.org/10.3390/math10132297