The Challenge of Deploying Failure Modes and Effects Analysis in Complex System Applications—Quantification and Analysis

Abstract

Failure Modes and Effects Analysis (FMEA) is a systematic approach for evaluating failure modes in a system. However, its current implementation in complex systems is marred by high resource requirements, a lack of available data and difficulty of deployment. Consequently, attempts to integrate FMEA with other systematic methodologies have yielded unclear outcomes. Therefore, this paper used a score-based metric and applied the ordered probit model to empirically identify challenges associated with deploying FMEA and these attempts’ impact on FMEA applications as well as the influence of other organisational parameters. Our findings reveal that Fault Tree Analysis and Axiomatic Design methodologies reduced the perceived level of challenge significantly in the investigated sample. Our research outcome is of value to the practitioner community by showing that the level of challenge associated with FMEA deployment appears independent of organisational parameters, and that such a co-adoption of complementary methodologies in complex systems can reduce this challenge.

1. Introduction

Manufacturing has grown increasingly sophisticated in the era of industry 4.0 as representing a vision for a more sustainable future. In the realm of industry 4.0, sensors, machines and production systems are interconnected to utilise the available resources and add value to customers. As the complexity of these systems grows, so does the potential for new risks to emerge. As a result, identifying, measuring and assessing new and emerging risks are critical to avoid their emergence or mitigate their consequences on the human, organizational and technical scales. As a core methodology in reliability, safety, design and quality engineering, Failure Modes and Effects Analysis (FMEA) was developed to evaluate the potential failure modes together with their causes and effects for any given product [1,2]. It is characterised by the Risk Priority Number (RPN) metric to analyse and prioritize failures modes. Three measures are involved in estimating the RPN score: (1) severity—how severe (S) the effects of a failure raised in a product will be, (2) occurrence—the probability of occurrence (O) of the failure causes and (3) detection—the probability of controlling (D) the failure identified. On a scale of 1 to 10, these measures are estimated to determine the value of RPN. However, it has been argued that the optimal value for implementing FMEA is during the product development phases [1,2,3,4]. In practice, FMEA is usually applied in three situations [1,2,5]:

• Functional Failure Modes and Effects Analysis (Functional FMEA) to evaluate those failures associated with functional requirements of products and systems;
• Design Failure Modes and Effects Analysis (Design FMEA) to analyse those failures associated with design elements;
• Process Failure Modes and Effects Analysis (Process FMEA) to assess the potential failures encountered in manufacturing and assembly processes.

Despite FMEA constituting a standard element in the reliability, safety, design and quality engineering toolkits, it still presents a range of issues in its application in complex systems, such as an aeroengine, which is made from different engineering disciplines combined with advanced technologies, and undoubtedly increases the challenge for implementing FMEA [6,7,8,9,10,11,12]. Three practical issues have been highlighted in particular: First, FMEA deployment can require an excessive use of resources, requiring significant monetary expenses, time commitment and personnel involvement [6,7,10]. Second, FMEA execution can lead to practical problems in terms of unambiguously identifying primary functional requirements and failure modes and, building on this, impede the systematic capture of these requirements in a detailed bottom-up approach [6,7,10,13]. Third, it may be difficult to meet the extensive information requirements to bridge the knowledge gap between FMEA procedures and their objects, potentially leading to an inability to trace failures [1,10,13,14,15,16]. The required data can take the form of design characteristics relating to failures (e.g., secondary functional requirements of a specified system) and empirical data relating to actual failures (i.e., causes, effects, control actions and RPNs). These issues are exacerbated in situations of high product complexity that may be characterised by the presence of a multitude of integrated functionalities, intricate hierarchical system structures and a large volume of design details [6,7,10,17]. An additional challenge in such situations might be posed by a lack of experience on the part of the FMEA practitioners [1,18,19].

In support of FMEA execution, Carlson [1] identifies two main tasks required for the effective management of FMEA deployment: decomposing system functions and identifying failure modes and synchronising FMEA procedures. However, a multiplicity of methodologies has been proposed in the literature to address each of these tasks with the aim of tackling the above-mentioned practical challenges of FMEA deployment observed in complex applications and developing sustainable FMEA practices. In the following, we present a brief overview of the current literature contributing to each of the FMEA management tasks identified by Carlson [1].

The first theme is the decomposition of system functions and the identification of failure modes concerning FMEA combined with other structured approaches. Recently, Peeters, Basten and Tinga [6] and Yu et al. [17] proposed a framework combining the Fault Tree Analysis (FTA) method with FMEA to guide the overall process and its influence on operational performance. Unlike FMEA, which follows inductive logic, FTA is a deductive failure analysis method that models system failures using Boolean logic (e.g., ‘and/or’ logical operators) in a top-down approach. Prior to this, Arcidiacono and Campatelli [20] studied the combination of the FTA method with FMEA in the light of using an Axiomatic Design (AD) methodology to improve the system decomposition and the synchronicity processes within Functional and Design FMEAs deployment. As a systematic design methodology, the AD method utilises a design matrix approach to logically transform customer needs into functional requirements, design and process domains [21]. AD involves four design domains operating as a logical succession: (1) the customer domain, (2) the functional domain, (3) the physical domain and (4) the process domain [21].

A range of further extensions has been proposed: Korayem and Iravani [22], White et al. [23] and Sharma, Kumar and Kumar [24] combined a Function Flow Boundary Diagram (FFBD) approach with FMEA to ensure that system functions are decomposed and contained within the deployment of FMEA. FFBDs consist of functional blocks used to display the functions in their logical and sequential relationships using gates (i.e., ‘and’/’or’, ‘go/no go’). Battirola Filho et al. [25] and Tang, Wang and Wang [26] combined the Analytical Hierarchy Process (AHP) method with FMEA to decompose the system functions and identify the associated failure modes. AHP method uses a hierarchical decomposition technique to accommodate complex information in multi-criterion decision making [27].

To synchronise FMEA processes, Gu, Cheng and Qiu [28] and Hassan et al. [29] merged the Quality Function Deployment (QFD) method with FMEA to form a hybrid decision tool. QFD is a quality management technique used to transform customer requirements to production processes by the means of a series of characteristic matrices called the House of Quality (HoQ). Furthermore, Augustine et al. [30] adopted a Knowledge-Based System (KBS) approach and proposed a cognitive-map system for simulating and identifying the links between different failure modes in the product development stages. Then, Renu et al. [31] developed a KBS to align FMEA procedures in the form of the Knowledge Discovery and Data mining (KDD) approach. However, Soufhwee et al. [13] proposed an approach to integrate Design with Process FMEAs as an element in the Lean Six Sigma toolkit following the “DAIREC” cycle (Define, Analyse, Improve, Recommend, Evaluate and Control). Moreover, Alruqi et al. [32] suggested that the application of the AD approach provides the basis for synchronising FMEA procedures; by integrating functional requirements (i.e., Functional FMEA) with design parameters and process variables (i.e., Design FMEA and Process FMEA) in a hierarchical way. Further to this, Arcidiacono and Campatelli (2004) suggested a similar idea for consolidating FTA, AD and FMEA methods into a working framework to decompose system functions and synchronise Functional with Design FMEAs.

The field of FMEA, however, has so far been exposed to limited empirical research on the effects of co-adopting FTA, FFBD, AHP, QFD, KBS, DAIREC and AD on the success of implementing FMEA. For example, the studies outlined above in this study have not demonstrated a clear quantification of the impacts of co-adopting FTA, FFBD, AHP, QFD, KBS, DAIREC and AD methodologies on overcoming the challenges encountered with deploying FMEA in complex applications. This provides the principal rationale for the orientation of this work, which aims to evaluate these approaches and determine the extent to which they could improve the practical execution of FMEA. To address this important gap in knowledge, this paper investigates the hypothesis that the co-adoption of methodologies, such as those outlined above, reduces the level of difficulty associated with the practical execution of FMEA. Other managerial factors including the experience level of FMEA practitioners, and the industry characteristics are also analysed in this study to investigate their influence on the performance of FMEA deployment.

Our contributions are twofold: first, we have advanced the literature on FMEA field by providing a deep methodological investigation into the associated impact of co-adopting FTA, FFBD, AHP, QFD, KBS, DAIREC and AD methodologies on FMEA implementation, as well as the influence of the managerial factors of the level of experience for the team members involved and level of details required to apply FMEA. On this basis, this paper presents an explanatory analysis of factors that could affect the challenge associated with implementing FMEA in complex systems. To the best of our knowledge, these aspects have not yet been subject to examination elsewhere. Therefore, this paper outcome can be regarded as significant since it helped to bring together and evaluate various experiences from different backgrounds and industries to validate the co-adoption of FTA, FFBD, AHP, QFD, KBS, DAIREC and AD methodologies and determine the extent to which these methodologies could improve the practice of FMEA in complex systems. In addition, this paper presents an analysis to improve the understanding of the association between the successful application of FMEA and the change in the managerial factors outlined above.

Second, we have employed the application of the statistical inference of the ordered probit model in the body of FMEA knowledge for the first time. The ordered probit model executed in this paper manifests a novel and practical model for measuring the performance of FMEA in any such development and any application. Thus, the designed model can be furthered and applied to evaluate the performance of toolkits used in the field of new product development, quality, reliability engineering and agile management.

The remainder of the paper is structured as follows. The next section presents the methodology employed for data collection, data preparation and statistical inference via an ordered probit model. The subsequent section presents the results obtained in terms of descriptive statistics together with the results of the ordered probit model estimation. The following section provides a critical discussion and evaluation of the results obtained, with reference to the relevant literature. The paper concludes with a summary and brief recommendations for future work.

2. Materials and Methods

To obtain a dataset on the difficulties of (co-)implementing FMEA in systems engineering environments, this research adopted a questionnaire-based data collection approach to gather information from a wide range of participants. The questionnaire was designed to capture the relationship between the co-adoption of approaches, such as those summarised above, and the level of challenge (LC) associated with deploying FMEA by prompting the participants from different aerospace and automotive organisations to identify methodologies supporting FMEA and their effectiveness. Additionally, the questionnaire surveyed the common challenges encountered in deploying FMEA within these organisations by identifying five challenge categories. This enabled the participants to identify particular challenges in the current practice of FMEA deployment.

Since the number of respondents was too small to perform a random sampling approach, the resulting dataset forms a convenience sample. Convenience sampling carries the benefit of a short lead time and reduces the associated costs incurred when forming a group of participants [33]. The targeted group for this study consists of engineers in design, manufacturing and quality roles in both the automotive and aerospace industries. It is acknowledged that convenience sampling carries the possibility of self-selection bias. To minimise this issue, one participant was then randomly selected from each location or organisation to represent the practice of FMEA deployment.

An invitation message containing a link to the online questionnaire was sent to 76 industry professionals. In conjunction with the online questionnaire and at the same time, the survey was completed in virtual and face-to-face meetings with 31 additional participants between 3 May 2018 and 24 August 2018, resulting in a sample size of n = 41. The observed response rate is in line with the researchers’ expectations—this is also due to the potentially sensitive nature of the information being sought.

2.1. Data Preparation

2.1.1. Construction of the Dependent Variable

The dependent variable for this study is the level of challenge (LC). This metric reflects the overall practical difficulties encountered in deploying FMEA in complex engineering systems as reported by the participants. Inspired by previous work analysing the challenges associated with manufacturing technology adoption [34], LC was constructed as the sum of the current perceived partial challenges in the deployment of FMEA. As far as this research is concerned, the literature has treated the challenges of implementing FMEA with the same level of importance. To design the dependent variable (LC), this study applied a scheme assuming each individual FMEA challenge is modular and has the same impact. The elements flowing into LC were obtained from binary ‘yes/no’ questions asking the participants to identify whether they had encountered the following five partial challenges in the deployment of FMEA: excessive use of resources, the applicability of results, capturing of interaction failures, the knowledge gap between design and manufacturing and inability to trace risks. The value of LC was then obtained as a score from the summation of these binary elements and can thus take six discrete values ranging from ‘0’ to ‘5’. This approach is summarised in

Table A1. Summary of the independent and dependent variables.

Variable Name	Symbol	Description	Information Type	Data Source and Remarks
Fault Tree Analysis	FTA_PERF	Performance value coded from Likert-scale	Ordered discrete	Questionnaire
Functional Flow Boundary Diagram	FFBD_PERF
Analytical Hierarchical Process	AHP_PERF
Quality Function Deployment	QFD_PERF
Knowledge-Based System	KBS_PERF
Define, Analysis, Improve, Recommend, Evaluate and Control	DAIREC_PERF
Axiomatic Design	AD_PERF
Experience Level	EXP	Coded as follows: EXP = 0, if the number of experiences = <9 EXP = 1, if the number of experiences > 9 and = <15 EXP = 2, if the number of experiences > 15 and = <23 EXP = 3, if the number of experiences > 23 and = <30 EXP = 4, if the number of experiences > 30	Continuous	LinkedIn profiles and face-to-face questions
Organisational size	SIZE	Coded as follows: SIZE = 0, if the number of employees <=20,000 SIZE = 1, if the number of employees > 20,000 and =<50,000 SIZE = 2, if the number of employees > 50,000 and =<100,000 SIZE = 3, if the number of employees > 100,000 and =<250,000 SIZE = 4, if the number of employees > 250,000		Annual reports of businesses
Industry membership	MEM	Industry membership values coded from participants fields of work	Binary	LinkedIn profiles and face-to-face questions
Level of challenge to deploy FMEA (dependent variable)	LC	Score variable consisting of the following challenges: Excessive use of resources Applicability Capture interaction failures between system, sub-system and components levels The knowledge gap between design and manufacturing phases Inability to trace risks	Ordered discrete	Questionnaire, composed from multiple questions

.

2.1.2. Construction of the Independent Variables

The independent variables capture the performance of FTA, FFBD, AHP, QFD, KBS, DAIREC and AD approaches that were suggested to support or complement FMEA in its deployment. To obtain the data on the performance of the implementation of these approaches, the survey contained Likert-scale questions prompting the participants to identify the level of effectiveness to which the above-mentioned approaches were being adopted within the participants’ organisation. On this basis, the following seven multinomial discrete variables reflecting the approach performance results were specified, each taking six possible values ranging from ’0’ to ‘5’: FTA_PERF, FFBD_PERF, AHP_PERF, QFD_PERF, KBS_PERF, DAIREC_PERF and AD_PERF. The coding method used is presented in Table A1.

Beyond the surveyed types of complementary approaches, the model included three additional variables to assess the impact of such organisational factors on LC. Industry membership (MEM) was included to represent the effects of industry characteristics, entered as a binary variable, with ‘0’ denoting aerospace membership and ‘1’ denoting automotive membership. Organisational size (SIZE), in terms of employee numbers in the respondent’s organisation, and experience level (EXP), measuring the respondents’ work experience in their role in years, were included to represent resources required for deploying FMEA. Both variables were entered in the investigation process in coded discrete form by converting continuous numerical information in a logarithmic fashion. The coding scheme is also summarised in Table A1.

2.2. Model Specification for Analysis

An ordered probit model was implemented to evaluate the impact of the dynamic change in the independent variables on the perceived level of challenge (LC). The ordered probit model is a statistical inference test that is considered useful in situations in which outcomes are represented in the form of discrete ordered numerical information [35]. In its implementation, this study follows the ordered probit model applications by Baumers, Tuck and Hague [34] and Chan, Miller and Tcha [36].

As the basis for the ordered probit model, Equation (1) shows a linear relationship between the unobservable variable $L{C}^{*}{}_i$ and the independent variables, with $L{C}^{*}{}_i$ being a single latent and continuous variable:

$$L{C}^{*}{}_i={x^{\prime }}_i {\beta }_i+{\epsilon }_i$$

(1)

where ${x^{\prime }}_i$ is the transpose of a vector of the independent variables for the $i^{th}$ observation; ${\beta }_i$ is a vector of parameters to be estimated (i.e., regression coefficients) measuring the effects of a change in ${x^{\prime }}_i$ on $L{C}^{*}{}_i$, and ${\epsilon }_i$ is the random error that is assumed to be normally distributed with 0 mean and unit variance ${\epsilon }_i$~N (0,1).

It should be noted that interpreting the relationship between the observable discrete variable $L{C}_i$, as a counterpart to the unobservable continuous variable $L{C}^{*}{}_i$, is not straightforward. However, the correspondence between $L{C}_i$ and $L{C}^{*}{}_i$ is determined by the following equation, where a set of threshold values ${\alpha }_k$ serves as estimable unknown parameters defining the limits of the observed categories ($L{C}_i=k)$ such that

$$L{C}_i=k if {\alpha }_{k-1}<L{C}^{*}{}_i<{\mathit{\alpha }}_k$$

(2)

Thus, for the given $k^{th}$ category of $L{C}_i$ and in a situation where $k\in \left\{0,1,2,3,4,5\right\}$, Equation (2) can be written as follows:

$$L{C}_i=\{\begin{array}{l}k=0, if-\infty <L{C}^{*}{}_i<{\alpha }_1\\ k=1, if {\alpha }_1<L{C}^{*}{}_i<{\alpha }_2\\ k=2, if {\alpha }_2<L{C}^{*}{}_i<{\alpha }_3\\ k=3, if {\alpha }_3<L{C}^{*}{}_i<{\alpha }_4\\ k=4, if {\alpha }_4<L{C}^{*}{}_i<{\alpha }_5\\ k=5, if {\alpha }_5<L{C}^{*}{}_i<\infty \end{array}$$

(3)

Therefore, by substituting (1) in (2),

$$L{C}_i=k, if {\alpha }_{k-1}<{x^{\prime }}_i {\beta }_i+{\epsilon }_i<{\alpha }_k$$

(4)

And by substituting (1) into (3),

$$L{C}_i=\{\begin{array}{l}k=0, if-\infty <{x^{\prime }}_i {\beta }_i+{\epsilon }_i<{\alpha }_1\\ k=1, if {\alpha }_1<{x^{\prime }}_i {\beta }_i+{\epsilon }_i<{\alpha }_2\\ k=2, if {\alpha }_2<{x^{\prime }}_i {\beta }_i+{\epsilon }_i<{\alpha }_3\\ k=3, if {\alpha }_3<{x^{\prime }}_i {\beta }_i+{\epsilon }_i<{\alpha }_4\\ k=4, if {\alpha }_4<{x^{\prime }}_i {\beta }_i+{\epsilon }_i<{\alpha }_5\\ k=5, if {\alpha }_5<{x^{\prime }}_i {\beta }_i+{\epsilon }_i<\infty \end{array}$$

(5)

Thus, the probability of observing $L{C}_i$ for the $k^{th}$ category can be determined as follows:

$$\mathrm{Pr}\left(L{C}_i=k\right)=\mathrm{Pr}({\alpha }_{k-1}<{x^{\prime }}_i {\beta }_i+{\epsilon }_i<{\alpha }_k)$$

(6)

This can be re-arranged to give

$$\mathrm{Pr}\left(L{C}_i=k\right)=\mathrm{Pr}\left({\alpha }_{k-1}-{x^{\prime }}_i {\beta }_i<{\epsilon }_i<{\alpha }_k-{x^{\prime }}_i {\beta }_i\right)$$

(7)

which, in turn, can be also re-arranged as follows:

$$\mathrm{Pr}\left(L{C}_i=k\right)=\mathrm{Pr}\left({\epsilon }_i<{\alpha }_k-{x^{\prime }}_i {\beta }_i\right)-\mathrm{Pr}({\epsilon }_i<{\alpha }_{k-1}-{x^{\prime }}_i {\beta }_i)$$

(8)

The normal distribution ${\epsilon }_i$ can be estimated through its cumulative distribution function $\mathsf{\Phi }\left(z\right)$, which determines the cumulative probability of observing a given z-value [37] such that

$$\mathsf{\Phi }\left(z\right)={{\displaystyle \int }}_{-\infty }^z\mathsf{\Phi }\left(t\right)dt$$

(9)

where

$$\mathsf{\Phi }\left(t\right)=\frac{e^{\frac{-t^2}{2}}}{\sqrt{2\pi }}$$

(10)

Therefore, Equation (8) can be re-written as follows:

$$\mathrm{Pr}\left(L{C}_i=k\right)=\mathsf{\Phi }\left({\alpha }_k-{x^{\prime }}_i {\beta }_i\right)-\mathsf{\Phi }\left({\alpha }_{k-1}-{x^{\prime }}_i {\beta }_i\right)$$

(11)

The full set of probabilities for all $k^{th}$ categories can be re-arranged and re-written as follows:

$$\mathrm{Pr}\left(L{C}_i=k\right)=\{\begin{array}{l}\mathrm{Pr}\left(k=0\right), if \mathsf{\Phi }({\alpha }_1-{x^{\prime }}_i {\beta }_i)-\mathsf{\Phi }\left(-\infty -{x^{\prime }}_i {\beta }_i\right)\\ \mathrm{Pr}\left(k=1\right), if \mathsf{\Phi }({\alpha }_2-{x^{\prime }}_i {\beta }_i)-\mathsf{\Phi }\left({\alpha }_1-{x^{\prime }}_i {\beta }_i\right)\\ \mathrm{Pr}\left(k=2\right), if \mathsf{\Phi }({\alpha }_3-{x^{\prime }}_i {\beta }_i)-\mathsf{\Phi }\left({\alpha }_2-{x^{\prime }}_i {\beta }_i\right)\\ \mathrm{Pr}\left(k=3\right), if \mathsf{\Phi }({\alpha }_4-{x^{\prime }}_i {\beta }_i)-\mathsf{\Phi }\left({\alpha }_3-{x^{\prime }}_i {\beta }_i\right)\\ \mathrm{Pr}\left(k=4\right), if \mathsf{\Phi }({\alpha }_5-{x^{\prime }}_i {\beta }_i)-\mathsf{\Phi }\left({\alpha }_4-{x^{\prime }}_i {\beta }_i\right)\\ \mathrm{Pr}\left(k=5\right), if \mathsf{\Phi }(\infty -{x^{\prime }}_i {\beta }_i)-\mathsf{\Phi }\left({\alpha }_5-{x^{\prime }}_i {\beta }_i\right)\end{array}$$

(12)

The estimation of ${\alpha }_k$ and ${\beta }_i$ values can be achieved by the Maximum Likelihood Estimation ($l$) [37,38]. Following Jackman [39], the likelihood contribution $l$ for each $i^{th}$ observation can be derived as follows:

$$l_i={{\displaystyle \prod }}_{k=1}^K\mathrm{Pr}{(L{C}_i=k|{x^{\prime }}_i)}^{Z_{ik}}$$

(13)

where $Z_{ik}$ is used for the purpose of computing the indicator function $Z_{ik}=1\left(L{C}_i=k\right)$, for $k=1$,…,S. This allows the statement of the likelihood contribution as follows:

$$l_i={{\displaystyle \prod }}_{k=1}^s{\left[\mathsf{\Phi }\left({\alpha }_k-{x^{\prime }}_i {\beta }_i\right)-\mathsf{\Phi }\left({\alpha }_{k-1}-{x^{\prime }}_i {\beta }_i\right)\right]}^{Z_{ik}}$$

(14)

Aggregating the likelihood contributions over the sample, the likelihood function (L) of parameters α and $\beta$ can thus be written as follows:

$$L\left(\alpha ,\beta \right)={{\displaystyle \prod }}_{i=1}^n{{\displaystyle \prod }}_{k=1}^K{\left[\mathsf{\Phi }\left({\alpha }_k-{x^{\prime }}_i {\beta }_i\right)-\mathsf{\Phi }\left({\alpha }_{k-1}-{x^{\prime }}_i {\beta }_i\right)\right]}^{Z_{ik}}$$

(15)

This can be simplified by taking logarithms:

$$\ln L\left(\alpha ,\beta \right)={\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{k=1}^K{z}_{ik}\left[\mathsf{\Phi }\left({\alpha }_k-{x^{\prime }}_i {\beta }_i\right)-\mathsf{\Phi }\left({\alpha }_{k-1}-{x^{\prime }}_i {\beta }_i\right)\right]$$

(16)

Equation (16) is maximised by the means of the Maximum Likelihood Estimation, which is reported to produce appropriate convergence results [35].

3. Results

3.1. Descriptive Statistics

Reflecting a discrete measure on the scale, a value of LC = ‘0’ indicates that there is no challenge in deploying FMEA, and a value of LC = ‘5’ indicates that deploying FMEA is highly challenging. As shown in the histogram presented in Figure 1, the LC values observed in the investigated sample appear to be approximately normally distributed.

Figure 2 displays the performance responses for the seven approaches (FTA_PERF, FFBED_PERF, AHP_PERF, QFD_PERF, KBS_PERF, DAIREC_PERF and AD_PERF). As can be seen, a zero-performance result was reported for all seven approaches, ranging from approximately 15% of the sample for the QFD method to over 80% of the sample for the AD approach. High levels of performance were also reported for all seven approaches, ranging from approximately 17% of the sample, for the AD method, to approximately 37% of the sample for the FFBD approach.

Table 1. Descriptive statistics.

	Mean	Standard Deviation	Minimum	Maximum	FTA_PERF	FFBD_PERF	AHP_PERF	QFD_PERF	KBS_PERF	DAIREC_PERF	AD_PERF	EXP	SIZE	MEM	LC
FTA_PERF	2.683	2.184	0	5	1.000
FFBD_PERF	3.024	2.091	0	5	0.072	1.000
AHP_PERF	2.439	2.086	0	5	0.245	0.141	1.000
QFD_PERF	3.439	1.732	0	5	−0.524	−0.065	−0.276	1.000
KBS_PERF	2.853	2.007	0	5	0.035	0.209	0.225	−0.154	1.000
DAIREC_PERF	2.268	2.292	0	5	0.197	0.395	0.236	−0.213	0.128	1.000
AD_PERF	0.854	1.905	0	5	0.337	0.089	0.124	−0.685	0.132	0.032	1.000
EXP	1.561	1.163	0	4	−0.213	−0.058	−0.073	0.197	−0.167	−0.114	0.004	1.000
SIZE	1.780	1.275	0	4	−0.044	0.078	0.124	−0.276	−0.112	0.056	0.237	0.018	1.000
MEM	0.536	0.505	0	1	−0.046	−0.178	−0.206	0.067	−0.069	−0.192	0.162	0.283	0.269	1.000
LC	2.780	1.255	0	5	−0.600	−0.303	−0.245	0.689	−0.162	−0.466	−0.599	0.172	−0.174	−0.046	1.000

displays the full correlation matrix for both the dependent and independent variables. As expected, most of the independent variables show overall negative correlations with LC. In particular, it was found that the FTA_PERF, DAIREC_PERF and AD_PERF variables are relatively strongly negatively correlated with LC (p = −0.600, p = −0.466 and p = −0.599, respectively). It is noteworthy that the EXP variable shows a weak positive correlation with LC, while the QFD_PERF variable surprisingly indicates a strong positive association with LC. The means and standard deviations for the dependent and independent variables are included in Table 1.

3.2. Model Results

The whole procedure for deploying the ordered probit model is shown in Figure 3. Obtaining the relative values of all variables indicated in this study is the first stage in this procedure. These numbers came from a variety of sources, including Likert scale measurements, binary data, continuous data and categorical data. The data on the level of challenge (LC) are then connected to a wide variety of engineering methods and other managerial factors in the next phase to highlight membership between these variables and LC. In the following step, the marginal effects associated with each change in all variables on LC are determined. Marginal effects analysis provides further insights and a useful measure for the degree of influence of the independent variables on LC_m improvement [36]. Finally, the estimated probabilities of membership in the various LC categories and a range of values that such selected variables can take are computed to get further insight into the relationships that these variables have with the level of challenge (LC). According to Chan, Miller and Tcha [36], the predicted probability analysis provides a good indicator with the objective to shed light on the actual levels with which the different levels of the FTA_m and AD_m variables are represented in LC_m categories.

Table 2. The ordered probit model estimation results.

Sample size: 41 Degree of freedom: 9 LR ${\mathrm{Chi}}^2$: 57.03 p-value > ${\mathrm{Chi}}^2$: 0.0000 Pseudo ${\mathrm{R}}^2$: 0.4335 Log-likelihood: −37.26
Variables	Coefficients	Standard Deviation	Significance	[95% Conf. Interval]
FTA_PERF	−0.249	0.109	**	-
FFBD_PERF	−0.158	0.103	-	-
AHP_PERF	−0.043	0.105	-	-
QFD_PERF	0.466	0.204	**	-
KBS_PERF	0.007	0.101	-	-
DAIREC_PERF	−0.293	0.099	***	-
AD_PERF	−0.477	0.182	***	-
EXP	0.186	0.178	-	-
SIZE	0.123	0.185	-	-
MEM	−0.798	0.467	*	-
${\alpha }_1$	−6.32	1.572	-	−9.425 → −3.226
${\alpha }_2$	−2.902	1.191	-	−5.246 → −0.558
${\alpha }_3$	−0.604	1.169	-	−2.895 → 1.686
${\alpha }_4$	0.704	1.161	-	−1.574 → 2.977
${\alpha }_5$	2.355	1.157	-	0.087 → 4.623

(*) indicates the significance of variable is at the 10% level; (**) indicates the significance of variable is at the 5% level; (***) indicates the significance of variable is at the 1% level; ${\alpha }_1$, ${\alpha }_2$, ${\alpha }_3$, ${\alpha }_4$ and ${\alpha }_5$ are threshold parameters.

represents the ordered probit model results, obtained by using the statistical package STATA (v.16). A Chi-square test produced a value of 57.03 with a p-value > ${\mathrm{Chi}}^2$ = 0.0000, indicating that this specified ordered probit model is statistically significant. A likelihood ratio test (Pseudo-${\mathrm{R}}^2$) was used to measure the goodness of fit of the observed variables; that is, quantifying the extent to which all independent variables included in the ordered probit model can describe LC. The obtained Pseudo-${\mathrm{R}}^2$ value for this model is 0.4335, indicating that the implementation of the specified ordered probit model carries an acceptable level of reliability. This is commensurate with the explanatory character of the study and the relatively small sample size.

To determine whether the differences between threshold mean values (${\alpha }_k$) are statistically significant and distinct, a confidence interval overlapping analysis was used to compare the boundary values of the confidence intervals coupled with threshold mean values (${\alpha }_k$). Confidence interval overlapping is an approach for determining the significant differences between groups [40,41]. At the 95% significance level, Table 2 shows that this model produced overlapping between confidence intervals associated with all threshold parameters. This may indicate that these parameters are not statistically significant and, therefore, are not distinct. However, it is imperative to clarify that the differences between the threshold parameters might still be significant.

In an attempt to satisfy the condition of the confidence interval overlapping between all threshold parameters, the original model was redesigned by minimising the scale of both the dependent and independent variables. Among various adjusted scenarios, the model was formed to a scale of four measures for both the dependent and most of the independent variables (FTA_PERF, FFBD_PERF, AHP_PERF, QFD_PERF, KBS_PERF, DAIREC_PERF, AD_PERF, EXP and SIZE). The applied coding approach for the optimised model, called the reduced model, is presented in

Table A2. Summary of the reduced independent and dependent variables.

Variable Name	Symbol	Description	Information Type	Data Source and Remarks
Fault Tree Analysis	FTA_m	Taking FTA_m as illustrative to represent all listed variables, the coding as follows: FTA_m = 0, if FTA_PERF = 0 FTA_m = 1, if FTA_PERF = 1 & 2 FTA_m = 2, if FTA_PERF = 3 & 4 FTA_m = 3, if FTA_PERF = 5	Ordered discrete	Original formed Likert scales
Functional Flow Boundary Diagram	FFBD_m
Analytical Hierarchical Process	AHP_m
Quality Function Deployment	QFD_m
Knowledge-Based System	KBS_m
Define, Analysis, Improve, Recommend, Evaluate and Control	DAIREC_m
Axiomatic Design	AD_m
Experience Level	EXP_m	Recoded as follows: EXP_m = 0, if EXP = 0 & 1 EXP_m = 1, if EXP = 2 EXP_m = 2, if EXP = 3 EXP_m = 3, if EXP = 4	Continuous
Organisational size	SIZE_m	Recoded as follows: SIZE_m = 0, if SIZE = 0 &1 SIZE_m = 1, if SIZE = 2 SIZE_m = 2, if SIZE = 3 SIZE_m = 3, if SIZE = 4
Industry membership (Kept same)	MEM	Industry membership values coded from participants fields of work	Binary	LinkedIn profiles and face-to-face questions
Level of challenge to deploy FMEA (dependent variable)	LC_m	Recoded as follows: LC_m = 0, if LC = 0 & 1 LC_m = 1, if LC = 2 LC_m = 2, if LC = 3 LC_m = 3, if LC = 4 & 5	Ordered discrete	Questionnaire, composed from multiple questions

.

At the 90% significance level,

Table 3. The reduced ordered probit model estimation results.

Sample size: 41 Degree of freedom: 9 LR ${\mathrm{Chi}}^2$: 51.41 p-value > ${\mathrm{Chi}}^2$: 0.0000 Pseudo ${\mathrm{R}}^2$: 0.4599 Log-likelihood: −30.18
Variables	Coefficients	Standard Deviation	Significance	[90% Conf. Interval]
FTA_m	−0.41	0.208	**	-
FFBD_m	−0.284	0.204	-	-
AHP_m	−0.017	0.196	-	-
QFD_m	0.927	0.36	**	-
KBS_m	−0.062	0.203	-	-
DAIREC_m	−0.426	0.182	**	-
AD_m	−0.71	0.288	**	-
EXP_m	0.007	0.237	-	-
SIZE_m	0.136	0.271	-	-
MEM	−0.471	0.49	-	-
${\alpha }_1$	−2.761	1.214	-	−4.759 → −0.763
${\alpha }_2$	−0.603	1.154	-	−2.502 → 1.295
${\alpha }_3$	0.725	1.139	-	−1.148 → 2.599

(**) indicates the significance of variable is at the 5% level; ${\alpha }_1$, ${\alpha }_2$ and ${\alpha }_3$ are threshold parameters.

presents the results of the model parameters obtained from the execution of the reduced ordered probit model. This model still addresses overlapping between confidence intervals coupled with all threshold parameters; however, the decision was made to retain this model since it produced narrower overlapping intervals and a better Pseudo-${\mathrm{R}}^2$ value. In addition, this model contains fewer regressand and regressor categories, which could carry meaningful effects with reduced margin error levels.

Using the ordered probit model, a positive coefficient denotes a higher probability of the membership to the maximum level of LC_m (i.e., LC_m = 3), whereas vice versa a negative coefficient indicates the higher probability of the membership to the minimum level of LC_m (i.e., LC_m = 0). This being indicated, the interpretation of the coefficients yielded by an ordered probit model requires further analysis to investigate the marginal effects and the predicted probabilities of the intermediate categories of LC_m [34,36].

As shown, the FTA_m, QFD_m, DAIREC_m and AD_m variables are statistically significant. The FTA_m variable (−0.41) suggests a higher probability of the membership to the lowest category of LC_m (i.e., LC_m = 0). This means that the co-adoption of the FTA method has the greatest positive influence on LC_m reduction by offering an effective method to decompose system functions and identify the coupled failure modes. Likewise, both the DAIREC_m and AD_m variables have nearly an equal relationship to the lowest category of LC_m. The similarity between the coefficients estimated for the DAIREC_m variable (−0.426) and AD_m variable (−0.710) is noteworthy since both approaches have the same positive impact and role in improving FMEA deployment (i.e., synchronising FMEA procedures). Compared to the DAIREC_m variable, the AD_m variable has a relatively higher positive impact on LC_m reduction. However, the QFD_m variable (0.927) indicates a positive relationship to the highest category of LC_m (i.e., LC_m = 3); this suggests an adverse impact of co-adopting QFD method on LC_m associated with FMEA deployment.

The next step in the analysis of the ordered probit model is the investigation of the marginal effects for each independent variable on its membership to all LC_m categories. The marginal effects, presented in

Table 4. The summary of the marginal effects.

Variables	Categories of LC_m for Deploying FMEA in Complex Systems
	3	2	1	0
FTA_m **	−7.148	−9.199	14.89	1.455
FFBD_m	−4.495	−6.365	10.30	1.007
AHP_m	−0.294	−0.379	0.61	0.06
QFD_m **	16.15	20.79	−33.65	−3.29
KBS_m	−1.084	−1.395	−2.226	−0.221
DAIREC_m **	−7.423	−9.554	15.46	1.511
AD_m **	−12.38	−15.93	25.79	2.521
EXP_m	0.121	0.155	−0.25	−0.024
SIZE_m	2.375	3.056	−4.947	−0.483
MEM	−8.443	−10.15	16.92	1.669

For expositional purposes, all coefficients were multiplied by 100, therefore values presented above is in the form of percentage point effects. Values across rows may not equal to 0 due to rounding errors. (**) indicates the significance of variable is at the 5% level.

, were calculated at the averages of the independent variables. As a result of a one-unit increase in the independent variables, this analysis allows evaluating the overall changes in the probability of observing LC_m categories at a particular level.

Thus, the results shown in Table 4 indicate that if the FTA_m variable increases by one unit, the probability of observing LC_m = 0 increases by more than 1%. With a one-unit increase in the FTA_m variable, the probabilities of being classified as ‘LC_m = 3’ and ‘LC_m = 2’ decrease by 7.148% and 9.199%, respectively. The above observations clearly show that the co-adoption of the FTA method is likely to reduce the level of challenge associated with FMEA deployment.

Likewise, the impacts of the DAIREC_m and AD_m variables on LC_m indicate that the co-adoption of DAIREC and AD approaches in the practice of deploying FMEA also contribute to the reduction of LC_m. A one-unit increase in the DAIREC_m and AD_m variables rise the probabilities of being classified as ‘LC_m = 1’ by 15.46% and 25.79%, respectively, while the probabilities of observing LC_m = 3 for both variables decrease by 7.423% and 12.38% correspondingly.

However, the effect of an increase in the QFD_m variable by a one-unit increases the chance of observing the highest level of challenge (LC_m = 3) by 16.15%. Running counter to this, the probability of observing LC_m = 1 resulting from a change in the QFD_m variable decreases by 33.65%. Overall, it is noteworthy that the change in all independent variables appears to increase or decrease the probability of observing the high level of challenge (LC_m = 3), whereas it appears to have a far lower effect when the level of the perceived challenge is low (LC_m = 0).

Finally, the FTA_m and AD_m variables were chosen for further examination due to their large influences on the reduction of LC_m. This was performed by calculating the predicted probabilities of the associations between all categories of LC_m and the FTA_m/AD_m variables.

Table 5. Summary of the predicted probabilities of all LC_m categories.

Variables	Value	The Predicted Probabilities for LC_m Categories
		3	2	1	0
FTA_m	0	65.36	26.82	7.729	0.074
	1	7.575	37.59	52.63	2.196
	2	22.57	26.78	39.71	10.92
	3	10.17	18.05	52.24	46.52
AD_m	0	38.54	29.49	28.14	3.818
	1	-	-	-	-
	2	-	-	-	-
	3	0.787	4.893	14.52	79.79

For exposition, all coefficients were multiplied by 100; therefore, values presented above are in the form of percentage points. Values across rows may not equal to 0 due to rounding errors.

presents the predicted probabilities for the memberships of all LC_m categories across the different discrete values of the FTA_m and AD_m variables, held at their mean values.

Table 5 indicates that the FTA_m and AD_m variables appear to have an influence on how the level of challenge (LC_m) associated with FMEA deployment is distributed. For example, the predicted probability for being classified as ‘LC_m = 3’ at FTA_m = 0 is 65.36%, and this drops to 10.17% at FTA_m = 3. Correspondingly, the predicted probability for being classified as ‘LC_m = 0’ increases from less than 1% at FTA_m = 0 to 46.52% at FTA_m = 3. This pattern is repeated even more pronouncedly for the AD_m variable, for which the predicted probabilities could only be obtained for the extreme values of the AD_m variable (i.e., AD_m = 0 and AD_m = 3). These results manifest the actual effectiveness of co-adopting both FTA and AD approaches on the deployment of FMEA in complex applications.

4. Discussion

The traditional model of FMEA implementation involves a bottom-up analysis of failure behaviour and is independently performed during the product development stages [1,2,3,4]. This results in difficulties for determining where to search for failure modes and how to pursue their effects [6,7,10,13]. In addition to this, the current mode of FMEA tends to ignore the significant benefit of synchronising the applications of Functional, Design and Process FMEAs [1,10,13,14]. This makes it difficult for FMEA practitioners to achieve sufficient depth of analysis, resulting in a generally time-consuming process [6,7,10]. The findings obtained in this paper provide useful insights into the methods and factors that may potentially overcome the aforementioned challenges and, therefore, lead to a successful application of FMEA in complex systems.

For example, the co-adoption of the FTA method was identified as a supporting element in the deployment of FMEA, allowing a top-down decomposition of complex systems. To address this issue, a structured framework for combining FTA and FMEA approaches was established and employed to evaluate the development of a metal additive manufacturing system [6]. Likewise, Yu et al. [17] and Arcidiacono and Campatelli [20] also suggest this concept of combining FTA and FMEA methods into a hybrid methodology to tackle the limitations involved with both approaches. However, these studies have only assessed the applicability of combining both methods into a hybrid framework. Therefore, the true impact of this integrated approach still requires further investigation to explore how this such integration can overcome the challenges of implementing FMEA.

Notwithstanding the above, the results presented in this paper do provide a unique quantitative indication of the extent to which co-adopting the FTA method can assist in the deployment of FMEA. This is confirmed by the negative coefficient of the FTA_m variable (–0.41), which was obtained by our model. This finding suggests that co-adopting the FTA method may be able to efficiently guide and lead the application of FMEA to the deepest level in the product design architecture. This can be seen through the hierarchical trees developed, allowing the identification of the top functional failure modes followed by an extension of these failure modes down to a detailed level. This process creates a robust, logical hierarchy that can be used to better understand the underlying failure mechanism [6,17,20]. Together with the results presented in this study, this indicates that the co-adoption of the FTA approach can have a positive impact in terms of reducing the system complexity analysis, which would save resources and ensure an effective application of FMEA. Arcidiacono and Campatelli [20] have drawn similar conclusions, noting that where FTA and AD methods combined with FMEA approach for system decomposition, a 50% saving in resources utilised in implementing FMEA was achieved.

To provide a method to control the synchronisation issue between Functional, Design and Process FMEAs, this paper shows that co-adopting AD method could support FMEAs deployment by integrating its applications, Functional, Design and Process FMEAs. The ability of the AD method to resolve FMEA synchronisation issues has also been documented by Arcidiacono and Campatelli [20] and Alruqi et al. [32]. These works, however, have only focused on the link between Functional and Design FMEAs. The empirical results, presented in this study, appear to be encouraging in terms of highlighting the ability of the AD method to integrate Functional, Design and Process FMEAs into a working architecture, therefore leading to a full comprehensive implementation of FMEA over the different product development stages. The negative coefficient of the AD_m variable (−0.71) suggests that combining the AD method in the application of FMEA affects LC_m relatively strongly. This indicates that the AD approach constitutes a suitable methodology to bridge the knowledge gap between all FMEA procedures, as it could be able to perform the role of system decomposition (i.e., through decomposition tree and design matrix) and interconnect FMEA procedures (i.e., through domains and zigzagging approaches). This is a key point, as AD implementation may facilitate clear modelling of the effects of the failure modes and also identify a detection of the cascading failures at diverse product development stages. As previously mentioned, these are areas where FMEA is currently underdeveloped and quite inefficient.

Another method examined in this study involved assessing the impact of co-adopting QFD approach in the application of FMEA. The QFD_m variable was identified as statistically significant and not reducing LC_m, due to its positive association with LC_m (0.927). It can be argued that QFD implementation works similarly to the AD approach by potentially interlinking FMEA procedures [42]. Although this may be true to an extent, others, such as Fargnoli and Sakao [43], argue that QFD implementation is not effective for modelling new designs (i.e., the aim of this study). It should be noted that QFD implementation is additionally viewed as a time-consuming methodology and is, therefore, impractical for treating large amounts of information within complex systems typically characterised by huge interrelationship matrices [43,44].

Other frameworks suggested for improving the deployment of FMEA in complex systems include combining the FFBD approach [22,23,24], AHP method [25,26] and KBS technique [30,31]. These approaches were also analysed in this paper to verify their impact on the implementation of FMEA. The impact of co-adopting these approaches on LC_m was identified as statistically insignificant. This is because the regression coefficients associated with these variables were numerically too small to influence LC_m. This might also be due to the small sample size used in this study and the fact that the complex systems, per se, present significant challenges for determining system integrations and managing a large scale of information, where FFBD and AHP methodologies seem unable to help in their original forms.

As for the KBS_m variable, its statistical insignificance could be attributed to the fact that implementing the KBS technique involves developing a set of rules to generate, utilise and match knowledge from diverse sources (e.g., in the context of FMEA: Functional to Design and then to Process FMEAs). Applying this technique would then lead to higher costs for the overall FMEA deployment through training and dedicating a specialised team. As indicated in Futia and Vetrò [45], there are also two main challenges for adopting KBS techniques: knowledge matching and explanations from different sources. These challenges would be exacerbated in the practice of FMEA, where there are already difficulties in its applications for identifying inputs include failures modes and other features from different sources and backgrounds as well as the gap in knowledge between Functional, Design and Process FMEAs.

As regards evaluating the impact of the change in other managerial factors, this study has investigated the effect of experience level (EXP), organisational size (SIZE) and industry membership (MEM) on the successful application of FMEA. These factors have been selected as possibly having a certain influence on the progress of FMEA deployment [1,18,19]. However, it is worth noting that there is a paucity of research linked to the impact of these factors on FMEA application.

The EXP_m, SIZE_m and MEM variables were not found to be statistically significant in our model in terms of their effect on LC_m. This is likely to be due to a lack of information on the number of system components included in FMEA deployment and the required depth of analysis representing the MEM variable, the characteristics of those people involved (i.e., skills and knowledge) representing the EXP variable and the number and influence of FMEA-tasked employees in the business representing the SIZE variable.

In the context of industry 4.0, data analysis and real-time processing are considered the key factors [46]. Santos et al. [47] have reported various challenges that hinder the successful applications of industry 4.0. One of the foremost challenges is the quality of data generated to steer the manufacturing operations [47]. Lo et al. [48] urged the introduction of FMEA in the realm of industry 4.0 to move the manufacturing environment into more agile and rapid product development. FMEA is a tool that can support decision makers by evaluating the collected data to improve the product design and manufacturability [46]. Considering the current issues associated with FMEA deployment, this research found that co-adopting FTA and AD approaches could produce such quality data that can sustain FMEA deployment and improve its outcomes in complex applications, and it also may help managers to practice industry 4.0. The FTA method is seen to identify and trace the failure causes top-down, providing useful input to FMEA deployment, where the AD method can chase and synchronise the flow of information (like design sequence) across different FMEA applications, understanding the product development operations and supporting sustainable manufacturing. This research provides a theoretical basis to estimate the potential of co-adopting other engineering methods to address the current issues associated with FMEA deployment and develop sustainability in its potential applications, including industry 4.0.

However, it can be expected that the narrow range of the independent variables included in the executed ordered probit model might limit the generalisability for the scope of the findings of this study, as does the small sample size. Thus, this study may not be fully representative of the industrial FMEA practice. As follow-up research, this study could be developed by acquiring more details regarding product complexity, the characteristics of people involved (i.e., skills and knowledge) and worker cost related to FMEA users, as well as examining other factors such as the number of mitigated failures. Given that the ordered probit model applied in this research can still present meaningful conclusions, it would also be worthwhile to apply this work to a larger sample size to improve the understanding of the associations between LC categories and the independent variables included in the analysis. Moreover, it is suggested that the presented model not only be validated on a broader subjective basis but also be applied to a real working situation to provide more understanding in a more practical and user-friendly form.

5. Conclusions

Research on FMEA deployment in complex systems has shown three main challenges: the excessive use of resources, deployment difficulty and data availability. This study developed a score-based metric for estimating the level of challenge (LC) associated with FMEA deployment. A review of the existing literature has also revealed limited empirical research on the impact of co-adopting FTA, FFBD, AHP, QFD, KBS, DAIREC and AD methods on FMEA deployment as well as the influence of experience level (EXP), organisational size (SIZE) and industry membership (MEM).

Therefore, this paper executed the ordered probit model to determine the association between the perceived level of challenge (LC) encountered with implementing FMEA and the co-adoption of the methods and managerial factors outlined above. The model results have improved the understanding of the impact of integrating these approaches and the change in these managerial factors on FMEA deployment. The results obtained from this study have revealed that the co-adoption of FTA and AD methods with FMEA application is negatively associated with LC. Therefore, this study infers that co-adopting FTA and AD approaches significantly improves FMEA practices.