Data-Driven Approach to Improving the Risk Assessment Process of Medical Failures

In recent decades, many researchers have focused on the issue of medical failures in the healthcare industry. A variety of techniques have been employed to assess the risk of medical failure and to generate strategies to reduce the frequency of medical failures. Considering the limitations of the traditional method—failure mode and effects analysis (FMEA)—for risk assessment and quality improvement, this paper presents two models developed using data envelopment analysis (DEA). One is called the slacks-based measure DEA (SBM-DEA) model, and the other is a novel data-driven approach (NDA) that combines FMEA and DEA. The relative advantages of the three models are compared. In this paper, an infant security case consisting of 16 failure modes at Western Wake Medical Center in Raleigh, North Carolina, U.S., was employed. The results indicate that both SBM-DEA and NDA may improve the discrimination and accuracy of detection compared to the traditional method of FMEA. However, NDA was found to have a relative advantage over SBM-DEA due to its risk assessment capability and precise detection of medical failures.


Introduction
In recent decades, medical failures, which are referred to as errors or adverse events in a medical service, have attracted much attention in the healthcare industry due to the increasing concern for patient safety [1]. The occurrence of medical failures may result in additional costs and a reduction in medical quality [2]. The U.S. Institute of Medicine reported that preventable medical failures result in 1,000,000 injuries and 44,000-98,000 deaths in hospitalized patients [3] and incur a loss of $17,000,000 [4] each year in America. In Taiwan, approximately 500 medical conflicts each year await legal mediation or trial [5]. Researchers suggest that reducing medical failures is critical for improving patient safety in healthcare systems. The prevention of medical failures may consist of two stages: risk assessment and quality improvement to monitor medical failures that may occur in a system. In general, risk assessment is processed by categorizing medical errors and predicting the probability of their occurrence [6]. The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) requires all accredited hospitals or other healthcare organizations to complete at least one proactive risk assessment annually to assess the risk of errors and to continuously improve quality [7].
The techniques of risk assessment and quality improvement involve a variety of methods to prevent medical failures, including Six Sigma, hazard analysis and critical control points (HACCP), failure mode and effect analysis (FMEA) or healthcare failure mode and effect analysis

Methodology
FMEA was first developed to solve reliability and safety problems in the aerospace industry in the late 1950s. Because FMEA emphasizes the proactive prevention of medical failures rather than solutions, it can assist managers in identifying failures and causes/effects and in eliminating failures by instituting corrective actions in the risk assessment process [16]. In FMEA, a documented method is employed, asking the manager to provide structural and formalized information for the risk control and assessment of potential failures in terms of what might go wrong, what might cause it to go wrong, and what effects it would have [17]. The prioritization of failure modes is determined by the RPN, which is calculated by multiplying the scales of S, O and D. Higher values of RPN imply that corrective action is more urgently needed. The calculation of the RPN is expressed as follows: DEA is a non-parametric analysis technique used to measure the relative efficiencies of decision-making units (DMUs). By using mathematical programming, DEA yields a composite efficiency score between zero and one for each DMU having multiple inputs and outputs. A DMU is said to be efficient if and only if it has an efficiency score of one. In DEA, an assumption of the weights for all of the productive indicators is not required. A set of weights is objectively generated via a programming process in which favorable weights for all DMUs under evaluation are determined by maximizing their efficiency scores. The result obtained from DEA may help decision-makers to identify the inefficient units and to consequently establish improvement strategies.
To overcome the weakness of FMEA, a numerous studies suggest that DEA may be an effective option for enhancing the assessment capability of FMEA [18][19][20]. DEA is a well-known data-driven approach for measuring the relative efficiencies among DMUs [21]. According to the efficiency perspective, DEA aggregates all productive indicators and yields a composite score to reveal the distance between a DMU's position and efficiency frontier. The frontier is formed by efficient DMUs, also called best -practices, representing the boundary condition for the whole system, which all DMUs can benchmark at their current production technology [22].
In this paper, a slacks-based measure (SBM) was used to assess the risk of failure modes in the healthcare industry. Suppose that there are n DMUs, each DMU j (j = 1, . . . , n) uses m inputs x ij (I = 1, . . . , m) to produce s outputs y rj (r = 1, . . . , s). Let DMU o be the DMU under evaluation, and its ith input and rth output are denoted by x io and y ro , respectively. The SBM score of DMU o , calculated using the input orientation, is expressed by the following programming model: Model (2) is commonly referred to as the envelopment form and is the dual problem of the multiplier form. In model (2), s − ∈ R m is the slack vector for the input, and λ j ∈ R n is the non-negative vector connecting all inputs and outputs to form an efficiency frontier. s − i denotes the excess amounts in the input that can be decreased non-radially comparing with efficient DMUs. The θ SBM * o is the SBM efficiency of DMU o . If s − * i > 0, DMU o is identified as an inefficient unit, i.e., θ SBM * o < 1; otherwise, it is an efficient unit. Compared with a traditional radial DEA model, SBM provides a clearer view for determining the specific input variable that causes the inefficiency. Through the optimal slack amount, s − * i , the direction for improving inefficient DMUs can also be easily obtained. Figure 1 depicts the process of risk assessment, including four steps: Step 1: Collect the risk index report of failure modes.
Step 2: Generate a dataset of risk indexes (severity, occurrence, and detection).
Step 3: Assess the risk of failure modes using modified DEA, including SBM-DEA and NDA, as discussed in this paper, and generate the risk rankings for all failure modes.
Step 4: Provide improvement strategies by mitigating risk for the prevention of medical failures. Because the S, O, and D dataset in FMEA has the property of "the lower, the better", which is similar to inputs, this paper considers the risk indexes of S, O, and D as inputs. Thus, by applying SBM, as depicted in Equation (2), the FMEA model yields the following:  Through the optimal slack amount, , the direction for improving inefficient DMUs can also be easily obtained. Figure 1 depicts the process of risk assessment, including four steps: Step 1: Collect the risk index report of failure modes.
Step 2: Generate a dataset of risk indexes (severity, occurrence, and detection).
Step 3: Assess the risk of failure modes using modified DEA, including SBM-DEA and NDA, as discussed in this paper, and generate the risk rankings for all failure modes.
Step 4: Provide improvement strategies by mitigating risk for the prevention of medical failures.  SBM-DEA may successfully improve the discriminatory problem of traditional RPN, i.e., some failure modes have the same RPN composed of different combinations of S, O and D, providing an alternative adjustment as quantitative information for each risky failure mode. However, SBM-DEA may present two chief shortcomings: (1) for safer failure modes, it is unable to generate the complete prioritization, as they all have a unity score θ RPN * o = 1; and (2) for risky failure modes, the improving targets waste of risk mitigation resources.
To overcome the problem arising from SBM-DEA, this paper presents the NDA, expressed in Equations (4) and (5). All failure modes are classified into two sets: safe failure modes and risky failure modes. We then apply the Super SBM model developed by Tone [23] and a minimum distance model developed by Aparicio et al. [24] to evaluate the RPN indexes of safe failure modes and risk failure modes, respectively.
For safer failure modes: Assume that there are n safe failure modes with θ RPN * o = 1. To differentiate them, the RPN for each safe failure mode is defined as the optimal value δ RPN * o in the following model.
For risky failure modes: o = 1 and λ j = 1. Thus, the RPN index for each risky failure mode can be solved by the following model.
where M is a large positive variable, and b j is the binary variable. The v S , v O and v D are the weights for severity, occurrence and detection, respectively, which comply with the corresponding constraint to the multiplier form.
Compared to SBM-DEA, in the NDA model, the safe failure modes to be evaluated are removed from the safe frontier (λ j , j = 0). In addition, all of the slacks (s S− , s O− and s D− ) in the constraints and objective function are modified from positive to negative. Thus, the RPN for each safe failure mode can be obtained as δ RPN * o ≥ 1.

Data Collection in the Healthcare Industry
In this section, we reuse a case from Western Wake Medical Center in Raleigh, North Carolina, which was first introduced by applying FMEA to mitigate the risk of preventing infant abduction in Reichert [25]. The dataset is tabulated in Table 1 and consists of 16 failure modes, which were identified by a managerial team in the medical service process.

Results
This paper applies the failure mode data in Table 1 to the three models: FMEA, SBM-DEA and NDA. The results are demonstrated in Figure 2 regarding the prioritization of failure modes among the three different approaches. For each approach, the horizontal axis displays the ranked set of failure modes sorted from highest to lowest risk, whereas the risk index is shown on the vertical axis. Additionally, we divided all ranked FMs into four quartiles (designated Q1 to Q4), with Q1 representing the highest 25th percentile of risk for FMs requiring urgent action. In Figure 2, we use the reciprocal of optimal scores obtained by SBM-DEA and the NDA model for simple ranking.  According to the results from the NDA model in Figure 2, all of the problems mentioned above were clearly solved. First, the weak discriminatory power of FMEA was improved upon. Risk rankings between FM 3 and FM 16 and between FM 7 and FM 9 were obtained using the NDA model. The risk index for FM 3 was (1/δ RPN * 3 = 1.333) greater than that for FM 16 (1/δ RPN * 16 = 1.25), and the risk index for FM 7 (1/δ RPN * 7 = 1.125) was smaller than that for FM 9 (1/δ RPN * 9 = 1.304). Second, three safe modes, including FM 4, FM 10 and FM 15, analyzed by SBM-DEA, showed the same unity score. Through the analysis depicted in Equation (4), the risk of these three failure modes was completely ranked using the NDA model. The priority ranking was determined as FM 15 FM 10 FM 4. Such a result helps risk managers to obtain full prioritization. Finally, the prioritization of Q1 = {FM 1, FM 11, FM 6, FM 13} and Q2 = {FM 3, FM 13, FM 8, FM 9} based on NDA is virtually a match, with the same results as FMEA, i.e., 100% in Q1 and 75% in Q2.

Discussion
The RPN value calculated using FMEA provides very limited information for establishing improvement strategies. Compared with the traditional FMEA, both SBM-DEA and the NDA model not only can generate a composite risk index from an efficiency perspective for each FM but also provide the risk control team with the quantitative information to set explicit targets for improving their strategies. This quantitative information can be determined from the difference between risky failure modes and their projection targets. However, these projection targets may differ between SBM-DEA and the NDA model because the former uses the maximum difference, while the latter concerns the minimum difference under programming. Table 2 provides a detailed list of optimal scores, projection targets and difference rates for all failure modes calculated from SBM-DEA and the NDA model. Both the SBM-DEA model (left-hand side) and the NDA model (right-hand side) identify that FM 4, FM 10 and FM 15 are relatively safe modes. For each, the SBM-DEA model yields a score of one and projection target of S, O and D that are equivalent to the original data, such that the reduction rates are all zero. In contrast, the NDA model provides a score of more than one that can be used as the basis for further ranking. Moreover, the positive rates of S, O and D express additional information on risk-taking ability, i.e., FM 4 can increase 100% of O, FM 10 can increase 40% of O and 50% of D, and FM 15 can increase 83% of S. For risky failure modes with scores of less than one, the reduction rate with respect to the corresponding original data provides the scales for improving the effort of S, O and D (see the fifth and tenth columns of Table 2). By following these rates, the risky failure modes can be made safer, i.e., either to achieve the projection targets or to produce a score of θ RPN * o = 1 for the SBM-DEA model and δ RPN * o ≥ 1 for the NDA model. Table 2 also shows large differences in optimal scores and reduction rates between the SBM-DEA model and the NDA model. On the basis of the optimal scores, the first three extreme cases include FM 16 (θ RPN *  In brief, SBM-DEA may yield biased measurements of risk and generate unrealistic solutions with two main shortcomings in comparison to the NDA model. First, SBM-DEA imperfectly generates a complete prioritization for all failure modes. Second, the improvement strategy generated by SBM-DEA for risk mitigation is feasible, but it results in a waste of resources.

Conclusions
Given the growing awareness of and pressure for healthcare quality and patient safety in today's healthcare environment, both academics and practitioners are increasingly concerned with risk management in medical services to avoid the effects of medical failures. FMEA is a well-known systematic procedure that is widely used to identify medical errors and to provide the necessary corrective actions. In this paper, we presented two models for improving the traditional FMEA method and compared the relative advantages among the three models. The analysis was based on an infant security case at Western Wake Medical Center in Raleigh, North Carolina, U.S.
Both the SBM-DEA model and the NDA model aim to solve the problem of discriminatory power arising from FMEA, which may result in the same RPN with different combinations of S, O and D for some failure modes. The results indicate that the SBM-DEA model and the NDA model, by integrating both two models and FMEA, may provide realistic solutions on optimal targets for safety improvement, whereas the risk mitigation strategies resulting from FMEA remain scarce. However, SBM-DEA might be imperfect, as the corrective direction generated from the optimizing process was In brief, SBM-DEA may yield biased measurements of risk and generate unrealistic solutions with two main shortcomings in comparison to the NDA model. First, SBM-DEA imperfectly generates a complete prioritization for all failure modes. Second, the improvement strategy generated by SBM-DEA for risk mitigation is feasible, but it results in a waste of resources.

Conclusions
Given the growing awareness of and pressure for healthcare quality and patient safety in today's healthcare environment, both academics and practitioners are increasingly concerned with risk management in medical services to avoid the effects of medical failures. FMEA is a well-known systematic procedure that is widely used to identify medical errors and to provide the necessary corrective actions. In this paper, we presented two models for improving the traditional FMEA method and compared the relative advantages among the three models. The analysis was based on an infant security case at Western Wake Medical Center in Raleigh, North Carolina, U.S.
Both the SBM-DEA model and the NDA model aim to solve the problem of discriminatory power arising from FMEA, which may result in the same RPN with different combinations of S, O and D for some failure modes. The results indicate that the SBM-DEA model and the NDA model, by integrating both two models and FMEA, may provide realistic solutions on optimal targets for safety improvement, whereas the risk mitigation strategies resulting from FMEA remain scarce. However, SBM-DEA might be imperfect, as the corrective direction generated from the optimizing process was unrealistic, resulting in a waste of resources. Additionally, SBM-DEA is incapable of providing complete prioritization for all failure modes.