Evaluation of the Operation Process of Medical Equipment to Enhance Ergonomic Reliability Based on FRAM–Moran’s I and CREAM

Abstract

To improve the ergonomic reliability of medical equipment design during the operation process, a method for evaluating the operating procedure of a medical equipment interface according to functional resonance analysis method (FRAM)-Moran’s I and cognitive reliability and error analysis method (CREAM) is proposed in this study. The novelty of this research is to analyze the ergonomic reliability of medical equipment in a more systematic manner and to minimize the impact of human subjectivity and individual differences on the evaluation results of the operation process. To solve the calculation problem of functional resonance in FRAM and to make the evaluation results more objective, Moran’s I was introduced to quantify the deviation degree caused by the individual differences of the subjects. By giving weights based on Moran’s I, the influence of individual differences and subjectivity on the evaluation results can be minimized, to a certain extent. Considering the importance of a special environment, which is not fully considered by the conventional CREAM, the weighting values based on Moran’s I, Delphi survey, and technique for order preference by similarity to an ideal solution (TOPSIS) were adopted to assign weights to common performance conditions (CPCs) in CREAM. The optimal design scheme was selected more objectively than in the conventional method. The validity and practicability of this operation process evaluation method was verified by a statistical method based on ergonomic reliability experiments.

1. Introduction

There are many methods used to evaluate ergonomic reliability. The first-generation research methods that focused on human behavior and classification of errors have appeared and developed into the second-generation methods seen later, such as cognitive reliability and error analysis method (CREAM). Like many new approaches, CREAM incorporates cognitive models representing the more complex human reliability analysis (HRA) [1]. Nowadays, ergonomic reliability is commonly applied in the medical field, and a study by Carayon et al. indicated that HRA would play a central role in medical security [2]. Sujan et al. noted that human–machine assessment can play an effective role in risk analysis in the healthcare field; however, the technique needs to be adapted to take into account the context of the field [3]. Zaitseva et al. applied the structure function in reliability engineering to analyze and evaluate human factors in the use of medical devices [4]. Foster et al. demonstrated the importance of human reliability assessment in laparoscopic ablation [5]. These studies indicate that reliability analysis is not limited to traditional mechanical field applications, such as the stress testing of mechanical structures etc. [6,7,8]; reliability analysis is also widely used in the medical field, nowadays, although some reliability analysis methods applied in this field are immature. In addition, CREAM is gradually being introduced into the medical field as a typical human reliability approach. Zheng et al. studied a derivative model based on CREAM to predict the dispensing error rate in pharmacies [9]; however, the model has limitations that do not take into account the source of system errors other than human factors. Modern engineering systems often operate under complex operating conditions [10]; thus, reliability analysis often involves multidisciplinary coupling and multi-level structure construction [11] with uncertain factors [12,13]. CREAM, which has been utilized in many optimization schemes, is very suitable for ergonomic reliability analysis in the above situations. Zhou et al. used a fuzzy analytic hierarchy process (FAHP) to evaluate the weight of common performance conditions (CPCs) in the shipping industry, and proposed a method where CREAM could be quantified to make it more suitable for the actual situation of personnel in the shipping industry [14]. Chen et al. studied the human error in diving operations by combining CREAM and Bayesian networks [15]. Based on the extended CREAM framework, He et al. optimized the CPC assessment under the chemical enterprise scenario on the basis of summarizing the safety management assessment of hundreds of hazardous chemical enterprises [16]. Shuen-Tai proposed a new fuzzy method to improve the logical relationship between CPC and cognitive control mode (COCOM) by adjusting the related weights, and verified it by taking an oil tanker as an example [17]. He et al. proposed a prospective quantitative process simplification of CREAM [18]. Marseguerra et al. mentioned that the decision of CPC level or score is disturbed by uncertainty and proposed a fuzzy set method to capture such uncertainty [19]. Wang et al. established a CREAM framework table for quantification of coal mine operation scenarios to solve its shortcomings, such as not considering the applicability of general performance conditions and the subjectivity of evaluating the CPC table [20]. The objects of systematic design optimization often have multi-objective characteristics, among which the determination and allocation of weights play an important role in the design result [21]; however, the predictions of traditional CREAM do not take into account how the system will change when targeted to the different cases. Compared with different improvement methods, CREAM can be better combined with the environment based on CPC weight allocation, which is also combined with other reliability algorithms, such as fault tree analysis (FTA), to solve human factor reliability problems about the source of system errors and targeted change methods. Zupancic and Marn applied the synthesis of CREAM and FTA in the reliability of distillation tower operating systems [22]. Alvarenga et al. mentioned that CREAM has made progress on issues related to human cognition; however, the interaction between these factors cannot be studied using this model. The functional resonance analysis method (FRAM), which is similar to but more advanced than traditional FTA, is a promising modern method to solve this problem. It can play a role in the correlation of each information transmission and enhance the influence of other parameters in the evaluation [23]. Reliability analysis needs to be systematic [24]. FTA is a top-down analysis based on workplace management [25], while FRAM is the opposite as a systemic ergonomics reliability analysis method used to analyze action behavior in a more in-depth and systematic way. Patriarca et al. studied and demonstrated that FRAM can solve problems by studying technology, people, organization [26], and other external factors that influence the boundaries of complex systems. França et al. used FRAM to build models and systematically analyzed human influence and non-technical technologies under the scenario of offshore drilling [27]. FRAM has also been applied in the medical field. Vahid Salehi et al. found that medical and health care is one of the most popular fields for using FRAM [28]; they studied sick and weak elderly people and modeled and analyzed their problems during the transition between hospital and home using this reliability analysis method [29]. Clay-Williams used FRAM in the medical field to bridge the gap between paper processes and actual work [30]. FRAM, as a reliability analysis approach, can provide additional insight into human factors and the environment [31] and is useful in evaluations. For example, Haddad and Rosa used FRAM to assess the risks of processes in the construction industry [32]. Gattola et al. discussed the application of FRAM in the assessment of socio-technical safety-related issues in manufacturing [33]. Anvarifar et al. explored the application of FRAM in qualitative risk assessment of multifunctional flood protection facilities [34]. In systems evaluation, FRAM has greater potential because it better simulates the environment under investigation [35].

However, relevant computing studies on FRAM are scarce at present. Most reliability analyses rely on big data [36,37], and some medical equipment is used in special conditions of high infection. When a sample size of hundreds or thousands cannot be reached, a method that reduces the bias caused by individual factors needs to be introduced. Moran’s index (Moran’s I) was proposed in 1950 and later used in spatial autocorrelation. Thompson et al. showed that when Moran’s I is used to quantify the degree and form of phase separation, it can provide more reliable quantitative measurements and good results can be obtained. Lima et al. used Moran’s I in their study to detect fraud in auction bids [38]. Mazhar et al. used Moran’s I to explore the adaptability of farming families in different regions, and showed that spatial analysis extending the Moran’s I can be a useful tool for decision makers to decide whether to conduct behavioral interventions [39]. Since FRAM can also demonstrate a spatial and behavioral relationship, Moran’s I can be applied to enhance it. Hirose and Sawaragi proposed a further FRAM in conjunction with CREAM, called “Fuzzy CREAM for systematic and quantitative FRAM analysis”, which was used to test the safety of drug dispensing procedures under specific circumstances [40]. In addition, Hirose and Sawaragi also applied this method in an air disaster and determined that the reason conclusion of the pilot deviated from the standard operating process [41]. Therefore, FRAM–Moran’s I amalgamated with CREAM can be useful in ergonomic reliability analysis.

With the development of information science and technology, there is still a lack of widely recognized simulation software for human reliability evaluation compared with the mechanical reliability field [42]. The verification of the ergonomic reliability method through experimental simulation requires necessary precision equipment [43,44], which is expensive; however, it is effective and practical under the conditions of use.

According to the situation analyzed above, to evaluate the operation process of medical equipment, this research proposes an approach based on FRAM–Moran’s I and CREAM. In Section 2, the ergonomic reliability analysis is elaborated deeply. Under the circumstances of high workload and a high-infectivity environment, a medical laboratory technician’s actions were broken down and the resonance relations between the action factors were dissected using the FRAM technique. Combining Moran’s I with FRAM is a novel way to reduce the unreasonable consequences resulting from the chain reaction caused by poor form, such as insomnia etc.. In addition, the related data provided by CREAM is more suitable for the human operation background of medical equipment through the regulation of weight values. In Section 3, there are three design schemes of the polymerase chain reaction (PCR) device as typical medical equipment that were assessed by the method described in this research. In Section 4, the effectiveness of the proposed method was verified by eye tracker and electromyogram experiments. In Section 5, our conclusions are presented.

2. Operation Process Evaluation of Medical Equipment Based on Ergonomic Reliability Analysis

2.1. Operation Process Analysis Based on FRAM–Moran’s I

FRAM can be systematically utilized to study the correlation between the detailed movement flow and the movements of medical personnel in terms of operating medical equipment. The hexagonal structure of FRAM is illustrated in Figure 1. I (inputs) offer connections to the previous functions. To produce outputs, inputs can be utilized by functions or converted. O (outputs) offer connections to follow-up functions, and R (resource) is an output requisite for processing inputs (energy, programs, software, hardware, and labor). C (controls) are used as the restrictions or control factors to assist in monitoring or restricting functions. P (prerequisites) serve as system conditions; they must be met before a function (another program or procedure, as well as particular conditions) can be performed. T (time) is a particular resource or restriction derived from the allowable amount of time.

In the evaluation of human factor reliability, the individual differences of operators can easily affect the entire evaluation. However, this phenomenon is rarely recognized, and the impact of individual differences on the entire evaluation is very difficult to quantify. In contrast, Moran’s I provides a reliable quantitative measurement method. Regarding the number of professionals who face difficulty in obtaining large-scale data, if the probability of failure of each module is particularly high and involves a chain reaction, it is caused by human differences and individual mistakes; therefore, Moran’s I, as a weight indicator for each operator, can be applied to measure the proficiency of the operator or the errors caused by poor status. In the process of evaluating medical device interfaces, the impact of an individual’s operational proficiency or poor state in the evaluation process needs to be minimized. If the value of Moran’s I is significantly higher, it implies that the individual has a significant influence; however, if the value of Moran’s I is relatively low or fluctuates around the mean value, the failure is caused by the interface design and not the individual. Based on the FRAM model, operational behaviors corresponding to spatial correlations can be systematically and comprehensively calculated using Moran’s I. The following equation reveals the calculation formula for the global Moran’s I:

$$I=\frac{{\displaystyle \sum _{p=1}^g{\displaystyle \sum _{q=1}^g{w}_{pq}(x_p-\overline{x}})(x_q-\overline{x})}}{S^2{\displaystyle \sum _{p=1}^g{\displaystyle \sum _{q=1}^g{w}_{pq}}}}$$

(1)

where, $S^2=\frac{{\displaystyle \sum _{p=1}^g{(x_p-\overline{x})}^2}}{g}$ is the sample variance, $w_{pq}$ is the spatial weight matrix of the $\left(p,q\right)$ factor (the distance from p to q is measured), and ${\displaystyle \sum _{p=1}^g{\displaystyle \sum _{q=1}^g{w}_{pq}}}$ is the sum total of the spatial weights. When the spatial weight matrix is row-normalized, ${\displaystyle \sum _{p=1}^g{\displaystyle \sum _{q=1}^g{w}_{pq}}}=g$ then the global Moran’s I should be expressed as follows:

$$I=\frac{{\displaystyle \sum _{p=1}^g{\displaystyle \sum _{q=1}^g{w}_{pq}(x_p-\overline{x}}})(x_q-\overline{x})}{{\displaystyle \sum _{p=1}^g{(x_p-\overline{x})}^2}}$$

(2)

Based upon the global Moran’s I, the weight of an individual’s influence on the entire evaluation can be obtained, as follows:

$$w_i^2=\frac{1-\frac{I_i}{{\displaystyle \sum _{i=1}^e{I}_i}}}{{\displaystyle \sum _{i=1}^e(1-\frac{I_i}{{\displaystyle \sum _{i=1}^e{I}_i}})}}$$

(3)

2.2. Ergonomics Reliability Analysis Based on Improved CREAM

The CREAM analysis process is divided into three main steps. Firstly, the CPCs are used to characterize the overall task, and this characterization is expressed by means of a combined CPC score. There are different levels with different numbers of levels in the different CPCs. Secondly, possible cognitive function failures must be identified. The cognitive control mode (COCOM) involves observation, interpretation, planning, and execution. Every cognitive activity should be described according to its related COCOM function, as illustrated in Figure 2. Thirdly, the failure probability is determined. Finally, the impacts of common performance conditions on cognitive failure probabilities (CFPs) are considered to obtain the event failure probabilities. In traditional CREAM, the detailed description of the C7 level is as follows: general time pressure levels for the task and state types, in addtion to how to better align task execution with the dynamic process [45]. C7 is divided into two levels: day-time and night-time. These two levels correlate with the guidance weights and control the degree of influence on failure probabilities for the four major failure modes in normal circumstances. As large amounts of medical equipment are used by doctors in laboratory medicine, they often face the problem of staying up late and losing sleep. Therefore, these factors, such as fatigue and insomnia, are added into C7 combined with the night-time level in the traditional CREAM framework, which means the adjusted values for night-time are used in this case based on medical background and are combined with the four kinds of cognitive function failures, as shown in Figure 2.

According to HRA techniques, such as the human error assessment and reduction technique (HEART), along with experiences from related studies [46], a database that specifies the applicable weighting factors for all cognitive function failures was built. The operation process evaluation of medical equipment under actual situations and environments, the weights of CPCs, and weighting factors based on COCOM were considered to adjust the cognitive function failures in order to improve the conventional CREAM, as illustrated in Figure 3.

2.2.1. Assessment of Common Performance Conditions

The conventional weighting value of the CPC factors of CREAM are obtained from the average of each factor. To make the assessment more reasonable and improve usability, according to the circumstances and case, the weighting value of each CPC factor should be adjusted under the different application environments based on the actual equipment. The Delphi method, developed by the RAND Corporation, is applied according to actual situations to consult experts [47]. Experts ranked the importance of the factors for CPCs. Based on this adjustment, the factors for CPC coefficients provided by CREAM fall under four major cognition control modes: observation, interpretation, planning, and execution. However, experts’ experiences are subjective to a certain extent based on their own rich experience [48]; this necessitates consensus and correlation tests to improve the reliability of the data provided by the experts [49].

The coefficient weights of COCOM functions corresponding to the expected effect of the CPCs on performance reliability should be adjusted.

The data collected via the Delphi exhibit has the hierarchical characteristics of multiple columns; thus, Kendall’s W is highly suitable for analyzing the consensus test according to this type of data. Based on the rankings given by E experts, the facilities are ranked as 1, 2, …, t, i.e., from small numbers to large numbers, according to their importance. In the h-th hierarchy, B₁^h, B₂^h, …, B_t^h denote the average ranking values. The test hypotheses are as follows:

Hypothesis 0 (H0).

The E rank orders are inconsistent.

Hypothesis 1 (H1).

The E rank orders are consistent.

The formula of Kendall’s W is expressed as:

$$W=\frac{12{\displaystyle \sum _{j=1}^t{T}_j{}^2-3E^{2}t{(t+1)}^2}}{E\left[Et(t^2-1)-{\displaystyle \sum (s^3-s)}\right]}$$

(4)

where E represents the number of participating judges; t is the rank order number of the weighting factors under one CPC, based on the different COCOMs; T_j is the sum of the numbers of the weighting factor rank orders; and s is the number of rank orders with the same importance for the weighting factors. The chi-squared result is calculated using Equation (5):

$${\chi }^2=E(t-1)W$$

(5)

The consistency of the rank orders of the weighting factors is determined by the chi-squared test. $\nu =t-1$ represents the number of degrees of freedom. When ${\chi }^2$_0.05,t−1 < ${\chi }^2$, P < 0.05, H1 should be accepted while H0 should be rejected.

After the consistency test, the correlation and reliability of the grader are tested based on grey correlation analysis, which is an approach used to judge the degree of correlation between different factors based on the geometric shape with a similar degree of curves. In addition, part of the relatively low degree of correlation should be removed [50]. After this low degree of correlation is removed, one more consistency is confirmed to improve the reliability of the data input to the evaluation matrix calculation. Grey correlation analysis is highly suitable for an uncertain system with a small sample of the part of information known in addition to the part of information unknown. The principles are as follows. There exists c weighting factors under one CPC based on the different COCOMs.

The mother sequence is expressed as in Equation (6):

$$X_0={\left(x_0(1),x_0(2),\cdots ,x_0(c)\right)}^T$$

(6)

The subsequence is as follows:

$$\begin{array}{l}{X}_1={\left({\mathrm{x}}_1(1),x_1(2),\cdots ,x_1(c)\right)}^T\\ \cdots \\ X_e={\left(x_e(1),x_e(2),\cdots ,x_e(c)\right)}^T\end{array}$$

(7)

$$y\left(x_0(k),x_i(k)\right)=\frac{\underset{i}{\min }\underset{k}{\min }|X_0(k)-X_i(k)|+\mathsf{\rho }\underset{i}{\max }\underset{k}{\max }|X_0(k)-X_i(k)|}{|X_0(k)-X_i(k)|+\mathsf{\rho }\underset{i}{\max }\underset{k}{\max }|X_0(k)-X_i(k)|}(i=1,2,\dots ,e;k=1,2,\dots ,c)$$

(8)

Then,

$$y\left(x_0,x_i\right)=\frac{1}{c}{\displaystyle \sum _{k=1}^{c}y}\left(x_0(k),x_i(k)\right)$$

(9)

is the grey correlation degree of $x_0$ and $x_i$.

After the consensus test based on Kendall’s W and the correlation test based on grey correlation analysis, the relatively objective and useful data are calculated in the next step of the analysis to obtain the weighting factor values. $B_m^h$ is the average rank under one weighting factor in the CPC, as expressed in Equation (10). Owning to the consistency matrix composed of the mean rank orders of the weighting factors of CPCs, it is not necessary to perform certain consistency tests once the judgment matrix is built, as in Equation (11).

$$a_{mn}^h=\raisebox{1ex}{$B_m^h$}\!\left/ \!\raisebox{-1ex}{$B_n^h$}\right.(m,n=1,2,\dots ,c)$$

(10)

$${\left(a^h\right)}_{c\times c}=\begin{array}{c}\begin{array}{ccccc}& O & I& P& E\end{array}\\ \begin{array}{c}O\\ I\\ P\\ E\end{array}\left(\begin{array}{cccc}1& a_{12}^h& \cdots & a_{1c}^h\\ a_{21}^h& 1& \cdots & a_{2c}^h\\ ⋮& ⋮& \ddots & ⋮\\ a_{c1}^h& a_{c2}^h& \cdots & 1\end{array}\right)\end{array}$$

(11)

Under the different weighting factors of the CPCs, O represents the related value under the observation mode, I represents the related value under the interpretation mode, P represents the related value under the planning mode, and E represents the related value under the execution mode. c is number of the weighting factors with different levels of one CPC under the four cognitive control modes [51]. In addition, the rank of a^h is 1, which is a consistent matrix; hence, Z is the only one non-zero eigenvalue of a^h. Then, the weight vector can be obtained from the normalized eigenvector of a^h, as indicated in Equation (12):

a^hw = Cw, w = (w₁, w₂, …, w_c)

(12)

where the non-zero vector w is the feature vector corresponding to the eigenvalue C of a^h.

$${W_m}^h=\raisebox{1ex}{$w_m$}\!\left/ \!\raisebox{-1ex}{${\displaystyle \sum _{m=1}^c{w}_m}$}\right.$$

(13)

In Equation (13), W_m^h, which is the adjusted weighting factor value of different levels of CPCs under different cognitive control modes, can be obtained by normalizing w.

The combined CPC scores are adjusted using the weights of the nine CPCs. Furthermore, the technique for order preference by similarity to an ideal solution (TOPSIS) is adopted for calculating the CPC weights. After the weight of each CPC is obtained using TOPSIS, it is then multiplied with the value of the weighting factor under different CPCs provided by the improved CREAM based on Delphi, in order to obtain a new weight factor value after modification; this can help determine the probability of ergonomic reliability in a more precise manner.

Considering the special environment in which the medical equipment is used, c CPCs are assessed with a normalized matrix of e assessment indexes based on experts’ research. The following is a decision matrix for the ergonomic reliability assessment constructed based on the TOPSIS analysis:

$$\mathit{H}=\left[\begin{array}{cccc}{h}_{11}& h_{12}& \cdots & h_{1e}\\ h_{21}& h_{22}& \cdots & h_{2e}\\ ⋮& ⋮& \ddots & ⋮\\ h_{c1}& h_{c2}& \cdots & h_{ce}\end{array}\right]$$

(14)

The following is the definition of the maximum value:

$${\mathit{H}}^{+}=\left(H_1^{+},H_2^{+},\cdots ,H_m^{+}\right)=\left(\max \left\{h_{11},h_{21},\cdots ,h_{c1}\right\},\max \left\{h_{12},h_{22},\cdots h_{c2}\right\},\cdots ,\max \left\{h_{1c},h_{2c},\cdots h_{ce}\right\}\right)$$

(15)

The following is the definition of the minimum value:

$${\mathit{D}}^{-}=\left(D_1^{-},D_2^{-},\cdots ,D_m^{-}\right)=\left(\min \left\{d_{11},d_{21},\cdots ,d_{c1}\right\},\min \left\{d_{12},d_{22},\cdots d_{c2}\right\},\cdots ,\min \left\{d_{1e},d_{2e},\cdots d_{ce}\right\}\right)$$

(16)

The distance from the maximum value to the value of the $k\text{-th}(k=1,2,\cdots ,c)$ evaluation index is defined in Equation (17). Here, w_ik is based on Moran’s I.

$$L_k^{+}=\sqrt{{\displaystyle \sum _{i=1}^e{w}_{ik}{\left(H_i^{+}-h_{ki}\right)}^2}}$$

(17)

The distance from the minimum value to the value of the $k\text{-th}(k=1,2,\cdots ,c)$ assessment index is defined as:

$$L_k^{-}=\sqrt{{\displaystyle \sum _{i=1}^e{w}_{ik}{\left(H_i^{-}-h_{ki}\right)}^2}}$$

(18)

In addition, the approximation degree of the $k\text{-th}(k=1,2,\cdots ,c)$ assessment index to the ideal solution is provided by:

$$w_k=\frac{\frac{L_k^{-}}{L_k^{+}+L_k^{-}}}{{\displaystyle \sum _{k=1}^c\left(\frac{L_k^{-}}{L_k^{+}+L_k^{-}}\right)}}$$

(19)

2.2.2. Probability Adjustment Based on Cognitive Failure

Figure 4 illustrates the four cognitive functions of the CPCs derived from CREAM. The relative cognitive activities and the corresponding weighting factors of every CPC are identified and calculated according to the basic weighting factors in Figure 4, which were used for a general situation. Thereafter, the multiplication result based on the modified weight values of the weighting factors for all hierarchy CPCs corresponding to each cognitive activity is obtained; the total weighting factor of cognitive activity is also determined. Next, according to different environments, the adjusted cognitive failure probability (CFP, the failure probability of the cognitive function) is derived in Equations (20) and (21):

$$CF{P}_{\mathit{modify}}=CF{P}_{calibration}\times {\displaystyle \prod _{f=1}^c{w}_f}\left(f\in c\right)$$

(20)

where $w_f$ is the revised weight of CPCs under different actual working environments based on the various cognitive behaviors:

$$w_f=\frac{w_k\times w_{meo}}{{\displaystyle \sum _{k=1}^c{w}_k}}\begin{array}{cc},& (\begin{array}{c}e\in \left\{\mathrm{Each}\mathrm{CPCs}\mathrm{level}\right\}\\ o\in \left\{\mathrm{OBS},\mathrm{INT},\mathrm{PLAN},\mathrm{EXE}\right\}\end{array})\end{array}$$

(21)

where $w_k$ is the weight of CPC achieved based on the TOPSIS method, and $w_{meo}$ is the weight factor value corresponding to the four different cognitive functions under the nine different CPCs.

3. Operation Process Evaluation in the Ergonomic Medical Equipment Interface

In this work, a PCR device, which is a piece of equipment typically used for virus detection, was considered. The procedures of medical personnel operating the PCR analysis software were recorded. The PCR interface operation process is mainly divided into three modules as follows: PCR program setup and operation, sample editing, and results analysis. In addition, PCR program setup and operation essentially includes the preparation, preincubation, amplification, melting curve, and cooling stages. According to the procedure, a FRAM model of the PCR interface operation process was built, as illustrated in Figure 5.

Moran’s I was introduced into FRAM to determine the spatial and temporal autocorrelations. There were three types of interface design schemes for the PCR control software. The analysis functions of these schemes were similar. Owing to limited conditions, a total of 36 experts were invited, and each scheme was operated by 12 experts. The failure times or number of errors of each expert operation were recorded. Based on these records, Moran’s I was introduced to identify the influence of the personal working status of the experts on the overall error operation. To reduce the influence of individual working statuses on the overall assessment, Moran’s I was used to assign weight to the experts. The global Moran’s I was obtained according to Equation (2). Furthermore, Figure 6 reveals the maximum values of Moran’s I based on Schemes A, B, and C. According to Equation (3), the weighting values from the experts’ operation errors were calculated based on Moran’s I. As shown in Figure 6, regarding Scheme A, the second expert was in the worst shape and clearly out of form, probably suffering from sleeplessness; thus the related Moran’s I value was the maximum. In order to minimize the impact of the data collected from the second expert on the ergonomic reliability results, the least weight was given as shown in Figure 6c. Similarly, regarding Scheme B, the least weight was given according to the fourth expert who had the maximum value of Moran’s I and was in the worst shape. Regarding Scheme C, the least weight was given based on the operation errors from the first expert.

Further, based on the improved CREAM proposed herein, using two rounds of consensus analysis based on Kendall’s W and a round of correlation analysis based on the grey correlation analysis, certain data from the Delphi technique were filtered out. Consequently, the objectivity of the data for analysis was improved, thereby overcoming the subjective defects of the Delphi survey.

As presented in

Table 1. The results of consistency tests before and after grey correlation analysis based on CPCs.

	C1 Levels	C2 Levels	C3 Levels	C4 Levels	C5 Levels	C6 Levels	C7 Levels	C8 Levels	C9 Levels
Value of Kendall’s W	0.102	0.164	0.112	0.117	0.121	0.186	0.075	0.148	0.147
Adjusted value of Kendall’s W	0.183	0.314	0.151	0.184	0.18	0.233	0.203	0.225	0.292
Value of chi-square	24.431	28.892	26.995	20.663	21.346	32.75	8.391	25.985	35.179
Adjusted value of chi-square	32.997	45.038	27.261	24.244	23.697	30.725	17.044	29.685	52.561
P values	0.058	0.002	0.029	0.037	0.03	0.001	0.299	0.007	0.002
Adjusted P values	0.05	<0.005	0.027	0.012	0.014	0.001	0.017	0.002	<0.005
Degrees of freedom	15	11	15	11	11	11	7	11	15

, based on the calculations using Equations (4)–(9), most of the p-values were significant. The chi-squared values were also elevated after the grey correlation analysis, except for in C6. This is because there were significant influence curves of CPCs in C6. There was less consensus in terms of consistency; however, it had a stronger impact on the final outcome. In this particular case, C6 had higher consistency before the grey correlation analysis; hence, the average ranks were applied in the evaluation matrix to obtain the importance of the weighting factors for C6. The COCOM function weight corresponding to different CPC levels can be calculated as illustrated in Figure 7. Those weights were calculated through the Delphi method to allow for the background of medical equipment operation and the features of interface design containing the expert experiences. In addition, the adjusted weighting factors for the CPC levels can be obtained using Equations (10)–(13). Figure 8a shows the basic weighting factors provided by the traditional CREAM aimed at the general situation, while Figure 8b presents the values summarized and calculated by considering the specific situation and research background based on improved CREAM.

The weighting values based on Moran’s I were used in TOPSIS. After testing and operating the interface, each expert was invited to fill in the important survey questionnaire of CPCs with a full score of 10 and to rate the importance of CPCs based on the PCR analysis software. According to Equations (14)–(19), the weights of the CPCs were calculated; the results are illustrated in Figure 9. The weight proportion of C3 was the largest, while that of C7 was the smallest. This may be because this research focuses on the interface design of the equipment.

In Figure 10, the three heatmaps reflect the users’ browsing and eye fixation. When the testers operated the medical equipment based on Scheme B, the fixation points mainly focused on some common keys in the test process. These hot spots do not form as a result of eyeballs wandering back and forth; thus, most of the hot spots are relatively evenly distributed. When Schemes A and C were observed, the eyes of testers moved back and forth between different non-important keys, causing the hot zones on the heatmaps to appear on the buttons that are not used frequently during testing. In addition, based on the adjusted weighting factors of the CPCs, the operation error times, and the basic values for cognitive function failures, the cognitive failure probability of the operation process can be calculated using Equations (20)–(21). For Scheme A, P_A = 0.04113; for Scheme B, P_B = 0.00504; and for Scheme C, P_C = 0.02956. From the perspective of ergonomics reliability, Scheme B is slightly better than Schemes A and C, and Scheme C is better than Scheme A. This is likely because several functional components of Schemes A and C are hidden in the drop-down menus and the inhuman layout, as illustrated in the heatmap in Figure 10.

4. Verification Based on Ergonomic Reliability Experiments

To verify that the pull-down and hidden tool menus are important factors resulting in the different degrees of ergonomic reliability in the operation process of the three schemes, the menu on the right of Scheme B was changed to a drop-down menu, similar to Scheme A, and a hidden menu, similar to Scheme C, as illustrated in Figure 11. Thirty-three subjects who were using the software for the first time participated in the experiments. During the user’s operation, an eye-tracking experiment was activated to record the observation orders of the subjects’ eyes and the number of mouse clicks during the process of searching for the virtual “save” button. The eye-tracking serial numbers recorded by the eye movement instrument with a steady frequency based on a typical subject are also shown in Figure 11. The larger the serial number is, the longer the time the subject spent trying to find the target button. The “save” button of Scheme A was found by the typical subject when the eye-tracking serial number was over 50; the “save” button of Scheme B was found when the serial number was 18; and the target button of Scheme C was found only when the eye tracking number reached approximately 40. In Scheme A, the virtual target button was located in the drop-down menu; in Scheme B, the virtual target button was designed as a large icon on the right; and in Scheme C, the virtual target button was located in the hidden menu on the right.

The data collection results of the eye-tracking serial numbers were contrasted in a one-way analysis of variance (ANOVA) based on 11 subjects that used the three schemes of the medical device software model for the first time. The F statistic was 22.15 (P < 0.005), as presented in

Table 2. Analysis of variance based on the serial numbers of eye tracking.

Source	Sum of Squares	Degree of Freedom	Mean Square	F Test Statistic	P Value
Between group	2403.87879	2	1201.93939	22.15	<0.005
Within group	1628	30	54.2666667
Total	4031.87879	32	125.996212

. Thus, the null hypothesis stating that the serial numbers of eye movement tracking under Schemes A, B, and C are all equal should be rejected. By contrast, the alternative hypothesis stating that the serial numbers of eye movement tracking under these three groups are not equal should be accepted. Thus, there are statistically significant differences in the serial numbers of eye tracking among the three groups. In addition, the one-premise condition of the variance analysis is that the total variance of each sample should be equal based on homogeneity of variance. Bartlett’s test ${\chi }^2$ was applied to test the homogeneity of variance, ${\chi }^2=1.7270$ (P = 0.422). This implies that, according to the inspection standards $\alpha =0.10$, the null hypothesis is not rejected, there is no statistical significance, and the three populations possess the homogeneity of variance; hence, the variance analysis applied to analyze the data is appropriate.

In order to further analyze which group had the largest mean coefficient of variation and which group has the smallest mean coefficient of variation among the three groups, the Bonferroni method was applied for the multiple comparisons, as presented in

Table 3. Multiple comparison results of three menu styles based on the Bonferroni method.

Comparison Group (1 and 2)	$\overline{{\mathit{x}}_2}-\overline{{\mathit{x}}_1}$
B and A	20.7273 (P = 0.000)
C and A	8.0000 (P = 0.049)
B and C	12.7273 (P = 0.001)

,. The results indicated that the button of Scheme C in the hidden menu is larger and easier to identify than that of Scheme A in the drop-down menu, which is more hidden and difficult to discover. By contrast, the button of Scheme B was the easiest to find.

Based on the three schemes, signals were collected from thenar muscle, the first lumbrical muscle, and flexor carpi radialis under the same screen angle as shown in Figure 12. Figure 12a illustrates the surface electromyogram (EMG) signals generated on these three muscles at a certain time. The root mean square (RMS) of surfaceEMG signals represents the instantaneous power of EMG signals and the effective value of muscle surface discharge, which is an indicator of the time domain characteristics of EMG signals [52]. RMS has excellent real-time performance and is often used to describe the activity state of muscles, mainly because its waveform is basically the same as that of the linear envelope of EMG signals. The amplitude variation characteristics of EMG signals can be reflected in detail in the time dimension, and there is an internal relationship between muscle load factors and muscle physiological and biochemical processes. RMS is correlated with the synchronization of motor unit recruitment and excitation rhythm [53]. Therefore, RMS was selected as a parameter to assess the degree of dynamic muscle activity. Figure 12b shows the surface EMG value processed by RMS in the same period; the RMS values of the first lumbrical muscle are large overall, while the values of thenar muscle are generally small.

Based on the experimental and statistical results, the RMS values of the surface EMG for the thenar muscle and the flexor carpi radialis under the three schemes showed no significant differences. However, the RMS values of the surface EMG for the first lumbrical muscle under the three schemes indicated significant differences. One-way ANOVA produced an F statistic of 7.25 (P = 0.0009). Bartlett’s test ${\chi }^2$ was used to test the homogeneity of variance ${\chi }^2=34.7319$ (P < 0.005). The homogeneity of variance was not met; therefore, the T test was applied to the two contrasts between the three schemes. The RMS of the EMG data collected based on Scheme A was greater than that for the data collected under Scheme C (T = 2.3632; P = 0.0098); the RMS of EMG data collected based on Scheme C was greater than that collected based on Scheme B (T = 1.2912, P = 0.0995); and the RMS of EMG data collected based on Scheme A was greater than that collected based on Scheme B (T = 3.4819, P = 0.0003). The EMG signals of each group were different and statistically significant. Moreover, Scheme B involved the lowest fatigue. This lower fatigue can enhance ergonomic reliability to a certain extent during the operation processes.

Using the two experimental designs, it was determined that the ergonomic reliability of Scheme B was relatively high. Although the costs of the experimental equipment are high, the validity of the ergonomics reliability evaluation approach presented herein was confirmed based on another perspective.

5. Conclusions

Among the existing human reliability analysis methods, CREAM is a relatively mature method that is suitable for the analysis of the operation processes of human computer interactions; however, analyzing the influence of individual states on the evaluation results for human reliability has not been studied adequately. The working status of each person was found to be different depending on the proficiency of training or the time periods, and this is difficult to quantify. This personal factor can lead to the misdiagnosis of design rationality for human–machine interfaces. When the human body is in a poor condition, a chain reaction of misbehavior can occur easily. Human operating behavior is systematic and complex. FRAM is ideal for analyzing complex integrated systems, and the degree of human chain reactions when operating the system can be reflected based on Moran’s I.

Considering the complexity of the operating system of certain medical equipment used in laboratories and the depth of function coupling, FRAM was employed to conduct an orderly, systematic, and detailed analysis of the operating process for medical equipment. The upstream–downstream coupling of the functions of each module of FRAM was identified based on Moran’s I to analyze the degree of chain reactions from the operator’s individual mistakes. The influence of individual operator differences on the evaluation results was minimized via Moran’s I to reduce the subjectivity of the evaluation results. Through three different design schemes and the ergonomic reliability statistical experiment, it was found that the novelty of Moran’s I’s worked together with FRAM is reasonable, and is a feasible and effective method to reduce the impact of the data of the subjects related to bad individual state on analysis results to improve the objectivity of the ergonomic reliability analysis of the interfaces of medical device operations. CREAM was adopted to classify the cognitive functions of operators when using medical equipment. These were then divided into four categories: observation, interpretation, planning, and execution. Each function had several failure modes. Based on the explanation of the 13 cognitive function failure modes and the basic value of the failure probability, the improved CPC weights and weighting factor values controlled by Moran’s I and the ergonomic reliability of different operation processes based on different design schemes were analyzed in order to identify excellent solutions. The rationality of the method based on FRAM–Moran’s I and CREAM was statistically verified via eye movement tracking and surface EMG acquisition experiments. This study found that the common button for high frequency use arranged on the main interface as much as possible, on the premise of not affecting contact efficiency between the human hand and the button, is more conducive to improving the ergonomic reliability of the operation process.