Reliability Growth Method for Electromechanical Products Based on Organizational Reliability Capability Evaluation and Decision-Making

Abstract

The reliability growth of electromechanical products is a continuous process of addressing reliability defects, which is very important for manufacturing enterprises. At present, research on the reliability growth of electromechanical products mostly focuses on the reliability defects of the products themselves, ignoring the fact that manufacturing enterprises are the executors of product reliability related work. Improving the organizational reliability capability of manufacturing enterprises can enhance the reliability of electromechanical products. In order to understand the current situation of organizational reliability capability (ORC) in electromechanical product manufacturing enterprises and make improvements, this paper establishes an ORC evaluation indicator framework for electromechanical product manufacturing enterprises and evaluates it using the grey evaluation method. Firstly, an evaluation indicator framework for ORC is established based on enterprise research. Secondly, the ORC of electromechanical product manufacturing enterprises is evaluated by combining the three-parameter interval grey number and projection index function. Then, the evaluation results are analyzed from multiple perspectives to understand the current situation and shortcomings of ORC and guide its improvement. Finally, the evaluation indicator framework and method are explained through practical application in CNC machine tool manufacturing enterprises, and the effectiveness of the framework and method is demonstrated through the MTBF growth of CNC machine tools.

1. Introduction

The quality of electromechanical products is directly related to the national economy and people’s livelihood, and product quality issues cause huge direct and indirect economic losses to the world every year. Reliability is a key quality characteristic of a product, and the growth of product reliability is crucial for enhancing enterprises’ market competitiveness, and reducing product maintenance time and economic costs. Generally, the reliability of a product depends on the combined effects of its design, manufacturing, testing, and other processes. At present, many scholars are conducting research on the reliability growth of products. In existing studies, scholars focus on reliability analysis, reliability optimization and other aspects of product reliability growth research. These studies mostly focus on the reliability of the product itself or a certain link in its lifecycle, and few studies study the reliability growth of products from the perspective of enterprises.

In IEEE Standard 1624-2008 [1], the organizational reliability capability (ORC) of enterprises is proposed and defined as the ability to organize reliability assurance work in order to ensure that product reliability meets or exceeds customer requirements. The ultimate effect of product design, manufacturing, and other processes is determined by the ORC of the enterprise. Therefore, improving the ORC can achieve the growth of product reliability.

Improving ORC has a positive impact on product reliability growth, and identifying weak links in ORC is the basis for ORC improvement decisions. To find out the weak links of the ORC, this paper tries to establish an ORC’s evaluation indicators framework and evaluate the ORC on the basis of the evaluation indicators. However, the ORC evaluation index system mentioned in IEEE Standard 1624-2008 [1] is quite complex, making it difficult for manufacturing enterprises to use. Therefore, to establish an ORC evaluation framework suitable for local electromechanical product manufacturing enterprises (hereinafter referred to as manufacturing enterprises), a local enterprise investigation is carried out, and the ORC composition is determined on the basis of the investigation and “8341” reliability methodology [2] which divided the reliability assurance capability into four aspects: design analysis, control, test improvement, and management assurance. Then, a general framework of ORC’s characteristic indicators is established as its evaluation indicators system based on the composition of ORC. At last, the grey evaluation method is used to evaluate the ORC on the basis of an evaluation indicators system. The main contribution of this paper is to study the product reliability growth method from the perspective of enterprise capabilities, establish an ORC evaluation framework suitable for local manufacturing enterprises, and provide methodological references for the evaluation and improvement of the decision-making of ORC in manufacturing enterprises.

In this paper, Section 3 clarifies the composition of the ORC. Section 4 establishes the evaluation indicators framework according to the enterprise investigation and “8341” reliability methodology. Section 5 evaluates ORC by combining the three-parameter interval grey number and projection index function, and analyzes the evaluation results. Section 6 illustrates the evaluation and analysis method of ORC with a practical application. Section 7 draws the conclusions.

2. Literature Review

We review the current literature focusing on reliability research on electromechanical products or systems and the management systems of the enterprises.

Reliability research involves the entire lifecycle of electromechanical products, mainly divided into design reliability, manufacturing reliability, and operational reliability. Design reliability refers to the prediction and allocation of reliability during the product design and development phase [3]. Research in this area primarily focuses on the division of product functional structure [4] and allocation strategies [5]. Zhang et al. [6] proposed an automated calculation method based on a functionality multi-graph to identify the structure of mechanical systems, and calculated system reliability based on the factoring theorem-based algorithm. Wang et al. [7] used the Analytic Hierarchy Process (AHP) method to quantitatively allocate the reliability of the mechanical system during the conceptual design phase. Operation reliability assessment is the study of the degradation of product reliability during use [8]. Common evaluation indicators for functional reliability include MTBF, MTTR, MTTFF [9], etc. For performance reliability, performance indicators such as kinematic accuracy [10] are often selected. Adrian Gill et al. [11] used the Failure Mode, Effects and Criticality Analysis (FMECA) method to study the failure modes of the engine, which is also one of the most commonly used analysis methods in operation reliability. Jiang et al. [12] selected the Wiener process model to describe the degradation process caused by wear in mechanical systems, and carried out the reliability analysis and the whole lifetime calculations.

The inherent reliability of a product is formed during the manufacturing phase, and the manufacturing process capability determines the initial value of product reliability. Therefore, it is necessary to study the manufacturing process reliability. Enterprises have various management systems to ensure their manufacturing capabilities. From the literature, there are many kinds of management systems in enterprises, such as the quality management system (QMS), risk management system, etc. Hong Jin Jeong et al. [13] established a hybrid QMS by combining the advantages of the packaged system and customized system to increase the system’s recyclability, decrease the required system building time and improve the suitability of the enterprise’s work process. Jesús Álvarez-Santos et al. [14] investigated the relationship between the TQM and the safety management system (SMS), and results of the research indicate that there is a positive and significant relationship between TQM and SMS: companies operating in TQM environments tend to develop the SMSs. Parvaneh Saeidi et al. [15] attempted to examine the influence of Enterprise Risk Management (ERM) on Competitive Advantage (CA) by the moderating role of information technology dimensions including Information Technology (IT) strategy and Information Technology (IT) structure by using the Partial Least Squares Structural Equation Modelling (PLS-SEM) approach to analyze the data from the questionnaires. Camila Fabrício Poltronieri et al. [16] investigated the relationship between the maturity of the integrated management system and sustainable performance, and the results demonstrate that management system integration is one way to achieve sustainability, and that the management systems’ maturity level of integration would lead to a higher level of performance in the sustainability. Alfonso Hernandez-Vivanco et al. [17] investigate the relationship between the integration of management systems (IMSs) and sustainable innovation, and this is one of the first articles to relate IMSs, the adoption of cleaner production technologies, and sustainable product innovation, studying a sample of companies in countries of diverse economic backgrounds. For more research about management systems in enterprises, we review the literature of Łukasz Grudzień et al. [18], María Luisa Cantonnet et al. [19], and Lisha Zhu et al. [20].

However, the capability evaluation of those manufacturing management systems is mostly qualitative, with a few quantitative indicators related to the machining quality of parts, such as the Process Capability Index [21], Defect Rate [22], Sigma Level [23], etc. Albassam et al. [24] considered the uncertainty of the manufacturing process when calculating the process capability index and proposed a new neutrosophic process capability index (NPCI). Psarommatis et al. [25] proposed a zero-defect manufacturing (ZDM) framework that mostly focuses on qualitative manufacturing improvements, without introducing new quantitative indicators other than failure rates. Shokri et al. [26] studied the combination of green manufacturing and Lean Six Sigma (LSS), and introduced a new perspective of Green LSS integration theory.

The current research on the reliability evaluation of manufacturing systems is still based on the processing quality of products. Liao et al. [27] provided the principles for selecting the product key reliability characteristic (KRC), and evaluated the reliability capability of the manufacturing system based on the product processing results. Zheng et al. [28] proposed monitoring the key dimensional parameters that affect product reliability, and utilized these monitoring data as indicators to measure the manufacturing process. Friederich et al. pointed out in the overview article [29] that, due to the complexity of the manufacturing system, it is necessary to propose a method for comprehensively evaluating various influencing factors in the manufacturing system. In view of this, this paper reflects the reliability assurance process of manufacturing systems through the manufacturing enterprises’ ORC, and comprehensively estimates the manufacturing systems’ reliability assurance capability by evaluating the ORC of manufacturing enterprises.

The research on product reliability growth mentioned above is summarized, and the key points are tabulated in

Table 1. Key points of research on product reliability growth.

Reference	Research Object	Method	Targeted Stages	Research Perspective
[6]	Mechanical system	Analytic Hierarchy Process (AHP) method	Design process	Complex systems perspective
[10]	Engine	Failure Mode, Effects and Criticality Analysis (FMECA)	Design process	Product perspective
[23]	Industrial process	Neutrosophic process capability index (NPCI)	Manufacturing process	Manufacturing process perspective
[26]	Product key reliability characteristic (KRC)	Exponentially weighted moving average control chart	Manufacturing process	Product perspective
[27]	Products	Integrated built-in reliability (BIR) approach	Design process	Product perspective

.

According to Table 1, existing research on product reliability growth is mostly conducted from the perspective of products, subsystems, or a certain link in the product lifecycle, using methods such as fault analysis and manufacturing process capability evaluation to achieve reliability growth. There are few studies on the reliability growth of products from the perspective of enterprise capability assessment and decision-making.

Grey relational analysis theory offers significant advantages in evaluating complex systems with multiple factors, as it can handle both subjective and objective weights simultaneously [30]. Although there are still many new evaluation methods available, simpler and more feasible methods are more practical for enterprises. Zhang et al. [31] employed the fuzzy hierarchical analysis and grey relational degree method to evaluate the ecological environment management system, and verified the effectiveness and feasibility of this method. Xiang et al. [32] conducted a hierarchical and quantitative evaluation for the distribution system using the grey relational analysis method, and validated its rationality through comparison with multiple other methods. Ulutaş et al. [33] established an integrated grey Multiple Criteria Decision-Making framework which was applied in the supply chain management system, and the case results validated the accuracy and reliability of the method. Therefore, we proposed a quantitative evaluation framework for the organizational reliability capability of manufacturing enterprises based on grey relational analysis theory.

3. Organizational Reliability Capability (ORC)

3.1. Composition of the ORC

Reliability assurance work runs through the whole life cycle of products, and therefore, ORC can be regarded as a kind of process capability.

To evaluate the process capability, it is necessary to clarify the composition of the process. According to “8341” reliability methodology (this provides four types of assurance capabilities for the reliability assurance process, and its main system architecture is shown in Figure 1) and the enterprise investigation, the reliability assurance work process can be roughly decomposed into the design analysis process (DA), manufacturing control process (MC), testing improvement process (TI), and after-sale maintenance process (AM); all of the processes are managed by the management assurance work process (MA). Therefore, the composition of the reliability assurance work process is shown as Figure 2.

Process capability is restricted by two factors. The first factor is the normalization of the processes established through the management work. The second factor is the executive capability of the process limited by the actual situation of the enterprise. Hence, ORC can be decomposed into reliability management capability (RMC) and reliability executive capability (REC).

3.2. Reliability Management Capability (RMC)

RMC is the capability to standardize and supervise the reliability assurance work in the whole work process. Various reliability management work supervises the whole life cycle of the product, so as to ensure the standardization of each work process. Therefore, RMC is reflected by the implementation effect of each reliability management work, which is determined by the capabilities of various reliability management work. The capability composition of the management work is shown as Figure 3.

Management organizational capability is the capability to establish the reliability management organization structure. Therefore, the rationality of the organization structure is the evaluation standard of management organizational capability.

The reliability growth system is the foundation of product reliability growth, it contains a large amount of reliability work. A reasonable configuration of the reliability work is the guarantee of reliability growth’s effect. Hence, the evaluation of the capability of the reliability growth system can proceed from the rationality of the growth system.

Reliability documentation and reliability data provide the basis for the reliability assurance work. Reliability documentation consists of standards, specifications, various process files, etc. The capability in the reliability documentation is mainly reflected in the comprehensiveness and rationality of documentations. However, reliability data mainly include the fault data, production process inspection data, and so on. Thus, the management capability of reliability data is mainly reflected in the comprehensiveness and accuracy of the data, as well as in the rationality of the data processing method.

Reliability assessment work is aimed at assessing the reliability of the product and judging whether it meets the standards or not. Reliability assessment work is seriously affected by executors, methods, and objects. Thus, the reliability assessment capability can be evaluated from these three factors.

The checklist is the most common, simple, and effective reliability assurance method and aims at prompting staff to carry out reliability assurance work in accordance with the standard process and to prevent omissions. A reasonable checklist system is the main purpose of checklist establishment work.

The management work of the reliability assurance work process aims to establish the reliability assurance work process of products. The rationality of the reliability assurance work process is the main factor that affects its execution.

3.3. Reliability Executive Capability (REC)

Reliability executive capability refers to the ability of manufacturing enterprises to perform reliability related assurance work processes such as reliability design and manufacturing throughout the entire product lifecycle.

According to Section 3.1, the reliability assurance work process is decomposed into DA, MC, TI, and AM. Therefore, the reliability executive capability is determined by the executive capability of the four processes.

3.3.1. Design Analysis Process

The design analysis process contains a large amount of design and analysis work, and it can be roughly decomposed into six stages: technical decision-making, preliminary design, technical design, working drawing design, prototype trial production, and small batch trial production.

Every stage has its corresponding output, as shown in Figure 4. Firstly, in the technical decision-making stage, enterprises investigate the reliability requirements of the products and determine the reliability design plan. Then, in the preliminary design, the establishment of the reliability task report and FRACAS system are completed, as well as the selection of outsourced parts. In the technical design stage, the system-level reliability block diagram and mathematical model are established. Furthermore, FTA and FMEA are also carried out in this stage. In the working drawing design stage, the part-level reliability block diagram and mathematical model are established. In addition, the reliability prediction analysis of the products is carried out. Finally, in the prototype trial production and small batch trial production stages, enterprises use the reliability test to verify and improve the design results.

To summarize, each stage of the reliability design analysis process has its own output. The effect of the outputs can reflect the executive capability of the design analysis process.

3.3.2. Manufacturing Control Process

The manufacturing process of the product is restricted by the factors of 5M1E (man, machine, material, method, environment, and measurement). In the manufacturing process, the reliability knowledge of the staff determines the preciseness of their operation, and the operation capability determines their execution capability in the manufacturing process; the processing capability of manufacturing equipment determines the upper limit of the product’s theoretical accuracy; the material performance determines the processing method and final performance of products; the manufacturing method determines the difficulty of the manufacturing process and the proximity degree between product precision and ideal value; the environment affects the operation process of the staff and the accuracy of the processing equipment; and the measurement work determines the accuracy of the inspection process.

A fishbone diagram is used to extract the influencing factors in the manufacturing control process, as shown in Figure 5.

3.3.3. Testing Improvement Process

The reliability test is an important component distributed in the design and manufacturing process. Most of the reliability tests are carried out on test benches by relevant testers. Therefore, the execution capability of the test process is determined by the test benches and testers. The rationality of the test benches and the capability of the testers are the main factors affecting the execution capability of the reliability testing improvement process.

3.3.4. After-Sale Maintenance Process

For the enterprises, maintenance is an effective mean to ensure products’ operational reliability and to enhance users’ perception reliability (i.e., the products’ reliability perceived by users). The maintenance process includes fault feedback, maintenance task allocation, maintenance operation and spare parts, and other processes. Execution efficiency and effectiveness are the most important reflection of maintenance process execution capability. Therefore, the executive capability of the after-sale maintenance process can be evaluated in these two aspects.

4. Indicator Framework for ORC

According to Section 3, the specific evaluation indicators for ORC are proposed and shown in

Table 2. Evaluation indicator framework for organizational reliability capability.

Process Area Classes	Process Area	Specific Practice	Notes
Management capability	Management organizational capability	Rationality of management organization structure M1	Qualitative
	Establishing capability of reliability growth system	Rationality of reliability growth system M2	Qualitative
	Establishing capability of reliability documentations	Comprehensiveness of management documents category M3	Qualitative
		Rationality of management files M4	Qualitative
	Reliability data management capability	Rationality of data source M5	Qualitative
		Comprehensiveness of data category M6	Integrity
		Rationality of data processing method M7	Qualitative
	Reliability assessment management capability	Rationality of assessment panel composition M8	Qualitative
		Rationality of review objects’ determination M9	Qualitative
		Rationality of assessment technology M10	Qualitative
	Establishing capability of checklist system	Rationality of checklist system M11	Qualitative
	Establishing capability of reliability assurance process	Rationality of design process M12	Qualitative
		Rationality of manufacturing process M13	Qualitative
		Rationality of test system M14	Qualitative
		Rationality of after-sale process M15	Qualitative
Executive capability	Reliability design and analysis process execution capability	Normality of design process execution E1	Qualitative
		Accuracy of requirement analysis E2	Satisfaction
		Rationality of reliability design documents E3	Qualitative
		Rationality of outsourcing parts selection E4	Qualitative
		Accuracy of reliability block diagram and mathematical model E5	Qualitative
		Rationality of FRACAS system E6	Qualitative
		Accuracy of reliability analysis report E7	Qualitative
		Effectiveness of prototype trail-production process E8	Growth rate
		Effectiveness of small batch trail-production E9	Growth rate
	Reliability manufacturing control process execution capability	Normality of manufacturing process execution E10	Qualitative
		Reliability consciousness of staff E11	Qualitative
		Operational capability of staff E12	Qualitative
		Process capability index of machining equipment E13	Cpk
		Reliability of the materials E14	Qualitative
		Rationality of processing technique E15	Qualitative
		Reliability of manufacturing environment E16	Qualitative
		Rationality of measurement method E17	Qualitative
		Rationality of measuring tool selection E18	Qualitative
	Reliability test improvement capability	Normality of testing process execution E19	Qualitative
		Capability of the testers E20	Qualitative
		Rationality of test benches E21	Qualitative
	Reliability after-sales maintenance capability	Normality of after-sale process execution E22	Qualitative
		Maintenance efficiency E23	MTTR
		Maintenance effectiveness E24	Recurrence probability

.

The indicator system consists of both qualitative and quantitative indicators. The qualitative indicators are evaluated by the experts, and the quantitative indicators are evaluated by specific numerical value. This paper introduced two quantitative indicators of integrity and recurrence probability, in addition to the four traditional quantitative indicators of customer satisfaction, growth rate, process capability index (Cpk), and mean time to repair (MTTR). The calculation methods are shown in

Table 3. Calculation methods for the quantitative index.

Indicator Name	Calculation Method	Symbol Meaning	Range	Indicator Meaning
Integrity $I$	$I={\displaystyle \frac{N_i}{N_s}}$	$N_i$—The number of the data categories is identical to the standard $N_s$—The total number of the process or document categories	[0, 1]	Integrity indicates the same ratio between an enterprise’s reliability data categories and the standard reliability data categories. The standard number of data categories are determined by experts.
Satisfaction $S_a$	$S_a={\displaystyle \frac{S_n}{N_r}}$	$S_n$—Number of satisfied users $N_r$—Total number of users	[0, 1]	Satisfaction indicates the percentage of users who are satisfied with the product’s reliability.
Growth rate $G_r$	$G_r={\displaystyle \frac{MTB{F}^{*}-MTBF}{MTBF}}$	$MTB{F}^{*}$—MTBF after the prototype test $MTBF$—MTBF before the prototype test	[0, 1]	Growth rate indicates the reliability improvement ratio after the prototype test.
Process capability index $C_{pk}$	$C_{pk}={\displaystyle \frac{T-2\epsilon }{6s}}$	$T$—Tolerance difference between its upper and lower limit $\epsilon$—Central offset $s$—Standard deviation of sample processing error	[0.67, 2]	Process capability index indicates the processing capability of the manufacturing equipment.
Mean time to repair $MTTR$	$MTTR={\displaystyle \frac{{\displaystyle \sum _{i=1}^n{T}_{Mi}}}{n}}$	$T_{Mi}$—The ith maintenance time $n$—Total account of the faults	$(T_{M\min },T_{M\max })$	Mean time to repair indicates the mean time for after-sale maintenance work.
Recurrence probability $P_r$	$P_r={\displaystyle \frac{N_r}{n}}$	$N_r$—Number of faults that recur within a specified period of time after maintenance $n$—The total number of faults	[0, 1]	Recurrence probability indicates the possibility that the same fault reappears in a short time after maintenance. The time is determined by experts according to the severity of the assessment.

.

5. Evaluation of Enterprises’ ORC

The reliability forming process of products is complex; every link can affect the final reliability of the products. In this paper, we assume the relationships among production links and the product’s reliability as a grey system, and it is feasible and appropriate to use the grey evaluation method to evaluate ORC.

In this paper, a three-parameter interval grey number is first used to represent the fuzzy information of each qualitative and quantitative indicator of ORC. Then, we evaluate the relative importance of each ORC indicator, establish a complementary matrix, and calculate the weights of each indicator. Finally, the projection index function is used to project the evaluation values of the ORC high-dimensional indicators along the weight vector direction into a low dimensional space for synthesis, obtaining the final evaluation result of the enterprise ORC. The flowchart of the evaluation method is shown in Figure 6.

5.1. Evaluation Matrix of the ORC

In cybernetics, the depth of color is usually used to indicate the clarity degree of the information. The system with partially known information and partially unknown information is called a “grey system” [34]. A grey number is a number for which the range is known but the value is unknown. The three-parameter interval grey number is the interval grey number for which the value with the greatest selection possibility is known, as shown below.

$$a(\otimes )\in [\underset{¯}{a},\tilde{a},\overline{a}](0\le \underset{¯}{a}\le \tilde{a}\le \overline{a})$$

(1)

where $\tilde{a}$ is the value of $a(\otimes )$ with greatest selection possibility, $\underset{¯}{a}$ is the lower limit of $a(\otimes )$, and $\overline{a}$ is the upper limit.

This paper uses three-parameter interval grey numbers to represent the evaluation values of various ORC evaluation indicators in manufacturing enterprises, in order to provide a fuzzy representation of the level of ORC evaluation indicators.

For the qualitative indicators, their evaluation values are represented by the degree of membership to an ideal situation. Evaluation values are scored by $k$ experts in the form of a three-parameter interval grey number according to enterprises’ actual situation.

The evaluation value of the ith qualitative indicator scored by the jth expert is as shown below.

$${\alpha }_{ij}(\otimes )\in [{\underset{¯}{\alpha }}_{ij},{\tilde{\alpha }}_{ij},{\overline{\alpha }}_{ij}];(0\le {\underset{¯}{\alpha }}_{ij}\le {\tilde{\alpha }}_{ij}\le {\overline{\alpha }}_{ij}\le 1;i=1,2,\cdots ,39;j=1,2,\cdots ,k)$$

(2)

Mean values of each qualitative indicator are used to represent the final evaluation values, shown as Equation (3).

$$\begin{array}{l}{\overline{\alpha }}_i(\otimes )={\displaystyle \frac{{\displaystyle \sum _{j=1}^k{\alpha }_{ij}(\otimes )}}{k}}\in \left[{\overline{\underset{¯}{\alpha }}}_i,{\overline{\tilde{\alpha }}}_i,{\overline{\overline{\alpha }}}_i\right]\\ {\overline{\underset{¯}{\alpha }}}_i={\displaystyle \frac{{\displaystyle \sum _{j=1}^k{\underset{¯}{\alpha }}_{ij}}}{k}},{\overline{\tilde{\alpha }}}_i={\displaystyle \frac{{\displaystyle \sum _{j=1}^k{\tilde{\alpha }}_{ij}}}{k}}\\ {\overline{\overline{\alpha }}}_i={\displaystyle \frac{{\displaystyle \sum _{j=1}^k{\overline{\alpha }}_{ij}}}{k}},(i=1,2,\cdots ,39)\end{array}$$

(3)

The row matrix $\overline{Q_l}(\otimes )$ is used to express the qualitative indicators, so the evaluation matrix of the qualitative indicators is as shown below according to Table 2.

$$\overline{Q_l}(\otimes )=[{\overline{M}}_1(\otimes )\cdots {\overline{M}}_5(\otimes ),{\overline{M}}_7(\otimes )\cdots {\overline{M}}_{18}(\otimes ),{\overline{E}}_1(\otimes ),{\overline{E}}_3(\otimes )\cdots {\overline{E}}_7(\otimes ),{\overline{E}}_{10}(\otimes )\cdots {\overline{E}}_{12}(\otimes ),{\overline{E}}_{14}(\otimes )\cdots {\overline{E}}_{22}(\otimes )]$$

(4)

For the quantitative indicators, their evaluation values are calculated according to enterprises’ actual situations. Before establishing the evaluation matrix, the calculation values need to be normalized by Equations (5) and (6), and the normalized values are used to represent the evaluation values of quantitative indicators.

The benefit-oriented evaluation indicators are as follows:

$$x_n={\displaystyle \frac{x-x_{\min }}{x_{\max }-x_{\min }}}$$

(5)

The cost-based evaluation indicators are as follows:

$$x_n={\displaystyle \frac{x_{\max }-x}{x_{\max }-x_{\min }}}$$

(6)

where $x$ is the evaluation value of the quantitative indicator, $x_{\min }$ is the lower limit of the quantitative indicator, and $x_{\max }$ is the upper limit of the quantitative indicator.

In this paper, the mean time to repair (MTTR) and recurrence probability are the cost-based indicators, and the other indicators are the benefit-oriented indicators.

The three-parameter interval grey number represents a specific number, when its upper limit and lower limit are equal to the value with the greatest selection possibility. Therefore, the quantitative indicators of ORC can be converted into a three-parameter interval grey number, as shown below.

$${\overline{\alpha }}_i(\otimes )=[{\alpha }_i,{\alpha }_i,{\alpha }_i],(i=1,2,\cdots 41)$$

(7)

Similarly, the row matrix $\overline{Q_t}(\otimes )$ is used to represent the quantitative indicators, and the valuation matrix of quantitative indicators is as shown below.

$$\overline{Q_t}(\otimes )=[{\overline{M}}_6(\otimes ),{\overline{E}}_2(\otimes ),{\overline{E}}_8(\otimes ),{\overline{E}}_9(\otimes ),{\overline{E}}_{13}(\otimes ),{\overline{E}}_{23}(\otimes ),{\overline{E}}_{24}(\otimes )]$$

(8)

Integrating $\overline{Q_l}(\otimes )$ and $\overline{Q_t}(\otimes )$, the evaluation matrix of ORC is obtained as below.

$$\overline{\alpha }(\otimes )=[\overline{Q_l}(\otimes ),\overline{Q_t}(\otimes )]=\left[\begin{array}{c}{\overline{\underset{¯}{M}}}_1\\ {\overline{\tilde{M}}}_1\\ {\overline{\overline{M}}}_1\end{array}\cdots \begin{array}{c}{\overline{\underset{¯}{M}}}_{18}\\ {\overline{\tilde{M}}}_{18}\\ {\overline{\overline{M}}}_{18}\end{array},\right.\left.\begin{array}{c}{\overline{\underset{¯}{E}}}_1\\ {\overline{\tilde{E}}}_1\\ {\overline{\overline{E}}}_1\end{array}\cdots \begin{array}{c}{\overline{\underset{¯}{E}}}_{23}\\ {\overline{\tilde{E}}}_{23}\\ {\overline{\overline{E}}}_{23}\end{array}\right]$$

(9)

5.2. Weight Vector of the Indicators

The weight vector of each indicator is established based on expert evaluation.

The relative importance of each indicator is evaluated by experts according to

Table 4. Fuzzy judgment scale.

Order	Importance of the Indicators	${\mathit{a}}_{\mathit{i}\mathit{j}}$
1	Indicator $i$ is as important as $j$	0.5
2	Indicator $i$ is slightly more important $j$	0.6
3	Indicator $i$ is more important than $j$	0.7
4	Indicator $i$ is extremely more important than $j$	0.8
5	Indicator $i$ is absolutely more important than $j$	0.9
6	Indicator $j$ is more important than $i$	1 − $a_{ij}$

, and then the mean values of corresponding evaluation values are converted into the matrix, so the two complementary matrices ${\tilde{A}}_M$ and ${\tilde{A}}_E$ are obtained as below, representing the complementary matrices of RMC and REC, respectively.

$${\tilde{A}}_i=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n_i}\\ a_{21}& a_{22}& \cdots & a_{2n_i}\\ ⋮& ⋮& \ddots & ⋮\\ a_{n_{i}1}& a_{n_{i}2}& \cdots & a_{n_i{n}_i}\end{array}\right],(i=M,E)$$

(10)

where $n_i$ is the indicator number of RMC or REC.

The weight vector is calculated by Equation (11) according to the complementary matrix as follows:

$$W_i=(w_{i1},w_{i2},\cdots ,w_{i{n}_i}),w_{ik}={\beta }^{{\displaystyle \frac{1}{n_i}}{\displaystyle \sum _{j=1}^{n_i}{\overline{a}}_{kj}}}/{\displaystyle \sum _{l=1}^{n_i}{\beta }^{\frac{1}{n_i}{\displaystyle \sum _{j=1}^{n_i}{\overline{a}}_{lj}}}}$$

(11)

where $\beta$ is the resolution parameter for the weight allocation and represents the experts’ resolution capability, $W_i$ is the weight vector of RMC or REC, $w_{ik}$ means the kth indicator’s weight of the RMC or REC, and the range of $j$ and $k$ is $[1,n_i]$.

5.3. Ideal Evaluation Matrix and Critical Evaluation Matrix

After obtaining the ORC indicator evaluation value and weight vector, it is necessary to use the projection index function to project the multidimensional ORC indicator into a low dimensional space, and use the projection result to represent the degree of closeness between the evaluation indicator and its ideal and critical values. Therefore, it is necessary to first establish an ideal and critical indicator evaluation matrix.

According to the evaluation of the qualitative indicators, their ideal value should be one, which means that the indicators are absolutely ideal; on the contrary, their critical value should be zero. Therefore, the ideal and critical evaluation matrices for qualitative indicators are shown in Equations (12) and (13).

$$D{X}^{+}(\otimes )=\left[\begin{array}{c}{\overline{\underset{¯}{DX}}}^{+}\\ {\overline{\widetilde{DX}}}^{+}\\ {\overline{\overline{DX}}}^{+}\end{array}\right]=\left[\begin{array}{cccc}1& 1& \cdots & 1\\ 1& 1& \cdots & 1\\ 1& 1& \cdots & 1\end{array}\right]$$

(12)

$$D{X}^{-}(\otimes )=\left[\begin{array}{c}{\overline{\underset{¯}{DX}}}^{-}\\ {\overline{\widetilde{DX}}}^{-}\\ {\overline{\overline{DX}}}^{-}\end{array}\right]=\left[\begin{array}{cccc}0& 0& \cdots & 0\\ 0& 0& \cdots & 0\\ 0& 0& \cdots & 0\end{array}\right]$$

(13)

Since the range of quantitative indicators are (0,1), for the benefit-oriented indicators, their ideal values should be one and the critical values should be zero, while the cost-oriented indicators are the opposite. The ideal and critical evaluation matrix of quantitative indicators are shown in Equations (14) and (15).

$$D{L}^{+}(\otimes )=\left[\begin{array}{c}{\overline{\underset{¯}{DL}}}^{+}\\ {\overline{\widetilde{DL}}}^{+}\\ {\overline{\overline{DL}}}^{+}\end{array}\right]=\left[\begin{array}{ccccccc}1& 1& 1& 1& 1& 0& 0\\ 1& 1& 1& 1& 1& 0& 0\\ 1& 1& 1& 1& 1& 0& 0\end{array}\right]$$

(14)

$$D{L}^{-}(\otimes )=\left[\begin{array}{c}{\overline{\underset{¯}{DL}}}^{-}\\ {\overline{\widetilde{DL}}}^{-}\\ {\overline{\overline{DL}}}^{-}\end{array}\right]=\left[\begin{array}{ccccccc}0& 0& 0& 0& 0& 1& 1\\ 0& 0& 0& 0& 0& 1& 1\\ 0& 0& 0& 0& 0& 1& 1\end{array}\right]$$

(15)

Integrating the corresponding matrix of quantitative and qualitative indicators, the ideal evaluation matrix and critical evaluation matrix of the evaluation indicators can be obtained as below.

$${\alpha }^{+}(\otimes )=\left[\begin{array}{cc}D{X}^{+}(\otimes )& D{L}^{+}(\otimes )\end{array}\right]$$

(16)

$${\alpha }^{-}(\otimes )=\left[\begin{array}{cc}D{X}^{-}(\otimes )& D{L}^{-}(\otimes )\end{array}\right]$$

(17)

5.4. Comprehensive Evaluation

According to the projection pursuit theory, the best projection vector of evaluation indicators is their weight vector. Regarding the weight vector $W$ as the projection direction, RMC and REC can be evaluated separately by Equation (18).

$$Y_i={\displaystyle \frac{{\left[(1-\gamma )Y_{1i}^{-}+Y_{2i}^{-}+\gamma Y_{3i}^{-}\right]}^{1/2}}{{\left[(1-\gamma )Y_{1i}^{-}+Y_{2i}^{-}+\gamma Y_{3i}^{-}\right]}^{1/2}+{\left[(1-\gamma )Y_{1i}^{+}+Y_{2i}^{+}+\gamma Y_{3i}^{+}\right]}^{1/2}}}$$

(18)

where $\gamma$ is the preference coefficient of the decision-maker, and $\gamma \in [0,1]$. When the decision-makers are radical, $\gamma =0$; when the decision-makers have a moderate mentality, $\gamma =1/2$; and when the decision-makers are cautious, $\gamma =1$.

$$\begin{array}{cc}& Y_{1i}^{+}={{\displaystyle \sum _{i=1}^{n_i}{w}_{ik}\left({\overline{\underset{¯}{\alpha }}}_{ik}-{\overline{\underset{¯}{\alpha }}}_{ik}^{+}\right)}}^2,Y_{1i}^{-}={{\displaystyle \sum _{i=1}^{n_i}{w}_{ik}\left({\overline{\underset{¯}{\alpha }}}_{ik}-{\overline{\underset{¯}{\alpha }}}_{ik}^{-}\right)}}^2\\ & Y_{2i}^{+}={{\displaystyle \sum _{i=1}^{n_i}{w}_{ik}\left({\overline{\tilde{\alpha }}}_{ik}-{\overline{\tilde{\alpha }}}_{ik}^{+}\right)}}^2,Y_{2i}^{-}={{\displaystyle \sum _{i=1}^{n_i}{w}_{ik}\left({\overline{\tilde{\alpha }}}_{ik}-{\overline{\tilde{\alpha }}}_{ik}^{-}\right)}}^2\\ & Y_{3i}^{+}={{\displaystyle \sum _{i=1}^{n_i}{w}_{ik}\left({\overline{\overline{\alpha }}}_{ik}-{\overline{\overline{\alpha }}}_{ik}^{+}\right)}}^2,Y_{3i}^{-}={{\displaystyle \sum _{i=1}^{n_i}{w}_{ik}\left({\overline{\overline{\alpha }}}_{ik}-{\overline{\overline{\alpha }}}_{ik}^{-}\right)}}^2\end{array}$$

(19)

where $w_{ik}$ is the RMC’s or REC’s weight of the kth indicator, ${{\alpha }_{ik}}^{+}(\otimes )$ is the RMC’s or REC’s ideal evaluation vector of the kth indicator, ${{\alpha }_{ik}}^{-}(\otimes )$ is the critical evaluation vector, and ${\alpha }_{ik}(\otimes )$ is the evaluation vector with the greatest selection possibility.

$Y_i$ shows the relative approaching degree between the evaluation vectors and their ideal evaluation vectors. The closer $Y_i$ is to one means the closer the evaluation vector is to the ideal evaluation vector, and the better the capability is.

This paper considers RMC and REC as equally important, so the evaluation result of ORC is expressed by the arithmetic average of RMC and REC as below:

$$Y={\displaystyle \frac{Y_M+Y_E}{2}}$$

(20)

5.5. Result Analysis and Weak Links Screening

In this paper, the evaluation results are analyzed from three aspects: ORC, process area classes (i.e., RMC and REC), and specific practices.

The comprehensive evaluation results of ORC indicate the approaching degree between the evaluation vector and ideal evaluation vector. When $Y$ is greater than 0.5, ORC is closer to the ideal situation, and on the contrary, ORC is closer to the critical situation. Following the form of Table 4, the criteria for evaluating the results are established, as shown in

Table 5. Judging criteria for the result of ORC.

Order	Evaluation Result	Range
1	Absolutely close to the ideal evaluation vector	(0.9, 1)
2	Extremely close to the ideal evaluation vector	(0.8, 0.9)
3	Very close to the ideal evaluation vector	(0.7, 0.8)
4	Close to the ideal evaluation vector	(0.6, 0.7)
5	Slightly close to the ideal evaluation vector	(0.5, 0.6)

.

Referring to Table 5, the evaluation results can be analyzed, and the issue of whether the capabilities are good or bad can be judged.

To analyze the specific practice indicators, it is necessary to set up additional evaluation criteria. For each qualitative indicators, 0.5 is used as the evaluation criterion to represent the critical line between the ideal and not ideal state of the indicator, and the evaluation values with greatest selection possibility of qualitative indicators are used as their evaluation values. For the quantitative indicators, their evaluation criteria should be evaluated by experts according to the actual situation and reliability objectives of enterprises.

According to the evaluation value of specific practice indicators, their relative difference from the corresponding criteria are calculated by Equation (20). Finally, the relative differences are sorted and the weak links are extracted according to the Pareto principle.

$$D={\displaystyle \frac{\tilde{\alpha }-{\alpha }^{\prime }}{{\alpha }^{+}-{\alpha }^{\prime }}}$$

(21)

where $\tilde{\alpha }$ means the evaluation value with greatest selection possibility, ${\alpha }^{\prime }$ means the criteria of indicators, and ${\alpha }^{+}$ means the ideal values.

6. Practical Application

6.1. ORC Comprehensive Evaluation of the CNC Machine Tool Manufacturing Enterprise

The CNC machine tool is the typical electromechanical product. So, this paper takes the ORC evaluation of a CNC machine tool manufacturing enterprise as an example to illustrate the evaluation and analysis method. The calculation process is illustrated by the calculation of RMC.

Firstly, the value of each quantitative indicator is calculated according to the actual situation of the enterprise, as shown below:

$${\overline{M}}_6(\otimes )=\left[0.89,0.89,0.89\right]$$

(22)

Each qualitative indicator is evaluated by five experts in the form of a three-parameter interval grey number, shown as follows:

$$\begin{array}{l}{\overline{M}}_1(\otimes )=\left[0.854,0.87,0.884\right], {\overline{M}}_2(\otimes )=\left[0.78,0.808,0.848\right], {\overline{M}}_3(\otimes )=\left[0.86,0.872,0.884\right],\\ {\overline{M}}_4(\otimes )=\left[0.818,0.836,0.864\right], {\overline{M}}_5(\otimes )=\left[0.83,0.852,0.874\right], {\overline{M}}_7(\otimes )=\left[0.832,0.85,0.882\right],\\ {\overline{M}}_8(\otimes )=\left[0.78,0.812,0.848\right], {\overline{M}}_9(\otimes )=\left[0.79,0.84,0.874\right], {\overline{M}}_{10}(\otimes )=\left[0.772,0.816,0.854\right],\\ {\overline{M}}_{11}(\otimes )=\left[0.768,0.802,0.822\right], {\overline{M}}_{12}(\otimes )=\left[0.826,0.856,0.88\right], {\overline{M}}_{13}(\otimes )=\left[0.812,0.83,0.856\right],\\ {\overline{M}}_{14}(\otimes )=\left[0.844,0.86,0.882\right], {\overline{M}}_{15}(\otimes )=\left[0.798,0.818,0.852\right]\end{array}$$

(23)

So, the evaluation matrix of RMC can be obtained as follows by integrating the qualitative and quantitative evaluation indicators.

$$\overline{M}(\otimes )=\left[\begin{array}{cc}\begin{array}{ccc}\begin{array}{ccc}\begin{array}{c}0.854\\ 0.87\\ 0.884\end{array}& \begin{array}{c}0.78\\ 0.808\\ 0.848\end{array}& \begin{array}{c}0.86\\ 0.872\\ 0.884\end{array}\end{array}& \begin{array}{ccc}\begin{array}{c}0.818\\ 0.836\\ 0.864\end{array}& \begin{array}{c}0.83\\ 0.852\\ 0.874\end{array}& \begin{array}{c}0.89\\ 0.89\\ 0.89\end{array}\end{array}& \begin{array}{c}0.832\\ 0.85\\ 0.882\end{array}\end{array}& \begin{array}{ccc}\begin{array}{cc}\begin{array}{c}0.78\\ 0.812\\ 0.848\end{array}& \begin{array}{ccc}\begin{array}{c}0.79\\ 0.84\\ 0.874\end{array}& \begin{array}{c}0.772\\ 0.816\\ 0.854\end{array}& \begin{array}{c}0.768\\ 0.802\\ 0.822\end{array}\end{array}\end{array}& \begin{array}{ccc}\begin{array}{c}0.826\\ 0.856\\ 0.88\end{array}& \begin{array}{c}0.812\\ 0.83\\ 0.856\end{array}& \begin{array}{c}0.844\\ 0.86\\ 0.882\end{array}\end{array}& \begin{array}{c}0.798\\ 0.818\\ 0.852\end{array}\end{array}\end{array}\right]$$

(24)

Then, the complementary matrix $A$ can be established by the experts according to the fuzzy scale in Table 3. Setting $\beta =0.8$, the weight vector of the indicators can be calculated by Equation (11).

$$\begin{array}{l}W=[0.9871,0.9998,1.0160,1.0363,0.9814,0.9971,1.0087,1.0313,1.0011,\\ 1.0191,0.9768,1.0216,0.9991,1.0137,0.9946,0.9653,0.9909,0.9600]\end{array}$$

(25)

The ideal and critical evaluation matrix is established, as shown below.

$$M^{+}(\otimes )=\left[\begin{array}{cc}\begin{array}{cccc}\begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{ccc}\begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}\end{array}& \begin{array}{ccc}\begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}\end{array}& \begin{array}{ccc}\begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}\end{array}\end{array}& \begin{array}{cc}\begin{array}{cccc}\begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}\end{array}& \begin{array}{c}1\\ 1\\ 1\end{array}\end{array}\end{array}\right]$$

(26)

$$M^{-}(\otimes )=\left[\begin{array}{cc}\begin{array}{cccc}\begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{ccc}\begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}\end{array}& \begin{array}{ccc}\begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}\end{array}& \begin{array}{ccc}\begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}\end{array}\end{array}& \begin{array}{cc}\begin{array}{cccc}\begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}\end{array}& \begin{array}{c}0\\ 0\\ 0\end{array}\end{array}\end{array}\right]$$

(27)

Finally, setting $\gamma =0.5$, the evaluation result of RMC is calculated by Equation (18).

$$Y_M=0.8406$$

(28)

Similarly, the evaluation result of the REC can be calculated, as shown below.

$$Y_E=0.7346$$

(29)

According to the evaluation results of RMC and REC, the evaluation value of ORC is calculated as follows.

$$Y={\displaystyle \frac{Y_M+Y_E}{2}}=0.7876$$

(30)

6.2. Results Analysis

According to the evaluation results, $Y$ is in the range of (0.7, 0.8), and indicates that the ORC is very close to the ideal situation. $Y_M$ and $Y_E$ are in the ranges of (0.8, 0.9) and (0.7, 0.8), respectively, and indicate that RMC is extremely close to the ideal situation and REC is very close to the ideal situation.

Taking ${\alpha }^{\prime }=0.5$ as the evaluation criterion of qualitative indicators, the evaluation criteria of RMC’s each quantitative indicator can be determined by the experts, as shown below.

$${{\alpha }_{M6}}^{\prime }=0.8$$

(31)

The relative difference of each indicator according to the evaluation criteria is calculated as shown in

Table 6. Relative difference for each indicator.

M1	M2	M3	M4	M5	M6	M7	M8
0.74	0.616	0.744	0.672	0.704	0.45	0.7	0.624
M9	M10	M11	M12	M13	M14	M15
0.68	0.632	0.604	0.712	0.66	0.72	0.636

.

Then, the weak links of ORC are screened by using the Pareto principle, as shown in Figure 7.

From the dashed box in Figure 7, the weak links of RMC in the machine tool enterprises are the comprehensiveness of data category, rationality of the checklist system, rationality of the reliability growth system and rationality of the assessment panel composition. Therefore, the improvement of RMC can start with these four aspects.

Based on the evaluation decision results, corresponding correction methods have been provided for the machine tool manufacturing enterprise through analysis. These methods include data type adjustment for collection and recording, file archiving strategy revision and the digitization of important data, etc. After taking relevant measures, the reliability of CNC machine tools has significantly increased, and the MTBF of key products can exceed 2000 h. Specifically, although the reliability growth of CNC machine tools is the result of multiple intervention measures, the results of its reliability growth still demonstrate the effectiveness of the aforementioned enterprise ORC evaluation indicator framework and evaluation method.

6.3. Method Comparison

At present, commonly used fuzzy evaluation methods include cloud models [35], interval intuitionistic trapezoidal fuzzy numbers [36], multi-criteria decision-making methods (MCDM) [37], etc. Compared with the fuzzy evaluation method based on three-parameter interval grey numbers used in this paper, the above method may be able to obtain more accurate results. However, for the application of evaluation methods in manufacturing enterprises, the calculation process in this paper is simpler and more efficient, and can quickly and efficiently perform the evaluation of and correction decision-making for the enterprise’s ORC. According to the actual application results, the effectiveness of the ORC evaluation indicator framework and evaluation method has been verified. Therefore, the evaluation method based on the three-parameter interval grey number used in this paper is more suitable for enterprises to evaluate and make corrective decisions on the ORC, which can reduce the time cost of enterprise capability evaluation and self-correction, and improve efficiency.

In addition, existing research on product reliability growth is mostly conducted from the perspective of products, subsystems, or a certain link in the product lifecycle, using methods such as fault analysis and manufacturing process capability evaluation to achieve reliability growth. There are few studies on the reliability growth of products from the perspective of enterprise capability assessment and decision-making. Therefore, this paper is innovative in studying product reliability growth from the perspective of enterprise ORC evaluation and improvement decision-making.

Comparing with IEEE standards, there are over 130 evaluation indicators in IEEE standard 1624-2008 [1], which are mainly aimed at organizations of electrical/electronic components or products. The indicator content is not entirely suitable for local manufacturing enterprises. In addition, for enterprises, the evaluation of 130 indicators is relatively complex and requires time and effort. Therefore, the indicator framework in IEEE standard 1624-2008 [1] is not suitable for local manufacturing enterprises to use. The ORC evaluation indicator framework proposed in this paper is based on the research results of local electromechanical product enterprises, with a total of 39 indicators. Therefore, the ORC evaluation indicator framework proposed in this paper is more suitable for local manufacturing enterprises.

7. Conclusions

This paper establishes the evaluation indicator framework for ORC first on the basis of the enterprise investigation and “8341” reliability methodology. Then, the ORC is evaluated and projected by combining the three-parameter interval grey number and projection index function together. Finally, the ORC evaluation indicator framework and evaluation method were explained through practical application, the weak links of ORC were extracted according to the evaluation results, and the effectiveness was verified by the growth of CNC machine tools’ MTBF.

Practical application has proven that a comprehensive evaluation of ORC can help enterprises understand the current situation of their ORC, sort the weaknesses, and make decisions on the improvement of the ORC. Compared with existing fuzzy evaluation methods, the comprehensive evaluation method adopted in this paper requires less time and complexity compared to other methods. For enterprises, the method proposed in this paper is more suitable for the characteristics of simplicity, directness, and efficiency that enterprises need. In summary, the comprehensive evaluation of ORC proposed in this paper is an effective measure to guide the reliability growth of products.

Practical application has proven that the evaluation indicator framework and evaluation method proposed in this paper can help enterprises understand the current situation and shortcomings of their ORC, and make decisions on improving ORC. Compared with IEEE’s evaluation indicator framework and existing evaluation methods, the framework and method proposed in this paper are simpler, more direct, and more suitable for manufacturing enterprises to conduct a self-assessment and self-determination of ORC. In summary, the framework and methods proposed in this paper are effective measures to guide the reliability growth of electromechanical products.

Meanwhile, further research can also make some detailed improvements to the evaluation indicator framework and evaluation methods of ORC. Firstly, the evaluation indicator framework can be fine-tuned according to the current situation of the enterprise, making it more universal and not only applicable to manufacturing enterprises. Secondly, enterprises can choose appropriate evaluation methods based on their own situation and combine them with the indicator framework of this paper for evaluation, which makes the evaluation indicator framework more suitable for enterprises. The detailed adjustments to the framework and methods will not affect the implementation process of the method.