In this section, the research framework was applied to a real case study. The company produces electromechanical breakers, it has 10 production lines distributed on 5 different plants, and a production capacity of about 1.3 million pcs per year.
For the application, a production line for the assembly of power transformers was chosen.
4.1. Implementation of Step 1
In this part, input variables need to be defined. The production cycle of the power transformers assembly line has the following operations (
Figure 3):
Assembling the primary and secondary cores in the case.
Soldering the input cable.
Application of resin in a vacuum chamber mixer.
Resin drying.
Assembling the output connection.
Insulation test.
Functional test (check the output current).
In this manufacturing process, the vacuum mixer machine is the most critical in terms of impact on the product quality rate; the impact of this operation on the total waste generated by the manufacturing process is about 85%, while the remaining part is due to soldering (operation #2). This operation concerns the manufacture of a resin layer between the cores of the current transformer that should guarantee the electrical insulation of the product, and it is made by a vacuum mixer machine, where the working cycle is composed of three phases: loading, degassing and dosing.
During this operation, the vacuum pressure in the tanks and in the chamber is significant for the success of the phase and then for the product quality, because the vacuum casting needs to create an insulating layer without voids, bubbles, and/or porosity (
Figure 4).
After the final assembly, the insulation of the product is tested at a high voltage according to the customer’s specifications.
Since the product quality in our case study is related to operation #3, this study focused on the maintenance in the mixer machine that runs this operation, and specifically of the vacuum pumps (VPs) that perform the working cycle.
Vacuum pumps have the function of decreasing the pressure of a gas in a certain volume. Consequently, they must remove some gas particles from the volume.
The VPs in our mixer machine consisted of two primary parts: an electric motor and a vacuum pump. The predictive maintenance and condition monitoring of vacuum pumps is the main topic of many papers; Mooney and Shelley [
61] summarize how pump predictive maintenance evolves through the use of networked monitoring systems. Konishi and Yamasawa [
70] considered the issue of process by-products accumulating in the pumping system. Deposits within the pump cause friction, exceeding current limits and blocking the pump; an ARMAX model was used to predict the vacuum pump motor current. Twiddle et al. [
62] tested the condition monitoring of a dry vacuum pump through a fuzzy logic scheme to identify mechanical inefficiency and exhaust system blockage through exhaust pressure. Butler et al. [
63] uses artificial neural networks to estimate the degradation and RUL (remaining useful life) of the pump, using as inputs pump process data regarding all the steps in the pump cycle. Muhaimin and Ghazali [
64] described a method for detecting vacuum leaks through thermography, based on IR thermography image analysis to detect leaks represented by a cold spot. Vinogradov and Kostrin [
71] investigate the aspects related to oil characteristics to be monitored to establish the correct frequency of oil replacement; a visual color scale is used to determine the oil condition and maintenance action. In contrast to these results, our study used predictive maintenance as a methodology to investigate possible pump failure after the IA model predicted a deviation in product quality connected to a difference of one or more parameters with respect to the normal operating ranges of the machine.
The machine that performs operation#3 in our production cycle is equipped with 4 pumps (
Figure 5):
Resin drum pump VP1.
Resin tank pump VP2.
Hardener tank pump VP3.
Dosing chamber pump VP4.
Figure 5.
Vacuum pumps in the mixer machine.
Figure 5.
Vacuum pumps in the mixer machine.
The vacuum level in the loading, degassing, and dosing phases ensures that the mixture is able to guarantee the insulation of the product. The lack of reliability of the pumps could be the cause of insufficient pressure in the tanks and in the dosing chamber, thus affecting the quality of the final product.
The pressures can be monitored by the control panel and are stored in the machine control unit. A typical maintenance indicator for vacuum pumps is the engine temperature that is stored in the machine control unit. Therefore, the parameters that allow us to define the machine’s status are pressure and temperature for each VP; the inputs of our model are presented in
Table 1.
The first pass yield (FPY) was chosen as the PQ variable; this index represents the percentage of pieces that do not need to be reworked, with respect to total daily production.
4.2. Implementation of Step 2
Step 2 concerns the prediction analysis, where the FPY is estimated based on the input variables, and a decision stage and a decision phase, which suggest a maintenance action based on the output of the previous stage (
Figure 6).
To choose the best ML technique to implement the predictive phase, the accuracies of a naive Bayes classifier (NBC), a nearest-neighbor classifier (NNC), and a bagged tree classifier (BTC) were compared because, from the literature review previously analyzed, it emerged that they are the most commonly used techniques for classification problems. Cakir et al. [
72] compared popular ML algorithms in the design of an IoT condition-based monitoring system.
The NBC is both a supervised learning method and a statistical method for classification [
73] and is used as a classifier in commercial and open source antispam e-mail filters [
74]. This classification is based on the Bayes theorem and allows us to determine uncertainty by calculating probabilities of outcomes.
The NNC is a supervised learning technique used for pattern recognition [
65]; this type of classifier has been used for demand forecasting production planning [
66], and also as a reference system to compare other classifiers. A NNC needs, in the design stage, a set of prototype vectors, a classification rule, and a neighborhood proximity measure. The prototypes are symbolic data used by the classifier to attribute class labels. NN classification essentially consists of selecting the label of the nearest neighbor of an unknown input vector.
The bagged tree classifier (BTC) uses bagging or bootstrap aggregations to improve the variance reduction in the prediction function [
75], and finds its main applications in statistical classification, predictions, and decision tree systems [
76]. Random forests consist of several decision-making trees, which work as a group of de-correlated and averaged trees [
67]. Averaging many noisy but roughly unbiased models allows us to reduce the variance and to use the ability of trees to capture complex interaction structures in the data.
The NBC, NNC, and BTC were trained through the MATLAB Classification Learner App with an 8 × 150 matrix input matrix and a 1 × 150 output matrix, representing the observation of 150 working days:
The pressure values of the pumps and the dosing chamber were stored on the machine control unit, the temperature values of the pumps were read by sensors and stored on the machine control unit, and the FPY index was calculated automatically by a line performance software, where operators enter data on daily production and non-compliant pieces.
The FPY output variable has been classified according to 3 categories:
FPY < 90%.
90% ≤ FPY ≤ 95%.
FPY > 95%.
Figure 7 shows the confusion matrix for the NBC trained thorough the IN and OUT matrices:
In this case, the FNR is high for classes 1 and 2 (representing, respectively, FPY = 2 and FPY = 1). This indicates that the naive Bayes classifier should not predict the correct output when the input variables should suggest an FPY index in these two classes.
In the scatter plot (p
4 vs. t
4) in
Figure 8, the yellow, orange, and blue dots represent, respectively, FPY = 1, FPY = 2, and FPY = 3. In addition, the yellow and orange dots overlap in some areas of the plot (X represents incorrect predictions).
This indicates the NNC is unable to predict the right output. The overall accuracy for the NBC is 75%.
Figure 9 shows the confusion matrix for an NNC (K = 1) trained with IN and OUT matrices.
In this case, the FNR is better for classes 1 and 2 (representing, respectively, FPY = 2 and FPY = 1), but the accuracy is lower in the prediction of class 3; the scatter plot (
Figure 10) shows overlapping for classes 1 and 2.
The overall accuracy for the NNC is 88%.
For the BTC, the best accuracy result (91%) was obtained, with 30 as the number of learners (the number of decision trees in the random forest) and 151 splits (the depth of the decision tree);
Figure 11 displays the confusion matrix for the BTC.
In this case, the FNR is better for class 3 with respect to the NBC and NNC.
The scatter plot (
Figure 12) shows overlapping for classes 1 and 2, and displays the confusion matrix for the BTC.
We compared the overall accuracy of the naive Bayes classifier and nearest neighbor classifier with that of the BTC (
Table 2).
The latter was chosen as the predictor of our model.
The predictive phase was completed by combining the bagged tree classifier (BTC) with a fuzzy inference engine (FIE). Variables that describe the state of the machine were used as the inputs of the BTC and FIE, and the predicted FPY (BTC output) represents an additional input to the FIE to determine the appropriate maintenance action for each pump (
Figure 13).
In summary, the BTC elaborates a prediction, based on the machine status, of the quality rate of the process, and the FIE suggests which corrective action should be carried out on the process according to the BTC output and to the machine status parameters. This strategy uses the BTC to predict the behavior of the system and the objectivity of the fuzzy rules to decide the action to be taken [
77].
In more detail, the FIE receives as an input normalized the pressure and temperature of each VP the and normalized FPY level estimated by the BTC, and elaborates a criticality index I
C according to a set of rules (
Figure 14).
The rules are designed to assess whether FPY index deviation is caused by human error or an actual anomaly in the pump. In the first case, a low-pressure value (VL), not corresponding to an increase in temperature (L), and an FPY = 2 (M) indicate a setup error (I
C = M), as expressed by the following rule:
In the second case, the pressure drop combined with a high temperature (VH), and an FPY = 1, leads to a high I
C (H):
Through the index value, a corrective action (after defuzzification) is suggested to the user as follows:
This portion of the study responds to the need, expressed through RQ1, to include the quality parameters of the product within the problem of predictive maintenance. In fact, the prediction of the FPY is used, together with the operating parameters of the machine, as one of the variables that guides the decision of the model with respect to the choice of maintenance actions.
4.3. Implementation of Step 3
In step 3, we analyze how the model predicts the quality rate and suggests a maintenance action through 3 different cases:
The parameters of all VPs are in the normal operating range:
In this case, no action is required on VP1, VP2, VP3, or VP4.
The pressure of VP
4 is low:
In this case, the FIE suggests checking the pressure setup of VP4 (the low pressure may be caused by an operator error).
The pressure of VP
4 is low and the temperature is high:
The FIE considers the association between the pressure drop and the temperature increase as an anomaly that causes the failure of the pump; the suggested action is to inspect the overheated area with a thermographic camera (
Figure 15).
In case 1, the model evaluates the input data as an optimal situation; in case 2 the low pressure, not being correlated to an increase in engine temperature, is judged as a setup error by the operator. In case 3, the temperature increase is considered as a signal of a possible pump failure.
The model was applied to the manufacturing process for a period of 6 months, corresponding to an observation period of 150 working days, each with two shifts.
During this period, 46 anomalies were detected in the 4 vacuum pumps, of which 32 were considered to be resolved with checking the machine parameters, and 14 were evaluated as possible pump failures.
Table 3 shows the difference between real events and the predicted events in the 2 categories of anomalies.
In summary, 96.9% of setup errors were correctly predicted by the model, and 92.8% of possible failures were reported and confirmed by a thermographic inspection of the vacuum pump. In one case, failure was not even predicted (with serious damage to the pump), because the anomaly had not generated an increase in temperature, and consequently the model was not able to anticipate failure.
An interesting result concerns the variations in MTBF and FPY by comparing the “before PdM” data and the period of the first application of the model to the manufacturing process, i.e., “after PdM”.
Table 4 shows the results and benefits, referring to a period of 150 days, in terms of number of maintenance events (NMEs), MTBM, MTTR, availability A, and FPY, where the MTBM is calculated as [
78] in (3):
and availability as in (4):
The comparison shows an evident improvement in the MTBM (from 37.8 to 48.6 h) (
Figure 16).
At the same time, the optimization of the FPY index (
Figure 17) confirms that the inclusion in the model of a production quality index, which must be monitored and optimized, effectively leads to an improvement in the continuity of operation of the machine (availability increases from 93.1% to 96.6%), when this index is dependent on some operating parameters of the machine itself (RQ
2).
Calculated as [
78] in (1): the naive Bayes classifier and nearest neighbor classifier.