# Remaining Useful Life (RUL) Prediction of Equipment in Production Lines Using Artificial Neural Networks

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- We developed a novel RUL prediction approach that utilizes the principal component analysis (PCA) feature selection algorithm, grid search parameter optimization algorithm, and multi-layer perceptron (MLP) machine learning algorithm.
- Since the RUL was not provided in training datasets, a polynomial function was fitted to HIs, and the interception between the polynomial and cycle axis was calculated as the failure point.

## 2. Background and Related Work

- Physics-based: Physical model, cumulative damage, hazard rate, proportional hazard rate, nonlinear dynamics
- Experimental-based
- Data-driven: Neural network (NN), support vector machine, Bayesian network, hidden (Markov, semi-Markov)
- Hybrid: Statistical model, Fourier transform with NN, statistical model with NN, fuzzy logic with NN, wavelet transform analysis with a statistical model, dynamic wavelet with NN.

- Expert model-based: Expert models, fuzzy logic
- Data-driven approaches
- ○
- Trend modeling methods: Machine learning, statistical models, stochastic models, deterministic models, probabilistic models
- ○
- Machine Learning

- Model-based approaches: Specific degradation models

## 3. Data Analysis

## 4. Methodology

#### 4.1. Data Pre-Processing

#### 4.2. Model Selection

#### 4.3. Interpolation and Model Training

#### 4.4. Evaluation

_{r}and T

_{p}stand for the real last cycle and the predicted last cycle, respectively.

_{RUL}by tuning hyperparameters. The model setting with the lowest MSE

_{RUL}is used for validation. After training, the model needs to be validated with the testing dataset. The testing data is processed with the steps, as in the case of training data. The testing data should have the same variables as the training data, and it is normalized with the training data mean and variance if Z-scores are performed. The testing data is transferred to PCA components with the training data eigenvector matrix.

_{RUL_Val}can be calculated based on Equation (5). The n equals the number of turbo units in the dataset. RULr and RULp stand for the real RUL of the test data and the predicted RUL, respectively.

## 5. Experimental Results

## 6. Discussion

#### 6.1. Main Discussion

#### 6.2. Threats to Validity

## 7. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Aghezzaf, E.H.; Najid, N.M. Integrated production planning and preventive maintenance in deteriorating production systems. Inf. Sci.
**2008**, 178, 3382–3392. [Google Scholar] [CrossRef] - Angius, A.; Colledani, M.; Yemane, A. Impact of condition based maintenance policies on the service level of multi-stage manufacturing systems. Control. Eng. Pract.
**2018**, 76, 65–78. [Google Scholar] [CrossRef] - Salonen, A.; Deleryd, M. Cost of poor maintenance. J. Qual. Maint. Eng.
**2011**, 17, 63–73. [Google Scholar] [CrossRef] - De Jonge, B.; Teunter, R.; Tinga, T. The influence of practical factors on the benefits of condition-based maintenance over time-based maintenance. Reliab. Eng. Syst. Saf.
**2017**, 158, 21–30. [Google Scholar] [CrossRef] - Seiti, H.; Tagipour, R.; Hafezalkotob, A.; Asgari, F. Maintenance strategy selection with risky evaluations using RAHP. J. Multi-Criteria Decis. Anal.
**2017**, 24, 257–274. [Google Scholar] [CrossRef] - Ribeiro, I.M.; Godina, R.; Pimentel, C.; Silva, F.J.G.; Matias, J.C.O. Implementing TPM supported by 5S to improve the availability of an automotive production line. Procedia Manuf.
**2019**, 38, 1574–1581. [Google Scholar] [CrossRef] - Alsyouf, I. Maintenance practices in Swedish industries: Survey results. Int. J. Prod. Econ.
**2009**, 121, 212–223. [Google Scholar] [CrossRef] [Green Version] - Ozcan, S.; Simsir, F. A new model based on Artificial Bee Colony algorithm for preventive maintenance with replacement scheduling in continuous production lines. Eng. Sci. Technol. Int. J.
**2019**, 22, 1175–1186. [Google Scholar] [CrossRef] - Percy, D.F.; Kobbacy, K.A.; Fawzi, B.B. Setting preventive maintenance schedules when data are sparse. Int. J. Prod. Econ.
**1997**, 51, 223–234. [Google Scholar] [CrossRef] - Gu, X.; Jin, X.; Guo, W.; Ni, J. Estimation of active maintenance opportunity windows in Bernoulli production lines. J. Manuf. Syst.
**2017**, 45, 109–120. [Google Scholar] [CrossRef] - Khorasgani, H.; Biswas, G.; Sankararaman, S. Methodologies for system-level remaining useful life prediction. Reliab. Eng. Syst. Saf.
**2016**, 154, 8–18. [Google Scholar] [CrossRef] - Susto, G.A.; Beghi, A.; De Luca, C. A predictive maintenance system for epitaxy processes based on filtering and prediction techniques. IEEE Trans. Semicond. Manuf.
**2012**, 25, 638–649. [Google Scholar] [CrossRef] - Mobley, R.K. An Introduction to Predictive Maintenance; Elsevier: Amsterdam, The Netherlands, 2002. [Google Scholar]
- Chan, G.K.; Asgarpoor, S. Optimum maintenance policy with Markov processes. Electr. Power Syst. Res.
**2006**, 76, 452–456. [Google Scholar] [CrossRef] - Duarte, Y.S.; Szpytko, J.; del Castillo Serpa, A.M. Monte Carlo simulation model to coordinate the preventive maintenance scheduling of generating units in isolated distributed Power Systems. Electr. Power Syst. Res.
**2020**, 182, 106237. [Google Scholar] [CrossRef] - Li, J.; Blumenfeld, D.E.; Huang, N.; Alden, J.M. Throughput analysis of production systems: Recent advances and future topics. Int. J. Prod. Res.
**2009**, 47, 3823–3851. [Google Scholar] [CrossRef] - Yang, S.K. A condition-based preventive maintenance arrangement for thermal power plants. Electr. Power Syst. Res.
**2004**, 72, 49–62. [Google Scholar] [CrossRef] - Wang, L.; Chu, J.; Wu, J. Selection of optimum maintenance strategies based on a fuzzy analytic hierarchy process. Int. J. Prod. Econ.
**2007**, 107, 151–163. [Google Scholar] [CrossRef] - Bülbül, P.; Bayındır, Z.P.; Bakal, İ.S. Exact and heuristic approaches for joint maintenance and spare parts planning. Comput. Ind. Eng.
**2019**, 129, 239–250. [Google Scholar] [CrossRef] - Rivera-Gómez, H.; Gharbi, A.; KennÉ, J.P.; Montaño-Arango, O.; Corona-Armenta, J.R. Joint optimization of production and maintenance strategies considering a dynamic sampling strategy for a deteriorating system. Comput. Ind. Eng.
**2020**, 140, 106273. [Google Scholar] [CrossRef] - Seiti, H.; Hafezalkotob, A.; Najafi, S.E.; Khalaj, M. Developing a novel risk-based MCDM approach based on D numbers and fuzzy information axiom and its applications in preventive maintenance planning. Appl. Soft Comput.
**2019**, 82, 105559. [Google Scholar] [CrossRef] - Wang, X.; Zhang, Y.; Wang, L.; Wang, J.; Lu, J. Maintenance grouping optimization with system multi-level information based on BN lifetime prediction model. J. Manuf. Syst.
**2019**, 50, 201–211. [Google Scholar] [CrossRef] - Susto, G.A.; Schirru, A.; Pampuri, S.; McLoone, S.; Beghi, A. Machine learning for predictive maintenance: A multiple classifier approach. IEEE Trans. Ind. Inform.
**2014**, 11, 812–820. [Google Scholar] [CrossRef] [Green Version] - Luo, R.C.; Wang, H. Diagnostic and Prediction of Machines Health Status as Exemplary Best Practice for Vehicle Production System. In Proceedings of the 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), Chicago, IL, USA, 27–30 August 2018. [Google Scholar]
- Fan, Y.; Nowaczyk, S.; Rögnvaldsson, T. Evaluation of Self-Organized Approach for Predicting Compressor Faults in a City Bus Fleet. Procedia Comput. Sci.
**2015**, 53, 447–456. [Google Scholar] [CrossRef] [Green Version] - Saxena, A.; Goebel, K. Turbofan Engine Degradation Simulation Data Set. 2008. Available online: https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/#turbofan (accessed on 20 November 2020).
- Ramasso, E.; Saxena, A. Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets. Int. J. Progn. Health Manag.
**2014**, 5, 1–15. [Google Scholar] - Borst, N.G. Adaptations for CNN-LSTM Network for Remaining Useful Life Prediction: Adaptable Time Window and Sub-Network Training. Master’s Thesis, Delft University of Technology, Delft, The Netherlands, August 2020. [Google Scholar]
- Garga, A.K.; McClintic, K.T.; Campbell, R.L.; Yang, C.C.; Lebold, M.S.; Hay, T.A.; Byington, C.S. Hybrid reasoning for prognostic learning in CBM systems. In Proceedings of the 2001 IEEE Aerospace Conference Proceedings (Cat. No. 01TH8542), Big Sky, MT, USA, 10–17 March 2001; Volume 6, pp. 2957–2969. [Google Scholar]
- Sikorska, J.Z.; Hodkiewicz, M.; Ma, L. Prognostic modelling options for remaining useful life estimation by industry. Mech. Syst. Signal Process.
**2011**, 25, 1803–1836. [Google Scholar] [CrossRef] - Ahmadzadeh, F.; Lundberg, J. Remaining useful life estimation. Int. J. Syst. Assur. Eng. Manag.
**2014**, 5, 461–474. [Google Scholar] [CrossRef] - Okoh, C.; Roy, R.; Mehnen, J.; Redding, L. Overview of Remaining Useful Life Prediction Techniques in Through-life Engineering Services. Procedia CIRP
**2014**, 16, 158–163. [Google Scholar] [CrossRef] [Green Version] - Hu, Y.; Liu, S.; Lu, H.; Zhang, H. Remaining useful life model and assessment of mechanical products: A brief review and a note on the state space model method. Chin. J. Mech. Eng.
**2019**, 32, 15. [Google Scholar] [CrossRef] [Green Version] - Si, X.S.; Wang, W.; Hu, C.H.; Zhou, D.H. Remaining useful life estimation–a review on the statistical data driven approaches. Eur. J. Oper. Res.
**2011**, 213, 1–14. [Google Scholar] [CrossRef] - Djeziri, M.A.; Benmoussa, S.; Zio, E. Review on Health Indices Extraction and Trend Modeling for Remaining Useful Life Estimation. In Artificial Intelligence Techniques for a Scalable Energy Transition; Springer: Berlin/Heidelberg, Germany, 2020; pp. 183–223. [Google Scholar]
- Li, H.; Zhao, W.; Zhang, Y.; Zio, E. Remaining useful life prediction using multi-scale deep convolutional neural network. Appl. Soft Comput.
**2020**, 89, 106113. [Google Scholar] [CrossRef] - Hou, M.; Pi, D.; Li, B. Similarity-Based Deep Learning Approach for Remaining Useful Life Prediction. Measurement
**2020**, 159, 107788. [Google Scholar] [CrossRef] - Cheng, H.; Kong, X.; Chen, G.; Wang, Q.; Wang, R. Transferable convolutional neural network based remaining useful life prediction of bearing under multiple failure behaviors. Measurement
**2020**, 168, 108286. [Google Scholar] [CrossRef] - Wang, B.; Lei, Y.; Yan, T.; Li, N.; Guo, L. Recurrent convolutional neural network: A new framework for remaining useful life prediction of machinery. Neurocomputing
**2020**, 379, 117–129. [Google Scholar] [CrossRef] - Chen, Y.; Peng, G.; Zhu, Z.; Li, S. A novel deep learning method based on attention mechanism for bearing remaining useful life prediction. Appl. Soft Comput.
**2020**, 86, 105919. [Google Scholar] [CrossRef] - Wu, J.; Hu, K.; Cheng, Y.; Zhu, H.; Shao, X.; Wang, Y. Data-driven remaining useful life prediction via multiple sensor signals and deep long short-term memory neural network. ISA Trans.
**2020**, 97, 241–250. [Google Scholar] [CrossRef] [PubMed] - Li, X.; Zhang, W.; Ma, H.; Luo, Z.; Li, X. Data alignments in machinery remaining useful life prediction using deep adversarial neural networks. Knowl. Based Syst.
**2020**, 197, 105843. [Google Scholar] [CrossRef] - Su, C.; Li, L.; Wen, Z. Remaining useful life prediction via a variational autoencoder and a time-window-based sequence neural network. Qual. Reliab. Eng. Int.
**2020**, 37, 34–46. [Google Scholar] [CrossRef] - Riad, A.; Elminir, H.; Elattar, H. Evaluation of neural networks in the subject of prognostics as compared to linear regression model. Int. J. Eng. Technol.
**2010**, 10, 52–58. [Google Scholar] - Cutler, A.; Cutler, D.; Stevens, J. Random Forests. Mach. Learn.
**2011**, 45, 157–176. [Google Scholar]

**Figure 11.**Retraining of the interpolated HI with a selected model (correlation coefficient r = 0.999).

Training Dataset | Testing Dataset | # of Conditions Engine | Fault Mode | ||||
---|---|---|---|---|---|---|---|

Dataset | Dimension | Dataset | Dimension | Dataset | Dimension | ||

FD001_train | 20,631 × 26 | FD001_test | 13,096 × 26 | RUL1 | 100 × 2 | 1 | HPC Degradation |

FD002_train | 53,759 × 26 | FD002_test | 33,991 × 26 | RUL2 | 259 × 2 | 6 | HPC Degradation |

FD003_train | 24,720 × 26 | FD003_test | 16,596 × 26 | RUL3 | 100 × 2 | 1 | HPC & Fan Degradation |

FD004_train | 61,249 × 26 | FD004_test | 41,214 × 26 | RUL4 | 248 × 2 | 6 | HPC & Fan Degradation |

Unit | Cycle | Setting1 | Setting2 | Setting3 | Sensor1 | …… | Sensor21 |
---|---|---|---|---|---|---|---|

Int | Int | Float | Float | Float | Float | Float | Float |

Unit | RUL |
---|---|

Int | Int |

Connection | Number of Units | Input Dimension | Activation Fun | |
---|---|---|---|---|

Input Layer | Dense | 24 | 24 | Tanh |

Hidden Layer-1 | Dense | 20 | - | Tanh |

Hidden Layer-2 | Dense | 5 | - | Tanh |

Output Layer | Dense | 1 | - | Linear |

Loss function | Optimizer | Learning Rate | Belta_1 | |

Compiling | MSE | Adam | 3 × 10^{−5} | 0.9 |

**Table 5.**(

**A**) Linear Regression, (

**B**) Linear Regression + PCA, (

**C**) MLP + PCA, (

**D**) RF + PCA, (

**E**) SVR + PCA; HI Training MSE stands for the MSE of the partial data training; HI Retrain MSE stands for the MSE of the retrain on the whole dataset; Training RUL MSE represents the evaluation of RUL with the training data; Validation RUL MSE represents the validation result of RUL estimation of the test data; Validation RUL MSE represents the validation of RUL in test data with cycle > 100.

Dataset | HI Training MSE | HI Retrain MSE | Training RUL MSE | Validation RUL_MSE | Validation RUL_MSE (cycle > 100) |
---|---|---|---|---|---|

FD001 | 3.18 × 10^{−3} | 8.60 × 10^{−4} | 20 | 668 | 499 |

FD002 | 3.87 × 10^{−2} | 2.90 × 10^{−3} | 26 | 1031 | 390 |

FD003 | 3.57 × 10^{−2} | 7.66 × 10^{−4} | 32 | 1332 | 1162 |

FD004 | 5.88 × 10^{−2} | 2.20 × 10^{−3} | 149 | 2181 | 1108 |

(A) | |||||

FD001 | 3.71 × 10^{−2} | 2.48 × 10^{−4} | 21 | 558 | 468 |

FD002 | 3.61 × 10^{−2} | 4.41 × 10^{−4} | 36 | 748 | 358 |

FD003 | 3.34 × 10^{−2} | 2.31 × 10^{−4} | 35 | 1387 | 1186 |

FD004 | 4.07 × 10^{−2} | 5.38 × 10^{−4} | 94 | 1904 | 1094 |

(B) | |||||

FD001 | 3.65 × 10^{−2} | 1.47 × 10^{−4} | 55 | 509 | 504 |

FD002 | 3.62 × 10^{−2} | 1.92 × 10^{−4} | 43 | 746 | 364 |

FD003 | 3.36 × 10^{−2} | 9.69 × 10^{−5} | 21 | 1259 | 1100 |

FD004 | 4.25 × 10^{−2} | 8.56 × 10^{−5} | 94 | 1427 | 1031 |

(C) | |||||

FD001 | 3.69 × 10^{−2} | 1.46 × 10^{−3} | 18 | 701 | 511 |

FD002 | 3.60 × 10^{−2} | 4.40 × 10^{−3} | 21 | 857 | 436 |

FD003 | 3.37 × 10^{−2} | 3.56 × 10^{−3} | 136 | 1895 | 1411 |

FD004 | 4.09 × 10^{−2} | 1.05 × 10^{−2} | 316 | 1994 | 1613 |

(D) | |||||

FD001 | 3.70 × 10^{−2} | 8.40 × 10^{−4} | 71 | 800 | 568 |

FD002 | 3.62 × 10^{−2} | 6.10 × 10^{−3} | 23 | 776 | 382 |

FD003 | 3.36 × 10^{−2} | 1.55 × 10^{−3} | 85 | 1089 | 947 |

FD004 | 4.29 × 10^{−2} | 1.01 × 10^{−3} | 162 | 1575 | 1199 |

(E) |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kang, Z.; Catal, C.; Tekinerdogan, B.
Remaining Useful Life (RUL) Prediction of Equipment in Production Lines Using Artificial Neural Networks. *Sensors* **2021**, *21*, 932.
https://doi.org/10.3390/s21030932

**AMA Style**

Kang Z, Catal C, Tekinerdogan B.
Remaining Useful Life (RUL) Prediction of Equipment in Production Lines Using Artificial Neural Networks. *Sensors*. 2021; 21(3):932.
https://doi.org/10.3390/s21030932

**Chicago/Turabian Style**

Kang, Ziqiu, Cagatay Catal, and Bedir Tekinerdogan.
2021. "Remaining Useful Life (RUL) Prediction of Equipment in Production Lines Using Artificial Neural Networks" *Sensors* 21, no. 3: 932.
https://doi.org/10.3390/s21030932