# Remaining Useful Life Prediction from 3D Scan Data with Genetically Optimized Convolutional Neural Networks

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Works

## 3. Materials and Methods

#### 3.1. Experimental Setup and Data Acquisitions

^{®}3210 (LMI Technologies Inc., Burnaby, BC, Canada) [41] sensor, tightly clamped on the punching machine structure (Figure 2B). The 3D scan sensor, equipped with a stereo camera of two megapixels and a blue LED projector, provides 3D point clouds in a single snapshot for accurate noncontact measurements down to 35 μm.

#### 3.2. D Scan Preprocessing

Algorithm 1. Point cloud registration | |

1. | ${\mathit{P}}_{0}\leftarrow \left\{{P}_{i}^{R}\in {\mathbb{R}}^{3},i=1,\dots ,{N}_{0}\right\}$;//Reference point cloud |

2. | $\mathit{P}\leftarrow \left\{{P}_{i}\in {\mathbb{R}}^{3},i=1,\dots ,N\right\}$;//Data point cloud to be registered ($N>{N}_{0}$) |

3. | $\epsilon =0.01;$//Tolerance between consecutive iterations |

4. | $MaxIterations=100$;//Maximum number of iterations |

5. | for $i=1;i\le {N}_{0};i++$ |

6. | ${\widehat{P}}_{i}^{0}\leftarrow {P}_{i}^{0}-\frac{1}{{N}_{0}}{\displaystyle \sum}_{j=1}^{{N}_{0}}{P}_{j}^{0}$; //Normalization of ${\mathit{P}}_{0}$ |

7. | end for |

8. | for $k=1$; $k\le MaxIterations$; $k++$ |

9. | $\mathit{P}\leftarrow Projectiectionof\mathit{P}onto{\mathit{P}}_{0}$;//Matching subset (from now $\left|\mathit{P}\right|={N}_{0}$) |

10. | for $i=1;i\le {N}_{0};i++$ |

11. | ${\widehat{P}}_{i}\leftarrow {P}_{i}-\frac{1}{{N}_{0}}{\displaystyle \sum}_{j=1}^{{N}_{0}}{P}_{j}$; //Normalization of $\mathit{P}$ |

12. | end for |

13. | Calculate the SVD decomposition of the matrix $H={\displaystyle \sum}_{j=1}^{{N}_{0}}{\widehat{P}}_{j}^{0}{\left({\widehat{P}}_{j}\right)}^{T}=U\mathsf{\Lambda}{V}^{T}$; |

14. | $R\leftarrow V{U}^{T}$;//Rotation matrix |

15. | $T\leftarrow \frac{1}{N}{\displaystyle \sum}_{j=1}^{{N}_{0}}{\widehat{P}}_{j}-R\frac{1}{N}{\displaystyle \sum}_{j=1}^{{N}_{0}}{\widehat{P}}_{j}^{0}$;//Translation matrix |

16. | if $\sum}_{j=1}^{{N}_{0}}{\widehat{P}}_{j}^{0}-\left(R{\widehat{P}}_{j}+T\right)\le \epsilon $ //Termination condition |

17. | break; |

18. | end if |

19. | end for |

#### 3.3. Evaluation Metrics

#### 3.4. DNN Architectures

#### 3.5. Genetic Optimization

^{®}Deep Learning Toolbox (v 14.2, R2021a, MathWorks Inc., Natick, MA, USA) [66]; whereas, genetic optimization was performed using the MathWorks

^{®}Optimization Toolbox (v 9.1, R2021a, MathWorks Inc., Natick, MA, USA) [67].

#### 3.6. SVR Based Estimation

## 4. Results

^{®}Core™ i7-5820K CPU @ 3.30 GHz, 16 GB DRAM, and NVIDIA GeForce GTX TITAN X GPU (Maxwell family) with 12 GB GRAM.

## 5. Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Atamuradov, V.; Medjaher, K.; Dersin, P.; Lamoureux, B.; Zerhouni, N. Prognostics and Health Management for Maintenance Practitioners—Review, Implementation and Tools Evaluation. Int. J. Progn. Health Manag.
**2017**, 8. [Google Scholar] [CrossRef] - Silvestri, L.; Forcina, A.; Introna, V.; Santolamazza, A.; Cesarotti, V. Maintenance transformation through Industry 4.0 technologies: A systematic literature review. Comput. Ind.
**2020**, 123, 103335. [Google Scholar] [CrossRef] - Kuo, C.-J.; Chien, C.-F.; Chen, J.-D. Manufacturing Intelligence to Exploit the Value of Production and Tool Data to Reduce Cycle Time. IEEE Trans. Autom. Sci. Eng.
**2010**, 8, 103–111. [Google Scholar] [CrossRef] - Chien, C.-F.; Chen, C.-C. Data-Driven Framework for Tool Health Monitoring and Maintenance Strategy for Smart Manufacturing. IEEE Trans. Semicond. Manuf.
**2020**, 33, 644–652. [Google Scholar] [CrossRef] - Si, X.-S.; Wang, W.; Hu, C.-H.; Zhou, D.-H. Remaining useful life estimation—A review on the statistical data driven approaches. Eur. J. Oper. Res.
**2011**, 213, 1–14. [Google Scholar] [CrossRef] - Liao, L.; Köttig, F. A hybrid framework combining data-driven and model-based methods for system remaining useful life prediction. Appl. Soft Comput.
**2016**, 44, 191–199. [Google Scholar] [CrossRef] - Wang, Y.; Zhao, Y.; Addepalli, S.N.P. Remaining Useful Life Prediction using Deep Learning Approaches: A Review. Procedia Manuf.
**2020**, 49, 81–88. [Google Scholar] [CrossRef] - Elattar, H.M.; Elminir, H.K.; Riad, A.M. Prognostics: A literature review. Complex Intell. Syst.
**2016**, 2, 125–154. [Google Scholar] [CrossRef] [Green Version] - Tran, V.T.; Pham, H.T.; Yang, B.-S.; Nguyen, T.T. Machine performance degradation assessment and remaining useful life prediction using proportional hazard model and support vector machine. Mech. Syst. Signal Process.
**2012**, 32, 320–330. [Google Scholar] [CrossRef] [Green Version] - Bengio, Y.; Courville, A.; Vincent, P. Representation Learning: A Review and New Perspectives. IEEE Trans. Pattern Anal. Mach. Intell.
**2013**, 35, 1798–1828. [Google Scholar] [CrossRef] - Zhao, R.; Yan, R.; Chen, Z.; Mao, K.; Wang, P.; Gao, R.X. Deep learning and its applications to machine health monitoring. Mech. Syst. Signal Process.
**2019**, 115, 213–237. [Google Scholar] [CrossRef] - Shrestha, A.; Mahmood, A. Review of Deep Learning Algorithms and Architectures. IEEE Access
**2019**, 7, 53040–53065. [Google Scholar] [CrossRef] - Pearson, K. On lines and planes of closest fit to systems of points in space. Philos. Mag. Ser.
**1901**, 2, 559–57223. [Google Scholar] [CrossRef] [Green Version] - Hotelling, H. Analysis of a complex of statistical vari-ables into principal components. J. Educ. Psychol.
**1933**, 24, 417–441. [Google Scholar] [CrossRef] - Fisher, R.A. The Use of Multiple Measurements in Taxonomic Problems. Ann. Eugen.
**1936**, 7, 179–188. [Google Scholar] [CrossRef] - Hinton, G.E.; Salakhutdinov, R.R. Reducing the dimensionality of data with neural networks. Science
**2006**, 313, 504–507. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Ma, J.; Su, H.; Zhao, W.-L.; Liu, B. Predicting the Remaining Useful Life of an Aircraft Engine Using a Stacked Sparse Autoencoder with Multilayer Self-Learning. Complexity
**2018**, 2018. [Google Scholar] [CrossRef] [Green Version] - Sun, C.; Ma, M.; Zhao, Z.; Tian, S.; Yan, R.; Chen, X. Deep transfer learning based on sparse autoencoder for re-maining useful life prediction of tool in manufacturing. IEEE Trans. Ind. Inform.
**2018**, 15, 2416–2425. [Google Scholar] [CrossRef] - Ren, L.; Sun, Y.; Cui, J.; Zhang, L. Bearing remaining useful life prediction based on deep autoencoder and deep neural networks. J. Manuf. Syst.
**2018**, 48, 71–77. [Google Scholar] [CrossRef] - Liao, L.; Jin, W.; Pavel, R. Enhanced Restricted Boltzmann Machine With Prognosability Regularization for Prognostics and Health Assessment. IEEE Trans. Ind. Electron.
**2016**, 63, 7076–7083. [Google Scholar] [CrossRef] - Haris, M.; Hasan, M.N.; Qin, S. Early and robust remaining useful life prediction of supercapacitors using BOHB optimized Deep Belief Network. Appl. Energy
**2021**, 286, 116541. [Google Scholar] [CrossRef] - Jiao, R.; Peng, K.; Dong, J.; Zhang, C. Fault monitoring and remaining useful life prediction framework for multiple fault modes in prognostics. Reliab. Eng. Syst. Saf.
**2020**, 203, 107028. [Google Scholar] [CrossRef] - Ma, M.; Sun, C.; Chen, X. Discriminative deep belief networks with ant colony optimization for health status as-sessment of machine. IEEE Trans. Instrum. Meas.
**2017**, 66, 3115–3125. [Google Scholar] [CrossRef] - Zhang, C.; Lim, P.; Qin, A.K.; Tan, K.C. Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics. IEEE Trans. Neural Netw. Learn. Syst.
**2016**, 28, 2306–2318. [Google Scholar] [CrossRef] [PubMed] - Frederick, D.K.; DeCastro, J.A.; Litt, J.S. User’s Guide for the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS. 2007. Available online: https://core.ac.uk/download/pdf/10566552.pdf (accessed on 27 July 2021).
- LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE
**1998**, 86, 2278–2324. [Google Scholar] [CrossRef] [Green Version] - Dhillon, A.; Verma, G.K. Convolutional neural network: A review of models, methodologies and applications to object detection. Prog. Artif. Intell.
**2020**, 9, 85–112. [Google Scholar] [CrossRef] - Babu, G.S.; Zhao, P.; Li, X.-L. Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life. In International Conference on Database Systems for Advanced Applications; Springer: Berlin/Heidelberg, Germany, 2016; pp. 214–228. [Google Scholar]
- Nectoux, P.; Gouriveau, R.; Medjaher, K.; Ramasso, E.; Chebel-Morello, B.; Zerhouni, N.; Varnier, C. PRONOSTIA: An experimental platform for bearings accelerated degradation tests. In Proceedings of the IEEE International Conference on Prognostics and Health Management, Denver, CO, USA, 18–21 June 2012; pp. 1–8. [Google Scholar]
- Heimes, F.O. Recurrent neural networks for remaining useful life estimation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA, 6–9 October 2008; pp. 1–6. [Google Scholar]
- Zheng, S.; Ristovski, K.; Farahat, A.; Gupta, C. Long Short-Term Memory Network for Remaining Useful Life estimation. In Proceedings of the 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), Piscataway, NJ, USA, 19–21 June 2017; pp. 88–95. [Google Scholar]
- Li, X.; Ding, Q.; Sun, J.-Q. Remaining useful life estimation in prognostics using deep convolution neural networks. Reliab. Eng. Syst. Saf.
**2018**, 172, 1–11. [Google Scholar] [CrossRef] [Green Version] - Malhotra, P.; TV, V.; Ramakrishnan, A.; Anand, G.; Vig, L.; Agarwal, P.; Shroff, G. Multi-sensor prognostics using an unsupervised health index based on LSTM encoder-decoder. arXiv
**2016**, arXiv:1608.06154. [Google Scholar] - Wu, X.; Liu, Y.; Zhou, X.; Mou, A. Automatic Identification of Tool Wear Based on Convolutional Neural Network in Face Milling Process. Sensors
**2019**, 19, 3817. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Marei, M.; El Zaatari, S.; Li, W. Transfer learning enabled convolutional neural networks for estimating health state of cutting tools. Robot. Comput. Manuf.
**2021**, 71, 102145. [Google Scholar] [CrossRef] - Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Li, F.F. Imagenet: A Large-Scale Hierarchical Image Database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
- Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Fei-Fei, L. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis.
**2015**, 115, 211–252. [Google Scholar] [CrossRef] [Green Version] - Krizhevsky, A.; Nair, V.; Hinton, G. Cifar-10 and Cifar-100 Datasets. 2009. Available online: https://www.cs.toronto.edu/~kriz/cifar.html (accessed on 30 June 2021).
- Mo, H.; Custode, L.L.; Iacca, G. Evolutionary neural architecture search for remaining useful life prediction. Appl. Soft Comput.
**2021**, 108, 107474. [Google Scholar] [CrossRef] - Li, X.; Jia, X.; Wang, Y.; Yang, S.; Zhao, H.; Lee, J. Industrial Remaining Useful Life Prediction by Partial Observation Using Deep Learning with Supervised Attention. IEEE/ASME Trans. Mechatron.
**2020**, 25, 2241–2251. [Google Scholar] [CrossRef] - LMI Technologies. Gocator 3210 Datasheet—Large Field of View 3D Snapshot Sensor. Available online: https://lmi3d.com/resource/gocator-3210-datasheet-large-field-view-3d-snapshot-sensor/ (accessed on 14 July 2021).
- Besl, P.J.; McKay, N.D. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell.
**1992**, 14, 239–256. [Google Scholar] [CrossRef] - Chen, Y.; Medioni, G. Object modelling by registration of multiple range images. Image Vis. Comput.
**1992**, 10, 145–155. [Google Scholar] [CrossRef] - Rousseeuw, P.J.; Leroy, A.M. Robust Regression and Outlier Detection; John Wiley & Sons: Hoboken, NJ, USA, 1987; Volume 589. [Google Scholar]
- Hoppe, H.; Derose, T.; Duchamp, T.; McDonald, J.; Stuetzle, W. Surface reconstruction from unorganized points. ACM SIGGRAPH Comput. Graph.
**1992**, 26, 71–78. [Google Scholar] [CrossRef] - LeCun, Y.; Bengio, Y. Convolutional networks for images, speech, and time series. In The Handbook of Brain Theory and Neural Networks; MIT Press: Cambridge, MA, USA, 1995; p. 3361. [Google Scholar]
- Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J. Mach. Learn. Res.
**2014**, 15, 1929–1958. [Google Scholar] - Zinkevich, M.; Weimer, M.; Smola, A.J.; Li, L. Parallelized stochastic gradient descent. NIPS
**2010**, 4, 4. [Google Scholar] - Ribani, R.; Marengoni, M. A Survey of Transfer Learning for Convolutional Neural Networks. In Proceedings of the 2019 32nd IEEE Conference on Graphics, Patterns and Images Tutorials (SIBGRAPI-T), Rio de Janeiro, Brazil, 28–31 October 2019; pp. 47–57. [Google Scholar]
- Iandola, F.N.; Han, S.; Moskewicz, M.W.; Ashraf, K.; Dally, W.J.; Keutzer, K. SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 MB model size. arXiv
**2016**, arXiv:1602.07360. [Google Scholar] - Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going Deeper with Convolutions. arXiv
**2014**, arXiv:1409.4842. [Google Scholar] - Szegedy, C.; Vanhoucke, V.; Ioe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer Vision. In Pro-ceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA, 27–30 June 2016; pp. 2818–2826. [Google Scholar]
- Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely Connected Convolutional Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2017, Honolulu, HI, USA, 21–21 July 2017; pp. 4700–4708. [Google Scholar]
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L.C. Mobilenetv2: Inverted residuals and linear bottle-necks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 4510–4520. [Google Scholar]
- He, K.M.; Zhang, X.Y.; Ren, S.Q.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Chollet, F. Xception: Deep Learning with Depthwise Separable Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; IEEE: New York, NY, USA, 2017; pp. 1251–1258. [Google Scholar]
- Szegedy, C.; Ioffe, S.; Vanhoucke, V.; Alemi, A.A. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 4–9 February 2017. [Google Scholar]
- Zhang, X.; Zhou, X.; Lin, M.; Sun, J. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 6848–6856. [Google Scholar]
- Zoph, B.; Vasudevan, V.; Shlens, J.; Le, Q.V. Learning Transferable Architectures for Scalable Image Recognition. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 8697–8710. [Google Scholar]
- Redmon, J. Darknet: Open-Source Neural Networks in c. 2013. Available online: https://pjreddie.com/darknet/ (accessed on 23 July 2021).
- Tan, M.; Le, Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA, 9–15 June 2019; pp. 6105–6114. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process.
**2012**, 25, 1097–1105. [Google Scholar] [CrossRef] - Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv
**2014**, arXiv:1409.1556. [Google Scholar] - Mitchell, M. An Introduction to Genetic Algorithms; MIT Press: Cambridge, MA, USA, 1998. [Google Scholar]
- Darwin, C. On the Origin of Species by Means of Natural Selection; J. Murray: London, UK, 1859. [Google Scholar]
- Beale, M.; Hagan, M.; Demuth, H. Deep Learning Toolbox™ Reference. MATLAB (r) R2021a. The MathWorks, Inc. 2021. Available online: https://it.mathworks.com/help/pdf_doc/deeplearning/nnet_ref.pdf (accessed on 27 July 2021).
- Optimization Toolbox™ User’s Guide. MATLAB (r) R2021a. The MathWorks, Inc. Available online: https://it.mathworks.com/help/pdf_doc/optim/optim.pdf (accessed on 27 July 2021).
- Cao, L.; Chua, K.; Chong, W.; Lee, H.; Gu, Q. A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine. Neurocomputing
**2003**, 55, 321–336. [Google Scholar] [CrossRef] - Sain, S.R.; Vapnik, V.N. The Nature of Statistical Learning Theory. Technometrics
**1996**, 38, 409. [Google Scholar] [CrossRef] - Smola, A.J.; Schölkopf, B. A tutorial on support vector regression. Stat. Comput.
**2004**, 14, 199–222. [Google Scholar] [CrossRef] [Green Version]

**Figure 2.**Punching machinery: (

**A**) punch tool and pump workpiece, (

**B**) Gocator

^{®}3210 3D scanning sensor.

**Figure 3.**Punch tool 3D model. (

**A**) Full view of the punch tool with working region circled with a dashed red line. (

**B**) Detail of the working region. (

**C**) 3D point cloud with grayscale patches of the working region obtained from a 3D scan snapshot.

**Figure 4.**3D point cloud segmented using the Euclidean distance between 3D points from different clusters. Segmented clusters are represented in different colors.

**Figure 7.**Surface normal vector representation of the working region obtained from the cropped point cloud provided in Figure 6.

**Figure 8.**(

**A**) Depth map. It is a false-color image: red color represents short distances and blue long ones. (

**B**) Normal map. It is a false-color image: dark-red color represents vectors parallel to z-axis (i.e., pointing out of the image plane) and light-blue perpendicular ones.

**Figure 9.**Longitudinal profile extraction. (

**A**) Longitudinal point-cloud region (red points) from which the longitudinal profile is estimated. (

**B**) Estimated longitudinal profile (red curve) from the point cloud projection on the YZ plane.

**Figure 10.**Single-head general architecture of CNN-based DNN for processing either depth or normal maps provided as color image inputs.

**Figure 11.**Double-head general architecture of CNN-based DNN for processing both depth and normal maps provided as color image inputs.

Network | Input Size | $\mathbf{Parameters}({10}^{6})$ |
---|---|---|

Squeezenet [50] | 227 × 227 × 3 | 1.2 |

Googlenet [51] | 224 × 224 × 3 | 7.0 |

Inceptionv3 [52] | 299 × 299 × 3 | 23.9 |

Densenet201 [53] | 224 × 224 × 3 | 20.0 |

Mobilenetv2 [54] | 224 × 224 × 3 | 3.5 |

Resnet18 [55] | 224 × 224 × 3 | 11.7 |

Resnet50 [55] | 224 × 224 × 3 | 25.6$s\times s$ |

Resnet101 [55] | 224 × 224 × 3 | 44.6 |

Xception [56] | 299 × 299 × 3 | 22.9 |

Inceptionresnetv2 [57] | 299 × 299 × 3 | 55.9 |

Shufflenet [58] | 224 × 224 × 3 | 1.4 |

Nasnetmobile [59] | 224 × 224 × 3 | 5.3 |

Nasnetlarge [59] | 331 × 331 × 3 | 88.9 |

Darknet19 [60] | 256 × 256 × 3 | 20.8 |

Darknet53 [60] | 256 × 256 × 3 | 41.6 |

Efficientnetb0 [61] | 224 × 224 × 3 | 5.3 |

Alexnet [62] | 227 × 227 × 3 | 61.0 |

Vgg16 [63] | 224 × 224 × 3 | 138.0 |

Vgg19 [63] | 224 × 224 × 3 | 144.0 |

Variable | ${\mathit{M}}_{\mathit{i}}$ | Lower Bound | Upper Bound |
---|---|---|---|

Filter size $\left({x}_{1},{y}_{1},{y}_{4}\right)$ | 4 | 1 | 2002 |

“ | 5 | 1 | 4004 |

“ | 6 | 1 | 6006 |

Number of filters $\left({x}_{2},{y}_{2},{y}_{5}\right)$ | 4 | 1 | 35 |

“ | 5 | 1 | 126 |

“ | 6 | 1 | 462 |

Pooling positions $\left({x}_{3},{y}_{3},{y}_{6}\right)$ | 4 | 1 | 16 |

“ | 5 | 1 | 32 |

“ | 6 | 1 | 64 |

Initial learning rate $\left({x}_{4},{y}_{7}\right)$ | 4, 5, 6 | ${10}^{-4}$ | $0.1$ |

Momentum $\left({x}_{5},{y}_{8}\right)$ | 4, 5, 6 | 0 | 1 |

Dropout probability $\left({y}_{9},{y}_{10}\right)$ | 4, 5, 6 | 0 | 1 |

Kernel | $\mathit{G}\left(\mathit{x},\mathit{y}\right)$ |
---|---|

Linear | $\langle x\xb7y\rangle $ |

Gaussian | $e-{\parallel x-y\parallel}^{2}$ |

Polynomial | ${\left(1+\langle x\xb7y\rangle \right)}^{q}$ with $q\in \mathbb{N}\backslash \left\{0,1\right\}$ |

**Table 4.**RUL prediction performance obtained with genetically optimized networks tested on both depth and normal maps.

Model No. | Model Name | MAPE | RMSE | SF | TT (Sec) |
---|---|---|---|---|---|

1 | go4normal | 0.063 | 0.036 | 0.301 | 63.793 |

2 | go5normal | 0.065 | 0.037 | 0.313 | 18.351 |

3 | go6normal | 0.083 | 0.035 | 0.282 | 16.928 |

4 | go4depth | 0.102 | 0.063 | 0.513 | 13.647 |

5 | go5depth | 0.083 | 0.049 | 0.393 | 18.603 |

6 | go6depth | 0.058 | 0.036 | 0.312 | 13.410 |

7 | go4+4 | 0.097 | 0.039 | 0.349 | 31.233 |

8 | go4+5 | 0.141 | 0.057 | 0.493 | 125.250 |

9 | go4+6 | 0.154 | 0.064 | 0.569 | 33.631 |

10 | go5+4 | 0.123 | 0.048 | 0.427 | 96.948 |

11 | go5+5 | 0.162 | 0.062 | 0.545 | 58.817 |

12 | go5+6 | 0.129 | 0.053 | 0.469 | 58.204 |

13 | go6+4 | 0.158 | 0.067 | 0.593 | 37.651 |

14 | go6+5 | 0.237 | 0.082 | 0.762 | 196.594 |

15 | go6+6 | 0.251 | 0.085 | 0.812 | 35.540 |

Model No. | Model Name | MAPE | RMSE | SF | FTT (Sec) |
---|---|---|---|---|---|

16 | squeezenet | 2.080 | 0.363 | 4.044 | 32.875 |

17 | googlenet | 0.452 | 0.112 | 1.077 | 18.947 |

18 | inceptionv3 | 3.402 | 0.367 | 3.839 | 121.734 |

19 | densenet201 | 1.802 | 0.559 | 5.453 | 758.300 |

20 | mobilenetv2 | 2.692 | 0.608 | 6.471 | 94.666 |

21 | resnet18 | 3.053 | 0.638 | 6.719 | 48.073 |

22 | resnet50 | 1.854 | 0.426 | 4.086 | 140.182 |

23 | resnet101 | 1.193 | 0.432 | 4.095 | 270.903 |

24 | xception | 1.825 | 0.365 | 1.550 | 833.500 |

25 | inceptionresnetv2 | 2.691 | 0.555 | 5.365 | 755.150 |

26 | shufflenet | 2.673 | 0.905 | 9.534 | 73.828 |

27 | nasnetmobile | 1.597 | 0.412 | 4.276 | 352.860 |

28 | darknet19 | 1.623 | 0.340 | 1.332 | 108.202 |

29 | darknet53 | 1.384 | 0.485 | 1.870 | 565.150 |

30 | efficientnetb0 | 1.408 | 0.338 | 3.306 | 195.668 |

31 | alexnet | 1.048 | 0.160 | 1.482 | 37.516 |

32 | vgg16 | 0.615 | 0.311 | 1.177 | 310.489 |

33 | vgg19 | 0.698 | 0.484 | 1.821 | 405.430 |

Model No. | Model Name | MAPE | RMSE | SF | FTT (Sec) |
---|---|---|---|---|---|

34 | squeezenet | 3.789 | 0.321 | 3.175 | 36.501 |

35 | googlenet | 0.416 | 0.114 | 1.142 | 56.326 |

36 | inceptionv3 | 2.200 | 0.312 | 3.067 | 267.350 |

37 | densenet201 | 1.657 | 0.527 | 5.387 | 785.250 |

38 | mobilenetv2 | 2.108 | 0.452 | 4.768 | 104.493 |

39 | resnet18 | 1.602 | 0.448 | 4.576 | 48.366 |

40 | resnet50 | 1.181 | 0.312 | 2.963 | 147.724 |

41 | resnet101 | 1.838 | 0.633 | 7.828 | 283.090 |

42 | xception | 1.790 | 0.409 | 1.725 | 915.500 |

43 | inceptionresnetv2 | 1.530 | 0.266 | 2.519 | 745.000 |

44 | shufflenet | 2.062 | 0.694 | 7.401 | 74.328 |

45 | nasnetmobile | 1.309 | 0.324 | 3.329 | 345.805 |

46 | darknet19 | 9.471 | 3.291 | 14.630 | 126.012 |

47 | darknet53 | 0.878 | 0.423 | 1.557 | 580.700 |

48 | efficientnetb0 | 0.870 | 0.274 | 2.761 | 183.381 |

49 | alexnet | 0.822 | 0.311 | 2.921 | 35.722 |

50 | vgg16 | 1.095 | 0.616 | 2.476 | 312.283 |

51 | vgg19 | 0.648 | 0.452 | 1.683 | 409.718 |

Model No. | Model Name | MAPE | RMSE | SF | TT (Sec) |
---|---|---|---|---|---|

52 | linear | 1.073 | 0.135 | 1.444 | 4.667 |

53 | gaussian | 1.190 | 0.182 | 1.673 | 4.117 |

54 | polynomial3 | 1.107 | 0.165 | 1.502 | 5.053 |

55 | polynomial4 | 0.909 | 0.124 | 1.180 | 5.726 |

56 | polynomial5 | 0.862 | 0.113 | 1.134 | 5.734 |

57 | polynomial6 | 0.857 | 0.120 | 1.179 | 5.807 |

Model No. | Model Name | Network Architecture | $\mathbf{Parameters}\left({10}^{6}\right)$ |
---|---|---|---|

1 | go4normal | S = (29,25,19,17) | 1.39 |

N = (32,32,32,32) | |||

P = (0,0,0,0) | |||

ILR = 0.0059 | |||

M = 0.8634 | |||

2 | go5normal | S = (5,7,19,23,29) | 4.95 |

N = (32,32,32,64,64) | |||

P = (1,0,0,0,0) | |||

ILR = 0.0068 | |||

M = 0.8049 | |||

3 | go6normal | S = (29,25,23,19,11,7) | 1.72 |

N = (16,32,32,32,64,64) | |||

P = (1,1,1,1,0,0) | |||

ILR = 0.0047 | |||

M = 0.7572 | |||

4 | go4depth | S = (29,23,21,15) | 0.347 |

N = (16,16,16,16) | |||

P = (1,1,1,1) | |||

ILR = 0.0074 | |||

M = 0.6636 | |||

5 | go5depth | S = (29,27,15,11,3) | 1.19 |

N = (32,32,32,32,32) | |||

P = (1,1,1,0,0) | |||

ILR = 0.0054 | |||

M = 0.6893 | |||

6 | go6depth | S = (3,5,15,21,23,27) | 7.93 |

N = (32,64,64,64,64,64) | |||

P = (1,0,0,0,0,0) | |||

ILR = 0.0077 | |||

M = 0.7702 |

Model No. | Model Name | Network Architecture | $\mathbf{Parameters}\left({10}^{6}\right)$ |
---|---|---|---|

7 | go4+4 | S1 = (5,15,21,29), S2 = (29,27,25,17)N1 = (16,16,16,32), N2 = (16,16,16,16)P1 = (1,1,1,1), P2 = (1,1,1,1)d1 = 0.6791, d2 = 0.2646 ILR = 0.0086 M = 0.6499 | 1.06 |

8 | go4+5 | S1 = (27,23,19,3), S2 = (29,25,23,13,7)N1 = (32,32,32,32), N2 = (16,64,64,64,64)P1 = (1,0,0,0), P2 = (0,0,0,0,0)d1 = 0.2452, d2 = 0.2494 ILR = 0.0084 M = 0.5046 | 4.73 |

9 | go4+6 | S1 = (5,23,27,29), S2 = (27,25,23,17,13,9)N1 = (16,32,32,32), N2 = (16,16,16,32,64,64)P1 = (1,0,0,0), P2 = (0,0,0,0,0,0)d1 = 0.2659, d2 = 0.4718 ILR = 0.0069 M = 0.6599 | 7.76 |

10 | go5+4 | S1 = (27,25,23,21,17), S2 = (29,25,11,5)N1 = (16,32,32,32,64), N2 = (32,32,32,32)P1 = (0,0,0,0,0), P2 = (0,0,0,0)d1 = 0.598, d1 = 0.282 ILR = 0.008 M = 0.804 | 2.81 |

11 | go5+5 | S1 = (25,21,19,15,7), S2 = (29,21,15,13,3)N1 = (64,64,64,64,64), N2 = (16,32,32,64,64)P1 = (1,1,1,0,0), P2 = (0,0,0,0,0)d1 = 0.1296, d2 = 0.1075 ILR = 0.0089 M = 0.4570 | 5.41 |

12 | go5+6 | S1 = (9,11,13,25,27), S2 = (7,11,17,19,21,27)N1 = (16,16,32,64,64), N2 = (16,32,64,64,64,64)P1 = (1,0,0,0,0), P2 = (0,0,0,0,0,0)d1 = 0.425, d1 = 0.355 ILR = 0.006 M = 0.464 | 11.32 |

13 | go6+4 | S1 = (5,15,19,21,23,29), S2 = (11,17,19,25)N1 = (16,16,32,32,32,32), N2 = (16,16,32,32)P1 = (1,0,0,0,0,0), P2 = (1,1,0,0)d1 = 0.7385, d2 = 0.5250 ILR = 0.0039 M = 0.1515 | 3.00 |

14 | go6+5 | S1 = (29,27,19,17,13,5), S2 = (7,17,21,27,29)N1 = (64,64,64,64,64,64),N2 = (16,32,32,32,32)P1 = (0,0,0,0,0,0), P2 = (1,0,0,0,0)d1 = 0.8581, d2 = 0.5764 ILR = 0.0074 M = 0.5507 | 8.82 |

15 | go6+6 | S1 = (3,13,15,21,25,29), S2 = (3,11,13,17,21,25)N1 = (16,16,16,32,64,64),N2 = (16,16,64,64,64,64)P1 = (1,0,0,0,0,0), P2 = (1,0,0,0,0,0)d1 = 0.7867, d2 = 0.6495 ILR = 0.0046 M = 0.6290 | 10.81 |

Study | Dataset | Degradation | Approach | MAPE | RMSE | TT/FTT |
---|---|---|---|---|---|---|

Wu et al. [32] | 8400 camera images | Tool wear | Transfer learning | 0.0476 | - | 2 ÷ 333 h |

Marei et al. [33] | 327 camera images | Tool wear | Transfer learning | - | 0.1654 | 3358 s |

Mo et al. [37] | 21 time series (engine sensors) | Turbofan engine | Genetically optimized | - | 11.28 | - |

Ren et al. [19] | 15 time series (vibration sensor) | Bearing wear | Manually engineered | - | 0.2 | - |

This study | 2014 scans | Punch tool | Transfer learning | 0.416 | 0.112 | 19 ÷ 916 s |

This study | 2014 scans | Punch tool | Genetically optimized | 0.058 | 0.035 | 13 ÷ 125 s |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Diraco, G.; Siciliano, P.; Leone, A.
Remaining Useful Life Prediction from 3D Scan Data with Genetically Optimized Convolutional Neural Networks. *Sensors* **2021**, *21*, 6772.
https://doi.org/10.3390/s21206772

**AMA Style**

Diraco G, Siciliano P, Leone A.
Remaining Useful Life Prediction from 3D Scan Data with Genetically Optimized Convolutional Neural Networks. *Sensors*. 2021; 21(20):6772.
https://doi.org/10.3390/s21206772

**Chicago/Turabian Style**

Diraco, Giovanni, Pietro Siciliano, and Alessandro Leone.
2021. "Remaining Useful Life Prediction from 3D Scan Data with Genetically Optimized Convolutional Neural Networks" *Sensors* 21, no. 20: 6772.
https://doi.org/10.3390/s21206772