Pre-trained 1DCNN-BiLSTM Hybrid Network for Temperature Prediction of Wind Turbine Gearboxes
Abstract
:1. Introduction
- A novel hybrid 1DCNN-BiLSTM model is designed in this paper. Thus, the most important spatial and periodic features contained in the temperature data can be completely extracted by the hybrid model; thus, higher prediction accuracy can be achieved.
- The pre-training method is creatively introduced into the model training. As a result, it is not necessary to randomly initialize the parameters of the hybrid model. This effectively prevents the training process from being trapped by local minima and significantly improves the prediction accuracy of the trained network.
- Several experiments are conducted to evaluate the effectiveness of the proposed model by the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset and the real wind turbine temperature dataset. Evidently, the proposed model outperforms certain classical models for temperature prediction.
2. Theoretical Background
2.1. One-Dimensional Convolutional Neural Network
2.1.1. Traditional Convolutional Neural Network (CNN)
2.1.2. The One-Dimensional Convolutional Neural Network
2.2. The Bi-Direction Long Short-Term Memory Network
2.3. The Basic Theory of Pre-Training
3. The Proposed Pre-trained 1DCNN-BiLSTM Model
3.1. Overview of the Proposed Hybrid Model
3.2. Data Collection and Processing
- The first n data points are selected as a training sample and the next data point as the label for this training sample. The value of n can be selected according to the complexity of the input data.
- Then, the first data point in the dataset is removed.
- The next data pair is generated by repeating steps one and two until the number of data points is less than or equal to n.
3.3. The Components of the Hybrid Model
3.4. Building the Hybrid Model
4. Case studies of Temperature Prediction
4.1. Case 1: Experiments Using the C-MAPSS Dataset
- M0: proposed pre-trained 1DCNN-BiLSTM model,
- M1: CNN model,
- M2: RNN model,
- M3: CNN-LSTM model,
- M4: 1DCNN-BiLSTM model without pre-training, and
- M5: residual shrink neural network.
4.2. Experiments Using Measured Dataset from Hejiashan Wind Farm
4.2.1. Case 2a: Wind Turbine Gearbox Temperature
4.2.2. Case 2b: Wind Turbine Bearing Temperature
5. Conclusions
- (a)
- The hybrid model retains the advantages of the two single models, and therefore, its more useful features can be extracted by the hybrid model, improving its performance at temperature prediction.
- (b)
- The pre-training method can help the model to obtain a better optimization path to obtain optimal parameters. Thus, the ability of the anti-interference of the model can be improved.
- (c)
- From the results of the experiments based on measured temperature data, the appropriateness of the proposed model in real applications is demonstrated.
- (a)
- The deep learning-based approach demonstrates strong generalization capabilities; it is reasonable to do further research on its universality. Employing the proposed method to predict the operational temperature of some other mechanical equipment, such as cement production machinery, aerospace engines, and so on, is worthy of further research.
- (b)
- The predictive accuracy of the proposed model in Case 1 is observed to be lower than in the other two cases. This is probably attributed to the insufficient data volume and the high complexity of the dataset. Therefore, in subsequent research, further refinement of the model’s structure and parameters can be performed. Additionally, designing a dynamic loss function to capture dynamic biases could be considered, eventually achieving higher prediction accuracy based on a small sample size dataset.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Han, J.; Chang, H. Development and opportunities of clean energy in China. Appl. Sci. 2022, 12, 4783. [Google Scholar] [CrossRef]
- Zhang, W.; Li, B.; Xue, R.; Wang, C.; Cao, W. A systematic bibliometric review of clean energy transition: Implications for low-carbon development. PLoS ONE 2021, 16, e0261091. [Google Scholar] [CrossRef] [PubMed]
- Liu, H.; Liang, D. A review of clean energy innovation and technology transfer in China. Renew. Sustain. Energy Rev. 2013, 18, 486–498. [Google Scholar] [CrossRef]
- Rinaldi, G.; Thies, P.R.; Johanning, L. Current status and future trends in the operation and maintenance of offshore wind turbines: A review. Energies 2021, 14, 2484. [Google Scholar] [CrossRef]
- Gao, Z.; Liu, X. An overview on fault diagnosis, prognosis and resilient control for wind turbine systems. Processes 2021, 9, 300. [Google Scholar] [CrossRef]
- Guo, Z.; Pu, Z.; Du, W.; Wang, H.; Li, C. Improved adversarial learning for fault feature generation of wind turbine gearbox. Renew. Energy 2022, 185, 255–266. [Google Scholar] [CrossRef]
- Heydari, A.; Garcia, D.A.; Fekih, A.; Keynia, F.; Tjernberg, L.B.; De Santoli, L. A hybrid intelligent model for the condition monitoring and diagnostics of wind turbines gearbox. IEEE Access 2021, 9, 89878–89890. [Google Scholar] [CrossRef]
- Yu, X.; Tang, B.; Zhang, K. Fault diagnosis of wind turbine gearbox using a novel method of fast deep graph convolutional networks. IEEE Trans. Instrum. Meas. 2021, 70, 6502714. [Google Scholar] [CrossRef]
- Zhang, X.; Zhong, J.; Li, W.; Bocian, M. Nonlinear dynamic analysis of high-speed gear pair with wear fault and tooth contact temperature for a wind turbine gearbox. Mech. Mach. Theory 2022, 173, 104840. [Google Scholar] [CrossRef]
- Liu, H.; Yu, C.; Yu, C. A new hybrid model based on secondary decomposition, reinforcement learning and SRU network for wind turbine gearbox oil temperature forecasting. Measurement 2021, 178, 109347. [Google Scholar] [CrossRef]
- Yang, Y.; Liu, A.; Xin, H.; Wang, J. Fault early warning of wind turbine gearbox based on multi-input support vector regression and improved ant lion optimization. Wind Energy 2021, 24, 812–832. [Google Scholar] [CrossRef]
- Wang, A.; Qian, Z.; Pei, Y.; Jing, B. A de-ambiguous condition monitoring scheme for wind turbines using least squares generative adversarial networks. Renew. Energy 2022, 185, 267–279. [Google Scholar] [CrossRef]
- Foley, A.M.; Leahy, P.G.; Marvuglia, A.; McKeogh, E.J. Current methods and advances in forecasting of wind power generation. Renew. Energy 2012, 37, 1–8. [Google Scholar] [CrossRef]
- Tran TT, K.; Bateni, S.M.; Ki, S.J.; Vosoughifar, H. A review of neural networks for air temperature forecasting. Water 2021, 13, 1294. [Google Scholar] [CrossRef]
- Baoyu, L.; Guoxing, L.; Guiyu, W.; Guofeng, Z.; Man, Y. Research on CART Model of Mass Concrete Temperature Prediction Based on Big Data Processing Technology. IEEE Access 2022, 10, 32845–32854. [Google Scholar] [CrossRef]
- Das, S.; Politis, D.N. Predictive inference for locally stationary time series with an application to climate data. J. Am. Stat. Assoc. 2021, 116, 919–934. [Google Scholar] [CrossRef]
- Wang, N.; Li, J.Y. Efficient multi-channel thermal monitoring and temperature prediction based on improved linear regression. IEEE Trans. Instrum. Meas. 2021, 71, 9500809. [Google Scholar] [CrossRef]
- Zou, L.; Lam, H.F.; Hu, J. Adaptive resize-residual deep neural network for fault diagnosis of rotating machinery. Struct. Health Monit. 2022, 22, 2193–2213. [Google Scholar] [CrossRef]
- Mu, R.; Zeng, X. A review of deep learning research. KSII Trans. Internet Inf. Syst. 2019, 13, 1738–1764. [Google Scholar] [CrossRef]
- Alzubaidi, L.; Zhang, J.; Humaidi, A.J.; Al-Dujaili, A.; Duan, Y.; Al-Shamma, O.; Santamaría, J.; Fadhel, M.A.; Al-Amidie, M.; Farhan, L. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. J. Big Data 2021, 8, 1–74. [Google Scholar] [CrossRef]
- Sezer, O.B.; Gudelek, M.U.; Ozbayoglu, A.M. Financial time series forecasting with deep learning: A systematic literature review: 2005–2019. Appl. Soft Comput. 2020, 90, 106181. [Google Scholar] [CrossRef]
- Zhao, Z.; Chen, W.; Wu, X.; Chen, P.C.; Liu, J. LSTM network: A deep learning approach for short-term traffic forecast. IET Intell. Transp. Syst. 2017, 11, 68–75. [Google Scholar] [CrossRef]
- Vlachas, P.R.; Byeon, W.; Wan, Z.Y.; Sapsis, T.P.; Koumoutsakos, P. Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks. Proc. R. Soc. A Math. Phys. Eng. Sci. 2018, 474, 20170844. [Google Scholar] [CrossRef] [PubMed]
- Jia, X.; Ji, Q.; Han, L.; Liu, Y.; Han, G.; Lin, X. Prediction of Sea Surface Temperature in the East China Sea Based on LSTM Neural Network. Remote Sens. 2022, 14, 3300. [Google Scholar] [CrossRef]
- Wu, X.; Li, J.; Jin, Y.; Zheng, S. Modeling and analysis of tool wear prediction based on SVD and BiLSTM. Int. J. Adv. Manuf. Technol. 2020, 106, 4391–4399. [Google Scholar] [CrossRef]
- Jiang, L.; Yan, C.; Zhang, X.; Zhou, B.; Cheng, T.; Zhao, J.; Gu, J. Temperature prediction of battery energy storage plant based on EGA-BiLSTM. Energy Rep. 2022, 8, 1009–1018. [Google Scholar] [CrossRef]
- Shahid, F.; Zameer, A.; Muneeb, M. Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM. Chaos Solitons Fractals 2020, 140, 110212. [Google Scholar] [CrossRef]
- Zheng, H.; Lin, F.; Feng, X.; Chen, Y. A hybrid deep learning model with attention-based conv-LSTM networks for short-term traffic flow prediction. IEEE Trans. Intell. Transp. Syst. 2020, 22, 6910–6920. [Google Scholar] [CrossRef]
- Kun, X.; Shan, T.; Yi, T.; Chao, C. Attention-based long short-term memory network temperature prediction model. In Proceedings of the 2021 7th International Conference on Condition Monitoring of Machinery in Non-Stationary Operations (CMMNO), Guangzhou, China, 11–13 June 2021; pp. 278–281. [Google Scholar] [CrossRef]
- Chen, X.; Zhang, B.; Gao, D. Bearing fault diagnosis base on multi-scale CNN and LSTM model. J. Intell. Manuf. 2021, 32, 971–987. [Google Scholar] [CrossRef]
- Li, T.; Hua, M.; Wu, X.U. A hybrid CNN-LSTM model for forecasting particulate matter (PM2. 5). IEEE Access 2020, 8, 26933–26940. [Google Scholar] [CrossRef]
- Qiao, W.; Li, Z.; Liu, W.; Liu, E. Fastest-growing source prediction of US electricity production based on a novel hybrid model using wavelet transform. Int. J. Energy Res. 2022, 46, 1766–1788. [Google Scholar] [CrossRef]
- Jun, T.J.; Eom, Y.; Kim, D.; Kim, C.; Park, J.H.; Nguyen, H.M.; Kim, Y.H.; Kim, D. TRk-CNN: Transferable ranking-CNN for image classification of glaucoma, glaucoma suspect, and normal eyes. Expert Syst. Appl. 2021, 182, 115211. [Google Scholar] [CrossRef]
- Ilesanmi, A.E.; Ilesanmi, T.O. Methods for image denoising using convolutional neural network: A review. Complex Intell. Syst. 2021, 7, 2179–2198. [Google Scholar] [CrossRef]
- Bai, Y.; Liu, S.; He, Y.; Cheng, L.; Liu, F.; Geng, X. Identification of MOSFET Working State Based on the Stress Wave and Deep Learning. IEEE Trans. Instrum. Meas. 2022, 71, 9003209. [Google Scholar] [CrossRef]
- Ye, W.; Jiang, Z.; Li, Q.; Liu, Y.; Mou, Z. A hybrid model for pathological voice recognition of post-stroke dysarthria by using 1DCNN and double-LSTM networks. Appl. Acoust. 2022, 197, 108934. [Google Scholar] [CrossRef]
- Liu, Z.; Wang, H.; Liu, J.; Qin, Y.; Peng, D. Multitask learning based on lightweight 1DCNN for fault diagnosis of wheelset bearings. IEEE Trans. Instrum. Meas. 2020, 70, 3501711. [Google Scholar] [CrossRef]
- Lara-Benítez, P.; Carranza-García, M.; Riquelme, J.C. An experimental review on deep learning architectures for time series forecasting. Int. J. Neural Syst. 2021, 31, 2130001. [Google Scholar] [CrossRef] [PubMed]
- Latifoğlu, L. A novel combined model for prediction of daily precipitation data using instantaneous frequency feature and bidirectional long short time memory networks. Environ. Sci. Pollut. Res. 2022, 29, 42899–42912. [Google Scholar] [CrossRef]
- Chen, B.; Zheng, H.; Wang, L.; Hellwich, O.; Chen, C.; Yang, L.; Liu, T.; Luo, G.; Bao, A.; Chen, X. A joint learning Im-BiLSTM model for incomplete time-series Sentinel-2A data imputation and crop classification. Int. J. Appl. Earth Obs. Geoinf. 2022, 108, 102762. [Google Scholar] [CrossRef]
- Zhou, D.; Zhuang, X.; Zuo, H.; Cai, J.; Zhao, X.; Xiang, J. A model fusion strategy for identifying aircraft risk using CNN and Att-BiLSTM. Reliab. Eng. Syst. Saf. 2022, 228, 108750. [Google Scholar] [CrossRef]
- Rangasamy, K.; As’ari, M.A.; Rahmad, N.A.; Ghazali, N.F. Hockey activity recognition using pre-trained deep learning model. ICT Express 2020, 6, 170–174. [Google Scholar] [CrossRef]
- Zhou, T.; Lu, H.; Yang, Z.; Qiu, S.; Huo, B.; Dong, Y. The ensemble deep learning model for novel COVID-19 on CT images. Appl. Soft Comput. 2021, 98, 106885. [Google Scholar] [CrossRef] [PubMed]
- Gao, J.; Song, Z.; Gui, J.; Yuan, S. Gas-bearing prediction using transfer learning and CNNs: An application to a deep tight dolomite reservoir. IEEE Geosci. Remote Sens. Lett. 2020, 19, 3001005. [Google Scholar] [CrossRef]
- Saxena, A.; Goebel, K.; Simon, D.; Eklund, N. Damage propagation modeling for aircraft engine run-to-failure simulation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA, 6–9 October 2008; pp. 1–9. [Google Scholar]
- Zhao, M.; Zhong, S.; Fu, X.; Tang, B.; Pecht, M. Deep residual shrinkage networks for fault diagnosis. IEEE Trans. Ind. Inform. 2019, 16, 4681–4690. [Google Scholar] [CrossRef]
The Definition of Layers | The Parameters of Layers |
---|---|
Input layer | Input size: (1, 5) |
Convolutional layer 1 | Channel number: 32, nuclear size: 1 |
Pooling layer 1 | Nuclear size: 2, step size: 2 |
Convolutional layer 2 | Channel number: 32, nuclear size: 1 |
Pooling layer 2 | Nuclear size: 2, step size: 2 |
Activation layer 1 | Activation function: ReLU |
Flatten layer | One-dimension feature vector |
Output layer | Output size: 1 |
The Definition of Layers | The Parameters of Each Layer |
---|---|
Input layer | Input size: (1, 16) |
BiLSTM layer | Channel number: 64, activation function: ReLU |
Output layer | Output size: 1 |
The Definition of Layers | The Parameters of Each Layer |
---|---|
Input layer | Input size: (1, 5) |
Convolutional layer 1 | Channel number: 32, nuclear size: 1 |
Pooling layer 1 | Nuclear size: 2, step size: 2 |
Convolutional layer 2 | Channel number: 32, nuclear size: 1 |
Pooling layer 2 | Nuclear size: 2, step size: 2 |
Activation layer 1 | Activation function: ReLU |
BiLSTM layer | Channel number: 64, activation function: ReLU |
Output layer | Output size: 1 |
Model | RMSE | MAE |
---|---|---|
M0 | 22.61107 | 17.46969 |
M1 | 27.59789 | 22.37025 |
M2 | 26.48077 | 21.16881 |
M3 | 30.95766 | 22.54774 |
M4 | 26.64763 | 20.69926 |
M5 | 23.58911 | 18.61125 |
Model | RMSE | MAE | R2 |
---|---|---|---|
M0 | 0.12252 | 0.07602 | 0.99959 |
M1 | 1.73922 | 1.59984 | 0.89635 |
M2 | 2.04201 | 1.85558 | 0.84766 |
M3 | 1.71908 | 1.45505 | 0.89737 |
M4 | 0.57687 | 0.52392 | 0.99061 |
M5 | 1.91532 | 1.55412 | 0.87133 |
Model | RMSE | MAE | R2 |
---|---|---|---|
M0 | 0.29843 | 0.06297 | 0.99349 |
M1 | 0.80648 | 0.62013 | 0.92599 |
M2 | 0.91263 | 0.72302 | 0.89618 |
M3 | 0.72388 | 0.51454 | 0.95233 |
M4 | 0.43028 | 0.22917 | 0.98541 |
M5 | 0.67138 | 0.57061 | 0.96228 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhuang, K.; Ma, C.; Lam, H.-F.; Zou, L.; Hu, J. Pre-trained 1DCNN-BiLSTM Hybrid Network for Temperature Prediction of Wind Turbine Gearboxes. Processes 2023, 11, 3324. https://doi.org/10.3390/pr11123324
Zhuang K, Ma C, Lam H-F, Zou L, Hu J. Pre-trained 1DCNN-BiLSTM Hybrid Network for Temperature Prediction of Wind Turbine Gearboxes. Processes. 2023; 11(12):3324. https://doi.org/10.3390/pr11123324
Chicago/Turabian StyleZhuang, Kejia, Cong Ma, Heung-Fai Lam, Li Zou, and Jun Hu. 2023. "Pre-trained 1DCNN-BiLSTM Hybrid Network for Temperature Prediction of Wind Turbine Gearboxes" Processes 11, no. 12: 3324. https://doi.org/10.3390/pr11123324
APA StyleZhuang, K., Ma, C., Lam, H.-F., Zou, L., & Hu, J. (2023). Pre-trained 1DCNN-BiLSTM Hybrid Network for Temperature Prediction of Wind Turbine Gearboxes. Processes, 11(12), 3324. https://doi.org/10.3390/pr11123324