Collaborative Multiple Players to Address Label Sparsity in Quality Prediction of Batch Processes
Abstract
:1. Introduction
- Rather than using only two base learners, we investigate the efficient number of different base learners used in co-training, which paves the way for capturing multi-channeled input features that are leveraged in pseudo-labels generation;
- Once the pseudo-labels complete the data augmentation, a sliding window is skillfully embedded preceding the feature engineering, to account for the unique 2D dynamics of batch processes;
- Leveraging the pseudo-labels inferred by the local feature similarity, a deep learning interface named 2D-GSTAE, is further connected to synthesize all the perspectives presented by the previous base learners, promoting a more comprehensive relationship between the input process data and the online estimated output.
2. Preliminary
2.1. Original Co-Training
2.2. Co-Training Regressors (Coreg)
3. Proposed Method
3.1. Collaborative Multiple-Player Structure to Infer the Labels in Individual Perspectives
3.2. Preprocess and 2D Slide Window Preceding the Modeling
3.3. Deep Learning: Fusing and Prediction
3.4. Evaluation Indicators
4. Case Study
4.1. Penicillin Fermentation Simulation Case
4.1.1. Experiment Design in Simulation Case
4.1.2. Parameter Settings in Simulation Case
4.1.3. Results and Analysis in Simulation Case
4.2. Real Lactic Acid Bacteria Fermentation Case
4.2.1. Experiment Design in Real Case
4.2.2. Parameter Settings in Real Case
4.2.3. Result and Analysis in Real Case
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
References
- Zhang, Z.; Zhu, J.; Zhang, S.; Gao, F. Process monitoring using recurrent Kalman variational auto-encoder for general complex dynamic processes. Eng. Appl. Artif. Intell. 2023, 123, 106424. [Google Scholar] [CrossRef]
- Zhu, J.; Gao, F. Improved nonlinear quality estimation for multiphase batch processes based on relevance vector machine with neighborhood component variable selection. Ind. Eng. Chem. Res. 2018, 57, 666–676. [Google Scholar] [CrossRef]
- Curreri, F.; Patanè, L.; Xibilia, M.G. Soft sensor transferability: A survey. Appl. Sci. 2021, 11, 7710. [Google Scholar] [CrossRef]
- Sun, Q.; Ge, Z. A survey on deep learning for data-driven soft sensors. IEEE Trans. Ind. Inform. 2021, 17, 5853–5866. [Google Scholar] [CrossRef]
- Zheng, J.; Shen, F.; Ye, L. Improved mahalanobis distance based JITL-LSTM soft sensor for multiphase batch processes. IEEE Access 2021, 9, 72172–72182. [Google Scholar] [CrossRef]
- Zhang, Z.; Jiang, Q.; Wang, G.; Pan, C.; Cao, Z.; Yan, X.; Zhuang, Y. Neural networks-based hybrid beneficial variable selection and modeling for soft sensing. Control Eng. Pract. 2023, 139, 105613. [Google Scholar] [CrossRef]
- Shen, F.; Zheng, J.; Ye, L.; Ma, X. LSTM soft sensor development of batch processes with multivariate trajectory-based ensemble just-in-time learning. IEEE Access 2020, 8, 73855–73864. [Google Scholar]
- Hua, L.; Zhang, C.; Sun, W.; Li, Y.; Xiong, J.; Nazir, M.S. An evolutionary deep learning soft sensor model based on random forest feature selection technique for penicillin fermentation process. ISA Trans. 2023, 136, 139–151. [Google Scholar] [CrossRef]
- Zhu, X.; Rehman, K.U.; Wang, B.; Shahzad, M. Modern soft-sensing modeling methods for fermentation processes. Sensors 2020, 20, 1771. [Google Scholar] [CrossRef]
- Brunner, V.; Siegl, M.; Geier, D.; Becker, T. Challenges in the development of soft sensors for bioprocesses: A critical review. Front. Bioeng. Biotechnol. 2021, 9, 722202. [Google Scholar] [CrossRef]
- Qiu, K.; Wang, J.; Wang, R.; Guo, Y.; Zhao, L. Soft sensor development based on kernel dynamic time warping and a relevant vector machine for unequal-length batch processes. Expert Syst. Appl. 2021, 182, 115223. [Google Scholar] [CrossRef]
- Wang, J.; Qiu, K.; Wang, R.; Zhou, X.; Guo, Y. Development of soft sensor based on sequential kernel fuzzy partitioning and just-in-time relevance vector machine for multiphase batch processes. IEEE Trans. Instrum. Meas. 2021, 70, 2509110. [Google Scholar] [CrossRef]
- Ji, C.; Ma, F.; Wang, J.; Sun, W. Profitability related industrial-scale batch processes monitoring via deep learning based soft sensor development. Comput. Chem. Eng. 2023, 170, 108125. [Google Scholar] [CrossRef]
- Yuan, X.; Wang, Y.; Yang, C.; Gui, W. Stacked isomorphic autoencoder based soft analyzer and its application to sulfur recovery unit. Inf. Sci. 2020, 534, 72–84. [Google Scholar] [CrossRef]
- Yuan, X.; Feng, L.; Wang, K.; Wang, Y.; Ye, L. Deep learning for data modeling of multirate quality variables in industrial processes. IEEE Trans. Instrum. Meas. 2021, 70, 2509611. [Google Scholar] [CrossRef]
- Jiang, Q.; Wang, Z.; Yan, S.; Cao, Z. Data-driven soft sensing for batch processes using neural network-based deep quality-relevant representation learning. IEEE Trans. Artif. Intell. 2022, 4, 602–611. [Google Scholar] [CrossRef]
- Ren, J.-C.; Liu, D.; Wan, Y. VMD-SEAE-TL-Based Data-Driven soft sensor modeling for a complex industrial batch processes. Measurement 2022, 198, 111439. [Google Scholar] [CrossRef]
- Sun, Y.-N.; Qin, W.; Xu, H.-W.; Tan, R.-Z.; Zhang, Z.-L.; Shi, W.-T. A multiphase information fusion strategy for data-driven quality prediction of industrial batch processes. Inf. Sci. 2022, 608, 81–95. [Google Scholar] [CrossRef]
- Gopakumar, V.; Tiwari, S.; Rahman, I. A deep learning based data driven soft sensor for bioprocesses. Biochem. Eng. J. 2018, 136, 28–39. [Google Scholar] [CrossRef]
- Ge, Z. Semi-supervised data modeling and analytics in the process industry: Current research status and challenges. IFAC J. Syst. Control 2021, 16, 100150. [Google Scholar] [CrossRef]
- Van Engelen, J.E.; Hoos, H.H. A survey on semi-supervised learning. Mach. Learn. 2020, 109, 373–440. [Google Scholar] [CrossRef]
- Ge, Z.; Song, Z.; Gao, F. Self-training statistical quality prediction of batch processes with limited quality data. Ind. Eng. Chem. Res. 2013, 52, 979–984. [Google Scholar] [CrossRef]
- Jin, H.; Li, Z.; Chen, X.; Qian, B.; Yang, B.; Yang, J. Evolutionary optimization based pseudo labeling for semi-supervised soft sensor development of industrial processes. Chem. Eng. Sci. 2021, 237, 116560. [Google Scholar] [CrossRef]
- Esche, E.; Talis, T.; Weigert, J.; Rihm, G.B.; You, B.; Hoffmann, C.; Repke, J.-U. Semi-supervised learning for data-driven soft-sensing of biological and chemical processes. Chem. Eng. Sci. 2022, 251, 117459. [Google Scholar] [CrossRef]
- Li, Z.; Jin, H.; Dong, S.; Qian, B.; Yang, B.; Chen, X. Semi-supervised ensemble support vector regression based soft sensor for key quality variable estimation of nonlinear industrial processes with limited labeled data. Chem. Eng. Res. Des. 2022, 179, 510–526. [Google Scholar] [CrossRef]
- Ning, X.; Wang, X.; Xu, S.; Cai, W.; Zhang, L.; Yu, L.; Li, W. A review of research on co-training. Concurr. Comput. Pract. Exp. 2023, 35, e6276. [Google Scholar] [CrossRef]
- Blum, A.; Mitchell, T. Combining labeled and unlabeled data with co-training. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, Madison, WI, USA, 24–26 July 1998; pp. 92–100. [Google Scholar]
- Zhou, Z.-H.; Li, M. Semi-supervised regression with co-training. In Proceedings of the IJCAI, Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, UK, 30 July–5 August 2005; pp. 908–913. [Google Scholar]
- Bao, L.; Yuan, X.; Ge, Z. Co-training partial least squares model for semi-supervised soft sensor development. Chemom. Intell. Lab. Syst. 2015, 147, 75–85. [Google Scholar] [CrossRef]
- Tang, Q.; Li, D.; Xi, Y. Soft sensor modeling based on cotraining-style kernel extreme learning machine. In Proceedings of the 2017 Chinese Automation Congress (CAC), Jinan, China, 20–22 October 2017; pp. 4028–4033. [Google Scholar]
- Zhao, L.; Zhu, J.; Zhang, Z.; Xie, Z.; Gao, F. A Novel Semi-supervised Two-dimensional Dynamic Soft Sensor for Quality Prediction in Batch Processes. In Proceedings of the 2023 5th International Conference on Industrial Artificial Intelligence (IAI), Shenyang, China, 21–24 August 2023; pp. 1–6. [Google Scholar]
- Sun, Q.; Ge, Z. Gated stacked target-related autoencoder: A novel deep feature extraction and layerwise ensemble method for industrial soft sensor application. IEEE Trans. Cybern. 2020, 52, 3457–3468. [Google Scholar] [CrossRef]
- Birol, G.; Ündey, C.; Cinar, A. A modular simulation package for fed-batch fermentation: Penicillin production. Comput. Chem. Eng. 2002, 26, 1553–1565. [Google Scholar] [CrossRef]
Number | Variable | Unit |
---|---|---|
1 | aeration rate | L/h |
2 | agitator power | W |
3 | substrate feed rate | L/h |
4 | substrate feed temperature | K |
5 | substrate concentration | g/L |
6 | culture volume | L |
7 | carbon dioxide concentration | g/L |
8 | pH | - |
9 | temperature | K |
10 | generated heat | kcal |
11 | cold water flow rate | L/h |
y | penicillin concentration | g/L |
Method | 5:1 | 10:1 | 20:1 | 25:1 | 40:1 | 50:1 | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
RMSE | R2 | RMSE | R2 | RMSE | R2 | RMSE | R2 | RMSE | R2 | RMSE | R2 | |
SAE | 0.0628 | 0.9814 | 0.0769 | 0.9737 | 0.1024 | 0.9541 | 0.1230 | 0.9319 | 0.2637 | 0.6980 | 0.7977 | −2.5510 |
SIAE | 0.0521 | 0.9878 | 0.0555 | 0.9862 | 0.0902 | 0.9657 | 0.1029 | 0.9528 | 0.2791 | 0.6203 | 0.6989 | −1.5870 |
GSTAE | 0.0183 | 0.9986 | 0.0207 | 0.9982 | 0.0333 | 0.9953 | 0.0423 | 0.9923 | 0.1060 | 0.9507 | 0.2119 | 0.8036 |
SS-GSTAE | 0.0157 | 0.9990 | 0.0184 | 0.9986 | 0.0293 | 0.9964 | 0.0363 | 0.9945 | 0.1023 | 0.9549 | 0.1568 | 0.8964 |
2D-GSTAE | 0.0119 | 0.9994 | 0.0156 | 0.9990 | 0.0293 | 0.9964 | 0.0345 | 0.9950 | 0.0927 | 0.9622 | 0.2027 | 0.8202 |
SS-2D-GSTAE | 0.0110 | 0.9995 | 0.0115 | 0.9994 | 0.0230 | 0.9977 | 0.0298 | 0.9951 | 0.0645 | 0.9819 | 0.1147 | 0.9402 |
Coreg | 0.0197 | 0.9984 | 0.0247 | 0.9975 | 0.0344 | 0.9951 | 0.0386 | 0.9938 | 0.0540 | 0.9878 | 0.0732 | 0.9777 |
CO-MP-DSSM | 0.0188 | 0.9985 | 0.0228 | 0.9978 | 0.0309 | 0.9960 | 0.0366 | 0.9943 | 0.0518 | 0.9888 | 0.0718 | 0.9786 |
Number | Variable | Unit |
---|---|---|
1 | temperature | K |
2 | pH | - |
3 | dissolved oxygen | - |
4 | agitator rate | r/min |
5 | acid supplement | mL |
6 | base supplement | mL |
7 | substrate supplement | mL |
y | lactic acid bacteria concentration | - |
Method | 30:1 | 60:1 | 120:1 | |||
---|---|---|---|---|---|---|
RMSE | R2 | RMSE | R2 | RMSE | R2 | |
SAE | 0.7204 | 0.9186 | 0.7804 | 0.9031 | 0.9759 | 0.8574 |
SIAE | 0.8215 | 0.8951 | 0.9429 | 0.8657 | 0.9597 | 0.8513 |
GSTAE | 0.7704 | 0.9111 | 0.8365 | 0.8965 | 0.9242 | 0.8746 |
SS-GSTAE | 0.6832 | 0.9314 | 0.7515 | 0.9163 | 0.8734 | 0.8880 |
2D-GSTAE | 0.6962 | 0.9287 | 0.7622 | 0.9143 | 0.9843 | 0.8576 |
SS-2D-GSTAE | 0.9266 | 0.8727 | 1.0061 | 0.8496 | 1.0604 | 0.8341 |
Coreg | 0.9731 | 0.8611 | 0.9558 | 0.8660 | 0.9722 | 0.8614 |
CO-MP-DSSM | 0.5593 | 0.9538 | 0.5698 | 0.9524 | 0.7455 | 0.9184 |
Number of Players | 30:1 | 60:1 | 120:1 | |||
---|---|---|---|---|---|---|
RMSE | R2 | RMSE | R2 | RMSE | R2 | |
1 | 0.6032 | 0.9465 | 0.6630 | 0.9349 | 0.7427 | 0.9188 |
2 | 0.5863 | 0.9490 | 0.6000 | 0.9471 | 0.7540 | 0.9165 |
3 | 0.5593 | 0.9538 | 0.5698 | 0.9524 | 0.7455 | 0.9184 |
4 | 0.5792 | 0.9505 | 0.6261 | 0.9422 | 0.7301 | 0.9218 |
5 | 0.5897 | 0.9487 | 0.7177 | 0.9239 | 0.7807 | 0.9106 |
6 | 0.6359 | 0.9406 | 0.6329 | 0.9412 | 0.7649 | 0.9142 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhao, L.; Zhang, Z.; Zhu, J.; Wang, H.; Xie, Z. Collaborative Multiple Players to Address Label Sparsity in Quality Prediction of Batch Processes. Sensors 2024, 24, 2073. https://doi.org/10.3390/s24072073
Zhao L, Zhang Z, Zhu J, Wang H, Xie Z. Collaborative Multiple Players to Address Label Sparsity in Quality Prediction of Batch Processes. Sensors. 2024; 24(7):2073. https://doi.org/10.3390/s24072073
Chicago/Turabian StyleZhao, Ling, Zheng Zhang, Jinlin Zhu, Hongchao Wang, and Zhenping Xie. 2024. "Collaborative Multiple Players to Address Label Sparsity in Quality Prediction of Batch Processes" Sensors 24, no. 7: 2073. https://doi.org/10.3390/s24072073
APA StyleZhao, L., Zhang, Z., Zhu, J., Wang, H., & Xie, Z. (2024). Collaborative Multiple Players to Address Label Sparsity in Quality Prediction of Batch Processes. Sensors, 24(7), 2073. https://doi.org/10.3390/s24072073