Research on Automatic Error Data Recognition Method for Structured Light System Based on Residual Neural Network
Abstract
:1. Introduction
2. Principles of Structured Light Measurement and Residual Neural Networks
2.1. Interference Factors in Structured Light Measurement Systems
2.2. Residual Neural Network
3. Methods of Classification and Establishing Data Sets
3.1. Evaluation of Structured Light Stripe Quality
3.2. Sample Data Collection
4. Experimental Results and Analysis
4.1. Structured Light Experiment System
4.2. Model Training
4.3. Testing and Evaluation
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Der Jeught, S.V.; Dirckx, J.J.J. Real-time structured light profilometry: A review. Opt. Lasers Eng. 2016, 87, 18–31. [Google Scholar] [CrossRef]
- Wang, Z.; Fu, Y.; Zhong, K.; Li, Y.; Bao, W. A direct calibration method for line structured light measurement system based on parallel lines. Opt. Commun. 2022, 508, 127699. [Google Scholar] [CrossRef]
- Liu, Y.; Fu, Y.; Zhuan, Y.; Zhou, P.; Guang, B. Large depth-of-field 3d measurement with a microscopic structured-light system. Opt. Commun. 2021, 481, 126540. [Google Scholar] [CrossRef]
- Deetjen, M.E.; Lentink, D. Automated calibration of multi-camera-projector structured light systems for volumetric high-speed 3d surface reconstructions. Opt. Express 2018, 26, 33278. [Google Scholar] [CrossRef] [PubMed]
- Bow, H.; Yang, X.; Chotai, S.; Feldman, M.; Yu, H.; Englot, D.J.; Miga, M.I.; Pruthi, S.; Dawant, B.M.; Parker, S.L. Initial experience with using a structured light 3d scanner and image registration to plan bedside subdural evacuating port system placement–sciencedirect. World Neurosurg. 2020, 137, 350–356. [Google Scholar] [CrossRef] [PubMed]
- Li, Y.; Zhao, B.; Zhou, J.; Ren, Y. A universal method for the calibration of swing-scanning line structured light measurement system. Optik 2021, 241, 166930. [Google Scholar] [CrossRef]
- Wang, M.; Zhang, Q.; Li, Q.; Wu, Z.; Chen, C.; Xu, J.; Xue, J. Research on morphology detection of metal additive manufacturing process based on fringe projection and binocular vision. Appl. Sci. 2022, 12, 9232. [Google Scholar] [CrossRef]
- Wang, Y.; Guo, W.; Zhao, S.; Xue, B.; Xing, Z. A scraper conveyor coal flow monitoring method based on speckle structured light data. Appl. Sci. 2022, 12, 6955. [Google Scholar] [CrossRef]
- Zhao, J.; Cheng, Y.; Cai, G.; Feng, C.; Liao, L.; Xu, B. Correction model of linear structured light sensor in underwater environment. Opt. Lasers Eng. 2022, 153, 107013. [Google Scholar] [CrossRef]
- Saita, M.T.; Dib, L.F.G.; Barbosa, E.A. Modeling of structured light projection on vibrating surfaces: Amplitude and phase mapping by fringe visibility evaluation. Optik 2022, 258, 168833. [Google Scholar] [CrossRef]
- Sun, P.P.; Xue, Q.; Ji, W.; Meng, H.; Yang, X. Analysis and compensation of lateral chromatic aberration of structured light 3d measurement system. Opt. Commun. 2021, 488, 126871. [Google Scholar] [CrossRef]
- Song, Z.; Chung, R.; Zhang, X.T. An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3-D. IEEE Trans. Ind. Electron. 2012, 60, 1023–1032. [Google Scholar] [CrossRef]
- Pham, D.; Ha, M.; Xiao, C. Color structured light Stripe edge detection method based on generative adversarial networks. Appl. Sci. 2023, 13, 198. [Google Scholar] [CrossRef]
- Steger, C. An unbiased detector of curvilinear structures. IEEE Trans. Pattern Anal. Mach. Intell. 1998, 20, 113–125. [Google Scholar] [CrossRef] [Green Version]
- Huang, K.; Yang, J.; Xu, D.; Sun, Y.; Chen, Y.; Li, X. Line laser based researches on a three-dimensional measuring system. In Proceedings of the 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chengdu, China, 15–17 March 2019. [Google Scholar]
- Wang, H.F.; Wang, Y.F.; Zhang, J.J.; Cao, J. Laser stripe center detection under the condition of uneven scattering metal surface for geometric measurement. IEEE Trans. Instrum. Meas. 2020, 69, 2182–2192. [Google Scholar] [CrossRef]
- Xi, J.; Bao, H.; Li, T. Research on the algorithm of noisy laser stripe center extraction. In Proceedings of the 2018 Chinese Control And Decision Conference (CCDC), Shenyang, China, 9–11 June 2018; pp. 6297–6300. [Google Scholar]
- Zhang, L.; Zhang, Y.; Chen, B. Improving the extracting precision of stripe center for structured light measurement. Optik 2020, 207, 163816. [Google Scholar] [CrossRef]
- Ji, W.; Xue, Q.; Sun, X.; Sun, P.; Meng, H. A structured light image quality evaluation method based on no-reference quality assessment. J. Phys. Conf. Ser. 2021, 1914, 012005. [Google Scholar] [CrossRef]
- Alazzawi, O.; Wang, D. A novel structural damage identification method based on the acceleration responses under ambient vibration and an optimized deep residual algorithm. Struct. Health Monit. 2022, 21, 2587–2617. [Google Scholar] [CrossRef]
- Ghannadi, P.; Kourehli, S.S. An effective method for damage assessment based on limited measured locations in skeletal structures. Adv. Struct. Eng. 2021, 24, 183–195. [Google Scholar]
- Ghannadi, P.; Kourehli, S.S. Data-driven method of damage detection using sparse sensors installation by SEREPa. J. Civil. Struct. Health Monit. 2019, 9, 459–475. [Google Scholar] [CrossRef]
- Kang, C.; Kim, T.; Kwon, O.S.; Song, J. Deep neural network-based regional seismic loss assessment considering correlation between EDP residuals of building structures. Earthq. Eng. Struct. Dyn. 2022, 1–21. [Google Scholar] [CrossRef]
- Huang, C.; Zhai, K.K.; Xie, X.; Tan, J. Deep residual network training for reinforced concrete defects intelligent classifier. Eur. J. Environ. Civ. Eng. 2022, 26, 7540–7552. [Google Scholar] [CrossRef]
- Qian, L.; Pan, Q.; Lv, Y.; Zhao, X. Fault Detection of Bearing by ResNet Classifier with Model-Based Data Augmentation. Machines 2022, 10, 521. [Google Scholar] [CrossRef]
- Fan, Z.; Lin, H.; Li, C.; Su, J.; Bruno, S.; Loprencipe, G. Use of Parallel ResNet for High-Performance Pavement Crack Detection and Measurement. Sustainability 2022, 14, 1825. [Google Scholar] [CrossRef]
- Zhong, C.; Gao, Z.; Wang, X.; Shao, S.; Gao, C. Structured light three-dimensional measurement based on machine learning. Sensors 2019, 19, 3229. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ding, C.; Tang, L.; Cao, L.; Shao, X.; Wang, W.; Deng, S. Knowledge-based automatic extraction of multi-structured light stripes. J. Real Time Image Process. 2020, 17, 1015–1027. [Google Scholar] [CrossRef]
- LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE Inst. Electr. Electron. Eng. 1998, 86, 2278–2324. [Google Scholar] [CrossRef] [Green Version]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef] [Green Version]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Srivastava, R.K.; Greff, K.; Schmidhuber, J. Highway networks. arXiv 2015, arXiv:1505.00387. [Google Scholar]
- Kiddee, P.; Fang, Z.; Tan, M. A practical and intuitive calibration technique for cross-line structured light. Optik 2016, 127, 9582–9602. [Google Scholar] [CrossRef]
- Ke, F.; Xie, J.; Chen, Y. A flexible and high precision calibration method for the structured light vision system. Optik 2016, 127, 310–314. [Google Scholar] [CrossRef]
- Chong, B.; Zhu, Y.K. Speckle reduction in optical coherence tomography images of human finger skin by wavelet modified bm3d filter. Opt. Commun. 2013, 291, 461–469. [Google Scholar] [CrossRef]
- Xue, Q.; Wang, Z.; Huang, J.; Gao, J. Improving the measuring accuracy of structured light measurement system. Opt. Eng. 2014, 53, 112204. [Google Scholar] [CrossRef]
- Huang, J.; Wang, Z.; Xue, Q.; Gao, J. Calibration of a camera–projector measurement system and error impact analysis. Meas. Sci. Technol. 2012, 23, 125402. [Google Scholar] [CrossRef]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
Layer Name | Output Size | 18-Layer | 34-Layer | 50-Layer | 101-Layer | 152-Layer |
---|---|---|---|---|---|---|
conv1 | ||||||
conv2_x | ||||||
Conv3_x | ||||||
Conv4_x | ||||||
Conv5_x |
Performance | 18-Layer | 34-Layer | 50-Layer | 101-Layer | 152-Layer |
---|---|---|---|---|---|
FLOPs | |||||
Top-1 Err. on ImageNet Validation Set (%) | 27.88 | 25.03 | 22.85 | 21.75 | 21.43 |
Method | Advantages | Disadvantages |
---|---|---|
Threshold Method | Fast and simple. | Low extraction accuracy. |
Extreme Value Method | Fast and simple. | Easily disturbed by noise. |
Curve Fitting Method | Less affected by noise, high accuracy. | Low robustness and slow calculation speed. |
Steger Algorithm | High accuracy and robustness. | Slow calculation speed. |
Grayscale Gravity Method | High accuracy, high noise resistance, and high efficiency. | Low sensitivity. |
Type | Number of Train Set Images | Number of Test Set Images |
---|---|---|
Defective stripes | 3320 | 830 |
Defect-free stripes | 1840 | 460 |
Performance | 34-Layer | 50-Layer | 101-Layer |
---|---|---|---|
Top-1 Accuracy on Validation Set (%) | 90.16 | 92.95 | 93.72 |
Elapsed Time for Each Epoch (s) | 1500 | 2880 | 4035 |
Grayscale Distribution in Figure 15. | Measurement Error (mm) | Prediction Result |
---|---|---|
(a) | 1.009 | Defective |
(b) | 1.097 | Defective |
(c) | 0.914 | Defective |
(d) | 0.988 | Defective |
(e) | 0.015 | No Defects |
(f) | 0.129 | No Defects |
(g) | 0.094 | No Defects |
(h) | 0.225 | No Defects |
Unprocessed | Processed by the Method in This Paper | Processed by the Method in Ref. [19] | |
---|---|---|---|
Mean/mm | 0.670 | 0.163 | 0.326 |
Std/mm | 0.786 | 0.106 | 0.264 |
Max/mm | 1.836 | 0.328 | 1.009 |
Unprocessed | Processed by the Method in This Paper | Processed by the Method in Ref. [19] | |
---|---|---|---|
Mean/mm | 0.278 | 0.050 | 0.155 |
Std/mm | 0.297 | 0.029 | 0.179 |
Max/mm | 0.968 | 0.091 | 0.530 |
Unprocessed | Processed by the Method in This Paper | Processed by the Method in Ref. [19] | |
---|---|---|---|
Mean/mm | 1.455 | 0.817 | 1.345 |
Std/mm | 1.759 | 0.273 | 0.633 |
Max/mm | 12.419 | 1.333 | 2.896 |
Method | Implementation Steps | Performance |
---|---|---|
Structured light stripe defect detection method in Ref. [19] | Setting the threshold; Extracting the gray centers of structured light stripes; Detect the quality of structured light stripes. | Slower processing speed, depending on the thresholds. |
Structured light stripe defect detection based on residual neural network | Directly detects the quality of structured light stripes. | Higher precision, faster processing speed, no threshold needs to be set. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ding, A.; Xue, Q.; Ding, X.; Sun, X.; Yang, X.; Ye, H. Research on Automatic Error Data Recognition Method for Structured Light System Based on Residual Neural Network. Appl. Sci. 2023, 13, 2920. https://doi.org/10.3390/app13052920
Ding A, Xue Q, Ding X, Sun X, Yang X, Ye H. Research on Automatic Error Data Recognition Method for Structured Light System Based on Residual Neural Network. Applied Sciences. 2023; 13(5):2920. https://doi.org/10.3390/app13052920
Chicago/Turabian StyleDing, Aozhuo, Qi Xue, Xulong Ding, Xiaohong Sun, Xiaonan Yang, and Huiying Ye. 2023. "Research on Automatic Error Data Recognition Method for Structured Light System Based on Residual Neural Network" Applied Sciences 13, no. 5: 2920. https://doi.org/10.3390/app13052920
APA StyleDing, A., Xue, Q., Ding, X., Sun, X., Yang, X., & Ye, H. (2023). Research on Automatic Error Data Recognition Method for Structured Light System Based on Residual Neural Network. Applied Sciences, 13(5), 2920. https://doi.org/10.3390/app13052920