Automatic Detection of Ballast Unevenness Using Deep Neural Network
Abstract
:1. Introduction
- We propose a simple criterion for the evaluation of the condition of the ballast based on the measurement of its unevenness in relation to the level of the sleeper;
- We treat ballast irregularity detection as an instance segmentation process. We examine two instance segmentation algorithms, Mask R-CNN and YOLACT, in terms of their application to ballast irregularity detection;
- The use of instance segmentation makes it possible to determine the unevenness of the ballast individually relative to each sleeper;
- We use segmentation-related metrics—mAP (Mean Average Precision), IoU (Intersection over Union) and Pixel Accuracy—to evaluate the quality of the detection of ballast irregularities.
2. Algorithm for Automatic Detection of Unevenness in Ballast
2.1. Detection of Sleepers
- -
- Classification of mask proposals (modest segmentation accuracy, slow and difficult to optimize training, slow testing, not suitable for real-time applications);
- -
- Detection followed by segmentation (relatively simple to train, better generalization, relatively faster, good segmentation accuracy);
- -
- Labelling pixels followed by clustering (relatively simpler techniques, lesser segmentation accuracy, intense computation necessitating high computational power, not suited for real-time applications);
- -
- Dense sliding window methods (modest segmentation accuracy, use complex algorithms, difficult to train and optimize, not suitable for real-time applications).
2.1.1. Mask R-CNN Network
2.1.2. YOLACT Network
2.2. Determination of the Area of Unevenness of the Ballast
Algorithm 1. Algorithm to determine areas of ballast unevenness in reference to sleeper levels |
1: for each sleeper detected by instance segmentation network in the intensity image do 2: Using range image calculate the mean level of the sleeper = ML 3: calculate Upper_Threshold := ML + Threshold 4: calculate Bottom_Threshold := ML − Threshold 5: Mask := ø 6: for each m from 1 to image_height do 7: for each n from 1 to image_width do 8: if (D(m, n) < Bottom_Threshold) or (D(m, n) > Upper_Threshold) then 9: Mask := Mask + (m, n) 10: end if 11: end for 12: end for 13: Mark points in the range image based on Mask 14: end for |
3. Experimental Results
3.1. Training and Testing Mask R-CNN Network
3.2. Training and Testing the YOLACT Network
- -
- Models are deep neural networks that are very large and their training is time-consuming. We prefer a statistical hypothesis test that enables us to compare the models on a single test data set instead of performing multiple training (k-fold cross-validation with a modified paired Student’s t-test);
- -
- Models are binary classifiers;
- -
- There are no assumptions about the type of data distribution;
- -
- The test data for models are the same.
3.3. Detection of the Unevenness of the Ballast
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Remennikov, A.M.; Kaewunruen, S. Experimental load rating of aged railway concrete sleepers. Eng. Struct. 2014, 76, 147–162. [Google Scholar] [CrossRef]
- Sadeghi, J.M.; Zakeri, J.A.; Najar, M.E.M. Developing Track Ballast Characteristic Guideline in order to Evaluate its Performance. IJR Int. J. Railw. 2016, 9, 27–35. [Google Scholar] [CrossRef]
- Scanlan, K.M. Evaluating Degraded Ballast and Track Geometry Variability along a Canadian Freight Railroad through Ballast Maintenance Records and Ground-Penetrating Radar. Ph.D. Thesis, Civil and Environmental Engineering, University of Alberta, Edmond, AB, Canada, 2018. [Google Scholar] [CrossRef]
- Wang, S.; Liu, G.; Jing, G.; Feng, Q.; Liu, H.; Guo, Y. State-of-the-Art Review of Ground Penetrating Radar (GPR) Applications for Railway Ballast Inspection. Sensors 2022, 22, 2450. [Google Scholar] [CrossRef] [PubMed]
- Silvast, M.; Nurmikolu, A.; Wiljanen, B.; Levomaki, M. An Inspection of Railway Ballast Quality Using Ground Penetrating Radar in Finland. Proc. Inst. Mech. Eng. Part F J. Rail Rapid Transit. 2010, 224, 345–351. [Google Scholar] [CrossRef]
- Massaro, A.; Dipierro, G.; Selicato, S.; Cannella, E.; Galiano, A.; Saponaro, A. Intelligent Inspection of Railways Infrastructure and Risks Estimation by Artificial Intelligence Applied on Noninvasive Diagnostic Systems. In Proceedings of the 2021 IEEE International Workshop on Metrology for Industry 4.0 & IoT (MetroInd4.0&IoT), Rome, Italy, 7–9 June 2021; pp. 231–236. [Google Scholar] [CrossRef]
- Benedetto, A.; Tosti, F.; Bianchini Ciampoli, L.; Calvi, A.; Brancadoro, M.G.; Alani, A.M. Railway ballast condition assessment using ground-penetrating radar—An experimental, numerical simulation and modelling development. Constr. Build. Mater. 2017, 140, 508–520. [Google Scholar] [CrossRef]
- Artagan, S.S.; Borecky, V. Advances in the nondestructive condition assessment of railway ballast: A focus on GPR. NDT E Int. 2020, 115, 102290. [Google Scholar] [CrossRef]
- Shangguan, P.; Al-Qadi, I.L.; Leng, Z. Ground-Penetrating Radar Data to Develop Wavelet Technique for Quantifying Railroad Ballast–Fouling Conditions. Transp. Res. Rec. J. Transp. Res. Board 2012, 2289, 95–102. [Google Scholar] [CrossRef]
- Lama, H.F.; Wongb, M.T. Railway Ballast Diagnose through Impact Hammer Test. Procedia Eng. 2011, 14, 185–194. [Google Scholar] [CrossRef]
- Wang, M.; Xiao, Y.; Li, W.; Zhao, H.; Hua, W.; Jiang, Y. Characterizing Particle-Scale Acceleration of Mud-Pumping Ballast Bed of Heavy-Haul Railway Subjected to Maintenance Operations. Sensors 2022, 22, 6177. [Google Scholar] [CrossRef] [PubMed]
- Liang, X.; Niu, X.; Liu, P.; Lan, C.; Yang, R.; Zhou, Z. Test on fouling detection of ballast based on infrared thermography. NDT E Int. 2023, 140, 102956. [Google Scholar] [CrossRef]
- Zarembski, A.M.; Grissom, G.T.; Euston, T.L. On the use of Ballast Inspection Technology for the Management of Track Substructure. Transp. Infrastruct. Geotechnol. 2014, 1, 83–109. [Google Scholar] [CrossRef]
- Sadeghi, J.; Najar, M.E.M.; Zakeri, J.A.; Kuttelwascher, C. Development of railway ballast geometry index using automated measurement system. Measurement 2019, 138, 132–142. [Google Scholar] [CrossRef]
- Aldao, E.; González-Jorge, H.; González-deSantos, L.M.; Fontenla-Carrera, G.; Martínez-Sánchez, J. Validation of Solid-State LiDAR Measurement System for Ballast Geometry Monitoring in Rail Tracks. Infrastructures 2023, 8, 63. [Google Scholar] [CrossRef]
- Sánchez-Rodríguez, A.; Riveiro, B.; Soilán, M.; González-deSantos, L.M. Automated detection and decomposition of railway tunnels from mobile laser scanning datasets. Autom. Constr. 2018, 96, 171–179. [Google Scholar] [CrossRef]
- Paixao, A.; Afonso, C.; Delgado, B.; Fortunato, E. Evaluation of ballast particle degradation under micro-deval testing using photogrammetry. Advances in Transportation Geotechnics IV. Lect. Notes Civ. Eng. 2022, 165, 113–124. [Google Scholar] [CrossRef]
- Lesiak, P.; Bojarczak, P.; Sokolowski, A. Algorithm for the extraction of selected rail track balast degradation using machine vision. Transp. Probl. 2023, 18, 129–141. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with Deep Convolutional Neural Networks. Commun. ACM 2012, 60, 1097–1105. [Google Scholar] [CrossRef]
- Alzubaidi, L.; Zhang, J.; Humaidi, A.J.; Al-Dujaili, A.; Duan, Y.; Al-Shamma, O.; Santamaría, J.; Fadhel, M.A.; Al-Amidie, M.; Farhan, L. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. J. Big Data 2021, 8, 1–74. [Google Scholar] [CrossRef] [PubMed]
- Modi, A.S. Review Article on Deep Learning Approaches. In Proceedings of the 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 14–15 June 2018; pp. 1635–1639. [Google Scholar] [CrossRef]
- Samek, W.; Montavon, G.; Lapuschkin, S.; Anders, C.J.; Müller, K.R. Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications. Proc. IEEE 2021, 109, 247–278. [Google Scholar] [CrossRef]
- Sharma, P.; Singh, A. Era of deep neural networks: A review. Proc. Int. Conf. Comput. Commun. Netw. Technol. 2017, 1–5. [Google Scholar] [CrossRef]
- Shrestha, A.; Mahmood, A. Review of Deep Learning Algorithms and Architectures. IEEE Access 2019, 7, 53040–53065. [Google Scholar] [CrossRef]
- Wu, H.; Liu, Q.; Liu, X. A review on deep learning approaches to image classification and object segmentation. Comput. Mater. Contin. 2019, 60, 575–597. [Google Scholar] [CrossRef]
- Zhao, Z.; Zheng, P.; Xu, S.; Wu, X. Object Detection with Deep Learning: A Review. IEEE Trans. Neural Netw. Learn. Syst. 2019, 30, 3212–3232. [Google Scholar] [CrossRef] [PubMed]
- Hafiz, A.M.; Bhat, G.M. A survey on instance segmentation: State of the art. Int. J. Multimed. Inf. Retr. 2020, 9, 171–189. [Google Scholar] [CrossRef]
- Gu, W.; Bai, S.; Kong, L. A review on 2D instance segmentation based on deep neural networks. Image Vis. Comput. 2022, 120, 104401. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations (ICLR 2015). arXiv 2014, arXiv:1409.1556. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 770–778. [Google Scholar] [CrossRef]
- Shelhamer, E.; Long, J.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 640–651. [Google Scholar] [CrossRef] [PubMed]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar] [CrossRef]
- Lin, T.-Y.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature Pyramid Networks for Object Detection. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 936–944. [Google Scholar] [CrossRef]
- Bolya, D.; Zhou, C.; Xiao, F.; Lee, Y. YOLACT: Real-time Instance Segmentation. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019; pp. 9157–9166. [Google Scholar] [CrossRef]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; Part I 14; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar] [CrossRef]
- Tan, C.; Sun, F.; Kong, T.; Zhang, W.; Yang, C.; Liu, C. A survey on Deep Transfer Learning. In Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, 4–7 October 2018; Part III 27; Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar] [CrossRef]
- Iman, M.; Arabnia, H.R.; Rasheed, K. A Review of Deep Transfer Learning and Recent Advancements. Technologies 2023, 11, 40. [Google Scholar] [CrossRef]
- Padilla, R.; Netto, S.L.; da Silva, E.A.B. A Survey on Performance Metrics for Object-Detection Algorithms. In Proceedings of the 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), Niteroi, Brazil, 1–3 July 2020; pp. 237–242. [Google Scholar] [CrossRef]
- Dietterich, T.G. Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Comput. 1998, 10, 1895–1923. [Google Scholar] [CrossRef] [PubMed]
Mean Average Precision | IoU | Dice Coefficient | Pixel ACCURACY | Mean Processing Time [ms] | |
---|---|---|---|---|---|
Resnet51 | 0.8571 | 0.7602 | 0.8046 | 0.7845 | 125 |
Resnet101 | 0.8901 | 0.78004 | 0.8304 | 0.8063 | 200 |
Mean Average Precision | IoU | Dice Coefficient | Pixel ACCURACY | Mean Processing Time [ms] | |
---|---|---|---|---|---|
Resnet51 | 0.8214 | 0.7226 | 0.7668 | 0.7737 | 44 |
Resnet101 | 0.8571 | 0.7380 | 0.7912 | 0.7956 | 61 |
Average Precision AP | IoU | Pixel Accuracy |
---|---|---|
0.91 | 0.93 | 0.91 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bojarczak, P.; Lesiak, P.; Nowakowski, W. Automatic Detection of Ballast Unevenness Using Deep Neural Network. Appl. Sci. 2024, 14, 2811. https://doi.org/10.3390/app14072811
Bojarczak P, Lesiak P, Nowakowski W. Automatic Detection of Ballast Unevenness Using Deep Neural Network. Applied Sciences. 2024; 14(7):2811. https://doi.org/10.3390/app14072811
Chicago/Turabian StyleBojarczak, Piotr, Piotr Lesiak, and Waldemar Nowakowski. 2024. "Automatic Detection of Ballast Unevenness Using Deep Neural Network" Applied Sciences 14, no. 7: 2811. https://doi.org/10.3390/app14072811
APA StyleBojarczak, P., Lesiak, P., & Nowakowski, W. (2024). Automatic Detection of Ballast Unevenness Using Deep Neural Network. Applied Sciences, 14(7), 2811. https://doi.org/10.3390/app14072811