Cloud-Based Monitoring of Thermal Anomalies in Industrial Environments Using AI and the Internet of Robotic Things
Abstract
:1. Introduction
2. Literature Review
3. Materials and Methods
3.1. Environment and Requirements
3.2. Proposed Robot Surveyor Operating System Architecture
3.3. Proposed Edge-Fog-Cloud Architecture for Industrial Monitoring
3.4. Thermal Anomalies Detection in Aluminium Factories
3.5. Robot Body and Magnetic Shielding
3.6. Proposed Self-Driving Deep Architecture for Steering Angle and Speed Regression
3.7. Robot Surveyor Localization and Navigation Using QR Code Detection
3.8. Remote Control and Collision Avoidance
3.9. Proposed Anomaly Localization Using Thermal to Visual Registration
4. Validation and Results
4.1. Objective Results for Proposed Self-Driving Network
4.2. Subjective Results for Proposed Self-Driving Network
4.3. QR Detection Localization Results
4.4. Obstacle Avoidance Response Time
- Scenario 1: An obstacle is detected from the front middle sensor. When the distance retrieved from the ultrasonic sensor centered at the front is small, indicating an obstacle exists in front of the robot, the robot moves to the stop state. After that, all motion is blocked except backward, as illustrated in Figure 18a.
- Scenario 2: An obstacle is detected from the front right sensor. When an obstacle exists on the front right side, the robot stops, and motion is restricted to backward Figure 18b.
- Scenario 3: Obstacles are detected from the front and rear sensors. As depicted in Figure 18c, when the front and backward sensors detect obstacles in rare cases, the robot stops, all motion commands are blocked.
4.5. Visual Results of Thermal Anomaly Segmentation and Visualization
4.6. End to End Integration Testing and Validation
4.7. Computational Complexity and Response Time Testing
4.8. Initial Setup Stage for New Environments
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Bekey, G. Autonomous Robots; Bradford Books: Cambridge, UK, 2017. [Google Scholar]
- Murphy, A. Industrial: Robotics Outlook 2025. Loup Ventures. 6 July 2018. Available online: https://loupventures.com/industrial-robotics-outlook-2025/ (accessed on 7 September 2020).
- Rembold, U.; Lueth, T.; Ogasawara, T. From autonomous assembly robots to service robots for factories. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’94), Munich, Germany, 12–16 September 1994; Volume 3, pp. 2160–2167. [Google Scholar] [CrossRef]
- Yin, S.; Rodriguez-Andina, J.J.; Jiang, Y. Real-Time Monitoring and Control of Industrial Cyberphysical Systems: With Integrated Plant-Wide Monitoring and Control Framework. IEEE Ind. Electron. Mag. 2019, 13, 38–47. [Google Scholar] [CrossRef]
- Tas, M.O.; Yavuz, H.S.; Yazici, A. Updating HD-Maps for Autonomous Transfer Vehicles in Smart Factories. In Proceedings of the 6th International Conference on Control Engineering & Information Technology (CEIT), Istanbul, Turkey, 25–27 October 2018; pp. 1–5. [Google Scholar] [CrossRef]
- Saeed, M.S.; Alim, N. Design and Implementation of a Dual Mode Autonomous Gas Leakage Detecting Robot. In Proceedings of the International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST), Dhaka, Bangladesh, 10–12 January 2019; pp. 79–84. [Google Scholar] [CrossRef]
- Rey, R.; Corzetto, M.; Cobano, J.A.; Merino, L.; Caballero, F. Human-robot co-working system for warehouse automation. In Proceedings of the 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Zaragoza, Spain, 10–13 September 2019; pp. 578–585. [Google Scholar] [CrossRef]
- Montano, L. Robots in challenging environments. In Proceedings of the 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Zaragoza, Spain, 10–13 September 2019; pp. 23–26. [Google Scholar] [CrossRef]
- Teja, P.R.; Kumaar, A.A.N. QR Code based Path Planning for Warehouse Management Robot. In Proceedings of the International Conference on Advances in Computing, Communications and Informatics (ICACCI), Bangalore, India, 19–22 September 2018; pp. 1239–1244. [Google Scholar] [CrossRef]
- Chie, L.C.; Juin, Y.W. Artificial Landmark-based Indoor Navigation System for an Autonomous Unmanned Aerial Vehicle. In Proceedings of the IEEE 7th International Conference on Industrial Engineering and Applications (ICIEA), Bangkok, Thailand, 16–21 April 2020; pp. 756–760. [Google Scholar] [CrossRef]
- Limeira, M.A.; Piardi, L.; Kalempa, V.C.; de Oliveira, A.S.; Leitão, P. WsBot: A Tiny, Low-Cost Swarm Robot for Experimentation on Industry 4.0. In Proceedings of the 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE), Rio Grande, Brazil, 23–25 October 2019; pp. 293–298. [Google Scholar] [CrossRef]
- Ciuccarelli, L.; Freddi, A.; Longhi, S.; Monteriu, A.; Ortenzi, D.; Pagnotta, D.P. Cooperative Robots Architecture for an Assistive Scenario. In Proceedings of the Zooming Innovation in Consumer Technologies Conference (ZINC), Novi Sad, Serbia, 30–31 May 2018; pp. 128–129. [Google Scholar] [CrossRef]
- Avola, D.; Foresti, G.L.; Cinque, L.; Massaroni, C.; Vitale, G.; Lombardi, L. A multipurpose autonomous robot for target recognition in unknown environments. In Proceedings of the IEEE 14th International Conference on Industrial Informatics (INDIN), Poitiers, France, 19–21 May 2016; pp. 766–771. [Google Scholar] [CrossRef]
- Merriaux, P.; Rossie, R.; Boutteau, R.; Vauchey, V.; Qin, L.; Chanuc, P.; Rigaud, F.; Roger, F.; Decoux, B.; Savatier, X. The VIKINGS Autonomous Inspection Robot: Competing in the ARGOS Challenge. IEEE Robot. Autom. Mag. 2019, 26, 21–34. [Google Scholar] [CrossRef] [Green Version]
- Dharmasena, T.; Abeygunawardhana, P. Design and Implementation of an Autonomous Indoor Surveillance Robot based on Raspberry Pi. In Proceedings of the International Conference on Advancements in Computing (ICAC), Malabe, Sri Lanka, 5–7 December 2019; pp. 244–248. [Google Scholar] [CrossRef]
- Mogaveera, A.; Giri, R.; Mahadik, M.; Patil, A. Self Driving Robot using Neural Network. In Proceedings of the International Conference on Information, Communication, Engineering and Technology (ICICET), Pune, India, 29–31 August 2018; pp. 1–6. [Google Scholar] [CrossRef]
- Omrane, H.; Masmoudi, M.S.; Masmoudi, M. Neural controller of autonomous driving mobile robot by an embedded camera. In Proceedings of the 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), Sousse, Tunisia, 21–24 March 2018; pp. 1–5. [Google Scholar] [CrossRef]
- Ebuchi, T.; Yamamoto, H. Vehicle/Pedestrian Localization System Using Multiple Radio Beacons and Machine Learning for Smart Parking. In Proceedings of the International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Okinawa, Japan, 11–13 February 2019; pp. 86–91. [Google Scholar] [CrossRef]
- Ullah, A.; Muhammad, K.; del Ser, J.; Baik, S.W.; de Albuquerque, V.H.C. Activity Recognition Using Temporal Optical Flow Convolutional Features and Multilayer LSTM. IEEE Trans. Ind. Electron. 2019, 66, 9692–9702. [Google Scholar] [CrossRef]
- Ullah, W.; Ullah, A.; Haq, I.U.; Muhammad, K.; Sajjad, M.; Baik, S.W. CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks. Multimed. Tools Appl. 2020. [Google Scholar] [CrossRef]
- Yin, S.; Yang, C.; Zhang, J.; Jiang, Y. A Data-Driven Learning Approach for Nonlinear Process Monitoring Based on Available Sensing Measurements. IEEE Trans. Ind. Electron. 2017, 64, 643–653. [Google Scholar] [CrossRef]
- Zhou, D.; Wei, M.; Si, X. A Survey on Anomaly Detection, Life Prediction and Maintenance Decision for Industrial Processes. Acta Autom. Sin. 2014, 39, 711–722. [Google Scholar] [CrossRef]
- Kammerer, K.; Hoppenstedt, B.; Pryss, R.; Stökler, S.; Allgaier, J.; Reichert, M. Anomaly Detections for Manufacturing Systems Based on Sensor Data—Insights into Two Challenging Real-World Production Settings. Sensors 2019, 19, 5370. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Pittino, F.; Puggl, M.; Moldaschl, T.; Hirschl, C. Automatic Anomaly Detection on In-Production Manufacturing Machines Using Statistical Learning Methods. Sensors 2020, 20, 2344. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kroll, B.; Schaffranek, D.; Schriegel, S.; Niggemann, O. System modeling based on machine learning for anomaly detection and predictive maintenance in industrial plants. In Proceedings of the IEEE Emerging Technology and Factory Automation (ETFA), Barcelona, Spain, 16–19 September 2014; pp. 1–7. [Google Scholar] [CrossRef]
- Kubota, T.; Yamamoto, W. Anomaly Detection from Online Monitoring of System Operations Using Recurrent Neural Network. Procedia Manuf. 2019, 30, 83–89. [Google Scholar] [CrossRef]
- Arkhipov, A.; Zarouni, A.; Baggash, M.J.I.; Akhmetov, S.; Reverdy, M.; Potocnik, V. Cell Electrical Preheating Practices at Dubal; Hyland, M., Ed.; Light Metals 2015; Springer: Cham, Switzerland, 2016; pp. 445–449. [Google Scholar] [CrossRef]
- Majid, N.A.; Taylor, M.; Chen, J.J.; Young, B.R. Aluminium Process Fault Detection and Diagnosis. Adv. Mater. Sci. Eng. 2015, 2015, 682786. [Google Scholar] [CrossRef] [Green Version]
- Gang, H.; Pyun, J. A Smartphone Indoor Positioning System Using Hybrid Localization Technology. Energies 2019, 12, 3702. [Google Scholar] [CrossRef] [Green Version]
- Styner, M.; Brechbuhler, C.; Szckely, G.; Gerig, G. Parametric estimate of intensity inhomogeneities applied to MRI. IEEE Trans. Med. Imaging 2000, 19, 153–165. [Google Scholar] [CrossRef] [PubMed]
Scenario | Obstacle Location | Obstacle Detection Time (s) | Obstacle Response Time (s) |
---|---|---|---|
Scenario 1 | front center | 0.00350 | 0.29350 |
Scenario 2 | front right | 0.00146 | 0.23146 |
Scenario 3 | front left | 0.00146 | 0.22146 |
Scenario 4 | rear center | 0.00350 | 0.21350 |
Scenario 5 | rear right | 0.00146 | 0.24146 |
Scenario 6 | rear left | 0.00146 | 0.26146 |
Scenario 7 | front and rear | 0.00350 | 0.20350 |
Network Architecture | Number of Layers | Iterations | Training Time (mins) | Validation RMSE | Frame Rate |
---|---|---|---|---|---|
VGG-19 | 47 | 13,200 | 989 | 0.17 | 22 |
Resenet-18 | 65 | - | 77 | 0.55 | - |
Proposed architecture | 28 | 6600 | 228 | 0.19 | 65 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ghazal, M.; Basmaji, T.; Yaghi, M.; Alkhedher, M.; Mahmoud, M.; El-Baz, A.S. Cloud-Based Monitoring of Thermal Anomalies in Industrial Environments Using AI and the Internet of Robotic Things. Sensors 2020, 20, 6348. https://doi.org/10.3390/s20216348
Ghazal M, Basmaji T, Yaghi M, Alkhedher M, Mahmoud M, El-Baz AS. Cloud-Based Monitoring of Thermal Anomalies in Industrial Environments Using AI and the Internet of Robotic Things. Sensors. 2020; 20(21):6348. https://doi.org/10.3390/s20216348
Chicago/Turabian StyleGhazal, Mohammed, Tasnim Basmaji, Maha Yaghi, Mohammad Alkhedher, Mohamed Mahmoud, and Ayman S. El-Baz. 2020. "Cloud-Based Monitoring of Thermal Anomalies in Industrial Environments Using AI and the Internet of Robotic Things" Sensors 20, no. 21: 6348. https://doi.org/10.3390/s20216348
APA StyleGhazal, M., Basmaji, T., Yaghi, M., Alkhedher, M., Mahmoud, M., & El-Baz, A. S. (2020). Cloud-Based Monitoring of Thermal Anomalies in Industrial Environments Using AI and the Internet of Robotic Things. Sensors, 20(21), 6348. https://doi.org/10.3390/s20216348