An Edge-Fog Architecture for Distributed 3D Reconstruction and Remote Monitoring of a Power Plant Site in the Context of 5G
Abstract
:1. Introduction
- Development of a distributed 3D reconstruction and imaging framework for remote monitoring that uses an optimized, scalable edge-fog architecture for computing resources and network communication.
- A study and analysis of the proposed approach in a real application of an electric power plant facility.
- A study on the application of a 5G private network and its benefits in the proposed framework to assist in the distributed processing scalability.
2. Background and Related Works
2.1. Remote Monitoring of Power Facilities, 3D Reconstructions, and Fog Computing
2.2. Edge-Fog-Cloud Applications in the IoT Context Using 5G
3. Materials and Methods
3.1. Robot Hardware
3.2. Robot Control and Algorithms
3.3. Proposed Architecture
3.4. Distributed Processing Methodology
4. Results in Experimental Environments
4.1. Improvements from the Edge-Fog Architecture When Compared to Edge-Based Approach
4.2. Application Scalability
4.3. Relevance of 5G Network for the Monitoring Scenario
- From the data gathered and shown in Figure 6, as a conservative approach, each robot is expected to require 5.67 MiBps of throughput, considering average plus one standard deviation, which is equivalent to 47.56 Mbps.
- Each Fog Node will be saturated from processing data of nine Edge Nodes due to the CPU utilization constraint.
- Nine parallel Edge Nodes will demand a throughput of 9 × 47.56 Mbps = 428.04 Mbps in each Fog Node connection.
- The value of 83 Mbps for uplink rate recorded in [63] is already enough for each Edge Node to send data according to our experimental first requirement, so any greater values will only help in network robustness and latency decrease.
5. Conclusions and Future Work
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Abbreviations
PPV | Pan point of view |
IoT | Internet of Things |
VR | Virtual reality |
ROS | Robot Operating System |
SLAM | Simultaneous localization and mapping |
IMU | Inertial measurement unit |
FR | Fog robotics |
CR | Cloud robotics |
SOR | Statistical outlier removal |
LiDAR | Light detection and ranging |
SfM | Structure from motion |
References
- Cai, X.; Ning, H.; Dhelim, S.; Zhou, R.; Zhang, T.; Xu, Y.; Wan, Y. Robot and its living space: A roadmap for robot development based on the view of living space. Digit. Commun. Networks 2020, 7, 505–517. [Google Scholar] [CrossRef]
- Xu, T.; Xu, L.; Li, X.; Yao, J. Detection of water leakage in underground tunnels using corrected intensity data and 3D point cloud of terrestrial laser scanning. IEEE Access 2018, 6, 32471–32480. [Google Scholar] [CrossRef]
- Biundini, I.Z.; Pinto, M.F.; Melo, A.G.; Marcato, A.L.; Honório, L.M.; Aguiar, M.J. A framework for coverage path planning optimization based on point cloud for structural inspection. Sensors 2021, 21, 570. [Google Scholar] [CrossRef] [PubMed]
- Gojcic, Z.; Zhou, C.; Wieser, A. Robust pointwise correspondences for point cloud based deformation monitoring of natural scenes. In Proceedings of the 4th Joint international symposium on deformation monitoring (JISDM), Athens, Greece, 15–17 May 2019. [Google Scholar]
- Wang, G.; Wu, J.; He, R.; Yang, S. A point cloud-based robust road curb detection and tracking method. IEEE Access 2019, 7, 24611–24625. [Google Scholar] [CrossRef]
- Zhang, H.; Aldana-Jague, E.; Clapuyt, F.; Wilken, F.; Vanacker, V.; Van Oost, K. Evaluating the potential of post-processing kinematic (PPK) georeferencing for UAV-based structure-from-motion (SfM) photogrammetry and surface change detection. Earth Surf. Dyn. 2019, 7, 807–827. [Google Scholar] [CrossRef] [Green Version]
- Chalupa, D.; Nielsen, P. Two strategies of two-level facility network design for autonomous ground vehicle operations. Prod. Manuf. Res. 2018, 6, 494–506. [Google Scholar] [CrossRef]
- Wrobel, A.; Placzek, M. Visualization systems for industrial automation systems. In Proceedings of the ModTech International Conference - Modern Technologies in Industrial Engineering VI, Constanta, Romania, 13–16 June 2018. [Google Scholar]
- Abdelhafidh, M.; Fourati, M.; Fourati, L.C.; Abidi, A. Remote Water Pipeline Monitoring System IoT-Based Architecture for New Industrial Era 4.0. In Proceedings of the 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), Hammamet, Tunisia, 30 October–3 November 2017. [Google Scholar]
- Salhaoui, M.; Guerrero-González, A.; Arioua, M.; Ortiz, F.J.; El Oualkadi, A.; Torregrosa, C.L. Smart industrial IoT monitoring and control system based on UAV and cloud computing applied to a concrete plant. Sensors 2019, 19, 3316. [Google Scholar] [CrossRef] [Green Version]
- Melo, A.G.; Pinto, M.F.; Honório, L.M.; Dias, F.M.; Masson, J.E. 3D Correspondence and Point Projection Method for Structures Deformation Analysis. IEEE Access 2020, 8, 177823–177836. [Google Scholar] [CrossRef]
- Vidal, V.F.; Honório, L.M.; Dias, F.M.; Pinto, M.F.; Carvalho, A.L.; Marcato, A.L. Sensors Fusion and Multidimensional Point Cloud Analysis for Electrical Power System Inspection. Sensors 2020, 20, 4042. [Google Scholar] [CrossRef]
- Xue, Y.; Zhang, S.; Zhou, M.; Zhu, H. Novel SfM-DLT method for metro tunnel 3D reconstruction and Visualization. Undergr. Space 2021, 6, 134–141. [Google Scholar] [CrossRef]
- Yu, D.; Ji, S.; Liu, J.; Wei, S. Automatic 3D building reconstruction from multi-view aerial images with deep learning. Isprs J. Photogramm. Remote. Sens. 2021, 171, 155–170. [Google Scholar] [CrossRef]
- Lin, J.; Zhang, F. R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package. arXiv 2021, arXiv:2109.07982. [Google Scholar] [CrossRef]
- Xu, W.; Cai, Y.; He, D.; Lin, J.; Zhang, F. Fast-lio2: Fast direct lidar-inertial odometry. arXiv 2021, arXiv:2107.06829. [Google Scholar] [CrossRef]
- Badat, L.; Vidal, V.; Pioli, L.; Mehaut, J.F.; Honorio, L.; Dantas, M.A.R. An IIoT Edge Environment as a Main Support to a 3D Reconstruction Virtualization Application. In Anais do XVIII Workshop em Clouds e Aplicações; SBC: Bento Gonçalves, Brazil, 2020; pp. 1–12. [Google Scholar] [CrossRef]
- Decamps, M.; Meháut, J.F.; Vidal, V.; Honorio, L.; Pioli, L.; Dantas, M.A. An Implementation Science Effort in a Heterogenous Edge Computing Platform to Support a Case Study of a Virtual Scenario Application. In Proceedings of the International Conference on P2P Parallel, Grid, Cloud and Internet Computing, Yonago, Japan, 28–30 October 2020. [Google Scholar]
- da Silva, L.A.; Vidal, V.F.; Honório, L.M.; Dantas, M.A.; Pinto, M.F.; Capretz, M. A Heterogeneous Edge-Fog Environment Supporting Digital Twins for Remote Inspections. Sensors 2020, 20, 5296. [Google Scholar] [CrossRef]
- Pinto, M.F.; Marcato, A.L.; Melo, A.G.; Honório, L.M.; Urdiales, C. A framework for analyzing fog-cloud computing cooperation applied to information processing of UAVs. Wirel. Commun. Mob. Comput. 2019, 2019, 7497924. [Google Scholar] [CrossRef] [Green Version]
- Singh, J.; Singh, P.; Gill, S.S. Fog computing: A taxonomy, systematic review, current trends and research challenges. J. Parallel Distrib. Comput. 2021, 157, 56–85. [Google Scholar] [CrossRef]
- Dias, V.; Moreira, R.; Meira, W.; Guedes, D. Diagnosing performance bottlenecks in massive data parallel programs. In Proceedings of the 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), Cartagena, Colombia, 16–19 May 2016. [Google Scholar]
- Yousefpour, A.; Fung, C.; Nguyen, T.; Kadiyala, K.; Jalali, F.; Niakanlahiji, A.; Kong, J.; Jue, J.P. All one needs to know about fog computing and related edge computing paradigms: A complete survey. J. Syst. Archit. 2019, 98, 289–330. [Google Scholar] [CrossRef]
- Etemadi, M.; Ghobaei-Arani, M.; Shahidinejad, A. Resource provisioning for IoT services in the fog computing environment: An autonomic approach. Comput. Commun. 2020, 161, 109–131. [Google Scholar] [CrossRef]
- Ivanov, D. Fog robotics distributed computing in a monitoring task. In Proceedings of the Computer Science On-line Conference, Zlin, Czech Republic, 15 July 2020. [Google Scholar]
- Sood, S.K.; Kaur, A.; Sood, V. Energy efficient IoT-Fog based architectural paradigm for prevention of Dengue fever infection. J. Parallel Distrib. Comput. 2021, 150, 46–59. [Google Scholar] [CrossRef]
- Li, L.; Ota, K.; Dong, M. Deep learning for smart industry: Efficient manufacture inspection system with fog computing. IEEE Trans. Ind. Inform. 2018, 14, 4665–4673. [Google Scholar] [CrossRef] [Green Version]
- Alves, J.M.; Honório, L.M.; Capretz, M.A.M. ML4IoT: A Framework to Orchestrate Machine Learning Workflows on Internet of Things Data. IEEE Access 2019, 7, 152953–152967. [Google Scholar] [CrossRef]
- Krishna Chand Gudi, S.L.; Ojha, S.; Johnston, B.; Clark, J.; Williams, M. Fog Robotics for Efficient, Fluent and Robust Human-Robot Interaction. In Proceedings of the 2018 IEEE 17th International Symposium on Network Computing and Applications (NCA), Cambridge, MA, USA, 1–3 November 2018. [Google Scholar]
- Raju, H.; Shenoy, S. Real-time remote monitoring and operation of industrial devices using IoT and cloud. In Proceedings of the 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), Noida, India, 14–17 December 2016. [Google Scholar]
- da Silva, A.F.; Ohta, R.L.; dos Santos, M.N.; Binotto, A.P. A cloud-based architecture for the internet of things targeting industrial devices remote monitoring and control. Ifac-Papersonline 2016, 49, 108–113. [Google Scholar] [CrossRef]
- Iorga, M.; Feldman, L.; Barton, R.; Martin, M.J.; Goren, N.S.; Mahmoudi, C. NIST Special Publications—Fog Computing Conceptual Model; Technical Report 325; National Institute of Standards and Technology: Gaithersburg, MD, USA, 2018.
- Mouradian, C.; Naboulsi, D.; Yangui, S.; Glitho, R.H.; Morrow, M.J.; Polakos, P.A. A comprehensive survey on fog computing: State-of-the-art and research challenges. IEEE Commun. Surv. Tutor. 2017, 20, 416–464. [Google Scholar] [CrossRef] [Green Version]
- Bellavista, P.; Berrocal, J.; Corradi, A.; Das, S.K.; Foschini, L.; Zanni, A. A survey on fog computing for the Internet of Things. Pervasive Mob. Comput. 2019, 52, 71–99. [Google Scholar] [CrossRef]
- Gomes, E.; Costa, F.; De Rolt, C.; Plentz, P.; Dantas, M. A Survey from Real-Time to Near Real-Time Applications in Fog Computing Environments. Telecom. Multidiscip. Digit. Publ. Inst. 2021, 2, 489–517. [Google Scholar] [CrossRef]
- Aazam, M.; Zeadally, S.; Harras, K.A. Offloading in fog computing for IoT: Review, enabling technologies, and research opportunities. Future Gener. Comput. Syst. 2018, 87, 278–289. [Google Scholar] [CrossRef]
- Sarker, V.; Queralta, J.P.; Gia, T.; Tenhunen, H.; Westerlund, T. Offloading slam for indoor mobile robots with edge-fog-cloud computing. In Proceedings of the 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), Dhaka, Bangladesh, 3–5 May 2019. [Google Scholar]
- Tian, N.; Tanwani, A.K.; Chen, J.; Ma, M.; Zhang, R.; Huang, B.; Goldberg, K.; Sojoudi, S. A fog robotic system for dynamic visual servoing. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019. [Google Scholar]
- Caprari, G.; Breitenmoser, A.; Fischer, W.; Hürzeler, C.; Tâche, F.; Siegwart, R.; Nguyen, O.; Moser, R.; Schoeneich, P.; Mondada, F. Highly compact robots for inspection of power plants. J. Field Robot. 2012, 29, 47–68. [Google Scholar] [CrossRef] [Green Version]
- de Faria, H., Jr.; Costa, J.G.S.; Olivas, J.L.M. A review of monitoring methods for predictive maintenance of electric power transformers based on dissolved gas analysis. Renew. Sustain. Energy Rev. 2015, 46, 201–209. [Google Scholar] [CrossRef]
- Zhao, L.; Matsuo, I.B.M.; Zhou, Y.; Lee, W.J. Design of an industrial IoT-based monitoring system for power substations. IEEE Trans. Ind. Appl. 2019, 55, 5666–5674. [Google Scholar] [CrossRef]
- Sato, Y.; Terasaka, Y.; Utsugi, W.; Kikuchi, H.; Kiyooka, H.; Torii, T. Radiation imaging using a compact Compton camera mounted on a crawler robot inside reactor buildings of Fukushima Daiichi Nuclear Power Station. J. Nucl. Sci. Technol. 2019, 56, 801–808. [Google Scholar] [CrossRef]
- Peng, H.; Wang, Q.; Yang, Y.; Bi, M. Research and Application of Digital 3D Modeling Technology in Substation Monitoring. In Proceedings of the 3rd International Conference on Engineering Technology and Application (ICETA 2016), Kyoto, Japan, 28–29 April 2016. [Google Scholar]
- Guo, L.; Zhang, Y.; Guo, J.; Shi, P.; Zhao, K. An Object-Oriented Based 3D Model for Substation Monitoring. In Proceedings of the 2020 IEEE/IAS Industrial and Commercial Power System Asia (I&CPS Asia), Weihai, China, 13–15 July 2020. [Google Scholar]
- Tanwani, A.K.; Mor, N.; Kubiatowicz, J.; Gonzalez, J.E.; Goldberg, K. A Fog Robotics Approach to Deep Robot Learning: Application to Object Recognition and Grasp Planning in Surface Decluttering. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019. [Google Scholar]
- Gudi, S.L.K.C.; Johnston, B.; Williams, M.A. Fog Robotics: A Summary, Challenges and Future Scope. arXiv 2019, arXiv:1908.04935. [Google Scholar] [CrossRef]
- Aleksy, M.; Dai, F.; Enayati, N.; Rost, P.; Pocovi, G. Utilizing 5G in industrial robotic applications. In Proceedings of the 2019 7th International Conference on Future Internet of Things and Cloud (FiCloud), Istanbul, Turkey, 26–28 August 2019. [Google Scholar]
- Shahzadi, R.; Niaz, A.; Ali, M.; Naeem, M.; Rodrigues, J.J.; Qamar, F.; Anwar, S.M. Three tier fog networks: Enabling IoT/5G for latency sensitive applications. China Commun. 2019, 16, 1–11. [Google Scholar]
- Bravo-Arrabal, J.; Toscano-Moreno, M.; Fernandez-Lozano, J.; Mandow, A.; Gomez-Ruiz, J.A.; García-Cerezo, A. The Internet of Cooperative Agents Architecture (X-IoCA) for Robots, Hybrid Sensor Networks, and MEC Centers in Complex Environments: A Search and Rescue Case Study. Sensors 2021, 21, 7843. [Google Scholar] [CrossRef] [PubMed]
- Lee, J.; Lee, K.; Yoo, A.; Moon, C. Design and Implementation of Edge-Fog-Cloud System through HD Map Generation from LiDAR Data of Autonomous Vehicles. Electronics 2020, 9, 2084. [Google Scholar] [CrossRef]
- Shin, S.; Kim, J.; Moon, C. Road Dynamic Object Mapping System Based on Edge-Fog-Cloud Computing. Electronics 2021, 10, 2825. [Google Scholar] [CrossRef]
- Livox Camera Lidar Calibration. Available online: https://github.com/Livox-SDK/livox_camera_lidar_calibration (accessed on 19 May 2022).
- Santos, M.; Pereira, V.; Ribeiro, A.; Silva, M.; do Carmo, M.; Vidal, V.; Honório, L.; Cerqueira, A.; Oliveira, E. Simulation and comparison between a linear and nonlinear technique applied to altitude control in quadcopters. In Proceedings of the 2017 18th International Carpathian Control Conference (ICCC), Sinaia, Romania, 28–31 May 2017. [Google Scholar]
- How to Use a KdTree to Search. Available online: https://pointclouds.org/documentation/tutorials/kdtree_search.html (accessed on 19 May 2022).
- The Point Cloud Library—Estimating Surface Normals in a PointCloud. Available online: https://pcl.readthedocs.io/en/latest/normal_estimation.html (accessed on 19 May 2022).
- The Point Cloud Library—Removing Outliers Using a StatisticalOutlierRemoval Filter. Available online: https://pcl.readthedocs.io/projects/tutorials/en/master/statistical_outlier.html (accessed on 19 May 2022).
- Habibi, P.; Farhoudi, M.; Kazemian, S.; Khorsandi, S.; Leon-Garcia, A. Fog computing: A comprehensive architectural survey. IEEE Access 2020, 8, 69105–69133. [Google Scholar] [CrossRef]
- Dustdar, S.; Avasalcai, C.; Murturi, I. Invited Paper: Edge and Fog Computing: Vision and Research Challenges. In Proceedings of the IEEE International Conference on Service-Oriented System Engineering (SOSE), San Francisco, CA, USA, 4–9 April 2019. [Google Scholar]
- Shi, W.; Cao, J.; Zhang, Q.; Li, Y.; Xu, L. Edge Computing: Vision and Challenges. IEEE Internet Things J. 2016, 3, 637–646. [Google Scholar] [CrossRef]
- Omoniwa, B.; Hussain, R.; Javed, M.A.; Bouk, S.H.; Malik, S.A. Fog/Edge Computing-Based IoT (FECIoT): Architecture, Applications, and Research Issues. IEEE Internet Things J. 2019, 6, 4118–4149. [Google Scholar] [CrossRef]
- Hayat, S.; Jung, R.; Hellwagner, H.; Bettstetter, C.; Emini, D.; Schnieders, D. Edge computing in 5G for drone navigation: What to offload? IEEE Robot. Autom. Lett. 2021, 6, 2571–2578. [Google Scholar] [CrossRef]
- Muzaffar, R.; Raffelsberger, C.; Fakhreddine, A.; Luque, J.L.; Emini, D.; Bettstetter, C. First experiments with a 5G-Connected drone. In Proceedings of the 6th ACM Workshop on Micro Aerial Vehicle Networks Systems, and Applications, Toronto, ON, Canada, 15 June 2020. [Google Scholar]
- Vasconcellos, V.; Cardoso, B.B.; Martins, K.A.; de Macedo, A.F.; Cecchetti, B.F.; Martins, M.A.I. On the Application of 5G to Energy Distribution Companies: A Brazilian Case Study. In Proceedings of the 2021 IEEE Latin-American Conference on Communications (LATINCOM), Santo Domingo, Dominican Republic, 17–19 November 2021. [Google Scholar]
Process Name | PU | IMS (MB) | OMS (MB) |
---|---|---|---|
Acquire data | 277.80 | 11.310 | 11.310 |
Filter misread points | 2.37 | 5.378 | 4.351 |
Send full HD image | 412.71 | 5.932 | 5.932 |
Lower image resolution | 13.80 | 5.932 | 0.370 |
Color point cloud | 14.69 | 4.721 | 5.438 |
Kd-tree neighbors search | 74.42 | 5.438 | 2.527 |
statistical outlier removal (SOR) Filter | 22.82 | 2.527 | 1.382 |
Normal estimation | 46.32 | 1.382 | 2.486 |
Accumulate final point cloud | 3.16 | 2.486 | - |
Edge-Based | Edge-Fog | ||
---|---|---|---|
Edge | Edge | Fog | |
CPU activity (%) | 91.4 | 63.2 | 30.2 |
RAM (%) | 88.3 | 41.8 | 5.9 |
Number of Robots | |||||
---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | |
CPU activity(%) | 30.2 | 33.6 | 42.9 | 53.3 | 63.7 |
RAM(%) | 5.9 | 11.0 | 16.2 | 19.9 | 27.1 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Vidal, V.; Honório, L.; Pinto, M.; Dantas, M.; Aguiar, M.; Capretz, M. An Edge-Fog Architecture for Distributed 3D Reconstruction and Remote Monitoring of a Power Plant Site in the Context of 5G. Sensors 2022, 22, 4494. https://doi.org/10.3390/s22124494
Vidal V, Honório L, Pinto M, Dantas M, Aguiar M, Capretz M. An Edge-Fog Architecture for Distributed 3D Reconstruction and Remote Monitoring of a Power Plant Site in the Context of 5G. Sensors. 2022; 22(12):4494. https://doi.org/10.3390/s22124494
Chicago/Turabian StyleVidal, Vinicius, Leonardo Honório, Milena Pinto, Mario Dantas, Maria Aguiar, and Miriam Capretz. 2022. "An Edge-Fog Architecture for Distributed 3D Reconstruction and Remote Monitoring of a Power Plant Site in the Context of 5G" Sensors 22, no. 12: 4494. https://doi.org/10.3390/s22124494