Developing a Flying Explorer for Autonomous Digital Modelling in Wild Unknowns
Abstract
:1. Introduction
- Developing the Aerial Robot Exploration (AREX) system, as shown in Figure 1, including its hardware and software framework, for autonomous digital modelling in unstructured and unknown environments.
- Developing a novel algorithm for autonomous exploration in unknown environments for multiple scenarios.
- Demonstration of the developed AREX system in real-world applications, showcases its effectiveness in several scenarios.
2. Materials and Methods
2.1. Robot Design
2.1.1. Hardware Design
2.1.2. Software Design
2.2. Robot Odometry
2.2.1. Visual Odometry
2.2.2. LiDAR Odometry
2.3. Robot Motion Planning
2.3.1. Environment Representation
2.3.2. Local Motion Planner
2.4. Robot Exploration Module
2.4.1. Environmental Modelling
2.4.2. Direction-Aware RRT Exploration
- represents the data frames from the LiDAR.
- represents the odometry information.
- represents the grid map.
- represents the frontier points.
- represents the expected exploration goal at time t.
- represents the the smoothed path.
- represents the flag for executing exploration
- Octomap_Server: accepts radar data frame information and odometer information for constructing a 2D occupancy map representing the scene exploration.
- Frontier_Detector: accepts the 2D occupancy map and odometer information constructed by Octomap Server and searches for boundary points of the unexplored environment by RRT and outputs a list of candidate target points.
- Revenue_Calculator: accepts the 2D occupancy map, odometer information, and candidate target points, calculates the value of the utility function of the candidate target points, and outputs the target point with the highest score.
- Path_Planner: accepts target points, radar data frames, and odometer information to plan a collision-free smooth path for the robot.
Algorithm 1: Direction-aware RRT exploration |
|
2.4.3. Evaluation Strategy
2.4.4. Stop Cretria
- No further candidate exploration target points are discovered. As shown below:
- The unknown area of the candidate region is below a preset threshold. As shown below, where represents the area of the unknown region near the corresponding target point, and s is the predefined threshold:
3. Results
3.1. Experiment Setups
3.2. Evaluation on Subsystems
3.2.1. Comparison of Visual and LiDAR Odometry
3.2.2. Evaluation on Robot Motion Planning
3.3. Field Experiments
3.3.1. Scene 1
3.3.2. Scene 2
3.4. Failure Analysis
3.4.1. Failure on Robot Odometry
3.4.2. Failure on Motion Planning
- Being entrapped within an obstacle group causes program judgment, leading to collision and subsequent planning failure. This behaviour stems from environmental perception inaccuracies due to LiDAR point cloud sparsity and measurement errors. Notably, the obstacle point cloud’s inconsistency, especially at object edges, causes sporadic jumps, resulting in the robot colliding with the obstacle cluster.
- Another prevalent failure involves obstacle avoidance. While achieving collision-free trajectories entails bypassing expanded obstacles in flight, determining an appropriate expansion coefficient in proportion to the airframe size is crucial. Excessively high expansion coefficients exacerbate the point cloud’s jumping phenomenon, leading to severe consequences.
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Grieves, M. Origins of the Digital Twin Concept; Florida Institute of Technology: Melbourne, FL, USA, 2016; Volume 8. [Google Scholar]
- Bisanti, G.M.; Mainetti, L.; Montanaro, T.; Patrono, L.; Sergi, I. Digital twins for aircraft maintenance and operation: A systematic literature review and an IoT-enabled modular architecture. Internet Things 2023, 24, 100991. [Google Scholar] [CrossRef]
- Hu, K.; Chen, Z.; Kang, H.; Tang, Y. 3D vision technologies for a self-developed structural external crack damage recognition robot. Autom. Constr. 2024, 159, 105262. [Google Scholar] [CrossRef]
- Kang, H.; Zang, Y.; Wang, X.; Chen, Y. Uncertainty-driven spiral trajectory for robotic peg-in-hole assembly. IEEE Robot. Autom. Lett. 2022, 7, 6661–6668. [Google Scholar] [CrossRef]
- Wang, Z.; Gupta, R.; Han, K.; Wang, H.; Ganlath, A.; Ammar, N.; Tiwari, P. Mobility digital twin: Concept, architecture, case study, and future challenges. IEEE Internet Things J. 2022, 9, 17452–17467. [Google Scholar] [CrossRef]
- Hu, K.; Wei, Y.; Pan, Y.; Kang, H.; Chen, C. High-fidelity 3D Reconstruction of Plants using Neural Radiance Field. arXiv 2023, arXiv:2311.04154. [Google Scholar]
- Sestras, P.; Bilașco, S.; Roșca, S.; Ilies, N.; Hysa, A.; Spalević, V.; Cîmpeanu, S.M. Multi-instrumental approach to slope failure monitoring in a landslide susceptible newly built-up area: Topo-Geodetic survey, UAV 3D modelling and ground-penetrating radar. Remote. Sens. 2022, 14, 5822. [Google Scholar] [CrossRef]
- Papachristos, C.; Mascarich, F.; Khattak, S.; Dang, T.; Alexis, K. Localization uncertainty-aware autonomous exploration and mapping with aerial robots using receding horizon path-planning. Auton. Robot. 2019, 43, 2131–2161. [Google Scholar] [CrossRef]
- Molina, S.; Cielniak, G.; Duckett, T. Robotic exploration for learning human motion patterns. IEEE Trans. Robot. 2021, 38, 1304–1318. [Google Scholar] [CrossRef]
- Hong, Z.W.; Shann, T.Y.; Su, S.Y.; Chang, Y.H.; Fu, T.J.; Lee, C.Y. Diversity-driven exploration strategy for deep reinforcement learning. Adv. Neural Inf. Process. Syst. 2018, 31. [Google Scholar] [CrossRef]
- Shen, S.; Michael, N.; Kumar, V. Stochastic Differential Equation-based Exploration Algorithm for Autonomous Indoor 3D Exploration with a Micro-Aerial Vehicle. Int. J. Robot. Res. 2012, 31, 1431–1444. [Google Scholar] [CrossRef]
- Naazare, M.; Rosas, F.G.; Schulz, D. Online next-best-view planner for 3D-exploration and inspection with a mobile manipulator robot. IEEE Robot. Autom. Lett. 2022, 7, 3779–3786. [Google Scholar] [CrossRef]
- Lindqvist, B.; Agha-Mohammadi, A.A.; Nikolakopoulos, G. Exploration-RRT: A multi-objective path planning and exploration framework for unknown and unstructured environments. In Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 27 September–1 October 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 3429–3435. [Google Scholar]
- Shao, T.; Li, Y.; Gao, W.; Lin, J.; Lin, F. A PAD-Based Unmanned Aerial Vehichle Route Planning Scheme for Remote Sensing in Huge Regions. Sensors 2023, 23, 9897. [Google Scholar] [CrossRef] [PubMed]
- Qin, T.; Cao, S.; Pan, J.; Li, P.; Shen, S. VINS-Fusion: An Optimization-Based Multi-Sensor State Estimator. 2019. Available online: https://github.com/HKUST-Aerial-Robotics/VINS-Fusion/tree/master (accessed on 18 January 2024).
- Xu, W.; Zhang, F. Fast-lio: A fast, robust lidar-inertial odometry package by tightly coupled iterated kalman filter. IEEE Robot. Autom. Lett. 2021, 6, 3317–3324. [Google Scholar] [CrossRef]
- Zhou, X.; Wang, Z.; Ye, H.; Xu, C.; Gao, F. Ego-planner: An esdf-free gradient-based local planner for quadrotors. IEEE Robot. Autom. Lett. 2020, 6, 478–485. [Google Scholar] [CrossRef]
- Xu, Z.; Deng, D.; Shimada, K. Autonomous UAV exploration of dynamic environments via incremental sampling and probabilistic roadmap. IEEE Robot. Autom. Lett. 2021, 6, 2729–2736. [Google Scholar] [CrossRef]
- Yang, K.; Keat Gan, S.; Sukkarieh, S. A Gaussian process-based RRT planner for the exploration of an unknown and cluttered environment with a UAV. Adv. Robot. 2013, 27, 431–443. [Google Scholar] [CrossRef]
- Naderi, K.; Rajamäki, J.; Hämäläinen, P. RT-RRT* a real-time path planning algorithm based on RRT. In Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games, Paris, France, 16–18 November 2015; pp. 113–118. [Google Scholar]
- Umari, H.; Mukhopadhyay, S. Autonomous robotic exploration based on multiple rapidly exploring randomized trees. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1396–1402. [Google Scholar]
- Yamauchi, B. A frontier-based approach for autonomous exploration. In Proceedings of the 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97. ‘Towards New Computational Principles for Robotics and Automation’, Monterey, CA, USA, 10–11 July 1997; IEEE: Piscataway, NJ, USA, 1997; pp. 146–151. [Google Scholar]
- Zhong, P.; Chen, B.; Lu, S.; Meng, X.; Liang, Y. Information-driven fast marching autonomous exploration with aerial robots. IEEE Robot. Autom. Lett. 2021, 7, 810–817. [Google Scholar] [CrossRef]
- Walker, V.; Vanegas, F.; Gonzalez, F. Multi-UAV Mapping and Target Finding in Large, Complex, Partially Observable Environments. Remote. Sens. 2023, 15, 3802. [Google Scholar] [CrossRef]
- Sun, Y.; Zhang, C. Efficient and safe robotic autonomous environment exploration using integrated frontier detection and multiple path evaluation. Remote. Sens. 2021, 13, 4881. [Google Scholar] [CrossRef]
- Pérez-Higueras, N.; Jardón, A.; Rodríguez, Á.; Balaguer, C. 3D exploration and navigation with optimal-RRT planners for ground robots in indoor incidents. Sensors 2019, 20, 220. [Google Scholar] [CrossRef]
- Amorín, G.; Benavides, F.; Grmapin, E. A novel stop criterion to support efficient Multi-robot mapping. In Proceedings of the 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE), Rio Grande, Brazil, 23–25 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 369–374. [Google Scholar]
- Lluvia, I.; Lazkano, E.; Ansuategi, A. Active mapping and robot exploration: A survey. Sensors 2021, 21, 2445. [Google Scholar] [CrossRef] [PubMed]
- Lu, L.; De Luca, A.; Muratore, L.; Tsagarakis, N.G. An Optimal Frontier Enhanced “Next Best View” Planner For Autonomous Exploration. In Proceedings of the 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), Ginowan, Japan, 28–30 November 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 397–404. [Google Scholar]
- Pan, Y.; Cao, H.; Hu, K.; Kang, H.; Wang, X. A Novel Mapping and Navigation Framework for Robot Autonomy in Orchards. arXiv 2023, arXiv:2308.16748. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, N.; Pan, Y.; Jin, Y.; Jin, P.; Hu, K.; Huang, X.; Kang, H. Developing a Flying Explorer for Autonomous Digital Modelling in Wild Unknowns. Sensors 2024, 24, 1021. https://doi.org/10.3390/s24031021
Zhang N, Pan Y, Jin Y, Jin P, Hu K, Huang X, Kang H. Developing a Flying Explorer for Autonomous Digital Modelling in Wild Unknowns. Sensors. 2024; 24(3):1021. https://doi.org/10.3390/s24031021
Chicago/Turabian StyleZhang, Naizhong, Yaoqiang Pan, Yangwen Jin, Peiqi Jin, Kewei Hu, Xiao Huang, and Hanwen Kang. 2024. "Developing a Flying Explorer for Autonomous Digital Modelling in Wild Unknowns" Sensors 24, no. 3: 1021. https://doi.org/10.3390/s24031021
APA StyleZhang, N., Pan, Y., Jin, Y., Jin, P., Hu, K., Huang, X., & Kang, H. (2024). Developing a Flying Explorer for Autonomous Digital Modelling in Wild Unknowns. Sensors, 24(3), 1021. https://doi.org/10.3390/s24031021