Flying Chameleons: A New Concept for Minimum-Deployment, Multiple-Target Tracking Drones
Abstract
:1. Introduction
2. State-of-the-Art and the Main Contributions
2.1. One UAV, Single or Multiple Targets
2.2. Multiple UAVs, Single or Multiple Targets
2.3. Visual Coverage
2.4. Shared Attention
2.5. Contributions of the Work and Main Areas of Application
3. Framework
3.1. Working Scenario
General Assumptions and Simplifications
3.2. Camera Model
- : sensor width .
- : sensor height .
- : number of pixels in horizontal direction .
- : number of pixels in vertical direction .
- f: focal length .
- s: skew factor.
- : coordinates of the sensor central point .
- : effective pixel width: .
- : effective pixel height: .
- : effective focal length (horizontal) .
- : effective focal length (vertical) .
3.3. Gimbal Structure
3.4. Target Geolocation
3.4.1. Geolocation Using Central Camera
3.4.2. Geolocation Using Tracking Cameras
3.5. Target Aiming
3.6. Initialization
4. Methods: Multi-Target Optimal Positioning
- Attempting to minimize variations in the appearance of the target. If no verticality constraints are set, this would leave total freedom to achieve both fully zenithal and fully horizontal views of the target. This would result in a potentially high variability in the target’s appearance, which would make the task of the tracking algorithms more difficult. It is expected that, by maintaining as vertical a view of each target as possible, the variability in the target appearance will be as low as possible.
- Attempting to minimize the occlusions. Given the nature of the proposed problem, in which several targets of unspecified height are moving around, it is clear that giving the possibility to track such targets without minimum verticality and altitude constrains, may lead to situations of more likely occlusions of the targets by other objects, between the targets themselves, or even that one of the onboard cameras gets in the line of sight of another one.
- In applications where it is specifically desired to monitor the targets with minimal intrusion and even where such monitoring should go unnoticed by the targets themselves, it is again interesting to establish a minimum altitude threshold, combined with an optimization of the verticality with respect to the set of targets.
4.1. Baseline Target Tracking: Naive Heuristic Strategies
Uniform Versus Periphery-Biased Baseline Strategies
4.2. Setting up a Suitable Optimization Index
4.2.1. Balancing Both Terms of the Optimization Index
4.2.2. Uniform Optimization Index
4.2.3. Approximate Uniform Optimization Index
4.2.4. Min–Max Optimization Index
5. Results and Discussion
6. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
Appendix A. Uniform and Min–Max Optimization Indices: Convexity Proofs
Appendix B. Approximate Uniform Optimization Index: Gradient Derivation and Closed-Form Solution
References
- Schwager, M.; Julian, B.J.; Angermann, M.; Rus, D. Eyes in the sky: Decentralized control for the deployment of robotic camera networks. Proc. IEEE 2011, 99, 1541–1561. [Google Scholar] [CrossRef]
- Miller, R.; Mooty, G.; Hilkert, J.M. Gimbal system configurations and line-of-sight control techniques for small UAV applications. In Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications X; Henry, D.J., Lange, D.A., von Berg, D.L., Rajan, S.D., Walls, T.J., Young, D.L., Eds.; International Society for Optics and Photonics, SPIE: Bellingham, WA, USA, 2013; Volume 8713, pp. 39–53. [Google Scholar] [CrossRef]
- Choi, H.; Kim, Y. UAV guidance using a monocular-vision sensor for aerial target tracking. Control Eng. Pract. 2014, 22, 10–19. [Google Scholar] [CrossRef]
- Farmani, N.; Sun, L.; Pack, D. An optimal sensor management technique for unmanned aerial vehicles tracking multiple mobile ground targets. In Proceedings of the 2014 International Conference on Unmanned Aircraft Systems (ICUAS), Orlando, FL, USA, 27–30 May 2014; pp. 570–576. [Google Scholar] [CrossRef]
- Helgesen, H.H.; Leira, F.S.; Fossen, T.I.; Johansen, T.A. Tracking of Ocean Surface Objects from Unmanned Aerial Vehicles with a Pan/Tilt Unit using a Thermal Camera. J. Intell. Robot. Syst. 2018, 91, 775–793. [Google Scholar] [CrossRef]
- Wang, S.; Jiang, F.; Zhang, B.; Ma, R.; Hao, Q. Development of UAV-Based Target Tracking and Recognition Systems. IEEE Trans. Intell. Transp. Syst. 2020, 21, 3409–3422. [Google Scholar] [CrossRef]
- Chen, C.; Tian, Y.; Lin, L.; Chen, S.; Li, H.; Wang, Y.; Su, K. Obtaining World Coordinate Information of UAV in GNSS Denied Environments. Sensors 2020, 20, 2241. [Google Scholar] [CrossRef]
- Bisagno, N.; Xamin, A.; De Natale, F.; Conci, N.; Rinner, B. Dynamic Camera Reconfiguration with Reinforcement Learning and Stochastic Methods for Crowd Surveillance. Sensors 2020, 20, 4691. [Google Scholar] [CrossRef]
- Wang, Z.M.; Xu, X.G. A survey on electronic image stabilization. J. Image Graph. 2010, 15, 470–480. [Google Scholar]
- Dahlin Rodin, C.; de Alcantara Andrade, F.A.; Hovenburg, A.R.; Johansen, T.A. A survey of practical design considerations of optical imaging stabilization systems for small unmanned aerial systems. Sensors 2019, 19, 4800. [Google Scholar] [CrossRef]
- Robin, C.; Lacroix, S. Multi-robot target detection and tracking: Taxonomy and survey. Auton. Robot. 2016, 40, 729–760. [Google Scholar] [CrossRef]
- Senanayake, M.; Senthooran, I.; Barca, J.C.; Chung, H.; Kamruzzaman, J.; Murshed, M. Search and tracking algorithms for swarms of robots: A survey. Robot. Auton. Syst. 2016, 75, 422–434. [Google Scholar] [CrossRef]
- Sharma, R.; Pack, D. Cooperative sensor resource management to aid multi target geolocalization using a team of small fixed-wing unmanned aerial vehicles. In Proceedings of the AIAA Guidance, Navigation, and Control (GNC) Conference, Boston, MA, USA, 19–23 August 2013; p. 4706. [Google Scholar]
- Zhao, Y.; Wang, X.; Wang, C.; Cong, Y.; Shen, L. Systemic design of distributed multi-UAV cooperative decision-making for multi-target tracking. Auton. Agents Multi-Agent Syst. 2019, 33, 132–158. [Google Scholar] [CrossRef]
- Baek, S.; York, G. Optimal Sensor Management for Multiple Target Tracking Using Cooperative Unmanned Aerial Vehicles. In Proceedings of the 2020 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece, 1–4 September 2020; pp. 1294–1300. [Google Scholar]
- Wierzbicki, D. Multi-Camera Imaging System for UAV Photogrammetry. Sensors 2018, 18, 2433. [Google Scholar] [CrossRef] [PubMed]
- Harmat, A.; Trentini, M.; Sharf, I. Multi-Camera Tracking and Mapping for Unmanned Aerial Vehicles in Unstructured Environments. J. Intell. Robot. Syst. 2015, 78, 291–317. [Google Scholar] [CrossRef]
- He, D.; Hsiu Min, C.; Chen, J.; Li, J.; Namiki, A. Real-Time Visual Feedback Control of Multi-Camera UAV. J. Robot. Mechatron. 2021, 33, 263–273. [Google Scholar] [CrossRef]
- Zhang, M.; Liu, H.H. Cooperative tracking a moving target using multiple fixed-wing UAVs. J. Intell. Robot. Syst. 2016, 81, 505–529. [Google Scholar] [CrossRef]
- Morbidi, F.; Mariottini, G.L. Active Target Tracking and Cooperative Localization for Teams of Aerial Vehicles. IEEE Trans. Control Syst. Technol. 2013, 21, 1694–1707. [Google Scholar] [CrossRef]
- Sung, Y.; Budhiraja, A.K.; Williams, R.K.; Tokekar, P. Distributed simultaneous action and target assignment for multi-robot multi-target tracking. In Proceedings of the 2018 IEEE International conference on robotics and automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 3724–3729. [Google Scholar]
- Toupet, O.; How, J. Collaborative sensor fusion and management for multiple UAVs. In Proceedings of the Infotech@Aerospace Conf2011, St. Louis, MI, USA, 29–31 March 2011. [Google Scholar] [CrossRef]
- Adamey, E.; Ozguner, U. A decentralized approach for multi-UAV multitarget tracking and surveillance. In Proceedings of the Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR III, International Society for Optics and Photonics, Baltimore, MD, USA, 24 May 2012; Volume 8389, p. 838915. [Google Scholar]
- Farmani, N.; Sun, L.; Pack, D.J. A Scalable Multitarget Tracking System for Cooperative Unmanned Aerial Vehicles. IEEE Trans. Aerosp. Electron. Syst. 2017, 53, 1947–1961. [Google Scholar] [CrossRef]
- Tokekar, P.; Isler, V.; Franchi, A. Multi-target visual tracking with aerial robots. In Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, 14–18 September 2014; pp. 3067–3072. [Google Scholar] [CrossRef]
- Ishat-E-Rabban, M.; Tokekar, P. Failure-Resilient Coverage Maximization With Multiple Robots. IEEE Robot. Autom. Lett. 2021, 6, 3894–3901. [Google Scholar] [CrossRef]
- Dan, H.; Yamauchi, J.; Hatanaka, T.; Fujita, M. Control Barrier Function-Based Persistent Coverage with Performance Guarantee and Application to Object Search Scenario. In Proceedings of the 2020 IEEE Conference on Control Technology and Applications (CCTA), Montréal, QC, Canada, 24–26 August 2020; pp. 640–647. [Google Scholar] [CrossRef]
- Xiao, X.; Dufek, J.; Woodbury, T.; Murphy, R. Uav assisted usv visual navigation for marine mass casualty incident response. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 6105–6110. [Google Scholar]
- Burke, C.; McWhirter, P.R.; Veitch-Michaelis, J.; McAree, O.; Pointon, H.A.; Wich, S.; Longmore, S. Requirements and limitations of thermal drones for effective search and rescue in marine and coastal areas. Drones 2019, 3, 78. [Google Scholar] [CrossRef]
- Helgesen, H.H.; Bryne, T.H.; Wilthil, E.F.; Johansen, T.A. Camera-Based Tracking of Floating Objects using Fixed-wing UAVs. J. Intell. Robot. Syst. 2021, 102, 80. [Google Scholar] [CrossRef]
- Fusini, L.; Fossen, T.I.; Johansen, T.A. Nonlinear Observers for GNSS- and Camera-Aided Inertial Navigation of a Fixed-Wing UAV. IEEE Trans. Control Syst. Technol. 2018, 26, 1884–1891. [Google Scholar] [CrossRef]
- Atkinson, M.P.; Kress, M.; Szechtman, R. Maritime transportation of illegal drugs from South America. Int. J. Drug Policy 2017, 39, 43–51. [Google Scholar] [CrossRef] [PubMed]
- Campana, P. Human Smuggling: Structure and Mechanisms. Crime Justice 2020, 49, 471–519. [Google Scholar] [CrossRef]
- Carling, J. Unauthorized Migration from Africa to Spain. Int. Migr. 2007, 45, 3–37. [Google Scholar] [CrossRef]
- Coluccello, S.; Massey, S. Out of Africa: The human trade between Libya and Lampedusa. Trends Organanized Crime 2007, 10, 77–90. [Google Scholar] [CrossRef]
- Ding, X.C.; Rahmani, A.R.; Egerstedt, M. Multi-UAV convoy protection: An optimal approach to path planning and coordination. IEEE Trans. Robot. 2010, 26, 256–268. [Google Scholar] [CrossRef]
- Mademlis, I.; Mygdalis, V.; Nikolaidis, N.; Montagnuolo, M.; Negro, F.; Messina, A.; Pitas, I. High-level multiple-UAV cinematography tools for covering outdoor events. IEEE Trans. Broadcast. 2019, 65, 627–635. [Google Scholar] [CrossRef]
- Sa, I.; Ahn, H.S. Visual 3D model-based tracking toward autonomous live sports broadcasting using a VTOL unmanned aerial vehicle in GPS-impaired environments. Int. J. Comput. Appl. 2015, 122, 1–7. [Google Scholar] [CrossRef]
- Mademlis, I.; Mygdalis, V.; Nikolaidis, N.; Pitas, I. Challenges in autonomous UAV cinematography: An overview. In Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME), San Diego, CA, USA, 23–27 July 2018; pp. 1–6. [Google Scholar]
- Bonatti, R.; Ho, C.; Wang, W.; Choudhury, S.; Scherer, S. Towards a robust aerial cinematography platform: Localizing and tracking moving targets in unstructured environments. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 4–8 November 2019; pp. 229–236. [Google Scholar]
- Gonzalez, L.F.; Montes, G.A.; Puig, E.; Johnson, S.; Mengersen, K.; Gaston, K.J. Unmanned Aerial Vehicles (UAVs) and Artificial Intelligence Revolutionizing Wildlife Monitoring and Conservation. Sensors 2016, 16, 97. [Google Scholar] [CrossRef]
- Panadeiro, V.; Rodriguez, A.; Henry, J.; Wlodkowic, D.; Andersson, M. A review of 28 free animal-tracking software applications: Current features and limitations. Lab. Anim. 2021, 50, 246–254. [Google Scholar] [CrossRef]
- Bozcan, I.; Kayacan, E. Au-air: A multi-modal unmanned aerial vehicle dataset for low altitude traffic surveillance. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 31 May–31 August 2020; pp. 8504–8510. [Google Scholar]
- Khan, M.A.; Ectors, W.; Bellemans, T.; Janssens, D.; Wets, G. Unmanned aerial vehicle–based traffic analysis: Methodological framework for automated multivehicle trajectory extraction. Transp. Res. Rec. 2017, 2626, 25–33. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Vargas, M.; Vivas, C.; Rubio, F.R.; Ortega, M.G. Flying Chameleons: A New Concept for Minimum-Deployment, Multiple-Target Tracking Drones. Sensors 2022, 22, 2359. https://doi.org/10.3390/s22062359
Vargas M, Vivas C, Rubio FR, Ortega MG. Flying Chameleons: A New Concept for Minimum-Deployment, Multiple-Target Tracking Drones. Sensors. 2022; 22(6):2359. https://doi.org/10.3390/s22062359
Chicago/Turabian StyleVargas, Manuel, Carlos Vivas, Francisco R. Rubio, and Manuel G. Ortega. 2022. "Flying Chameleons: A New Concept for Minimum-Deployment, Multiple-Target Tracking Drones" Sensors 22, no. 6: 2359. https://doi.org/10.3390/s22062359
APA StyleVargas, M., Vivas, C., Rubio, F. R., & Ortega, M. G. (2022). Flying Chameleons: A New Concept for Minimum-Deployment, Multiple-Target Tracking Drones. Sensors, 22(6), 2359. https://doi.org/10.3390/s22062359