Next Article in Journal
Active Monitoring of Fatigue Crack in the Weld Zone of Bogie Frames Using Ultrasonic Guided Waves
Next Article in Special Issue
Subdiffusive Source Sensing by a Regional Detection Method
Previous Article in Journal
Privacy-Preserving Multi-Receiver Certificateless Broadcast Encryption Scheme with De-Duplication
Previous Article in Special Issue
Low-Slow-Small (LSS) Target Detection Based on Micro Doppler Analysis in Forward Scattering Radar Geometry
Open AccessArticle

Deep Learning-Based Real-Time Multiple-Object Detection and Tracking from Aerial Imagery via a Flying Robot with GPU-Based Embedded Devices

School of Mechanical & Convergence System Engineering, Kunsan National University, 558 Daehak-ro, Gunsan 54150, Korea
*
Author to whom correspondence should be addressed.
Sensors 2019, 19(15), 3371; https://doi.org/10.3390/s19153371
Received: 30 May 2019 / Revised: 17 July 2019 / Accepted: 26 July 2019 / Published: 31 July 2019
(This article belongs to the Special Issue Sensors In Target Detection)
In recent years, demand has been increasing for target detection and tracking from aerial imagery via drones using onboard powered sensors and devices. We propose a very effective method for this application based on a deep learning framework. A state-of-the-art embedded hardware system empowers small flying robots to carry out the real-time onboard computation necessary for object tracking. Two types of embedded modules were developed: one was designed using a Jetson TX or AGX Xavier, and the other was based on an Intel Neural Compute Stick. These are suitable for real-time onboard computing power on small flying drones with limited space. A comparative analysis of current state-of-the-art deep learning-based multi-object detection algorithms was carried out utilizing the designated GPU-based embedded computing modules to obtain detailed metric data about frame rates, as well as the computation power. We also introduce an effective target tracking approach for moving objects. The algorithm for tracking moving objects is based on the extension of simple online and real-time tracking. It was developed by integrating a deep learning-based association metric approach with simple online and real-time tracking (Deep SORT), which uses a hypothesis tracking methodology with Kalman filtering and a deep learning-based association metric. In addition, a guidance system that tracks the target position using a GPU-based algorithm is introduced. Finally, we demonstrate the effectiveness of the proposed algorithms by real-time experiments with a small multi-rotor drone. View Full-Text
Keywords: multi-target detection and tracking; multi-copter drone; aerial imagery; image sensor; deep learning; GPU-based embedded module; neural computing stick; image processing multi-target detection and tracking; multi-copter drone; aerial imagery; image sensor; deep learning; GPU-based embedded module; neural computing stick; image processing
Show Figures

Figure 1

MDPI and ACS Style

Hossain, S.; Lee, D.-J. Deep Learning-Based Real-Time Multiple-Object Detection and Tracking from Aerial Imagery via a Flying Robot with GPU-Based Embedded Devices. Sensors 2019, 19, 3371.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop