Next Article in Journal
Using Inertial Sensors to Quantify Postural Sway and Gait Performance during the Tandem Walking Test
Next Article in Special Issue
A Fast Calibration Method for Photonic Mixer Device Solid-State Array Lidars
Previous Article in Journal
FeinPhone: Low-cost Smartphone Camera-based 2D Particulate Matter Sensor
Previous Article in Special Issue
Single-Pixel Imaging and Its Application in Three-Dimensional Reconstruction: A Brief Review
Article

Deep Attention Models for Human Tracking Using RGBD

1
Networked Robotics and Sensing Laboratory, School of Engineering Science, Simon Fraser University, Burnaby, BC V5A 1S6, Canada
2
School of Computing Science, Simon Fraser University, Burnaby, BC V5A 1S6, Canada
3
School of Mechatronic Systems Engineering, Simon Fraser University, Burnaby, BC V5A 1S6, Canada
*
Authors to whom correspondence should be addressed.
Sensors 2019, 19(4), 750; https://doi.org/10.3390/s19040750
Received: 21 December 2018 / Revised: 31 January 2019 / Accepted: 3 February 2019 / Published: 13 February 2019
(This article belongs to the Special Issue Depth Sensors and 3D Vision)
Visual tracking performance has long been limited by the lack of better appearance models. These models fail either where they tend to change rapidly, like in motion-based tracking, or where accurate information of the object may not be available, like in color camouflage (where background and foreground colors are similar). This paper proposes a robust, adaptive appearance model which works accurately in situations of color camouflage, even in the presence of complex natural objects. The proposed model includes depth as an additional feature in a hierarchical modular neural framework for online object tracking. The model adapts to the confusing appearance by identifying the stable property of depth between the target and the surrounding object(s). The depth complements the existing RGB features in scenarios when RGB features fail to adapt, hence becoming unstable over a long duration of time. The parameters of the model are learned efficiently in the Deep network, which consists of three modules: (1) The spatial attention layer, which discards the majority of the background by selecting a region containing the object of interest; (2) the appearance attention layer, which extracts appearance and spatial information about the tracked object; and (3) the state estimation layer, which enables the framework to predict future object appearance and location. Three different models were trained and tested to analyze the effect of depth along with RGB information. Also, a model is proposed to utilize only depth as a standalone input for tracking purposes. The proposed models were also evaluated in real-time using KinectV2 and showed very promising results. The results of our proposed network structures and their comparison with the state-of-the-art RGB tracking model demonstrate that adding depth significantly improves the accuracy of tracking in a more challenging environment (i.e., cluttered and camouflaged environments). Furthermore, the results of depth-based models showed that depth data can provide enough information for accurate tracking, even without RGB information. View Full-Text
Keywords: computer vision; visual tracking; attention model; RGBD; Kinect; deep network; convolutional neural network; Long Short-Term Memory computer vision; visual tracking; attention model; RGBD; Kinect; deep network; convolutional neural network; Long Short-Term Memory
Show Figures

Figure 1

MDPI and ACS Style

Rasoulidanesh, M.; Yadav, S.; Herath, S.; Vaghei, Y.; Payandeh, S. Deep Attention Models for Human Tracking Using RGBD. Sensors 2019, 19, 750. https://doi.org/10.3390/s19040750

AMA Style

Rasoulidanesh M, Yadav S, Herath S, Vaghei Y, Payandeh S. Deep Attention Models for Human Tracking Using RGBD. Sensors. 2019; 19(4):750. https://doi.org/10.3390/s19040750

Chicago/Turabian Style

Rasoulidanesh, Maryamsadat, Srishti Yadav, Sachini Herath, Yasaman Vaghei, and Shahram Payandeh. 2019. "Deep Attention Models for Human Tracking Using RGBD" Sensors 19, no. 4: 750. https://doi.org/10.3390/s19040750

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop