Next Article in Journal
Numerical Simulation of a Scanning Illumination System for Deep Tissue Fluorescence Imaging
Previous Article in Journal
Automatic Inspection of Aeronautical Mechanical Assemblies by Matching the 3D CAD Model and Real 2D Images
Open AccessArticle

Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition

1
School of Energy and Electronic Engineering, University of Portsmouth, Portsmouth PO1 3DJ, UK
2
School of Computing, University of Portsmouth, Portsmouth PO1 3DJ, UK
3
School of Computing, Engineering and Physical Sciences, University of the West of Scotland, Paisley PA1 2BE, UK
*
Author to whom correspondence should be addressed.
J. Imaging 2019, 5(10), 82; https://doi.org/10.3390/jimaging5100082
Received: 15 May 2019 / Revised: 13 October 2019 / Accepted: 17 October 2019 / Published: 21 October 2019
Human action recognition (HAR) is an important yet challenging task. This paper presents a novel method. First, fuzzy weight functions are used in computations of depth motion maps (DMMs). Multiple length motion information is also used. These features are referred to as fuzzy weighted multi-resolution DMMs (FWMDMMs). This formulation allows for various aspects of individual actions to be emphasized. It also helps to characterise the importance of the temporal dimension. This is important to help overcome, e.g., variations in time over which a single type of action might be performed. A deep convolutional neural network (CNN) motion model is created and trained to extract discriminative and compact features. Transfer learning is also used to extract spatial information from RGB and depth data using the AlexNet network. Different late fusion techniques are then investigated to fuse the deep motion model with the spatial network. The result is a spatial temporal HAR model. The developed approach is capable of recognising both human action and human–object interaction. Three public domain datasets are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the art algorithms. View Full-Text
Keywords: action recognition; transfer learning; multi-resolution; feature fusion action recognition; transfer learning; multi-resolution; feature fusion
Show Figures

Graphical abstract

MDPI and ACS Style

Al-Faris, M.; Chiverton, J.; Yang, Y.; Ndzi, D. Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition. J. Imaging 2019, 5, 82. https://doi.org/10.3390/jimaging5100082

AMA Style

Al-Faris M, Chiverton J, Yang Y, Ndzi D. Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition. Journal of Imaging. 2019; 5(10):82. https://doi.org/10.3390/jimaging5100082

Chicago/Turabian Style

Al-Faris, Mahmoud; Chiverton, John; Yang, Yanyan; Ndzi, David. 2019. "Deep Learning of Fuzzy Weighted Multi-Resolution Depth Motion Maps with Spatial Feature Fusion for Action Recognition" J. Imaging 5, no. 10: 82. https://doi.org/10.3390/jimaging5100082

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop