Next Article in Journal
Numerical Investigations on the Blade Tip Clearance Excitation Forces in an Unshrouded Turbine
Previous Article in Journal
Analysis of the DC Bias Effects on the UHV Autotransformer with Rated Load Based on a Reduced-Scale Model Experiment
Previous Article in Special Issue
Multi-View Interactive Visual Exploration of Individual Association for Public Transportation Passengers
Open AccessArticle

Body-Part-Aware and Multitask-Aware Single-Image-Based Action Recognition

College of Information Technology, Gachon University, Seongnam 13120, Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(4), 1531; https://doi.org/10.3390/app10041531
Received: 13 January 2020 / Revised: 19 February 2020 / Accepted: 20 February 2020 / Published: 24 February 2020
(This article belongs to the Special Issue Computing and Artificial Intelligence for Visual Data Analysis)
Action recognition is an application that, ideally, requires real-time results. We focus on single-image-based action recognition instead of video-based because of improved speed and lower cost of computation. However, a single image contains limited information, which makes single-image-based action recognition a difficult problem. To get an accurate representation of action classes, we propose three feature-stream-based shallow sub-networks (image-based, attention-image-based, and part-image-based feature networks) on the deep pose estimation network in a multitasking manner. Moreover, we design the multitask-aware loss function, so that the proposed method can be adaptively trained with heterogeneous datasets where only human pose annotations or action labels are included (instead of both pose and action information), which makes it easier to apply the proposed approach to new data on behavioral analysis on intelligent systems. In our extensive experiments, we showed that these streams represent complementary information and, hence, the fused representation is robust in distinguishing diverse fine-grained action classes. Unlike other methods, the human pose information was trained using heterogeneous datasets in a multitasking manner; nevertheless, it achieved 91.91% mean average precision on the Stanford 40 Actions Dataset. Moreover, we demonstrated the proposed method can be flexibly applied to multi-labels action recognition problem on the V-COCO Dataset. View Full-Text
Keywords: convolutional neural network; deep learning; action recognition; pose estimation; multitask learning convolutional neural network; deep learning; action recognition; pose estimation; multitask learning
Show Figures

Figure 1

MDPI and ACS Style

Bhandari, B.; Lee, G.; Cho, J. Body-Part-Aware and Multitask-Aware Single-Image-Based Action Recognition. Appl. Sci. 2020, 10, 1531.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop