Next Article in Journal
RSI-CB: A Large-Scale Remote Sensing Image Classification Benchmark Using Crowdsourced Data
Next Article in Special Issue
Deep Active Learning for Surface Defect Detection
Previous Article in Journal
Fast, Resource-Saving, and Anti-Collaborative Attack Trust Computing Scheme Based on Cross-Validation for Clustered Wireless Sensor Networks
Previous Article in Special Issue
Color-Guided Depth Map Super-Resolution Using a Dual-Branch Multi-Scale Residual Network with Channel Interaction
Open AccessArticle

Multi-Person Pose Estimation using an Orientation and Occlusion Aware Deep Learning Network

College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga 525-8577, Japan
Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan
Author to whom correspondence should be addressed.
Sensors 2020, 20(6), 1593;
Received: 7 February 2020 / Revised: 9 March 2020 / Accepted: 10 March 2020 / Published: 12 March 2020
Image based human behavior and activity understanding has been a hot topic in the field of computer vision and multimedia. As an important part, skeleton estimation, which is also called pose estimation, has attracted lots of interests. For pose estimation, most of the deep learning approaches mainly focus on the joint feature. However, the joint feature is not sufficient, especially when the image includes multi-person and the pose is occluded or not fully visible. This paper proposes a novel multi-task framework for the multi-person pose estimation. The proposed framework is developed based on Mask Region-based Convolutional Neural Networks (R-CNN) and extended to integrate the joint feature, body boundary, body orientation and occlusion condition together. In order to further improve the performance of the multi-person pose estimation, this paper proposes to organize the different information in serial multi-task models instead of the widely used parallel multi-task network. The proposed models are trained on the public dataset Common Objects in Context (COCO), which is further augmented by ground truths of body orientation and mutual-occlusion mask. Experiments demonstrate the performance of the proposed method for multi-person pose estimation and body orientation estimation. The proposed method can detect 84.6% of the Percentage of Correct Keypoints (PCK) and has an 83.7% Correct Detection Rate (CDR). Comparisons further illustrate the proposed model can reduce the over-detection compared with other methods. View Full-Text
Keywords: pose estimation; body orientation; multi-person; multi-task pose estimation; body orientation; multi-person; multi-task
Show Figures

Figure 1

MDPI and ACS Style

Gu, Y.; Zhang, H.; Kamijo, S. Multi-Person Pose Estimation using an Orientation and Occlusion Aware Deep Learning Network. Sensors 2020, 20, 1593.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

Search more from Scilit
Back to TopTop