Next Article in Journal
The Algorithms of Image Processing and Analysis in the Textile Fabrics Abrasion Assessment
Next Article in Special Issue
Design of Micro-Phasor Measurement System to Identify Phasor of Distribution Transformers
Previous Article in Journal
PDDL Planning with Natural Language-Based Scene Understanding for UAV-UGV Cooperation
Previous Article in Special Issue
A Novel Bidirectional Wireless Power Transfer System for Mobile Power Application
Open AccessArticle

Non-Touch Sign Word Recognition Based on Dynamic Hand Gesture Using Hybrid Segmentation and CNN Feature Fusion

School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Fukushima 965-8580, Japan
*
Author to whom correspondence should be addressed.
Appl. Sci. 2019, 9(18), 3790; https://doi.org/10.3390/app9183790
Received: 30 July 2019 / Revised: 29 August 2019 / Accepted: 6 September 2019 / Published: 10 September 2019
(This article belongs to the Special Issue Intelligent System Innovation)
Hand gesture-based sign language recognition is a prosperous application of human– computer interaction (HCI), where the deaf community, hard of hearing, and deaf family members communicate with the help of a computer device. To help the deaf community, this paper presents a non-touch sign word recognition system that translates the gesture of a sign word into text. However, the uncontrolled environment, perspective light diversity, and partial occlusion may greatly affect the reliability of hand gesture recognition. From this point of view, a hybrid segmentation technique including YCbCr and SkinMask segmentation is developed to identify the hand and extract the feature using the feature fusion of the convolutional neural network (CNN). YCbCr performs image conversion, binarization, erosion, and eventually filling the hole to obtain the segmented images. SkinMask images are obtained by matching the color of the hand. Finally, a multiclass SVM classifier is used to classify the hand gestures of a sign word. As a result, the sign of twenty common words is evaluated in real time, and the test results confirm that this system can not only obtain better-segmented images but also has a higher recognition rate than the conventional ones. View Full-Text
Keywords: human–computer interaction; hybrid segmentation; sign word recognition; convolutional neural network (CNN); support vector machine (SVM) human–computer interaction; hybrid segmentation; sign word recognition; convolutional neural network (CNN); support vector machine (SVM)
Show Figures

Figure 1

MDPI and ACS Style

Rahim, M.A.; Islam, M.R.; Shin, J. Non-Touch Sign Word Recognition Based on Dynamic Hand Gesture Using Hybrid Segmentation and CNN Feature Fusion. Appl. Sci. 2019, 9, 3790.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop