Next Article in Journal
Comparative Study of Wheel–Rail Contact Impact Force for Jointed Rail and Continuous Welded Rail on Light-Rail Transit
Previous Article in Journal
Early Detection of Change by Applying Scale-Space Methodology to Hyperspectral Images
Article

A Dynamic Gesture Recognition Interface for Smart Home Control based on Croatian Sign Language

Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, Laboratory for Smart Environment Technologies, University of Split, 21000 Split, Croatia
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Appl. Sci. 2020, 10(7), 2300; https://doi.org/10.3390/app10072300
Received: 3 February 2020 / Revised: 11 March 2020 / Accepted: 23 March 2020 / Published: 27 March 2020
(This article belongs to the Section Computing and Artificial Intelligence)
Deaf and hard-of-hearing people are facing many challenges in everyday life. Their communication is based on the use of a sign language, and the ability of the cultural/social environment to fully understand such a language defines whether or not it will be accessible for them. Technology is a key factor that has the potential to provide solutions to achieve a higher accessibility and therefore improve the quality of life of deaf and hard-of-hearing people. In this paper, we introduce a smart home automatization system specifically designed to provide real-time sign language recognition. The contribution of this paper implies several elements. Novel hierarchical architecture is presented, including resource-and-time-aware modules—a wake-up module and high-performance sign recognition module based on the Conv3D network. To achieve high-performance classification, multi-modal fusion of RGB and depth modality was used with the temporal alignment. Then, a small Croatian sign language database containing 25 different language signs for the use in smart home environment was created in collaboration with the deaf community. The system was deployed on a Nvidia Jetson TX2 embedded system with StereoLabs ZED M stereo camera for online testing. Obtained results demonstrate that the proposed practical solution is a viable approach for real-time smart home control. View Full-Text
Keywords: sign language; multimodal gesture recognition; home automatization; 3D convolution sign language; multimodal gesture recognition; home automatization; 3D convolution
Show Figures

Figure 1

MDPI and ACS Style

Kraljević, L.; Russo, M.; Pauković, M.; Šarić, M. A Dynamic Gesture Recognition Interface for Smart Home Control based on Croatian Sign Language. Appl. Sci. 2020, 10, 2300. https://doi.org/10.3390/app10072300

AMA Style

Kraljević L, Russo M, Pauković M, Šarić M. A Dynamic Gesture Recognition Interface for Smart Home Control based on Croatian Sign Language. Applied Sciences. 2020; 10(7):2300. https://doi.org/10.3390/app10072300

Chicago/Turabian Style

Kraljević, Luka, Mladen Russo, Matija Pauković, and Matko Šarić. 2020. "A Dynamic Gesture Recognition Interface for Smart Home Control based on Croatian Sign Language" Applied Sciences 10, no. 7: 2300. https://doi.org/10.3390/app10072300

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop