Next Article in Journal
Transient Liquid Phase Bonding of Copper Using Sn Coated Cu MWCNT Composite Powders for Power Electronics
Next Article in Special Issue
Joint Detection and Classification of Singing Voice Melody Using Convolutional Recurrent Neural Networks
Previous Article in Journal
Analytical Solutions for the Propagation of UltraShort and UltraSharp Pulses in Dispersive Media
Previous Article in Special Issue
Creative Chord Sequence Generation for Electronic Dance Music
Open AccessArticle

Real-Time Musical Conducting Gesture Recognition Based on a Dynamic Time Warping Classifier Using a Single-Depth Camera

Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei 10607, Taiwan
*
Author to whom correspondence should be addressed.
Appl. Sci. 2019, 9(3), 528; https://doi.org/10.3390/app9030528
Received: 11 October 2018 / Revised: 18 January 2019 / Accepted: 30 January 2019 / Published: 4 February 2019
(This article belongs to the Special Issue Digital Audio and Image Processing with Focus on Music Research)
Gesture recognition is a human–computer interaction method, which is widely used for educational, medical, and entertainment purposes. Humans also use gestures to communicate with each other, and musical conducting uses gestures in this way. In musical conducting, conductors wave their hands to control the speed and strength of the music played. However, beginners may have a limited comprehension of the gestures and might not be able to properly follow the ensembles. Therefore, this paper proposes a real-time musical conducting gesture recognition system to help music players improve their performance. We used a single-depth camera to capture image inputs and establish a real-time dynamic gesture recognition system. The Kinect software development kit created a skeleton model by capturing the palm position. Different palm gestures were collected to develop training templates for musical conducting. The dynamic time warping algorithm was applied to recognize the different conducting gestures at various conducting speeds, thereby achieving real-time dynamic musical conducting gesture recognition. In the experiment, we used 5600 examples of three basic types of musical conducting gestures, including seven capturing angles and five performing speeds for evaluation. The experimental result showed that the average accuracy was 89.17% in 30 frames per second. View Full-Text
Keywords: human–computer interaction; dynamic gesture recognition; depth camera; palm tracking; dynamic time warping; musical gesture; musical conductor human–computer interaction; dynamic gesture recognition; depth camera; palm tracking; dynamic time warping; musical gesture; musical conductor
Show Figures

Figure 1

MDPI and ACS Style

Chin-Shyurng, F.; Lee, S.-E.; Wu, M.-L. Real-Time Musical Conducting Gesture Recognition Based on a Dynamic Time Warping Classifier Using a Single-Depth Camera. Appl. Sci. 2019, 9, 528.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop