Next Article in Journal
Evaluation of In Vitro Bio-Activities Effects of WST (Wushanshencha)
Previous Article in Journal
Analytical Solution and Experimental Validation of the Electromagnetic Field in an IPT System
Previous Article in Special Issue
Real-Time Musical Conducting Gesture Recognition Based on a Dynamic Time Warping Classifier Using a Single-Depth Camera
Open AccessArticle

Joint Detection and Classification of Singing Voice Melody Using Convolutional Recurrent Neural Networks

Music and Audio Computing Lab, Graduate School of Culture Technology, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2019, 9(7), 1324; https://doi.org/10.3390/app9071324
Received: 1 February 2019 / Revised: 24 March 2019 / Accepted: 25 March 2019 / Published: 29 March 2019
(This article belongs to the Special Issue Digital Audio and Image Processing with Focus on Music Research)
Singing melody extraction essentially involves two tasks: one is detecting the activity of a singing voice in polyphonic music, and the other is estimating the pitch of a singing voice in the detected voiced segments. In this paper, we present a joint detection and classification (JDC) network that conducts the singing voice detection and the pitch estimation simultaneously. The JDC network is composed of the main network that predicts the pitch contours of the singing melody and an auxiliary network that facilitates the detection of the singing voice. The main network is built with a convolutional recurrent neural network with residual connections and predicts pitch labels that cover the vocal range with a high resolution, as well as non-voice status. The auxiliary network is trained to detect the singing voice using multi-level features shared from the main network. The two optimization processes are tied with a joint melody loss function. We evaluate the proposed model on multiple melody extraction and vocal detection datasets, including cross-dataset evaluation. The experiments demonstrate how the auxiliary network and the joint melody loss function improve the melody extraction performance. Furthermore, the results show that our method outperforms state-of-the-art algorithms on the datasets. View Full-Text
Keywords: melody extraction; singing voice detection; joint detection and classification; convolutional recurrent neural network melody extraction; singing voice detection; joint detection and classification; convolutional recurrent neural network
Show Figures

Figure 1

MDPI and ACS Style

Kum, S.; Nam, J. Joint Detection and Classification of Singing Voice Melody Using Convolutional Recurrent Neural Networks. Appl. Sci. 2019, 9, 1324. https://doi.org/10.3390/app9071324

AMA Style

Kum S, Nam J. Joint Detection and Classification of Singing Voice Melody Using Convolutional Recurrent Neural Networks. Applied Sciences. 2019; 9(7):1324. https://doi.org/10.3390/app9071324

Chicago/Turabian Style

Kum, Sangeun; Nam, Juhan. 2019. "Joint Detection and Classification of Singing Voice Melody Using Convolutional Recurrent Neural Networks" Appl. Sci. 9, no. 7: 1324. https://doi.org/10.3390/app9071324

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop