Automatic detection of Voice Disorders: recent literature advancements

Abstract

A short review of some recent findings in the field of automatic voice disorders detection and classification is provided in this article. The matter is getting more and more interest due to appealing non-invasiveness of the methods as well as the good achievable performances. An increasing role is played by Artificial Neural Networks (ANN), especially Deep ones, despite the need for large amounts of data for such networks, that are not always available for the task in question. The research in this field is directed in other directions too, including the investigation of new features and the capability to process running speech other than sustained sounds.

1. Introduction

The analysis of abnormal voice patterns or voice disorders plays an important role in diagnosing/treating several diseases and reducing the impact on the individual’s communication skills. Voice disturbances include alterations in the quality, pitch, or amplitude, among other characteristics, that diverge from voices of similar age, gender, and social groups. Voice pathology detection and classification can be accomplished by means of automatic and a noninvasive method, by capturing patient’s voice samples by a microphone, a smartphone, or any voice recorder, then submitting such samples to digital systems running specific software. Even though the terms detection and classification are used interchangeably very often, they actually refer to different tasks. Voice disorder detection refers to determining whether the given voice sample was produced by a person having a voice disorder or not; voice disorder classification task involves the ability to infer the type of disorder.

In this paper, some of the most recent studies in the field of automatic detection and classification of voice disorders are outlined. The found researches cover the use of sustained vowel as well as running speech, the investigation of novel features as well as novel classification methods to improve the accuracy of the outcome. Artificial Neural Networks (ANN) are more and more employed but traditional Machine Learning classifiers such as Support Vector Machines (SVM) still have their own advantages.

The remaining sections of the paper are organized as follows. The Section 2 reviews the most popular databases used to “learn” pathological and normal speech characteristics. Section 3 and Section 4 respectively present some representative recent research papers on automatic voice disorders detection (including mobile apps solutions) and classification. Section 5 focuses on speech impairments by central nervous system disorders, while Section 6 present some articles about Specific language impairment (SLI), also known as development dysphasia. Section 7 present our conclusions.

2. Databases

2.1. Massachusetts Eye and Ear Infirmary (MEEI) database

The SVD database is publicly available via the Internet, (Barry and Putzer 2020) contains not only voice samples but also electroglottographic (EGG) signals. The signals contain the information of the glottis movement during voice phonation. The materials include vowels /a/, /i/, /u / pronounced at different pitch (low, normal and high) for 1-3 s., the sentence "Guten Morgen, wie geht es Ihnen?’’ (Hello, how are you?), and EGG. The files have averages of around 1 and 3 s for sustained vowels and voice samples were sampled at 50 kHz with 16 bits of resolution.

2.3. Arabic voice pathology database (AVPD)

The AVPD (Mesallam et al. 2017) was recently developed at King Saud University, Riyadh. The database contains samples of sustained vowels, words, and paragraphs. All the speakers were native to Arabic language. Dysphonic patients suffering from five different types of organic voice disorders (cysts, nodules, polyps, paralysis and sulcus) were included in the database. The database contains repeated vowels, a running speech, Arabic digits and some common words. All subjects, including patients and normal persons, were recorded after clinical evaluation.

2.4. VOice ICar fEDerico II (VOICED)

The freely available VOICED database (Cesari et al. 2018) has been realized by the “Institute of High-Performance Computing and Networking of the National Research Council of Italy (IC-AR-CNR)” and the Hospital University of Naples “Federico II”. It consists of 208 healthy and pathological voices collected during a clinical study performed following the guidelines of the medical SIFEL (Società Italiana di Foniatria e Logopedia) protocol and the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013 Statement. For each subject, the database contains a recording of the vowel /a/ of five seconds in length, lifestyle information, the medical diagnosis, and the results of two specific medical questionnaires.

2.5. The Cantonese perceptual evaluation of voice (CanPEV) database

The CanPEV database (Law et al. 2010) was developed by the Division of Speech Therapy, the Chinese University of Hong Kong (CUHK). It consists of speech recordings from 232 native Cantonese speakers with either normal or pathological voices. The speech was recorded with a close-talking microphone in a quiet room at 44,100 Hz sampling rate. Each subject was required to produce repetitions of sustained vowels /a/, /i/ and /u/ (each one about 3 to 5 seconds long), 30–90 seconds of read speech, spontaneous speech from few seconds to few minutes.

2.6. The HUPA Database

This database was recorded at the Príncipe de Asturias hospital in Alcalá de Henares, Madrid, Spain (Moro-Velázquez et al. 2015; Arias-Londoño et al. 2011). The dataset contains sustained phonations of the vowel /a/ by 439 adult Spanish speakers (239 healthy and 200 pathological). Originally, the data was recorded with a sampling frequency of 50 kHz and later downsampled to 25 kHz. Pathological voices contain a wide variety of organic pathologies such as nodules, polyps, oedemas and carcinomas. More details of the database can be found also in Godino-Llorente et al. (2008).

2.7. The Advanced Voice Function Assessment Database (AVFAD)

The AVFAD database (Jesus et al. 2017) contains 363 healthy voices (253 females and 110 males) and 346 abnormal voices (247 females and 99 males). All clinical conditions were registered according to the Classification Manual of Voice Disorders-I (Verdolini et al. 2006). Participants were audiorecorded, producing the following vocal tasks: sustaining vowels /a, i, u/; reading of six CAPE-V sentences; reading a phonetically balanced text; spontaneous speech (Behlau 2003).

2.8. The Voice disordered and Healthy Adults Speech Database

The recordings for this database (Tulics et al. 2019) were collected at the Outpatients’ Department of the Head and Neck Surgery Department of the National Institute of Oncology, Budapest, Hungary, during consultations. The most common recordings are from patients with functional dysphonia and recurrent paresis. Samples from healthy people were recorded as well, these are used as comparison. All the participants had to read out loud the same eight sentence long text, titled ‘The North Wind and the Sun’. This folk tale is frequently used in phoniatrics as a demonstration of continuous speech. The text was read in its Hungarian translation. The database contains a total of 450 recordings, 257 from patients with voice disorders (156 females and 101 males) and 193 people with a healthy voice (108 females and 85 males).

2.9. The LANNA children speech corpus

This database by the Laboratory of Artificial Neural Network Applications (Grill and Tučková 2016) contains speech samples of children suffering from SLI and healthy controls, i.e. normally developing children with no language or speech disorders diagnosed. The patients’ group consists of 54 Czech children diagnosed with SLI in the 4 to 13 age group. Their speech was recorded in private speech and language therapist’s office and doctor’s office in Motol University Hospital. No information on the severity of the disorder is provided. The other (controls) group consists of 44 healthy Czech children in the 4 to 10 age group. The utterances from them were collected in school-rooms. The healthy controls subset comprises 1658 samples, while the SLI children subset comprises 2103 samples. All the recordings contain background noise as they were registered in situations simulating environment natural to the children to ensure their natural behaviour. The corpus consists of seven types of utterances: vowels, consonants, one-, two-, three-, and four-syllable words, and difficult words.

2.10. Parkinson’s UI Machine Learning database

Acoustic data in this database (Little et al. 2009) consists of 195 sustained vocal phonations of 31 male and female subjects, of which 23 were diagnosed with Parkinson’s disease. The age of the patients varies between 46 and 85 years (average of 65.8, standard deviation of 9.8). For each patient, averages of six phonations were recorded, with a length ranging from 1 to 36 s.

3. Some representative recent research papers on automatic voice disorders detection

AL-Dhief et al. (2020) presented a voice pathology detection system using Online Sequential Extreme Learning Machine (OSELM) to classify the voice signal into healthy or pathological, based on Mel-Frequency Cepstral Coefficients (MFCCs) as input feature. OSELM combines the advantage of good generalization performance at extremely fast learning speed of ELM (feedforward neural networks, “invented” in 2006 by G. Huang, 2006) with the capability to handle data samples obtained within packets over time (Abbas et al. 2017), instead of all at once. The voice samples for the vowel /a/ were collected equally from Saarbrücken voice database (SVD). The obtained results show that the maximum accuracy, sensitivity and specificity are 85%, 87% and 87%, respectively, showing that the proposed approach can differentiate healthy and pathological voices effectively.

While most approaches rely on feature extraction of the analysed signal with features subsequently fed into a classifier, Georgopoulos (2020), investigated the direct use a timefrequency distribution (namely, the WignerVille Distribution) of the voice signal and a deep learning classification method to automatically classify voice signals as normal or pathological. The time-frequency distribution is used as an image representation of the signal. The classification method is based on transfer learning of GoogleNet (Wu et al. 2018)) a welltrained Convolutional Neural Network (CNN) on large-scale natural images (unrelated to this problem) available in ImageNet. Voice data came from KAY Elemetrics (now Pentax Medical) database, developed by the MEEI Voice and Speech Lab. The samples used for analysis here is sustained phonation of vowel /a/. Achieved accuracy ranged from 69% to 74%. Kadiri and Alku (2020), presented a systematic analysis of glottal source features in normal and pathological voice and investigated their effectiveness in voice pathology detection. Voice pathology detection experiments were carried out using the HUPA and the SVD databases. The glottal source features were derived from three signals: from the glottal flows estimated with the quasi-closed phase (QCP) glottal inverse filtering method (Airaksinen et al., 2014), from the approximate source signals computed with the zero frequency filtering (ZFF) method (Sri et al. 2008) and directly from acoustic voice signals. The QCP method is based on the principles of closed phase (CP) analysis which estimates the vocal tract model from few speech samples located in the CP of the glottal cycle using linear prediction (LP) analysis. In contrast to the CP method, QCP takes advantage of all the speech samples of the analysis frame in computing the vocal tract model. ZFF is based on the fact that the effect of an impulse-like excitation (that occurs at the instant of glottal closure) is present throughout the spectrum including the zero frequency, while the vocal tract characteristics are mostly reflected in resonances at much higher frequencies. In this method, the acoustic speech signal is passed through a cascade of two zero frequency resonators and the resulting signal is equivalent to integration of the signal four times. Hence, the output grows or decays as a polynomial function of time. The trend is removed by subtracting the local mean computed over the average pitch period at each sample and the resulting output signal is referred as the zero-frequency filtered (ZFF) signal. Analysis of features revealed that glottal source features help in discriminating normal voice from pathological voice. A Support Vector Machine (SVM) with a radial basis function (RBF) kernel has been used as classifier. The studied glottal source features provide better discrimination compared to spectral features such as MFCCs and perceptual linear prediction (PLPs) features. Further, the combination of the existing spectral features with the glottal source features resulted in improved detection performance, indicating the complementary nature of features. The best achieved accuracy was about 78%.

Oliveira et al. (2020), investigated the feasibility of combining sustained vowels for computerbased pathological voice characterization. The Authors conducted experiments on samples of sustained vowels /a/, /i/ and /u/ from SVD and AVFAD datasets, exploring the wavelet decomposition levels in the range of 4 to 18, revealing that wavelet coefficients extracted from the combination of vowels improved signal description and, hence, identification of subtle features of pathological voices, using Random Forest classifier (Breiman 2001). They also showed that the Haar wavelet-based features (Shia and Jayasree 2017) extracted from combined vowels achieved accurate voice classification with fewer decomposition levels. This approach enabled accuracy improvements of at least 15.61 and 2.61% for SVD and AVFAD datasets, respectively, regardless of the biological gender, achieving a final accuracy ranging from 78% to 83%.

Tulics et al. (2019) investigated two types of input vectors (acoustic features and Automatic Speech Recognition -ASR- posterior probabilities) with a SVM- and DNN-based classifiers, using read text materials from Hungarianspeaking patients suffering from multiple types of diseases from the Voice disordered and Healthy Adults Speech Database. They found that using acoustic parameters instead of the use phone-specific posteriors as input features increases the accuracy for the detection and classification of disordered voices. The most important parameters, as suggested by the employed Forward Feature Selection (FFS) algorithm, were the mean of MFCCs, the range of SPI on voiced plosives and affricates, the standard deviation of HNR, the range of IMF on nasals, the mean and standard deviation of jitter, the standard deviation of MFCCs, the mean of SPI on voiced plosives and affricates, the range of SPI on the vowel [E] and the standard deviation of SPI on voiced plosives and affricates. Also, the DNN approach outperformed the SVM classifier. Later, the same Authors (Tulics et al. 2020) examined the combination of the two input vectors can contribute to improve classification accuracy. They concluded that it is not worthwhile to calculate ASR phone posteriors, as it has no significant impact on classification outcome, but it can greatly complicate and slow down a diagnosis support system that models the cognitive decision-making process of an expert. ASR phone posterior derived features are less effective in the automatic classification of healthy and dysphonic voices, than using the acoustic feature set directly. Adding ASR phone posterior derived features to the acoustic features does not significantly improve the automatic classification accuracy of healthy and dysphonic voices.

They explained this finding by the fact that ASR acoustic models are trained with the objective of being robust to variation, which is likely to shade the differences between dysphonic and non-dysphonic voices. If the training corpora used to train the ASR models were controlled and labelled w.r.t. dysphonia, phone posteriors could become useful for detecting dysphonia, however, as ASR training relies on very large datasets (ideally several hundred, rather thousands of hours of speech), this requirement is quasi hard to fulfil in practice.

3.1. Mobile apps for automatic detection of voice disorders

Ilapakurti et al. (2019), aiming at developing mobile diagnostic voice disorder app, investigated Mel-Scale Spectrogram and MFCCs as input features for several NN architectures: a 5-layer plain network, 5-layer CNN and a Recurrent Neural Network (RNN). Voice samples were obtained from a voice clinic in a tertiary teaching hospital (Far Eastern Memorial Hospital, FEMH), which included 50 normal voice samples and 150 samples of com-mon voice disorders, including vocal nodules, polyps, and cysts (collectively referred to Phono trauma); glottis neoplasm; unilateral vocal paralysis. Voice samples of a 3-second sustained vowel sound /a:/ were recorded at a comfortable level of loudness. The best model was a 5-layer CNN trained with MFCC and Mel-Spectrogram. It had a Sensitivity: 96% & Specificity: 18% on the test data.

Verde et al. (2019) propose a machine learning (ML)-based mobile voice disorder detection system. A trained model was directly embedded in a mobile application, allowing the user to evaluate the health of his/her own voice anywhere and at any time, without the necessity of trans-mitting user data to or storing user data on any server. This constitutes, at the time of writing, a significant innovation on account of the fact that most of the existing studies in literature limit the use of the mobile device to the tasks of acquiring the useful signal, transmitting it to an external server to be analysed and visualizing and communicating the results obtained to the users (Alhussein and Muhammad 2018; Muhammad et al. 2018). Unfortunately, the transmission of these patient data can be subject to a security attacks on security or privacy violations. The proposed mobile system, instead, has no need to transmit any data, so limiting the probability of any security attack.

4. Automatic classification of voice disorders

Liu et al. (2019) investigated phone posterior probabilities from a large-vocabulary ASR system trained with normal speech (Cantonese) to classify spoken utterances of 80 subjects extracted from the CanPEV database, which were already rated and divided into several categories: normal+mild, moderate, and severe. In addition to the proposed ASR voice features, the effectiveness of a set of conventional voice features that can be extracted from the utterance without using acoustical model has been investigated. The Authors adopted a minimalistic acoustic parameter set for voice analysis, known as eGeMAPS, which is implemented with the OpenSMILE toolkit (Eyben et al. 2016). Given the sequence of posterior vectors, the Authors proposed to use it to compute four types of feature parameters (a total of 18-dimension features), which are used to locate and quantify irregular posterior variations at specific speech sounds: PPV (Phone Posterior Variation), GOP (Goodness of Pronunciation), GOPV (GOP Variation) and BFR (Blurred Frame Ratio). By combining the contributions from the ASR voice features and conventional voice features, a subject-level prediction accuracy of over 80% on three severity classes has been achieved. Subjects with mild disorder and those with severe disorder could be perfectly distinguished by the proposed method. (Miliaresi et al. 2021) addressed the task to classify functional dysphonia, phonotrauma, laryngeal neoplasm and vocal paralysis and showed that it is possible to treat MFCC derived features and data from medical records as two different input sources to a single neural network architecture consisting of two subnetworks. The first one, a CNN is used to treat the acoustic signal as an image, that captures spectral shape by operating on MFFC derived features and simple filterbank outputs. The second (feed-forward) network analyses an enhanced input vector, consisting of the demographic parameters and mid-term signal features. The outputs of the aforementioned subnetworks are concatenated and fed to a dense layers with 1024 nodes, with each node’s output being processed by a Rectified Linear Unit (ReLu) activation function. Finally, a softmax output layer of four units is used to produce posterior probability estimations of the four classes of the problem under study.

5. Speech impairments by central nervous system disorders.

Lauraitis et al. (2020) adopted Bidirectional Long Short-Term Memory (BiLSTM) neural network and Wavelet Scattering Transform with Support Vector Machine (WST-SVM) classifier for detecting speech impairments of patients at the early stage of central nervous system disorders (CNSD). The study includes 339 voice samples collected from 15 subjects: 7 patients with early stage CNSD (3 Huntington, 1 Parkinson, 1 cerebral palsy, 1 post stroke, 1 early dementia), other 8 subjects were healthy. Speech data is collected using voice recorder from Neural Impairment Test Suite (NITS) mobile app. Features are extracted from pitch contours, Mel-frequency cepstral coefficients (MFCC), Gammatone cepstral coefficients (GTCC), Gabor (analytic Morlet) wavelet and auditory spectrograms. 94.50% (BiLSTM) and 96.3% (WST-SVM) accuracy is achieved for solving healthy vs. impaired classification problem. The developed method can be applied for automated CNSD patient health state monitoring and clinical decision support systems as well as a part of Internet of Medical Things (IoMT).

5.1. Parkinons’ desease (PD)

Kodrasi and Bourlard (2020), proposed to use the spectro-temporal sparsity characterization as a robust feature for dysarthric speech detection, based on the motivation that since dysarthric speech of patients suffering from PD is breathy, semi-whispery, and is characterized by abnormal pauses and imprecise articulation, it can be expected that its spectro-temporal sparsity differs from the spectro-temporal sparsity of healthy speech. The Authors first provided a numerical analysis of the suitability of different non-parametric and parametric measures (i.e., l1-norm, kurtosis, Shannon entropy, Gini index, shape parameter of a Chi distribution, and shape parameter of a Weibull distribution) for sparsity characterization. It is shown that kurtosis, the Gini index, and the parametric sparsity measures are advantageous sparsity measures, whereas the l1-norm and entropy measures fail to robustly characterize the temporal sparsity of signals with a different number of time frames. Second, they proposed to characterize the spectral sparsity of an utterance by initially timealigning it to the same utterance uttered by a (arbitrarily selected) reference speaker using dynamic time warping. Experimental results on a Spanish database of healthy and dysarthric speech showed that estimating the spectrotemporal sparsity using the Gini index or the parametric sparsity measures and using it as a feature in a support vector machine results in a high classification accuracy of 83.3%.

Asmae et al. (2020), used ANN and K-Nearest Neighbours (KNN) algorithms, in the purpose of distinguishing between PD patient and healthy individual. Voice data in the Parkinson’s UI Machine Learning has been used. Standard features derived from fundamental frequency, jitter and shimmer, have been used, as well as non-standard features such as Correlation Dimension (Kantz and Schreiber 2003), Recurrence Period Density Entropy and Detrended Fluctuation Analysis (Dixit 1988): 22 features in total. The ANN has two hidden layers and the Levenberg-Marquardt (LM) has been used as training optimization algorithm (Hagan and Menhaj 1994). Experimental results have showed that the ANN classifier achieved higher average performance than the KNN classifier in term of accuracy. The established system can distinguish healthy people from an acceptable range of people with PD with an accuracy rate of 96.7% by using ANN, and 79.3% by using KNN when the number of neighbours taken was k=1, by using the cosine distance.

6. Specific language impairment (SLI)

Several very recent researches regarded Specific language impairment (SLI), also known as development dysphasia.

Kotarba and Kotarba (2020), proposed an efficient approach to automatic detection of SLI based on log-power spectrograms of speech samples. The utterances from the LANNA children speech corpus were used to calculate the normalized log-power spectrograms. Deep neural network algorithm based on ResNet architecture (He et al. 2016) was used to perform the classification task. The accuracy rate of proposed SLI detection method exceeds 99% in the speaker independent scenario.

Reddy et al. (2020), proposed a method for SLI detection in children that utilizes time- and frequency-domain glottal parameters, which are extracted from the voice source signal obtained using quasi-closed phase (QCP). In addition, 12 MFCCs and openSMILE based acoustic features are also extracted from speech utterances, including min (or max) value and its relative position, standard deviation, range, median, skewness, kurtosis, 2 linear regression coefficients, and quadratic error of the following features: root mean square (RMS) energy, zero crossing rate, pitch and voicing probability. SVMs with RBF kernel and feed-forward neural network (FFNN), are trained separately for the MFCCs, openSMILE and glottal features. A leave-fourteen-speakers-out cross-validation strategy is used for evaluating the classifiers. The experiments are conducted using the LANNA corpus. Experimental results show that the glottal parameters contain significant discriminative information required for identifying children with SLI. Furthermore, the complementary nature of glottal parameters is investigated by independently combining these features with the MFCCs and openSMILE acoustic features. The overall results indicate that the glottal features when used in combination with MFCCs feature set provides the best performance with the FFNN classifier in the speaker-independent scenario (98.82%). Sharma and Singh (2020) used sustained phonation of vowel /a/ uttered by children, from the LANNA database, to detect and classify control (healthy) and experimental (SLI) group using linear predictive coding (LPC) feature set. LPC order was set to 8, and a set of 408 features was build using 17 statistical function applied to the 8 coefficients, their delta and delta-delta (“delta” refers to the difference between two consecutive feature frames). A standard nonparametric Mann-Whitney non-parametric Utest was applied to filter the significant features for 95% level of confidence. The top-20 and top-10 features were then selected by computing the Spearman’s rank correlation coefficients. Naïve-Bayes (NB) and SVM were employed for machine learning task. The best accuracies were obtained from NB classifiers i.e. 97.9% (for top-20 LPC features) and 97.8% (for top-10 LPC features) with 5-fold crossvalidation protocol.

7. Conclusions

From this unexhaustive review of the most recent attempts to improve automatic voice disorders detection and classification, it is clear that the role of ANN is going to increase in the near future. More and more complex networks are investigated such as Online Sequential Extreme Learning Machine (OSELM) and GoogleNet and transfer learning is applied to face the availability of limited amount of data compared to the amount that would be required to train from scratch the most complex networks. Accuracy achievable with ANN can be higher than 90%, with peaks close to 99% in some specific tasks.

One of the most promising approach is to embed the capability to detect voice disorders in mobile devices as software app, allowing a portable and usable solution to monitor the quality of voice in real time.

An intense research activity is also devoted to find and select new features to augment or replace the classical one to improve the recognition accuracy, including a trend to use directly spectrographic or time-frequency representations of voice samples as images to feed ANN good at image-recognition.

Finally, running speech is increasingly considered instead of sustained vowels to take into account more realistic speech scenarios.