A Hybrid U-Lossian Deep Learning Network for Screening and Evaluating Parkinson’s Disease

Abstract

Speech impairment analysis and processing technologies have evolved substantially in recent years, and the use of voice as a biomarker has gained popularity. We have developed an approach for clinical speech signal processing to demonstrate the promise of deep learning-driven voice analysis as a screening tool for Parkinson’s Disease (PD), the world’s second most prevalent neurodegenerative disease. Detecting Parkinson’s disease symptoms typically involves an evaluation by a movement disorder expert, which can be difficult to get and yield varied findings. A vocal digital biomarker might supplement the time-consuming traditional manual examination by recognizing and evaluating symptoms that characterize voice quality and level of deterioration. We present a deep learning based, custom U-lossian model for PD assessment and recognition. The study’s goal was to discover anomalies in the PD-affected voice and develop an automated screening method that can discriminate between the voices of PD patients and healthy volunteers while also providing a voice quality score. The classification accuracy was evaluated on two speech corpora (Italian PVS and own Lithuanian PD voice dataset) and we have found the result to be medically appropriate, with values of 0.8964 and 0.7949, confirming the proposed model’s high generalizability.

1. Introduction

Parkinson’s disease (PD) is the second most prevalent neurodegenerative disease [1]. PD mostly affects people over 50 years of age, although the number continues to decline, and the number of affected subjects only increases as the population increases. With more than 10 million people diagnosed each year in the world, 1 million in the USA and comparable rates of occurrence have been observed in Europe [2]. The pathogenetic mechanisms of the disease are variable and not yet fully understood [3]. The disease presents itself with a wide spectrum of motor and non-motor symptoms, including those that directly affect a person’s capability to operate. Clinically manifested PD is preceded by a prodromal period, lasting for decades, when there are no clinical signs or unspecific symptoms, such as constipation, apathy, daytime sleepiness, inattention, depression, anxiety, smell impairment, pain, motor slowing, etc., may be present [4,5,6,7]. Motor symptoms appear relatively late in the course of neurodegeneration, only when compensatory mechanisms of smooth motor control are exhausted and do not manifest until more than half of the dopaminergic neurons in the substantia nigra region in the midbrain are dead [8]. Nevertheless, they are the signs that make the disease recognizable. The cardinal symptoms of PD are bradykinesia, which represents impairment of voluntary motor control and is characterized by slowness and gradually decreasing range of movement, muscle rigidity, and tremor [9]. To date, the diagnosis of PD is exceptionally clinical and faces many challenges. Firstly, it comes late in the context of the neurodegenerative process. Secondly, the diagnosis requires experience in recognizing clinical signs, especially when motor symptoms are mild, as mild motor symptoms as well as non-motor symptoms are prevalent in the healthy elderly population [10]. The symptoms of PD overlap with other neurodegenerative, toxic, or vascular diseases that might have a different prognosis and require different approaches to treatment, and even with aging, as mentioned above.

Early identification of PD is important, as disabling motor symptoms, which otherwise may be attributed to aging or other causes, could be treated highly effectively when recognized. Earlier disease intervention may result in longer uncompromised working capacity, as well as the prolonged quality time of the patient’s life. The prospect of neuroprotective treatment that is expected to emerge in the near future demands identification of those who need it even earlier, before the symptoms appear. Many studies in the area of neurodegeneration have recently been directed toward finding the disease biomarkers [11]. Speech is very vulnerable to degeneration of neural structures [12] and quality decline has been extensively reported in the PD research literature [13,14]. It is therefore not surprising that speech acoustic analysis in PD has been receiving an exponentially growing scientific interest in recent years as it has the potential to reveal a lot of information about fine motor control. The ease and broad availability of the recording technique makes it an excellent candidate for becoming a diagnostic biomarker as well as a progression marker for PD.

Fundamental frequency variability has been discovered as early as 5 years before diagnosable symptoms [15]. Rush et al. have repeatedly shown that subjects with early PD [16,17] and even with rapid eye movement behavior disorder (RBD) [18] (which is recognized as the strongest predictor of neurodegeneration, including Parkinson’s disease) manifest various articulatory, phonatory, and prosodic speech deficits, a combination of which can discriminate between the people with PD and the control group. Early detection and diagnosis can help with therapy, but diagnosis usually necessitates an interview with a healthcare professional or the completion of a formal diagnostic questionnaire. As a result, inconspicuous techniques to monitor depression symptoms in daily life could be quite useful in diagnosing and screening PD and determining whether or not it requires professional treatment [19].

Speech therefore must be constantly reevaluated to improve the patients’ quality of life; therefore, PD-impacted speech must be monitored and screened on a regular basis. Deterioration is linked to swallowing problems in PD [20], which in turns raises the possibility of choking, aspiration pneumonia, and untimely death. Swallowing problems are often underreported by patients with PD, not least related to the lack of suitable screening methods. Monitoring for speech deterioration with a simple and even remotely available acoustic analysis method could serve as a tool to schedule a visit to the doctor. Speech disorders impair patients’ capacity to communicate, as they may talk slowly and cannot convey themselves effectively. Their speech is sometimes breathy and mumbled towards the end of a phrase. As a result, individuals are unable to express emotions when speaking. It also has an impact on their socialization capacity, a case that has been especially pronounced during and after the COVID-19 pandemic [21]. Healthy persons may alter their voices to produce a variety of sounds that require delicate coordination and control of the articulatory and respiratory muscles. Patients with PD, on the other hand, have impaired neuromotor control, which affects the vocal mechanism and, as a result, the sounds generated. Speech analysis by machines might help in early identification of PD, however there are large inter- and intra-individual differences, making this difficult. Recent development of speech-based PD screening tools based on speech analysis show the acceptable classification rates in distinguishing between healthy participants and PD patients. However, in these investigations, the data used to build the classification model included voice recordings from both early and late-stage PD patients with varying degrees of speech impairment, resulting in unrealistic results for real-life application cases. In a more realistic scenario, healthy participants or early PD patients with moderate speech impairment will employ an early screening method in questionable situations [22]. The study [23] showed that vocal tract length acquired from phoneme recording using a smartphone may reliably identify persons with Parkinson’s disease.

Our study’s goal was to discover anomalies in the PD-affected voice and develop an automated screening method capable of distinguishing between the voices of PD patients and healthy individuals while also providing a voice quality score. We look at whether speech recordings might be utilized as a simple, low-cost method of PD assessment and screening, using deep learning for expert score prediction and evaluation. Acoustic investigations enable high-throughput screening that, if the screening results are aberrant, can be followed by a comprehensive medical evaluation. We developed an automated screening method that can discriminate between the voices of patients with Parkinson’s disease and healthy participants, as well as provide a voice quality score, to detect anomalies in the voice impacted by Parkinson’s disease. The main challenge was getting the algorithm to work with different forms of voice input. The Italian PD speech samples (training material) used sustained vowel phonation, while the Lithuanian samples used phonetically balanced phrase analysis. However, because sustained vowel phonation is a type of phonation that is manufactured, that feature establishes the same limitation in the meaning of the sustained vowel signal analysis. Connected speech, on the other hand, is more common in ordinary conversation and may be considered more “ecologically legitimate”. As a result, we have raised a hypothesis to design a dedicated deep learning based PD voice screening system, that is, if an objective voice evaluation is to be considered robust and ecologically valid for screening purposes, acoustic measurements should ideally be performed using both speaking patterns—sustained phonation and running speech.

The manuscript is further organized as follows: initially, we offer a state-of-the-art review of Parkinson disease voice signal processing, then the dataset and the methodology of our U-lossian deep neural network approach, followed by the experimental evaluation, discussion, and conclusions.

2. State of the Art Review of Signal Analysis Based Approaches for Analyzing Parkinson’s Disease

Given the technological nature of the paper, this section aims to provide a technological review of numerous computer-driven methodologies utilized for technological Parkinson disease analysis and investigation, categorization, and screening.

The great majority of patients diagnosed with PD have vocal performance impairment [22], which may often be one of the first signs of the onset of the diseases and there is a clear demand for objective symptom assessment and screening [24]. Speech faults caused by Parkinson’s disease include, among other things, decreased voice loudness [25], mono pitch [26], and improper consonant articulation as was defined by Tukalova’s team [27]. Recent research have created tools to distinguish PD from controls and track speech rehabilitation in PD using high-quality voice recordings [28,29]. Recent research have also looked at the practicality and usefulness of employing smartphone technology [30] to help with the clinical diagnosis of Parkinson’s disease, which expanded the use of speech data to include four more tests for dexterity, postural sway, gait, and response times [31]. Modern AI-powered screening was shown to be hardware-independent, yielding accurate findings. According to a research conducted by Rusz et al. [32], smartphones can identify speech problems in those who are at high risk of acquiring PD.

Acoustic measurements such as fundamental frequency variability, pause interval duration, and speech timing rate extracted from spontaneous speech were sensitive enough to distinguish between groups and demonstrated a strong correlation and reliability between the professional microphone and the smartphone. Ehsan et al. [33] found a similar reliable results when using the k-nearest neighbor (KNN) algorithm. Bot et al. [34] also introduced an iPhone app to assess the PD patient’s memory, tapping, speech, and walking. Zhan et al. [35] created an app to assess the feasibility of remotely monitoring PD symptoms, and to use an AI-based approach to distinguish between measures before and after medication. Lipsmeier et al. [36] compared smartphone outcome metrics to traditional clinical practice with trials that substantially distinguished PD from healthy persons with $p<0.005$.

Regarding the applications of artificial intelligence (AI), recent research has shown that the use of machine learning techniques to identify PD from non-PD using extracted speech data is successful [37,38]. Behroozi et al. [39] developed a multi-classifier framework to distinguish Parkinson’s disease patients from healthy controls. Tsanas et al. in Refs. [30,40] studied the link between the characteristics of speech signals and the motor impairment score in Parkinson’s disease patients. Perez et al. [41] created an autonomous feature extraction system to diagnose Parkinson’s disease and follow its development. Khan et al. [42] evaluated the severity of PD using audio recordings of the vocal function. Ensemble learning was also important in boosting PD classification accuracy. As demonstrated by Mohammadi et al. [43], stacking results of conventional classifiers are time-efficient, low-cost, and more accurate, with an achieved accuracy of 95–97%. While these and related studies show great classification accuracy, they have numerous limits as they progress upward. Small cohort sizes are a more well-known issue, as they limit the generalizability of conclusions to a wider and more diversified population. Some researches also disregarded the fact that a person with PD may have greater trouble pronouncing some words than others. As a result, summarizing the voice examination may result in the loss of vital information [44]. To overcome this issue, the categorization setting must take into consideration what has been communicated. Identity confounding is a less well-known issue in which several voice samples are obtained from each subject and these samples appear in both training and testing data, therefore because the model has trained to identify features of certain subjects and is utilizing that knowledge to identify the label in the test set, the model’s performance is too optimistic. Furthermore, adopting a simple classifier does not lead to high prediction accuracy, thus deep learning based techniques for improving model performance include data normalization [45], feature selection, and feature extraction [46]. This is a sort of data leakage in which information is transferred between the training and test sets accidentally, resulting in exaggerated performance measures [47].

More fundamentally, studies that merely categorize patients as having Parkinson’s disease or not having it have limited value in terms of enhancing their quality of life. Although techniques to improve diagnostic accuracy are needed, diagnosis is usually achieved only after the disease has advanced to a more severe stage, i.e., when symptoms appear. Ali et al. [48] proposed to address the above-mentioned issues, with the aim of creating a hybrid system capable of automatically performing acoustic analysis of voice signals to identify PD. The suggested smart system employs linear discriminant analysis to reduce dimensionality and a genetic algorithm to optimize the hyperparameters of a neural network utilized as a predictive model. Furthermore, to eliminate subject overlap, they excluded one participant from validation and attained a 95% accuracy. Arora et al. [31] attempted to investigate the scalability of voice as a population screening tool for PD, and performed the Parkinson’s Speech Initiative (PVI) research, a PD characterization study using telephone-quality voice. PVI was the first large-scale study of its sort to gather speech data from PwP and control participants in free-living acoustic circumstances, with the goal of distinguishing PD individuals from controls using phonations acquired in non-acoustically controlled situations. Viswanathan et al. [49] used two complexity measures: fractal dimension (FD) and normalized mutual information (NMI) to compare the voices of Parkinson’s disease (PD) and age-matched control (CO) subjects when uttering three phonemes. They found that the voices of PD patients have lower FD and NMI between voice recordings of PD–CO and PD–PD is higher than CO–CO. This shows that using NMI from the sample voice in combination with the known CO and PD groups can be utilized to detect PD sounds. Hirens et al. [50] proposed an ensemble of convolutional neural networks for detecting PD from speech samples. For the dataset, the solution obtained 99% accuracy. To tackle the problem of imbalanced data samples, Polat et al. [51] employed a Synthetic Minority Over-Sampling Technique, followed by a Random Forest model to categorize the samples. Gabriel et al. [52] achieved an accuracy of roughly 94% for a Wrappers feature subset selection preceding an SVM. Tuncer et al. [53] acquired a greater efficiency by using a KNN technique followed by minimal average maximum tree and singular value decomposition as feature extractors. According to the findings of Pah et al. [54], automated analysis of various phonemes should be used. An approach for categorizing PD that is sensitive to privacy has been presented by Laganas et al. [55]. Voice characteristics collected from running speech signals were recovered from passively recorded voice calls. To fuse and predict speech characteristics, language-aware training of multiple- and single-instance learning classifiers was used, yielding an AUC of 0.84.

Overview of Deep Learning Based Approaches to PD Speech Analysis

Deep learning (DL) architectures and algorithms are rapidly being employed in research to handle complex PD voice signal processing issues [56]. The initial step is to convert the input speech stream into voice feature vectors or tensors that DL models can assess. As previously indicated in the main body of the state-of-the-art review, numerous aspects of voice are included in the voice characteristics of Parkinson’s disease patients, further varying in relation to the native tongue of the speaker. The second step in DL-based approaches is to apply a classification or analysis to the retrieved voice characteristics, as vocal biomarkers are a potentially useful technique of monitoring symptoms and severity [57]. This section thus examines DL-related models and the effectiveness of strategies used for PD identification from voice.

Krishna et al., tested multiple architectures of the 1D CNN model to accurately detect the illness using extracted speech data and obtained about 87% accuracy [58]. Sparse kernel transfer learning was proposed by Zhang et al. [59] to extract the effective structural information of PD speech characteristics from public datasets as source domain data. They also used a quick alternating direction multiplier iteration strategy to improve information extraction performance, attaining an overall accuracy of 86.7%. Similarly, Ma et al. proposed a deep dual-side learning ensemble model with a weighted fusion mechanism to fuse the classification models into a classification ensemble model, attaining 98.4% accuracy [60]. On the Max Little dataset, Ouhmida et al. [61] used traditional CNN with an accuracy of 93.10%. An ensemble of convolutional neural networks with Gaussian blurring allowed a further increase in accuracy [62]. Similarly, the spectrum of the audio recordings was computed and utilized as an image input to the pre-trained ResNet18 architecture using the ImageNet and SVD databases with an accuracy of 97.1% in Ref. [63]. Alexnet adapted by Ref. [64] showed a very similar performance, while Densnet had lower accuracy, albeit on a different dataset [65]. Combining the CNN classifier with additional iterative adaptive inverse filtering (IAIF) and quasi-closed phase (QCP) glottal inverse filtering algorithms can assist in utilizing the baseline and glottal information generated from each spoken utterance and the related healthy/PD classifications [66]. Anisha et al. [67] proposed adopting a method in which weak learners are bagged and boosted before making predictions, which will certainly provide better outcomes than basic ensemble voting and stacking. The suggested technique achieved an accuracy of 94.12% by enhancing ensemble classifiers. Grover et al. [68] using UCI’s Parkinson’s Telemonitoring Voice Data Set of patients, suggested an approach for predicting Parkinson’s disease severity using deep neural networks with 82% accuracy in controlled conditions, while a very similar approach with mRMR feature selection improved the accuracy levels on the same dataset [69]. Danish et al. [70] evaluated the potential of a deep neural network (DNN) and long short-term memory (LSTM) network-based model for predicting Parkinson’s illness using speech samples from a person with 97.1% accuracy. Quan et al., proposed an improvement, using a Bidirectional long-short term memory (LSTM) model to identify PD by capturing time-series dynamic aspects of a speech stream [71]. The authors of Ref. [72] developed an SSWA-based attention-based LSTM that provided improved performance with 92.5% accuracy on an Indian language dataset. The experiments by Zhangs et al. [73] showed that DF-EMD can be utilized to identify PD efficiently since the high-frequency section of the speech signal carries more information concerning PD. Nagasubramanian and Sankayya investigated the capability of acoustic-based DL approaches, discovering that combining these techniques results in around a 3% improvement in performance [74].

3. Materials and Methods

3.1. Dataset

This study was approved by the Kaunas Regional Ethics Committee for Biomedical Research (No. BE-2-49). Voice samples were obtained from 104 PD subjects examined in the Lithuanian University of Health Sciences (Lithuania). Serial numbers were assigned to each participant at the time of inclusion to protect their identity.

Speech recordings of the phonetically balanced Lithuanian sentence “Turėjo senelė žilą oželį” (‘The grandmother had a little grey goat’) were obtained using a T-series silent room for hearing testing (T-room, CA Tegner AB, Bromma, Sweden) via a D60S Dynamic Vocal (AKG Acoustics, Vienna, Austria) microphone placed 10.0 cm from the mouth with an about 90${}^{\circ }$ microphone-to-mouth angle. Speech recordings were made at a rate of 44,100 samples per second and were exported as uncompressed 16-bit deep WAV audio files.

The phonetically balanced Lithuanian sentence “Turėjo senelė žilą oželį” (‘The grandmother had a little grey goat’) was recorded using a T-series silent room for hearing testing and a D60S Dynamic Vocal (AKG Acoustics, Vienna, Austria) microphone placed 10.0 cm from the mouth with an approximately 90${}^{\circ }$ microphone-to-mouth angle. Speech was recorded at 44,100 samples per second and produced as uncompressed 16-bit deep WAV audio files. Speech samples were obtained from patients diagnosed with Parkinson’s disease at the Lithuanian University of Health Sciences Hospital, Kaunas Clinics by an experienced neurologist according to the UK Parkinson’s Disease Society Brain Bank criteria [75]. Only physically independent patients up to stage 3 on modified Hoehn and Yahr scale (H-Y), without any concomitant neurological or hearing disorders, respiratory tract infection, or any voice problems unrelated to PD at the time of the recording, were invited to participate in the study. The time of the recording was adjusted to best suit the schedule of the patient as well as the availability of the examiner, therefore, not all the recordings were made in best “on” state of the patients, who took medication for PD. The sample consisted of 61 patients (28 males and 33 females), aged 39–84 (mean age 64.9 (SD 9.7)), with a duration of the disease of up to 14 years (mean 3.6 (SD 3.6)), based on the time of diagnosis. Most participants were in H-Y stage 2 (29.5%), followed by H-Y stage 3 (26.2%), and H-Y stage 2.5 (19.5%). The normal speech subgroup was composed of 43 healthy volunteers who had no present or pre-existing speech, hearing, neurological, or laryngeal disorders and considered their speech as normal. Volunteers were free of upper respiratory infection at the time of the recording. Laryngeal endoscopy was done using the XION EndoSTROB DX device (XION GmbH, Berlin, Germany) with a 70-degree rigid endoscope without topical anesthesia for both the patient and control groups. Subjects with any pathological alterations of the laryngeal fold, such as polyps, granulomas, paralysis etc., were not included in the study. This dataset was used only for the validation of our U-lossian network.

For the training of the algorithm we have used the publicly accessible ItalianPVS corpora [76], where each speaker recorded five sustained vowel sounds with two iterations each. These samples were contributed by 22 (10 men and 12 women) healthy volunteers (HC) aged 66–77 years and 28 (19 men and 9 women) patients with PD, aged 40–80 years. The healthy subgroup did not report particular speech or language disorders. The patients reported no speech or language disorders that were not related to their Parkinson’s disease. All patients received antiparkinsonian treatment. The disease severity was classified by the specialists as <4 on the modified Hoehn and Yahr scale for 25 patients, stage 4 for 2 patients, and stage 5 for 1 patient. All of the speakers in this corpus were recorded in Bari, Puglia, Italy. Each recording session was conducted in a controlled atmosphere in a quiet, echo-free room, and speech samples were recorded after a short explanation by the specialist. Elements such as room temperature, microphone distance (15 to 25 cm), time of day, and a discussion with the subject to warm up their vocal muscles were taken into account. The sample frequency was set to 16 kHz. The paper [77] has further details.

3.2. U-Lossian Deep Learning Network

We sought a customized strategy since one of the limitations of traditional loss function implementation is that it equally weights false positive (FP) and false negative (FN) identification. In practice, this leads to voice segmentation maps with great accuracy but low recall. FN detections must be weighted higher than FPs to improve the recall rate for extremely imbalanced data and smaller ROIs, such as the small Lithuanian Parkinson dataset. Another problem with typical loss functions is that they have difficulties segmenting small ROIs since they do not contribute much to the loss. To address this, we propose using a custom loss function that is parametrized to control the difference between easy background and tough ROI training samples. To focus on speech samples recognized with lesser probability, we propose applying a parameter that exponentiates the cross-entropy loss. We perform fast Fourier transformation (FFT) to the vocalized audio sample with 201 bins to build the spectrogram. Mel-frequency cepstral coefficients (MFCCs) are then calculated using 40 mel scale coefficients from the FFTs, as recommended by Ref. [78]. As a consequence, an MFCC picture with a height of 40 pixels and a length equal to the audio duration is produced.

The implementation flow is represented in Figure 1 and the process sequence diagram is given in Figure 2. A doctor has two possibilities when using the PD screening application: (1) utilize a previously recorded audio file, or (2) record live. The signal is then converted to a waveform (any system compatible codec is permitted as it has no discernible effect on classification effectiveness). The signal is then preprocessed for 16 kHz, which speeds up the computation, and is then transformed to mono. The unique characteristics indicating the logarithmic perception of a PD patient’s unique intensity and tone were extracted using MFCC. The actions for assessing MFCC features must be conducted in the sequence of each speech signal, and then these characteristics were used to train the CNN algorithm, which also extracts features. Within each vocal outline, a feature vector was saved to hold the features related to the specific spectrogram “frame”. This also aided in reducing the complexity of the model and achieving improved recognition accuracy. The doctor receives a result along with numerous calculated voice parameters.

To achieve a better balance of accuracy and recall in the processed voice, we presented a modified Hybrid Mask U-Net architecture with an adaptive custom loss function (further named U-lossian). Our approach was inspired by the popular architectures Mask U-Net (often used for “segmentation” of sounds, e.g., [79]) and Resnet18 (often used for “classification” of sounds, e.g., [80]). We have aimed for an architecture design to operate well with a modest number of training samples, making it appropriate for less popular language datasets such as Lithuanian. The network is made up of two paths: one that contracts to extract locality characteristics and one that expands to resample the image maps with contextual information. Skip connections are used to integrate high-resolution local characteristics with low-resolution global features in order to get more relevant results. During our research, we noticed that while training a negative gradient over-suppression, for example, each positive sample of one class in softmax or sigmoid cross-entropy might be interpreted as a negative sample for other classes, resulting in more suppressed gradients for tail classes. To prevent gradient over-suppression, we developed a custom loss function with better regularization. It may also be used to compare the expected and actual sampling frequencies of each class, and then use the division of these frequencies to re-weight loss values for different classes. The custom loss function prioritizes less accurate misclassified predictions. When we detect over-suppression of this custom loss, it typically means that the model is near convergence. We tested with high learning rates and found that the learning rate of $1\times {10}^7$ produced the best results; therefore we trained all experiments with it. To prevent loss function over-suppression, we trained intermediate layers with the usual U-net strategy but supervised the final layer with the loss function to reduce sub-optimal convergence. The input of our U-lossian model is an image representing the voice spectrogram, which is denoted as X. The network learns to map the input to the prediction Y according to a convolutional feature given by Equation (1):

$$Y=F_n\left(X_n\right|{\Theta }_n)=h(W{X}_n+b),{\Theta }_n=[W,b]$$

(1)

here $X_n$ is a one-dimensional input matrix of N feature maps, W is a set of N one-dimensional kernels that is used to extract a set of features from the input values, h is an activation function, and b is a bias vector.

Two maximum pooling layers are included in the U-lossian model. The maximum pooling approach is used by the pooling layer in the contraction route to extract prominent characteristics in a region. The pooling size is $2\times 2$ and the step size is 2, which is the same as downsampling twice. Two-layer fusion layers are also included in the model. Combining the features retrieved by the contraction path’s convolutional layer with the output results of the expansion path’s upsampling layer, and merging the information lost in feature extraction into the expansion path, can improve the output result.

One-layer Dropout layers are included in the model. We improve the model’s prediction performance on the test set, and employ the Dropout layer in the U-lossian model to speed up the convergence. Two layers of upsampling are also included in the model. In the expansion path, the features of the same size retrieved by the respective contraction layer are combined, and up-sampling is performed twice to generate an output of the same size as the original input.

A loss function, denoted by $\ell :H\times Z\to {\mathbb{R}}_{+}:=[0,+\infty )$, measures the difference between a predicted label and a true label, e.g., $L^2$ loss: $\ell (f_{\theta },z)=\frac{1}{2}{(f_{\theta }\left(x\right)-y)}^2$, where $z=(x,y)$.

Training loss for a set $S={(x_i,y_i)}_{i=1}^n$ is denoted by $L_S\left(\theta \right)$ or $L_n\left(\theta \right)$ or $R_n\left(\theta \right)$ or $R_S\left(\theta \right)$,

$$L_S\left(\theta \right)=\frac{1}{n}\sum _{i=1}^n\ell (f_{\theta }\left(x_i\right),y{}_i)+\lambda W.$$

(2)

here $\lambda W$ is a regularization term that penalizes the complexity of the model.

The loss function used in the study is the cross-entropy loss function, which is defined as follows:

$$\begin{array}{cc}\hfill L(Y,P(Y/X\left)\right)& =-logP(Y/X)\hfill \\ \hfill & =-\frac{1}{N}\sum _{i=1}^N\sum _{j=1}^M{y}_{ij}log\left(p_{ij}\right).\hfill \end{array}$$

(3)

The loss function is L, the output variable is Y, and the input variable is X. $y_{ij}$ is a binary index showing whether category j is the real category of input instance $xi$. N is the input sample size, M is the number of potential categories, and $y_{ij}$ is a binary index indicating whether category j is the real category of input instance $x_i$. The probability that the model prediction input instance $x_i$ corresponds to category j is represented by $p_{ij}$. The optimal model weights and bias settings are determined by minimizing the loss function, allowing the model to better correctly forecast the PD.

A good optimization strategy may speed up model convergence, improve data feature learning, fine-tune the neural network’s weight and bias parameters, and reduce the loss function to the maximum degree possible. The strategy of applying an adaptive learning rate will be more favorable to model training and prediction accuracy; also, the model network will be deeper and converge faster. Therefore, the Adaptive Moment Estimation (Adam) [81] algorithm is used to minimize the loss function of the U-lossian model. The classification model is presented in Figure 3 and the hyper-parameters are given in

Table 1. Hyper-parameters of the U-lossian network.

Parameter	Value
initial learning rate	$1\times {10}^3$
min learning rate	$1\times {10}^7$
scheduler	cosine annealing with warm restarts—200 epochs
batch size	8
optimizer	AdamW

.

4. Experimental Validation

4.1. Performance Evaluation

We use the following metrics for the evaluation of the performance, which were calculated according to Ref. [82]:

Sensitivity or true positive rate (TPR) is the probability that a predicted result will be positive when the disease is present.

$$Sensitivity=a/(a+b)$$

(4)

Specificity or true negative rate (TNR) is the probability that a predicted result will be negative when the disease is not present.

$$Specificity=d/(c+d)$$

(5)

where a represents true positives (TP), participants who have the disease and positive test findings; d represents subjects who do not have the disease and the test agrees (TN); b represents individuals who do not have the disease but the test shows ‘disease’ (FP); and c represents false negatives (FN).

The positive likelihood ratio (PLR) is the ratio of the likelihood of a positive test result given the existence of the illness to the likelihood of a positive test result given the absence of the disease, i.e.,

$$PLR=TPR/FPR=Sensitivity/(1-Specificity)$$

(6)

The negative likelihood ratio (NLR) is the ratio of the likelihood of a negative test result in the presence of disease to the likelihood of a negative test result in the absence of disease, i.e.,

$$NLR=FNR/TNR=(1-Sensitivity)/Specificity$$

(7)

When screening people, predictive values are more important than sensitivity and specificity. The positive predictive value (PPV) is the probability that the disease is present when the prediction is positive.

$$PPV=a/(a+b)$$

(8)

The negative predictive value (NPV) is the probability that the disease is not present when the prediction is negative.

$$NPV=d/(c+d)$$

(9)

Accuracy is the overall probability that a voice sample is correctly classified.

$$Accuracy=Sensitivity\times Prevalence+Specificity\times (1--Prevalence)$$

(10)

The misclassification rate (MCR) is a measure that indicates the ratio of outcomes predicted incorrectly by a classification model.

$$MCR=(b+c)/(a+b+c+d)$$

(11)

Youden presented an index to assess the quality of a screening test. The index is defined using the false positive and false negative values as

$$Youden=sensitivity+specificity-1$$

(12)

The false discovery rate (FDR) is the predicted proportion of incorrect positive classifications to total positive classifications (rejections of the negative class).

$$FDR=\left(FP\right)/(FP+TP)$$

(13)

The false omission rate (FOR) measures the ratio of false negatives that are incorrectly rejected.

$$FOR=\left(FN\right)/(FN+TN)$$

(14)

Balanced accuracy (BA) is the average of the recall scores acquired for each class, that is, the general average of the recall scores per class.

$$BA=(sensitivity+specificity)/2$$

(15)

The F1-Measure is the harmonic mean of the precision and recall.

$$F_1=2\times precision\times recall/(precision+recall)$$

(16)

The G-measure (or Fowlkes–Mallows index) is a metric for measuring the confusion matrices that is used to quantify the similarity between two clusterings (clusters formed following a clustering technique), where each cluster represents a different class.

$$G=\sqrt{PPV\times TPR}$$

(17)

Matthews index is a balanced measure that yields a number between 1 and +1 as a correlation coefficient between predicted and true binary classifications. A coefficient of +1 implies a flawless forecast, a coefficient of 0 shows no better than random prediction, and a coefficient of 1 indicates absolute disagreement between prediction and observation.

$$Matthews=\sqrt{PPV\times TPR\times TNR\times NPV}--\sqrt{FDR\times FNR\times FPR\times FOR}$$

(18)

The critical success index (CSI) is a categorical prediction performance verification metric equal to the total number of correct predictions divided by the total number of predictions plus the number of misses.

$$CSI=a/(a+b+c)$$

(19)

Cohen’s kappa $\kappa$ is used to evaluate binary classifications,

$$\kappa =2\times (TP\times TN--FN\times FP)/\left[\right(TP+FP)\times (FP+TN)+(TP+FN)\times (FN+TN\left)\right]$$

(20)

The Yule coefficient Y is a measure of association between two binary variables.

$$Y=(\sqrt{a\times d}-\sqrt{b\times c})/(\sqrt{a\times d}+\sqrt{b\times c})$$

(21)

Diagnostic odds ratio (DOR) is a measure of the efficacy of a diagnostic test. It is defined as the ratio of the probabilities of a positive test if the subject has a disease to the odds of a positive test if the subject does not have the condition.

$$DOR=[sensitivity\times specificity]/\left[\right(1-sensitivity)\times (1-specificity\left)\right]$$

(22)

Discriminant Power is the degree of precision with which a set of predictor factors categorizes the outcomes.

The confidence ranges for accuracy, sensitivity, and specificity are “exact” Clopper–Pearson confidence ranges [83]. The “Log technique”, as described in Ref. [84], is used to compute confidence ranges for likelihood ratios. The standard logit confidence intervals offered by Mercaldo et al. [85] are used to calculate the confidence intervals for the prediction values.

To visualize the results, the precision-recall (PR) curve, receiver operating characteristic (ROC) curve, sensitivity-specificity plot, and alluvial plot of confusion matrix are used.

When different probability thresholds are utilized, ROC curves describe the trade-off between a predictive model’s true positive rate and false positive rate. When different probability thresholds are utilized, precision-recall (PR) curves describe the trade-off between a predictive model’s actual positive rate and positive predictive value.

When the observations in each class are balanced, ROC curves are acceptable, but precision-recall curves are appropriate for unbalanced datasets. An alluvial diagram is used to show associations between categorical variables making the confusion matrix.

4.2. Results

The classification results using the ItalianPVS dataset are presented in Figure 4 and Figure 5 (confusion matrix). The results show that the network model achieved good results with an F1-score of 0.8974, and Area Under Curve (AUC) of 0.9433.

The classification results using the Lithuanian voice dataset are presented in Figure 6 and Figure 7 (confusion matrix).

Figure 8a display the alluvial plot for the Italian dataset, while Figure 8b display the aluavian plot for the Lithuanian dataset. The classifier has achieved worse results on the Lithuanian voice dataset as it is a smaller dataset, achieving F1-score of 0.7778, and Area Under Curve (AUC) of 0.8179. Perhaps, better results could have been achieved if more voice data would be available for network, training, which is supported by the shape of PR and ROC curves, which are not perfect.

The sensitivity-specificity plots for both datasets are presented in Figure 9a,b, respectively. They show that the best performance of the classifier model is achieved, when the cutoff value is equal to $0.1$ for the ItalianPVS dataset, and $0.8$ for the Lithuanian voice dataset.

The time performance of the proposed deep learning model on both datasets is compared in Figure 10. An excellent time performance of processing one voice record in just 4 ms was demonstrated, which experimentally validated the characteristics presented in

Table 2. Size of the U-lossian model.

Parameter	Value
Total params:	11,692,525
Trainable params:	11,692,525
Non-trainable params:	0
Model size (params + buffers):	44.65 Mb
Framework and CUDA overhead:	1942.21 Mb
Total RAM usage:	1986.86 Mb
Floating Point Operations on forward:	1.70 GFLOPs
Multiply-Accumulations on forward:	850.23 MMACs
Direct memory accesses on forward:	863.85 MDMAs

. The results show that the proposed model can be used for real-time PD screening.

The performance results of both datasets are presented in

Table 3. Summary of the performance results.

Parameter	Value (Confidence Range) Italian Dataset	Value (Confidence Range) Lithuanian Dataset
Sensitivity	0.9543 (0.9283–0.9720)	0.7671 (0.6914–0.8300)
False Negative rate (FNR)	0.0457 (0.0285–0.0712)	0.2329 (0.1706–0.3080)
Specificity	0.8440 (0.8047–0.8771)	0.8193 (0.7479–0.8753)
False Positive rate (FPR)	0.1560 (0.1231–0.1951)	0.1807 (0.1255–0.2513)
Positive Likelihood Ratio (PLR)	6.1188 (5.7392–6.5236)	4.2447 (3.6150–4.9842)
Negative Likelihood Ratio (NLR)	0.0541 (0.0508–0.0577)	0.2842 (0.2421–0.3338)
Precision	0.8468 (0.8077–0.8796)	0.7887 (0.7147–0.8489)
False discovery rate (FDR)	0.1532 (0.1206–0.1921)	0.2113 (0.1518–0.2847)
Negative Predictive Value (NPV)	0.9534 (0.9272–0.9713)	0.8000 (0.7269–0.8587)
False omission rate (FOR)	0.0466 (0.0292–0.0724)	0.2000 (0.1420–0.2724)
Accuracy	0.8964 (0.8620–0.9235)	0.7949 (0.7213–0.8543)
Mis-classification Rate (MCR)	0.1036 (0.0768–0.1377)	0.2051 (0.1464–0.2780)
Balanced Accuracy (BA)	0.8992 (0.8651–0.9259)	0.7932 (0.7195–0.8528)
F1-measure	0.8974 (0.8631–0.9244)	0.7778 (0.7029–0.8393)
G-measure (Fowlkes–Mallows index)	0.8990 (0.8649–0.9258)	0.7779 (0.7029–0.8394)
Matthews index:	0.8996 (0.8656–0.9263)	0.7938 (0.7201–0.8533)
Critical success index (CSI)	0.8139 (0.7723–0.8496)	0.6364 (0.5552–0.7110)
Cohen’s Kappa	0.7935 (0.7351–0.8519)	0.5874 (0.4599–0.7148)
Yule’s coefficient	0.9825 (0.9628–0.9918)	0.8745 (0.7453–0.9404)
Critical Diagnostic Odds Ratio (DOR)	1.1326	1.2643
Discriminant Power	2.6066	1.4906

and summarized visually in Figure 11. These results show lower performance for the Lithuanian voice dataset with a higher variability (i.e., wider confidence range), which can be explained by the smaller size of the Lithuanian voice dataset.

4.3. Statistical Evaluation

We evaluated the results statistically using Bayesian credibility assessment by Ref. [86] to assess the credibility of our results considering the existing evidence. The odds ratio had to be smaller than $0.96$ so that our results could be considered credible. It specifically indicates if the new finding, when integrated with existing information, may be considered to have genuinely proved efficacy at the 95% confidence level. At the 95% confidence threshold, such a finding is deemed to be credible.

For the ItalianPVS dataset, the Critical Error Odds Ratio (EOR) is $1.6841$. Error Odds Ratio (EOR) is $3.8599$. As $EOR>COR$, the test is credible at the 95%. Critical Diagnostic Odds Ratio (DOR) is $1.1326$. Diagnostic Odds Ratio (DOR) is $113.0458$. As $DOR>COR$, the test is credible at the 95%.

For the Lithuanian voice dataset, Critical Diagnostic Odds Ratio (DOR) is $1.2643$. Diagnostic Odds Ratio (DOR) is $14.9333$—the classifier discriminates between diseased and not diseased. As $DOR>COR$, the test is credible at the 95%.

4.4. Summary of the Results

It is worth noting that the current work employed two quite distinct datasets of PD voices (Italian and Lithuanian) for analysis and algorithm building. The Italian PD voice samples were based on sustained vowel phonation, whereas the Lithuanian samples were based on phonetically balanced phrase analysis. However, because sustained vowel phonation is an artificial sort of phonation, that feature defines the same constraint in the meaning of the sustained vowel signal analysis. Connected speech, on the other hand, is more typical of everyday speaking and might be deemed more “ecologically valid” (i.e., more representative of daily speech and voice use patterns). As a result, if an objective voice evaluation is to be deemed robust and physiologically valid for screening purposes, the acoustic measurements should preferably be done utilizing both speaking patterns–sustained phonation and running speech (used in our approach).

As a result, the PD classification findings shown on speech samples from two independent datasets must be regarded with caution. The Italian dataset represented Parkinson’s disease patients with a minor incidence of advanced PD stages according to the H-Y scale, and hence more pronounced voice abnormalities. This characteristic may have improved classification accuracy, resulting in the somewhat superior PD classification results presented on the Italian dataset.

The current study, on the other hand, shows that the suggested Hybrid U-lossian deep learning network technique is successful in analyzing and classifying PD voice samples of various sorts, hence expanding PD screening options. It is vital to note that the recommended algorithm is not currently regarded a medical device or diagnostic tool. Nonetheless, the suggested technique has the potential to be applied in clinical settings as a sensitive tool for the screening of Parkinson’s disease-related voice and speech abnormalities.

5. Discussion

Despite extensive research of the potential for acoustic analysis in the classification of PD voices, the methods used, as well as parameters considered, vary immensely among studies. The standardization of study protocols as well as reproducibility studies are lacking. The acoustic speech signal carries a massive amount of information. Nevertheless, the direct link between voice production and acoustics has not been explicitly studied [87]. Moreover, the patophysiology of speech production itself is not yet fully understood in PD [88]. Speech function is well known to deteriorate with disease progression up to unintelligible levels in the latest stages. However, only a fraction of studies analyze speech in the early phase of PD. The phonetics of spoken languages differ in some aspects and are shared in others. Therefore, some acoustic parameters may have similar values and meanings in different populations [89], while others are not to be universalized directly. Acoustic studies in different languages may shed some light on the topic and the achieved results. Moro-Velasquez [90] declared his group the first to successfully implement the cross-corpora trials for AI PD detection. They used databases of Czech and two dialects of Spanish speakers and only used diadochokinetic speech task for training with one and testing with another database, as these utterances were supposed to not differ across the databases. Our approach of a cross-corpus trial was driven by the fact that Lithuanian is a language, spoken by as little as 3 million native speakers, and is largely underinvestigated. Considering the sample size, losing part of the subjects to testing was chosen not an option. The bold approach to test on other language speech samples, was encouraged by the work of Rush et al. [89] who had implemented a composite dysarthria score for testing early PD and RBD speakers of 7 different languages with comparable results and an overall AUC of 0.8. Therefore, It can be considered that PD differently affects certain phonetic groups, with those requiring the narrowest contact between the articulators (such as fricatives), being influenced the most, correlating with results shown by Ref. [90]. Still, naturally, a substantial effect on other phonetic groups cannot be overlooked as the disease has been shown to impair the articulatory sequence as a whole [90]. A comparison of the effectiveness of different deep learning approaches is offered in

Table 4. Comparison of different methods of PD voice analysis.

Method	Accuracy	Dataset (Language, Availability)
Sparse kernel transfer learning [59]	86.72%	UCI repository v.2013 (English, open on request) [91]
Linear DL network [68]	81.67%	Telemonitoring voice dataset (Unspecified, closed)
Linear DL network + mRMR feature selection [69]	99.1%	UCI repository v.2019 (English, open on request) [92]
Ensemble classifier [67]	94.12%	UCI repository v.2019 (English, open on request) [92]
Dual-side learning ensemble [60]	98.41%	LSVT voice rehabilitation dataset [93]
LSTM [70]	97.12%	UCI repository v.2013 (English, open on request) [91]
1D CNN [58]	87.76%	UCI repository v.2013 (English, open on request) [91]
ADCNN [74]	98.11%	UCI repository v.2013 (English, open on request) [91]
Bidirectional LSTM [71]	75.56%	GYENNO SCIENCE Parkinson Disease Research Center dataset (Chinese, closed)
LSTM with SSWA-based attention [72]	92.5%	Proprietary (Indian, closed)
ResNet18 [63]	91.7%	PC-GITA database (Spanish, open) [94]
Alexnet [64]	91.7%	Dataset of Department of Neurology at the Medical University of Warsaw (Polish, closed)
DenseNet161 [65]	89.75%	mPower PD dataset (English, open) [34]
CNN-ANN [61]	93.10%	Max Little dataset (English, open)
Ensemble CNN [62]	97.3%	PC-GITA database (Spanish, open) [94]
CNN with IAIF and QCP [66]	97.3%	PC-GITA database (Spanish, open) [94]
EDF-EMD [73]	92.59%	Dataset-CPPDD (Chinese, closed)
Hybrid U-lossian network (our approach)	89.64%	LSMU Lithuanian dataset (Lithuanian, open on request)

.

With the above-mentioned studies, to date there is no established universal approach to study speech impairment in PD. Speech acoustic analysis has not been included in the latest Movement disorder society (MDS) criteria for PD [95,96], nor MDS Research Criteria for Prodromal Parkinson’s Disease [97]. As was stated by Godino-Llorente in his editorial for [98], automatic systems to assess PD will benefit from new knowledge generated in the research area to develop more accurate and robust systems, as was also shown in our experimental overview.

Finally, the fact that hypokinetic dysarthria is not the integral symptom of PD, with an estimated prevalence of up to 90% of cases with rising expression in the latest stages [99], the accuracy of classification obtained on both the Italian and Lithuanian corpora is satisfactory from a medical perspective. Furthermore, our proposed approach reaches an accuracy not markedly lower in the cross corpora–cross linguistic trial, indicating the generalizability of the method to different populations and recording conditions.

6. Conclusions

In this paper, we introduced a method for speech signal processing to illustrate the utility of deep learning-driven voice analysis as a screening tool for Parkinson’s disease (PD). We investigated whether voice recordings might be used to provide a simple, low-cost approach to the assessment and screening of Parkinson’s disease, using deep learning to predict and evaluate expert scores. As a consequence, we created a Deep U-lossian model, a modified Hybrid Mask U-Net architecture with an adaptive custom loss function, for PD evaluation and detection in order to achieve a better balance of accuracy and recall in processed speech. The classification accuracy on two speech corpora (ItalianPVS and Lithuanian voice dataset), 0.8964 and 0.7949, respectively, is satisfactory from a medical aspect, confirming the proposed model’s excellent generalizability. Furthermore, the proposed model has demonstrated exceptional performance in terms of speed, ensuring real-time performance.

The method has a strong potential to be employed in clinical settings as a sensitivity test for the screening of Parkinson’s disease-related voice and speech alterations and as a result, the future work of the study will be a proper clinical validation in the hospital environment, with the idea being in line to create a system that could analyze a large number of telephone voice calls (patients calling the Department of Otorhinolaryngology for remote consultation, allowing doctors to acquire live data on a patient while he is still speaking).