Next Article in Journal
School-Based TGfU Volleyball Intervention Improves Physical Fitness and Body Composition in Primary School Students: A Cluster-Randomized Trial
Previous Article in Journal
How Tasmanian Emergency Departments ‘Choose Wisely’ When Investigating Suspected Pulmonary Embolism
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Parkinson’s Disease Detection from Voice Recordings Using Associative Memories

by
Irving Luna-Ortiz
1,
Mario Aldape-Pérez
1,*,
Abril Valeria Uriarte-Arcia
1,
Alejandro Rodríguez-Molina
2,
Antonio Alarcón-Paredes
3 and
Elías Ventura-Molina
1
1
Instituto Politécnico Nacional, Center for Computing Innovation and Technological Development (CIDETEC), Computational Intelligence Laboratory (CIL), Mexico City 07700, Mexico
2
Tecnológico Nacional de México/IT de Tlalnepantla, Research and Postgraduate Division, Tlalnepantla de Baz 54070, Mexico
3
Instituto Politécnico Nacional, Center for Computing Research (CIC), Computational Intelligence Laboratory (CIL), Mexico City 07700, Mexico
*
Author to whom correspondence should be addressed.
Healthcare 2023, 11(11), 1601; https://doi.org/10.3390/healthcare11111601
Submission received: 24 March 2023 / Revised: 21 May 2023 / Accepted: 26 May 2023 / Published: 30 May 2023
(This article belongs to the Section Artificial Intelligence in Medicine)

Abstract

:
Parkinson’s disease (PD) is a neurological condition that is chronic and worsens over time, which presents a challenging diagnosis. An accurate diagnosis is required to recognize PD patients from healthy individuals. Diagnosing PD at early stages can reduce the severity of this disorder and improve the patient’s living conditions. Algorithms based on associative memory (AM) have been applied in PD diagnosis using voice samples of patients with this health condition. Even though AM models have achieved competitive results in PD classification, they do not have any embedded component in the AM model that can identify and remove irrelevant features, which would consequently improve the classification performance. In this paper, we present an improvement to the smallest normalized difference associative memory (SNDAM) algorithm by means of a learning reinforcement phase that improves classification performance of SNDAM when it is applied to PD diagnosis. For the experimental phase, two datasets that have been widely applied for PD diagnosis were used. Both datasets were gathered from voice samples from healthy people and from patients who suffer from this condition at an early stage of PD. These datasets are publicly accessible in the UCI Machine Learning Repository. The efficiency of the ISNDAM model was contrasted with that of seventy other models implemented in the WEKA workbench and was compared to the performance of previous studies. A statistical significance analysis was performed to verify that the performance differences between the compared models were statistically significant. The experimental findings allow us to affirm that the proposed improvement in the SNDAM algorithm, called ISNDAM, effectively increases the classification performance compared against well-known algorithms. ISNDAM achieves a classification accuracy of 99.48%, followed by ANN Levenberg–Marquardt with 95.89% and SVM RBF kernel with 88.21%, using Dataset 1. ISNDAM achieves a classification accuracy of 99.66%, followed by SVM IMF1 with 96.54% and RF IMF1 with 94.89%, using Dataset 2. The experimental findings show that ISNDAM achieves competitive performance on both datasets and that statistical significance tests confirm that ISNDAM delivers classification performance equivalent to that of models published in previous studies.

1. Introduction

Parkinson’s disease (PD) is a neurological condition that is chronic and worsens over time. It affects between 2 and 3 percent of the world’s population who are over the age of 65 [1,2], which presents a challenging diagnosis [3,4]. An accurate diagnosis is required to differentiate healthy individuals from PD patients. Studies have shown that PD can be diagnosed at early stages and that an early diagnosis of this neurodegenerative condition can lessen the impact of PD and improve patient’s living conditions [5,6]. At an early stage of this disease, a patient’s face may show no expression, and speech may become incomprehensible [7]. The first symptom may be a barely noticeable rhythmic shaking in only one hand. The symptoms of PD worsen as the condition progresses. Over the past few years, the use of computational intelligence (CI) techniques to diagnose PD has experienced rapid growth [8].
Almeida et al. [9] used feature extraction techniques on voice signals and machine learning methods for PD detection. They concluded that phonation analysis is a more efficient alternative to detect this progressive neurodegenerative disorder than speech tasks.
Polat and Nour [10] analyzed informative features from voice signals using a data-preprocessing method for PD recognition. They concluded that the proposed approach could be applied as a data-preprocessing method on computational intelligence algorithms for PD recognition, using informative features.
Pereira et al. [11] presented a methodical study of current enabling technologies that can be used to diagnose PD as well as to ameliorate the life quality of individuals with this condition. The authors also included a closer look into innovative and future approaches to achieve this goal.
Sakar et al. [12] presented a methodical study of voice signal processing for PD detection. The authors extracted features from voice signals of PD patients using a tunable wavelet transform approach (TQWT). The authors concluded that the performance of TQWT is superior to other cutting-edge speech signal processing approaches that are utilized in PD classification.
Pahuja and Nagabhushan [13] presented an analysis of the currently available supervised machine learning methods applied to Parkinson’s Disease recognition. They compared three types of supervised machine learning approaches: SVM, K-NN, and ANN. They concluded that a feed-forward ANN combined with the Levenberg–Marquardt method improves its performance and allows the highest classification rates to be achieved when applied to the voice dataset.
Alzubaidi et al. [14] investigated and provided a summary of neural networks applied to Parkinson’s disease diagnosis. The authors analyzed 91 studies to identify the function that a feed-forward artificial neural network plays in the process of PD diagnosis. The authors concluded that the early detection of PD through voice sample analysis occurs in a significant proportion of the 91 works analyzed. The authors also argue that ANNs are a viable option for performing this task.
Sechidis et al. [15] introduced a cross-domain transfer-learning model for speech emotion recognition and applied it to a Parkinsonian speech corpus. They evaluated distinctive voice patterns of patients with PD and concluded that there is a relationship between PD and emotional scores, where patients with this disease are often perceived as expressing an emotion of sadness.
Ma et al. [16] proposed an ensemble model for Parkinson speech recognition based on a dual-side learning ensemble that applies sample selection to remove useless samples and deep feature transformation to generate high-quality features for PD recognition. They concluded that the weighted fusion mechanism that merges classification algorithms is the crucial component that helps to achieve higher performance than the state-of-art relevant algorithms.
Haq et al. [17] presented a methodical review of the literature of deep learning methods frequently used in Parkinson detection. They explored current reviews with their advantages and limitations. They also presented numerical results comparison for current relevant methods.
Zhang et al. [18] introduced a feature called EDF-EMD and assessed its effectiveness on two datasets. The authors analyzed the frequency behavior of voice signals and concluded that the high-frequency segment of the speech signal provides relevant components related to the diagnosis and recognition of PD. They concluded that classification performance results are outstanding, using feature extraction from IMF1. The experiments suggest that the EDF-EMD feature can be utilized to recognize this medical condition effectively.
Mei et al. [19] presented a thorough analysis of the literature to study and enhance global understanding of supervised machine learning algorithms of practical application to Parkinson’s disorder recognition. The authors reviewed datasets, mathematical methods, data types, machine learning algorithms, data sources and the related results of 209 studies. The authors concluded that there is a strong potential of machine learning approaches in clinical decision making, which will lead to a more informed and methodical diagnosis of PD.
Ngo et al. [20] presented a methodical review of the literature of the last decade to investigate signal features and Parkinson’s disease recognition methods. They reported on the signal recording protocols, the different datasets used by supervised learning algorithms and signal analysis methods frequently used in Parkinson speech recognition. Based on classification and correlation results, they concluded that voice signal features as well as information extracted from speech recordings have a potential utility for Parkinson’s disorder recognition and assessment.
Quan et al. [21] presented a convolutional neural network (CNN) approach for Parkinson’s disorder recognition using voice samples. This approach is based on the successive application of CNN on voice samples. First, a 2D CNN is applied to obtain time series dynamic features, then a 1D CNN is used to obtain the dependencies between these time series. The authors concluded that higher frequencies of the Mel-spectrogram are less significant than low-frequency regions.
Madruga et al. [22] analyzed the impact of the variability that exists between different voice signal acquisition devices and the negative effects on voice signal recordings specifically oriented for Parkinson’s recognition. They proposed a methodology to increase the robustness against the variability between different devices. They concluded that this approach improves the capacity of supervised learning models to detect PD patients from healthy individuals, even using different voice signal acquisition devices.
Coelho et al. [23] analyzed the performance effects of Hjorth parameters extracted from electroencephalogram (EEG) signals for Parkinson’s disease detection. They evaluated the differences between healthy individuals’ and PD patients’ brain’s cerebral cortex. They concluded that there are substantial distinctions between healthy individuals’ and Parkinson’s disease patients’ brain’s lobes; thus, they can be used as Parkinson’s disease biomarkers.
Dixit et al. [24] presented a comprehensive overview of many AI-based machine learning and deep learning approaches to diagnose PD, as well as their impact on the development of new research areas. This review additionally provides an in-depth look at the current state of PD diagnostics as well as forthcoming applications of data-driven AI technology.
As evidenced by recent research, PD diagnosis continues to be a current topic of interest, and furthermore, its detection through voice samples using computational intelligence techniques is still a major challenge.
This paper presents an improvement to the smallest normalized difference associative memory (SNDAM) algorithm and shows performance measurement results achieved by this new proposal, called improved smallest normalized difference associative memory (ISNDAM), when it is applied to PD diagnosis. The experimental findings confirm that the proposed improvement in the SNDAM algorithm, called ISNDAM, is efficient and effectively increases the classification performance compared against well-known algorithms.

2. Materials and Methods

The learning matrix concept was conceived more than six decades ago [25,26,27], and since then, major research groups worldwide have shifted their focus to this promising research area [28,29,30,31,32]. The concept of an array that stores the relationship between patterns has evolved into what is known today as associative memories (AM).
Associative memories are used to simulate the basic learning activity of the human brain. Thus, this learning paradigm associates input and output patterns.
Definition 1.
Let x μ be the μ-th input pattern and y μ be the μ-th output pattern, then the learning set is represented as follows:
{ ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . . , ( x μ , y μ ) , . . . , ( x p , y p ) }
In this way, an associative memory M is created through the association of output patterns with input patterns. This stage is known as the training phase.
x μ M y μ
Once associative memory M has been built, a test pattern x ˜ will be presented to associative memory M to obtain an output pattern y ϑ that will indicate the class label that corresponds to test pattern x ˜ . This stage is known as the operation phase.
x ˜ M y ϑ

2.1. Smallest Normalized Difference Associative Memory

This model was proposed in [33] to overcome the shortcomings of ABAM introduced in [34,35]. The ABAM model proposed two operators for its functionality: α and β . These operators were conceived in a binary space. Thus, there are certain drawbacks to this, such as the fact that all patterns have to be binary encoded [36], which increases the processing complexity but does not necessarily increase the recall capacity of the ABAM model [37].
Briefly stated, the main contribution of the SNDAM model is that it extended α R and β R operators to the R domain, which reduced the complexity of the binary encoding of the fundamental set of patterns as well as the time consumed for model training, while the robustness to deal with subtractive, additive or combined alterations in input patterns was preserved.
Definition 2.
The following describes the α R operation:
α R ( w , v ) = w v + 1
Definition 3.
The following describes the β R operator:
β R ( w , q ) = w i f w q w + q 1 i f w > q
The second contribution of this model is to use the smallest normalized difference to evaluate the similarity between an unknown recalled pattern and another one present in the training phase. This removes the ambiguity in the class label assignment; as a consequence, classification performance is improved.
The central core of SNDAM is based on operations between vectors to produce pattern associations, stored as a two-dimensional array (learning matrix) [33]. One crucial advantage of the SNDAM model is that it can fully recover the learning set when the learning phase is carried out in an autoassociative mode. Another aspect that distinguishes the SNDAM model is the robustness against additive or subtractive noise in training patterns. This turns out to be an advantage since an unknown pattern x ω can be considered as a training pattern x μ altered by noise (additive or subtractive).
Definition 4.
Let M be the SNDAM M A X type, which is obtained as follows:
M = μ = 1 p α R x μ , x μ
Definition 5.
Let W be the SNDAM min type, which is obtained as follows:
W = μ = 1 p α R x μ , x μ
Definition 6.
Let A = R , let ⋁ be the maximum operator, and let x M A X A n be an n-dimensional vector that contains the highest value of each of the n components of all the p instances in the learning set:
x i M A X = μ = 1 p x i μ
Definition 7.
Let y ϑ be the recalled pattern from a given input instance x ˜ that was not present in the learning set, using M :
y ϑ = β R ( M , x ˜ )
Definition 8.
Let y ϑ be the recalled pattern from a given input instance x ˜ that was not present in the learning set, using W :
y ϑ = β R ( W , x ˜ )
Definition 9.
Let δ ϑ μ represent the magnitude of the similarity between each of the p instances x μ in the learning set and the recalled pattern y ϑ :
δ ϑ μ = i = 1 n y i ϑ x i μ x i m a x

2.2. Improved Smallest Normalized Difference Associative Memory

Even though the SNDAM model [33] overcame some of the shortcomings of ABAM [34,35], in this section, an improvement to the SNDAM model is proposed. This model is known as improved smallest normalized difference associative memory (ISNDAM), and it is based on the incorporation of a relevance identification phase, which is executed before the testing phase with unknown patterns, as is shown in Figure 1.

2.2.1. ISNDAM Algorithm

The operation of ISNDAM requires three phases, namely: training, relevance identification and testing. As a result of the first phase, a learning matrix is obtained using those patterns present in the learning set. After that, an iterative search process is carried out in order to identify those characteristics that provide more information for the purposes of classification, as is shown in Figure 1. Those features that are identified as relevant are binary encoded and are stored in a reinforcement vector, as is shown in Figure 2. Finally, the smallest normalized difference is applied to test the unknown patterns; thus, the performance indicators of the proposed model are obtained.

2.2.2. Training Phase

At first, a learning matrix M is built by associating all p instances in the learning set. After classifying these instances, a performance measure of the learning set is obtained. This first indicator sets the classification performance lower bound. Algorithm 1 describes the steps required to implement this phase.
Algorithm 1: Training Phase
Data: p , n , { ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . . , ( x p , y p ) }
Result: M , y ϑ
Initialization:
Step 1: Generate p matrices, one for each fundamental pattern association ( x μ , x μ ) , using Equation (6).
Step 2: Compare the previously generated p matrices, and keep the maximum value (⋁). The result is a learning matrix M that contains the maximum value of all p generated matrices.
Step 3: Compare the n components of all the p instances in the learning set, and keep the maximum value (⋁), using Equation (8). The result is an n-dimensional vector that contains the highest value of each of the n components of all the the p instances x μ in the learning set.
Step 4: Recall all the p instances x μ in the learning set, using Equation (9).
Step 5: Obtain the magnitude of the similarity δ ϑ μ between each of the p instances x μ in the learning set and the recalled pattern y ϑ , using Equation (11).
Step 6: Identify μ value, using the smallest normalized similarity value δ ϑ μ .
Step 7: Assign x μ class label to the recalled pattern y ϑ .
End of Training Phase

2.2.3. Relevance Identification Phase

The relevance identification phase consists of an iterative search process to identify and to select those features that improve ISNDAM classification performance. Algorithm 2 describes the steps required to implement this phase. The selected subset of features retains relevant information that sufficiently describes the problem to be effectively solved. This optimal subset of features is represented by a binary encoded n-dimensional vector. Those features that improve classifier performance are coded with a value of 1, while those features that negatively affect the performance of the classifier are coded with a value of 0, as is shown in Figure 2.
Definition 10.
Let A = { 0 , 1 } and let e r be a binary coded n-dimensional column vector that represents the identification of those relevant features that improve the performance of the model, with the r value represented in binary form:
e r = e 1 r e 2 r e n r A n
Algorithm 2: Relevance Identification Phase
Healthcare 11 01601 i002

2.2.4. Testing Phase

In this third phase, those features that benefit the performance of ISNDAM have already been identified. The relevance identification vector e r ignores those features that negatively affect the classification performance; thus, the testing phase is executed considering only relevant features. In this phase, test instances are recalled, and the smallest normalized difference value obtained is computed using all training patterns to assign a class label. Algorithm 3 describes the steps required to implement this phase.
Algorithm 3: Testing Phase
Data: M , p , n , e r , x ˜
Result: y ϑ
Initialization:
Step 1: Use memory M , previously generated in the training phase, to recall y ϑ from a given input instance x ˜ that was not present in the learning set, using Equation (9).
Step 2: Obtain the magnitude of the similarity δ ϑ μ between each of the p instances x μ in the learning set and the recalled pattern y ϑ , using Equation (11).
Step 3: Identify μ value, using the smallest normalized similarity value δ ϑ μ .
Step 4: Assign x μ class label to the recalled pattern y ϑ .
End of Testing Phase

2.2.5. ISNDAM Numerical Example

In order to clarify the implementation of the ISNDAM model, the operation of the proposed model is illustrated by way of an example. Numeric values in our example correspond to patterns taken from the widely known iris plants dataset [38]. Each instance has four features, and there are an equal number of instances in each class, specifically three instances of each class.
Example. 
Given six instances in the learning set, each described by n = 4 features, where x 1 , x 2 , x 3 belong to class 1 and x 4 , x 5 , x 6 belong to class 2, recall all instances and obtain a classification performance measure.
x 1 = 5.1 3.5 1.4 0.2 x 2 = 4.9 3.0 1.4 0.2 x 3 = 4.7 3.2 1.3 0.2
x 4 = 7.0 3.2 4.7 1.4 x 5 = 6.4 3.2 4.5 1.5 x 6 = 6.9 3.1 4.9 1.5
Generate p matrices, one for each fundamental pattern association x μ , x μ , using Equation (6). Thus, each matrix element can be obtained using the following expression:
m i j = α R ( x i μ , x j μ )
Once all p matrices have been obtained, the maximum operator ⋁ is applied according to Equation (6). This means that each m i j component of each matrix must be compared and the maximum value is stored, so that finally a single matrix containing maximum values is obtained. In our example, the SNDAM autoassociative MAX-type obtained is as follows:
M = 1 4.8 4.7 6.6 0.5 1 3.1 4.3 0.9 2.8 1 4.4 3.5 0.6 0.1 1
Use memory M , previously generated in the training phase, to recall y ϑ from a given input instance x ˜ that was not present in the learning set, using Equation (9). Then, obtain the magnitude of the similarity δ ϑ μ between each of the p instances x μ in the learning set and the recalled pattern y ϑ , using Equation (11). Thus, each component of the recalled pattern y ϑ is obtained using the following expression:
y i ϑ = j = 1 n β R ( m i j , x ˜ j )
Firstly, the M matrix is taken, and the Beta operator β R is applied to the first instance to be tested. In our case, we take the first instance of the numerical example x 1
y ϑ = j = 1 n β R 1.0 4.8 4.7 6.6 0.5 1.0 3.1 4.3 0.9 2.8 1.0 4.4 3.5 0.6 0.1 1.0 5.1 3.5 1.4 0.2
Subsequently, the minimum operator ⋀ is applied, and as a result, we obtain vector y ϑ .
y ϑ = 5.1 7.3 5.1 5.8 3.6 3.5 3.5 3.5 3.2 5.3 1.4 3.6 0.6 1.9 0.3 0.2 = 5.1 3.5 1.4 0.2 y ϑ = 5.1 3.5 1.4 0.2
Once vector y ϑ has been recalled, what follows is to obtain the magnitude of the similarity δ ϑ μ between each of the p instances x μ in the learning set and the recalled pattern y ϑ , using Equation (11)
For this example, y ϑ will be compared with x 1 , x 2 , , x 6   and the smallest normalized difference is obtained, according to (11).
δ 11 8.41 E - 18 δ 12 0.30 δ 13 0.48 δ 14 2.72 δ 15 2.09 δ 16 2.65
As can be seen, the smallest normalized difference δ ϑ μ between y ϑ and each of the patterns present in the training phase occurs in the first position δ 11 . This result is intuitive because the recalled values of y ϑ are present in the x 1 pattern. Therefore, since y ϑ has a greater similarity with x 1 , the class label of x 1 is applied to y ϑ . This implies that y ϑ belongs to class 1. It can be verified that all patterns x 1 , x 2 , , x 6 that are present in the training phase are correctly recalled, that is, recalled patterns y 1 , y 2 , y 3 belong to class 1 and y 4 , y 5 , y 6 belong to class 2.
What follows is to test the ISNDAM model with unknown patterns. At this stage, we are interested in obtaining the class label that corresponds to test pattern x ˜ , which is achieved by measuring the similarity between each one of those patterns that were present in the training phase and the recalled pattern y ˜ ϑ .
The test pattern x ˜ is the following:
x ˜ = 6.8 3.2 4.7 1.5
Firstly, the M matrix is taken, and the Beta operator β R is applied to the unknown pattern x ˜ to be tested.
y ˜ ϑ = j = 1 n β R 1 4.8 4.7 6.6 0.5 1 3.1 4.3 0.9 2.8 1 4.4 3.5 0.6 0.1 1 6.8 3.2 4.7 1.5
Subsequently, the minimum operator ⋀ is applied, and as a result, we obtain vector y ˜ ϑ .
y ˜ ϑ = 6.8 7.0 8.4 7.1 5.3 3.2 6.8 4.8 4.9 5.0 4.7 4.9 2.3 1.6 3.6 1.5 = 6.8 3.2 4.7 1.5 y ˜ ϑ = 6.8 3.2 4.7 1.5
Once vector y ˜ ϑ has been recalled, what follows is to obtain the smallest normalized difference δ ϑ μ between each pattern in the learning set and the recalled pattern y ˜ ϑ . For these purposes, y ˜ ϑ will be compared with x 1 , x 2 , , x 6 , and the smallest normalized difference will be obtained, according to Equation (11).
δ 11 2.49 δ 12 2.79 δ 13 2.97 δ 14 0.23 δ 15 0.39 δ 16 0.16
As can be seen, the smallest normalized difference δ ϑ μ between y ˜ ϑ and each of the patterns present in the training phase occurs in the sixth position δ 16 . Therefore, since y ˜ ϑ has a greater similarity with x 6 , the class label of x 6 is applied to y ˜ ϑ . This implies that y ˜ ϑ belongs to class 2.
In summary, the ISNDAM model takes advantage of the properties of the learning matrices trained in auto-associative mode and uses the smallest normalized difference δ ϑ μ to measure the similarity between those patterns present in the training phase and some unknown pattern y ˜ ϑ .
  • One of the contributions of the ISNDAM model is the simplification of the Beta operator β R to a single case, which eliminates ambiguity in class assignment and consequently increases classification performance.
  • The second contribution of this model is the relevance identification phase, which identifies all the relevant characteristics for classification purposes through a wrapper-based feature selection approach applied to SNDAM, as is shown in Figure 1.

2.3. Datasets

In this section, information about the datasets used for the validation of the performance measurements of ISNDAM model is provided. Table 1 summarizes the most relevant characteristics of them. More information about these datasets can be found in public data repositories [39,40].
  • The first dataset was created at Oxford University to differentiate PD patients from healthy individuals. It was generated from voice signal analysis measurements from thirty-one individuals, where twenty-three have PD. This dataset consists of 195 instances with 23 attributes. More details on how the recordings were gathered as well as the feature extraction process can be found at [41].
  • The second dataset was created at Istanbul University, from voice recordings of forty individuals, where twenty of them are healthy individuals and the remaining twenty patients have PD. This dataset consists of 1040 instances with 27 attributes. More details on how the recordings were gathered as well as the feature extraction process can be found at [12].

2.4. Performance Metrics

In this section, a succinct description of each performance metric is presented. First, let P be the number of individuals who have a certain condition, let N be the number of individuals who do not have such condition, let T P be the true positive value that refers to a test outcome that accurately indicates the existence of a particular condition in a patient, let F P be the false positive value that refers to a test outcome that wrongly indicates the existence of a particular condition in a patient, let T N be the true negative value that refers to a test outcome that accurately indicates the non-existence of a certain condition in a patient, and let F N be the false negative value that refers to a test outcome that wrongly indicates the non-existence of a certain condition in a patient.
A confusion matrix contrasts predicted and actual classes to visually show the performance of a model [42]. A particular case of confusion matrices appears when there is a problem where there are only two classes (negative and positive). This is exemplified as follows:
Healthcare 11 01601 i001
  • Sensitivity represents a test’s capacity to accurately identify all individuals who have a certain condition, also known as recall:
    S e n s i t i v i t y = T P R = T P T P + F N
  • Accuracy refers to a model’s performance. It is computed as the proportion of tests that were properly predicted by all of the predictions:
    A c c u r a c y = T N + T P N + P
    A c c u r a c y = T N + T P T P + F P + F N + T N
  • Specificity indicates a test’s capacity to correctly detect every individual who does not have a certain condition:
    S p e c i f i c i t y = T N F P + T N
  • False positive rate (FPR) indicates a test’s capacity to incorrectly detect healthy individuals who do not have a certain condition:
    F P R = F P F P + T N = 1 T N R
  • Precision refers to the reliability of a model in predicting a positive test result. It represents the proportion of tests that were accurately predicted as positive to the total number of tests that were forecast as positive:
    P r e c i s i o n = T P F P + T P
  • Area under the ROC curve (AUC) represents how well a binary classification algorithm is able to identify the difference between two classes [43]:
    A U C = 1 + T P R F P R 2 = T P R + T N R 2
    A U C = T P R + T N R 2 = S e n s i t i v i t y + S p e c i f i c i t y 2
  • Geometric mean (G-Mean) estimates the balance of classification performance between the minority and majority classes:
    G - M e a n = S p e c i f i c i t y × S e n s i t i v i t y
  • F1-score is determined by finding the harmonic mean of the assessments for sensitivity and precision. It represents sensitivity and precision into a single statistic in a symmetrical form:
    F 1 - S c o r e = 2 × S e n s i t i v i t y × P r e c i s i o n S e n s i t i v i t y + P r e c i s i o n

2.5. Statistical Hypothesis Tests

In statistical hypothesis testing, the objective is to provide evidence that the performance metric findings are representative of the overall behavior of classifiers. In addition to performance assessments obtained from the metrics described in Section 2.4, there are two additional aspects to consider: the first is whether the reported findings can be attributed to the properties of the classifiers or whether they could have occurred by chance, and the second is whether the apparent superiority in performance of one algorithm over another is not due to random selection of training or testing samples. To address these two factors, a statistical significance analysis is required [33].
As suggested in [44,45], the two-matched-samples t test can be useful to find whether the difference between two means is meaningful [46], which is applicable in the case of two independent samples [47]. Hypothesis testing consists of assuming two hypotheses: an alternative hypothesis, H 1 , and a null hypothesis, H 0 , which is generally the opposite of what we want to prove. Rejecting the null hypothesis implies, based on evidence, that the alternative hypothesis is accepted, which gives us confidence in believing that the observations were not by chance. Algorithm 4 outlines the steps necessary to conduct a statistical significance test.
Definition 11.
Let H 0 be a null hypothesis; it is assumed to be true until evidence indicates otherwise and particularly states: classifiers A and B have the same performance.
Algorithm 4: Statistical Significance Test
Healthcare 11 01601 i003

3. Results

The performance metrics, briefly described in Section 2.4, were computed for each compared algorithm using two datasets: Dataset 1 [41] and Dataset 2 [12]. The comparison of the obtained results is divided into two sections.
  • First: The efficiency of ISNDAM was evaluated and compared to the efficiency of seventy different models that are included in the WEKA workbench [48,49]. All experiments were performed using 5 × 2 cross-validation, as suggested in [50,51]. Additionally, a statistical significance test was conducted to verify if there existed statistically significant differences in the performance of each compared algorithm included in the WEKA workbench. Performance outcomes using Dataset 1 are shown in Table 2, while Table 3 shows the results of the statistical significance analysis for Dataset 1. Similarly, performance outcomes using Dataset 2 are shown in Table 4, while the statistical significance analysis for Dataset 2 is presented in Table 5.
  • Second: The performance of ISNDAM was compared to that of previous studies [13,19,24,52] using Dataset 1, as well as to that of previous studies [11,12,18] using Dataset 2. The performance results of ISNDAM compared to that of previous studies [13,24] using Dataset 1 are shown in Table 6. In the same way, the performance results of ISNDAM compared to that of previous studies [19,52] using Dataset 1 are shown in Table 7. Similarly, the performance results of ISNDAM compared to that of previous studies [11,12,18] using Dataset 2 are shown in Table 8.
As is shown in Table 2 and Table 4, several performance metrics, briefly described in Section 2.4, were used for comparison purposes: accuracy, G-Mean, specificity, F1-score, precision, AUC and sensitivity. Statistical significance test results for the paired measurements between the ISNDAM classification accuracy results and the performance outcomes of all of the algorithms compared are presented in Table 3 and Table 5. In accordance with Table 2, it is clear that ISNDAM achieves the best performance, followed by SNDAM and IBk, using Dataset 1. The first two models belong to the associative model-based classifier family, while IBk belongs to the lazy classifiers family. The default number of nearest neighbors used by IBk is (k = 1), while the default distance function is the Euclidean. As suggested in [44], the two-matched-samples t test can be useful in finding if the difference between two means is meaningful [46]. Table 3 shows the statistical significance test results using Dataset 1 for the paired measurements between the ISNDAM classification accuracy and the classification assessment results of each compared algorithm, with a confidence interval percentage of 95%, where p < 0.05 establishes the statistical significance threshold. Given the conditions of the two-matched-samples t test, classification accuracy results pairwise comparisons have to be made [45]. Therefore, we have seven p value measurements to be evaluated. Thus, a p value smaller than the statistical significance threshold implies that less evidence exists to support the null hypothesis. In each pairwise comparison of Table 3, it can be observed that the alternative hypothesis is accepted. This allows us to affirm that there exists a statistically significant difference between ISNDAM classification accuracy and the performance achieved by all the other compared models. This implies that even when performance differences are small, they are not due to random selection of training or testing samples; that is, the performance of the ISNDAM model is superior to the performance achieved by the other models that appear in Table 2 using Dataset 1 and 5 × 2 cross-validation.
Following the same analysis scheme, according to Table 4, it is clear that ISNDAM achieves the best performance, followed by SNDAM and random forest, using Dataset 2. The first two models belong to the associative model-based classifier family, while random forest belongs to the decision tree classifiers family. Table 5 shows the statistical significance test results using Dataset 2 for the paired measurements between the ISNDAM classification accuracy and the classification assessment results of each compared algorithm, with a confidence interval percentage of 95%, where p < 0.05 establishes the statistical significance threshold. We have seven p value measurements to be evaluated. In each pairwise comparison of Table 5, it can be observed that the alternative hypothesis is accepted. This allows us to affirm that there exists a statistically significant difference between ISNDAM classification accuracy and the performance achieved by all the other compared models. This implies that the performance of the ISNDAM model is superior to the performance achieved by the other models that appear in Table 4 using Dataset 2 and 5 × 2 cross-validation.
With the purpose of expanding the experimental outcomes interpretation, the performance of ISNDAM was compared to that of previous studies [13,19,24,52] using Dataset 1, as well as to that of previous studies [11,12,18] using Dataset 2.
As it is shown in Table 6 the performance of ISNDAM was compared to that of previous studies [13,24]. It is clear that ISNDAM achieves the best performance with a classification accuracy of 99.48%, followed by ANN Levenberg–Marquardt with 95.89% and SVM RBF kernel with 88.21%, using Dataset 1.
Similarly, Table 7 shows the performance of ISNDAM compared to that of previous studies [19,52]. It is clear that ISNDAM achieves the best performance with a classification accuracy of 99.48% using the raw features of Dataset 1. However, if the weighted features approach [52] is applied to Dataset 1, the highest classification accuracy is achieved by LS-SVM, PNN and GRNN.
In the same way, Table 8 shows the classification accuracy of ISNDAM compared to that of previous studies [11,12,18]. It is clear that ISNDAM achieves the best performance with a classification accuracy of 99.66%, followed by SVM IMF1 with 96.54% and RF IMF1 with 94.89%, using Dataset 2.

4. Discussion

One of the primary goals that this study intends to accomplish focuses on the development of a machine learning model that assists the Parkinson’s disease detection process with competitive performance. To achieve this, it was necessary to show, through frequently used performance metrics, that a slight modification to the SNDAM model can improve PD detection from voice samples.
Another objective was to find whether the performance of the ISNDAM model differed significantly from that of other machine learning models. The fulfillment of this objective was accomplished using two computational platforms, namely: the WEKA workbench and IBM SPSS software platform. It was possible to verify that the proposed model presents a superior classification performance when compared to other machine learning models implemented in the WEKA workbench. This statement is based on a statistical significance analysis, where in all pairwise comparisons, the null hypothesis is rejected.
Another objective was to compare the performance of the ISNDAM model with classification models published in previous studies. For such purposes and in order to conduct a coherent comparison, the performance analysis was restricted to classification algorithms that used the same datasets and cross-validation schemes; therefore, classification performance of ISNDAM was compared to that of previous studies [13,19,24,52] using Dataset 1, as well as to that of previous studies [11,12,18] using Dataset 2.
While using Dataset 1, the ISNDAM model performance was superior in the following studies: Pahuja and Nagabhushan [13] and Dixit et al. [24].
Using raw features of Dataset 1, the performance of the ISNDAM model was superior in all the cases that appear in the following studies: Hariharan et al. [52] and Mei et al. [19]. However, using weighted features of Dataset 1, the highest classification accuracy was achieved by LS-SVM, PNN and GRNN.
While using Dataset 2, the ISNDAM model performance was superior in each of the following studies: Pereira et al. [11], Sakar et al. [12] and Zhang et al. [18].
There are several reasons why the proposed model offers competitive experimental performance. The first is because of its great learning capacity, specifically because it has been trained in autoassociative mode. The second is because of its high tolerance for noise in test patterns. The third is the ability of the proposed model to retrieve patterns when the associative memory has been trained in autoassociative mode. The last and perhaps the most important is the use of the smallest normalized distance, which helps to retrieve the class label of a pattern that was present in the training phase and assign it to an unknown (test) pattern that has the highest similarity with respect to some instance that was present in the training phase. It basically assigns a class label considering the maximum similarity between the unknown instance and the training instances.

5. Conclusions

This paper introduces an efficient method for performing Parkinson’s disease detection. This model is known as improved smallest normalized difference associative memory (ISNDAM) algorithm. It is the outcome of an improvement to the original SNDAM model. A feature selection stage was added to the original SNDAM model, which enhances the classification performance.
After performing an analysis on the outcomes of the experimental phase, it can be affirmed that the ISNDAM algorithm is an efficient and effective option for detecting Parkinson’s disease from voice samples.

Author Contributions

Conceptualization, I.L.-O., M.A.-P. and A.V.U.-A.; methodology, M.A.-P., A.V.U.-A. and A.R.-M.; software, A.A.-P. and E.V.-M.; validation, M.A.-P., A.A.-P. and E.V.-M.; formal analysis, M.A.-P. and A.R.-M.; investigation, I.L.-O. and A.V.U.-A.; resources, M.A.-P.; data curation, I.L.-O. and A.V.U.-A.; writing—original draft preparation, I.L.-O., M.A.-P. and A.V.U.-A.; writing—review and editing, A.R.-M., A.A.-P. and E.V.-M.; visualization, I.L.-O.; supervision, A.V.U.-A. and E.V.-M.; project administration, M.A.-P.; funding acquisition, M.A.-P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors of the present paper would like to thank the following institutions for their economical support and facilities provided to develop this work: the Secretaría de Investigación y Posgrado (SIP), the Comisión de Operación y Fomento de Actividades Académicas (COFAA), the Instituto Politécnico Nacional through project no. 20230433, the Dirección de Posgrado, Investigación e Innovación of the Instituto Tecnológico Nacional de México (TecNM) through project no. 13585.22-P and no. 14301.22-P and from the Sistema Nacional de Investigadores (SNI) of the Consejo Nacional de Humanidades Ciencias y Tecnologías (CONAHCYT), México.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Poewe, W.; Seppi, K.; Tanner, C.M.; Halliday, G.M.; Brundin, P.; Volkmann, J.; Schrag, A.E.; Lang, A.E. Parkinson’s disease. Nat. Rev. Dis. Prim. 2017, 3, 17013. [Google Scholar] [CrossRef] [PubMed]
  2. Radhakrishnan, D.M.; Goyal, V. Parkinson’s disease: A review. Neurol. India 2018, 66, 26–35. [Google Scholar] [CrossRef]
  3. Rossi, M.; Perez-Lloret, S.; Merello, M. How much time is needed in clinical practice to reach a diagnosis of clinically established Parkinson’s disease? Park. Relat. Disord. 2021, 92, 53–58. [Google Scholar] [CrossRef] [PubMed]
  4. Tolosa, E.; Garrido, A.; Scholz, S.W.; Poewe, W. Challenges in the diagnosis of Parkinson’s disease. Lancet Neurol. 2021, 20, 385–397. [Google Scholar] [CrossRef]
  5. Li, Z.; Yang, J.; Wang, Y.; Cai, M.; Liu, X.; Lu, K. Early diagnosis of Parkinson’s disease using Continuous Convolution Network: Handwriting recognition based on off-line hand drawing without template. J. Biomed. Inform. 2022, 130, 104085. [Google Scholar] [CrossRef]
  6. Nilashi, M.; Abumalloh, R.A.; Yusuf, S.Y.M.; Thi, H.H.; Alsulami, M.; Abosaq, H.; Alyami, S.; Alghamdi, A. Early diagnosis of Parkinson’s disease: A combined method using deep learning and neuro-fuzzy techniques. Comput. Biol. Chem. 2023, 102, 107788. [Google Scholar] [CrossRef]
  7. Razali, R.; Ahmad, F.; Rahman, F.N.A.; Midin, M.; Sidi, H. Burden of care among caregivers of patients with Parkinson’s disease: A cross-sectional study. Clin. Neurol. Neurosurg. 2011, 113, 639–643. [Google Scholar] [CrossRef]
  8. Dash, S. Chapter 22—A systematic review of adaptive machine learning techniques for early detection of Parkinson’s disease. In Artificial Intelligence for Neurological Disorders; Abraham, A., Dash, S., Pani, S.K., García-Hernández, L., Eds.; Academic Press: Cambridge, MA, USA, 2023; pp. 361–385. [Google Scholar] [CrossRef]
  9. Almeida, J.S.; Rebouças Filho, P.P.; Carneiro, T.; Wei, W.; Damaševičius, R.; Maskeliūnas, R.; de Albuquerque, V.H.C. Detecting Parkinson’s disease with sustained phonation and speech signals using machine learning techniques. Pattern Recognit. Lett. 2019, 125, 55–62. [Google Scholar] [CrossRef]
  10. Polat, K.; Nour, M. Parkinson’s disease classification using one against all based data sampling with the acoustic features from the speech signals. Med. Hypotheses 2020, 140, 109678. [Google Scholar] [CrossRef]
  11. Pereira, C.R.; Pereira, D.R.; Weber, S.A.; Hook, C.; de Albuquerque, V.H.C.; Papa, J.P. A survey on computer-assisted Parkinson’s Disease diagnosis. Artif. Intell. Med. 2019, 95, 48–63. [Google Scholar] [CrossRef]
  12. Sakar, C.O.; Serbes, G.; Gunduz, A.; Tunc, H.C.; Nizam, H.; Sakar, B.E.; Tutuncu, M.; Aydin, T.; Isenkul, M.E.; Apaydin, H. A comparative analysis of speech signal processing algorithms for Parkinson’s disease classification and the use of the tunable Q-factor wavelet transform. Appl. Soft Comput. 2019, 74, 255–263. [Google Scholar] [CrossRef]
  13. Pahuja, G.; Nagabhushan, T.N. A Comparative Study of Existing Machine Learning Approaches for Parkinson’s Disease Detection. IETE J. Res. 2021, 67, 4–14. [Google Scholar] [CrossRef]
  14. Alzubaidi, M.S.; Shah, U.; Dhia Zubaydi, H.; Dolaat, K.; Abd-Alrazaq, A.A.; Ahmed, A.; Househ, M. The Role of Neural Network for the Detection of Parkinson’s Disease: A Scoping Review. Healthcare 2021, 9, 740. [Google Scholar] [CrossRef]
  15. Sechidis, K.; Fusaroli, R.; Orozco-Arroyave, J.R.; Wolf, D.; Zhang, Y.P. A machine learning perspective on the emotional content of Parkinsonian speech. Artif. Intell. Med. 2021, 115, 102061. [Google Scholar] [CrossRef]
  16. Ma, J.; Zhang, Y.; Li, Y.; Zhou, L.; Qin, L.; Zeng, Y.; Wang, P.; Lei, Y. Deep dual-side learning ensemble model for Parkinson speech recognition. Biomed. Signal Process. Control 2021, 69, 102849. [Google Scholar] [CrossRef]
  17. Ul Haq, A.; Li, J.P.; Agbley, B.L.Y.; Mawuli, C.B.; Ali, Z.; Nazir, S.; Din, S.U. A survey of deep learning techniques based Parkinson’s disease recognition methods employing clinical data. Expert Syst. Appl. 2022, 208, 118045. [Google Scholar] [CrossRef]
  18. Zhang, T.; Zhang, Y.; Sun, H.; Shan, H. Parkinson’s disease detection using energy direction features based on EMD from voice signal. Biocybern. Biomed. Eng. 2021, 41, 127–141. [Google Scholar] [CrossRef]
  19. Mei, J.; Desrosiers, C.; Frasnelli, J. Machine Learning for the Diagnosis of Parkinson’s Disease: A Review of Literature. Front. Aging Neurosci. 2021, 13. [Google Scholar] [CrossRef]
  20. Ngo, Q.C.; Motin, M.A.; Pah, N.D.; Drotár, P.; Kempster, P.; Kumar, D. Computerized analysis of speech and voice for Parkinson’s disease: A systematic review. Comput. Methods Programs Biomed. 2022, 226, 107133. [Google Scholar] [CrossRef]
  21. Quan, C.; Ren, K.; Luo, Z.; Chen, Z.; Ling, Y. End-to-end deep learning approach for Parkinson’s disease detection from speech signals. Biocybern. Biomed. Eng. 2022, 42, 556–574. [Google Scholar] [CrossRef]
  22. Madruga, M.; Campos-Roca, Y.; Pérez, C.J. Addressing smartphone mismatch in Parkinson’s disease detection aid systems based on speech. Biomed. Signal Process. Control 2023, 80, 104281. [Google Scholar] [CrossRef]
  23. Coelho, B.F.O.; Massaranduba, A.B.R.; dos Santos Souza, C.A.; Viana, G.G.; Brys, I.; Ramos, R.P. Parkinson’s disease effective biomarkers based on Hjorth features improved by machine learning. Expert Syst. Appl. 2023, 212, 118772. [Google Scholar] [CrossRef]
  24. Dixit, S.; Bohre, K.; Singh, Y.; Himeur, Y.; Mansoor, W.; Atalla, S.; Srinivasan, K. A Comprehensive Review on AI-Enabled Models for Parkinson’s Disease Diagnosis. Electronics 2023, 12, 783. [Google Scholar] [CrossRef]
  25. Steinbuch, K. Die Lernmatrix. Kybernetik 1961, 1, 36–45. [Google Scholar] [CrossRef]
  26. Steinbuch, K.; Frank, H. Nichtdigitale lernmatrizen als perzeptoren. Kybernetik 1961, 1, 117–124. [Google Scholar] [CrossRef]
  27. Steinbuch, K. Adaptive networks using learning matrices. Kybernetik 1964, 2, 148–152. [Google Scholar] [CrossRef]
  28. Aldape-Pérez, M.; Yáñez-Márquez, C.; Camacho-Nieto, O.; Argüelles-Cruz, A.J. A New Tool for Engineering Education: Hepatitis Diagnosis using Associative Memories. Int. J. Eng. Educ. 2012, 28, 1399–1405. [Google Scholar]
  29. Alarcón-Paredes, A.; Pogrebnyak, O.; Argüelles-Cruz, A.J. Image Transform based on Alpha-Beta Associative Memories. Comput. Y Sist. 2013, 17, 527–541. [Google Scholar]
  30. Valle, M.E.; Sussner, P. Quantale-based autoassociative memories with an application to the storage of color images. Pattern Recognit. Lett. 2013, 34, 1589–1601. [Google Scholar] [CrossRef]
  31. López-Yáñez, I.; Sheremetov, L.; Yáñez-Márquez, C. A novel associative model for time series data mining. Pattern Recognit. Lett. 2014, 41, 23–33. [Google Scholar] [CrossRef]
  32. Aldape-Pérez, M.; Yáñez-Márquez, C.; Camacho-Nieto, O.; López-Yáñez, I.; Argüelles-Cruz, A.J. Collaborative learning based on associative models: Application to pattern classification in medical datasets. Comput. Hum. Behav. 2015, 51, 771–779. [Google Scholar] [CrossRef]
  33. Ramírez-Rubio, R.; Aldape-Pérez, M.; Yáñez-Márquez, C.; López-Yáñez, I.; Camacho-Nieto, O. Pattern classification using smallest normalized difference associative memory. Pattern Recognit. Lett. 2017, 93, 104–112. [Google Scholar] [CrossRef]
  34. Yáñez-Márquez, C. Associative Memories based on Order Relations and Binary Operators. Ph.D. Thesis, Instituto Politécnico Nacional, Instituto Politécnico Nacional, México, 2002. (In Spanish). [Google Scholar]
  35. Yáñez-Márquez, C.; López-Yáñez, I.; Aldape-Pérez, M.; Camacho-Nieto, O.; Argüelles-Cruz, A.J.; Villuendas-Rey, Y. Theoretical Foundations for the Alpha-Beta Associative Memories: 10 Years of Derived Extensions, Models, and Applications. Neural Process. Lett. 2017, 48, 811–847. [Google Scholar] [CrossRef]
  36. Acevedo-Mosqueda, M.E.; Yáñez-Márquez, C.; López-Yáñez, I. Complexity of Alpha-Beta Bidirectional Associative Memories. In Proceedings of the MICAI 2006: Advances in Artificial Intelligence, Apizaco, Mexico, 13–17 November 2006; Gelbukh, A., Reyes-Garcia, C.A., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 357–366. [Google Scholar]
  37. Acevedo-Mosqueda, M.E.; Yáñez-Márquez, C.; López-Yáñez, I. Alpha-Beta bidirectional associative memories: Theory and applications. Neural Process. Lett. 2007, 26, 1–40. [Google Scholar] [CrossRef]
  38. Fisher, R.A. The use of multiple measurements in taxonomic problems. Ann. Eugen. 1936, 7, 179–188. [Google Scholar] [CrossRef]
  39. Dheeru, D.; Karra Taniskidou, E. UCI Machine Learning Repository. 2017. Available online: http://archive.ics.uci.edu/ml (accessed on 25 May 2023).
  40. Sculley, D. Kaggle Datasets. 2023. Available online: https://www.kaggle.com/datasets (accessed on 25 May 2023).
  41. Little, M.A.; McSharry, P.E.; Hunter, E.J.; Spielman, J.; Ramig, L.O. Suitability of Dysphonia Measurements for Telemonitoring of Parkinson’s Disease. IEEE Trans. Biomed. Eng. 2009, 56, 1015–1022. [Google Scholar] [CrossRef]
  42. Stehman, S.V. Selecting and interpreting measures of thematic classification accuracy. Remote Sens. Environ. 1997, 62, 77–89. [Google Scholar] [CrossRef]
  43. Meyer-Baese, A.; Schmid, V. Chapter 7—Foundations of Neural Networks. In Pattern Recognition and Signal Analysis in Medical Imaging, 2nd ed.; Meyer-Baese, A., Schmid, V., Eds.; Academic Press: Oxford, UK, 2014; pp. 197–243. [Google Scholar] [CrossRef]
  44. Dietterich, T.G. Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Comput. 1998, 10, 1895–1923. [Google Scholar] [CrossRef]
  45. Japkowicz, N.; Shah, M. Evaluating Learning Algorithms: A Classification Perspective; Cambridge University Press: New York, NY, USA, 2011. [Google Scholar]
  46. Howell, D.C. Statistical Methods for Psychology, 8th ed.; Cengage Learning: Belmont, CA, USA, 2012. [Google Scholar]
  47. Hill, T.; Lewicki, P. Statistics: Methods and Applications, 1st ed.; StatSoft, Inc.: Tulsa, OK, USA, 2005. [Google Scholar]
  48. Witten, I.; Frank, E. Data Mining: Practical Machine Learning Tools and Techniques, 2nd ed.; Morgan Kaufmann Series; Morgan Kaufmann Publishers Inc.: Cambridge, MA, USA, 2005. [Google Scholar]
  49. Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I.H. The WEKA Data Mining Software: An Update. SIGKDD Explor. 2009, 11, 10–18. [Google Scholar] [CrossRef]
  50. Kohavi, R. A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI 95), Montreal, QC, Canada, 20–25 August 1995; pp. 1137–1145. [Google Scholar]
  51. Kohavi, R.; John, G.H. Wrappers for Feature Subset Selection. Artif. Intell. 1997, 97, 273–324. [Google Scholar] [CrossRef]
  52. Hariharan, M.; Polat, K.; Sindhu, R. A new hybrid intelligent system for accurate detection of Parkinson’s disease. Comput. Methods Programs Biomed. 2014, 113, 904–913. [Google Scholar] [CrossRef] [PubMed]
Figure 1. A wrapper-based feature selection approach applied to SNDAM. In a wrapper-based feature selection approach, performance is evaluated with all possible feature combinations. After an exhaustive search for relevant features, the set of features that increase classification performance is retained.
Figure 1. A wrapper-based feature selection approach applied to SNDAM. In a wrapper-based feature selection approach, performance is evaluated with all possible feature combinations. After an exhaustive search for relevant features, the set of features that increase classification performance is retained.
Healthcare 11 01601 g001
Figure 2. Example of a binary encoded relevance identification vector. The value 1 is assigned to features that contribute to improving classification performance, while the value 0 is assigned to those that do not.
Figure 2. Example of a binary encoded relevance identification vector. The value 1 is assigned to features that contribute to improving classification performance, while the value 0 is assigned to those that do not.
Healthcare 11 01601 g002
Table 1. Main attributes of the datasets used for performance measurement validation.
Table 1. Main attributes of the datasets used for performance measurement validation.
No.DatasetInstancesAttributes
1.Parkinson’s Dataset (University of Oxford) [41]19523
2.Parkinson’s Disease Classification Dataset (Istanbul University) [12]104027
Table 2. Parkinson’s disease classification using Dataset 1, 5 × 2 cross-validation. The highest performance metric value is highlighted in boldface.
Table 2. Parkinson’s disease classification using Dataset 1, 5 × 2 cross-validation. The highest performance metric value is highlighted in boldface.
No.AlgorithmDataset 1
AccuracySensitivitySpecificityPrecisionAUCG-Mean
1.IBk96.4196.6095.8398.6196.2296.21
2.NaiveBayes69.2361.9091.6795.7976.7975.32
3.MultilayerPerceptron90.7792.5285.4295.1088.9788.89
4.RandomForest91.7995.9279.1793.3887.5487.14
5.RandomTree84.6289.8068.7589.8079.2778.57
6.SMO87.1899.3250.0085.8874.6670.46
7.SNDAM96.6196.9396.8398.6396.8896.87
ISNDAM99.4899.9899.3198.7599.6599.64
Table 3. Statistical hypothesis tests using Dataset 1: Two-matched-samples t test, confidence interval percentage of 95%, p < 0.05 establishes statistical significance threshold.
Table 3. Statistical hypothesis tests using Dataset 1: Two-matched-samples t test, confidence interval percentage of 95%, p < 0.05 establishes statistical significance threshold.
No.Compared AlgorithmsDataset 1
p ValueNull HypothesisAlternative Hypothesis
1.ISNDAM—IBk4.82 × 10 14
2.ISNDAM—NaiveBayes2.20 × 10 16
3.ISNDAM—MultilayerPerceptron5.10 × 10 12
4.ISNDAM—RandomForest4.41 × 10 13
5.ISNDAM—RandomTree7.51 × 10 14
6.ISNDAM—SMO9.31 × 10 11
7.ISNDAM—SNDAM1.37 × 10 04
Table 4. Parkinson’s disease classification using Dataset 2, 5 × 2 cross-validation. The highest performance metric value is highlighted in boldface.
Table 4. Parkinson’s disease classification using Dataset 2, 5 × 2 cross-validation. The highest performance metric value is highlighted in boldface.
No.AlgorithmDataset 2
AccuracySensitivitySpecificityPrecisionAUCG-Mean
1.IBk67.9672.2462.3171.7267.2767.09
2.NaiveBayes59.8573.9841.1562.4557.5755.17
3.MultilayerPerceptron68.0569.9165.5872.8867.7467.71
4.RandomForest73.1079.9464.0474.6371.9971.54
5.RandomTree66.1471.8058.6569.6865.2364.89
6.SMO66.2384.7441.7365.8063.2359.46
7.SNDAM98.2898.1898.1997.9598.1998.19
ISNDAM99.6699.2399.9899.9899.6199.60
Table 5. Statistical hypothesis tests using Dataset 2: Two-matched-samples t test, confidence interval percentage of 95%, p < 0.05 establishes statistical significance threshold.
Table 5. Statistical hypothesis tests using Dataset 2: Two-matched-samples t test, confidence interval percentage of 95%, p < 0.05 establishes statistical significance threshold.
No.Compared AlgorithmsDataset 2
p ValueNull HypothesisAlternative Hypothesis
1.ISNDAM—IBk1.73 × 10 14
2.ISNDAM—NaiveBayes2.20 × 10 16
3.ISNDAM—MultilayerPerceptron5.86 × 10 13
4.ISNDAM—RandomForest1.12 × 10 11
5.ISNDAM—RandomTree1.08 × 10 12
6.ISNDAM—SMO6.54 × 10 13
7.ISNDAM—SNDAM1.95 × 10 03
Table 6. Comparing the effectiveness of our proposed method to previous studies [13,24] using Dataset 1. The highest performance metric value is highlighted in boldface.
Table 6. Comparing the effectiveness of our proposed method to previous studies [13,24] using Dataset 1. The highest performance metric value is highlighted in boldface.
No.AlgorithmsDataset 1
AccuracySensitivitySpecificityAUCG-Mean
1.KNN Cityblock distance69.7466.6770.7568.7168.68
2.KNN Euclidean distance72.3168.7573.4771.1171.07
3.SVM Polynomial kernel81.0379.1787.7683.4683.35
4.SVM Linear kernel82.9087.3378.5682.9482.83
5.SVM RBF kernel88.2191.6777.5584.6184.31
6.ANN Scaled conjugate gradient85.1270.0096.5983.3082.23
7.ANN Levenberg–Marquardt95.8993.7596.5995.1795.16
ISNDAM99.4899.9899.3199.6599.64
Table 7. Comparing the effectiveness of our proposed method to previous studies [19,52] using Dataset 1. Raw features (RF), weighted features (WF), general regression neural network (GRNN), least-square support vector machine (LS-SVM) and probabilistic neural network (PNN). The highest performance metric value is highlighted in boldface.
Table 7. Comparing the effectiveness of our proposed method to previous studies [19,52] using Dataset 1. Raw features (RF), weighted features (WF), general regression neural network (GRNN), least-square support vector machine (LS-SVM) and probabilistic neural network (PNN). The highest performance metric value is highlighted in boldface.
No.AlgorithmsDataset 1
Feature Reduction/SelectionAccuracySensitivitySpecificityAUCG-Mean
1.LS-SVM22 RF95.3896.4492.0594.2494.21
2.LS-SVM22 WF100.00100.00100.00100.00100.00
3.PNN22 RF95.4997.9988.4793.2393.10
4.PNN22 WF100.00100.00100.00100.00100.00
5.GRNN22 RF95.4998.1388.2093.1693.03
6.GRNN22 WF100.00100.00100.00100.00100.00
ISNDAM22 RF99.4899.9899.3199.6599.64
Table 8. Comparing the effectiveness of our proposed method to previous studies [11,12,18] using Dataset 2. Random forests (RF), support vector machine (SVM), intrinsic mode functions (IMF). The highest performance metric value is highlighted in boldface.
Table 8. Comparing the effectiveness of our proposed method to previous studies [11,12,18] using Dataset 2. Random forests (RF), support vector machine (SVM), intrinsic mode functions (IMF). The highest performance metric value is highlighted in boldface.
No.AlgorithmsDataset 2
Accuracy Sensitivity Precision F1
1.SVM IMF196.5492.0199.4595.58
2.SVM IMF293.1687.7695.7191.56
3.SVM IMF384.6882.2383.2482.73
4.SVM IMF483.0075.0081.4978.11
5.SVM IMF577.5474.8972.8973.87
6.SVM IMF663.3354.2563.0558.31
7.RF IMF194.8994.7895.2395.00
8.RF IMF291.6794.1284.2188.88
9.RF IMF381.5270.0088.4478.14
10.RF IMF479.6766.0081.2572.83
11.RF IMF577.0870.2273.6871.90
12.RF IMF672.9260.0070.5964.86
ISNDAM99.6699.2399.9899.60
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Luna-Ortiz, I.; Aldape-Pérez, M.; Uriarte-Arcia, A.V.; Rodríguez-Molina, A.; Alarcón-Paredes, A.; Ventura-Molina, E. Parkinson’s Disease Detection from Voice Recordings Using Associative Memories. Healthcare 2023, 11, 1601. https://doi.org/10.3390/healthcare11111601

AMA Style

Luna-Ortiz I, Aldape-Pérez M, Uriarte-Arcia AV, Rodríguez-Molina A, Alarcón-Paredes A, Ventura-Molina E. Parkinson’s Disease Detection from Voice Recordings Using Associative Memories. Healthcare. 2023; 11(11):1601. https://doi.org/10.3390/healthcare11111601

Chicago/Turabian Style

Luna-Ortiz, Irving, Mario Aldape-Pérez, Abril Valeria Uriarte-Arcia, Alejandro Rodríguez-Molina, Antonio Alarcón-Paredes, and Elías Ventura-Molina. 2023. "Parkinson’s Disease Detection from Voice Recordings Using Associative Memories" Healthcare 11, no. 11: 1601. https://doi.org/10.3390/healthcare11111601

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop