Abstract
This paper introduces a modified soft margin optimal hyperplane (MSMOH) algorithm, which enhances the linear separating properties of support vector machines (SVMs) by placing higher penalties on large misclassification errors. This approach improves margin symmetry in both balanced and asymmetric data distributions. The research is divided into two main stages. The first stage evaluates MSMOH for synthetic data classification and its application in heart disease diagnosis. In a cross-validation setting with unknown data, MSMOH demonstrated superior average performance compared to the standard soft margin optimal hyperplane (SMOH). Performance metrics confirmed that MSMOH maximizes the margin and reduces the number of support vectors (SVs), thus improving classification performance, generalization, and computational efficiency. The second stage applies MSMOH as a novel synthesis algorithm to design a neural associative memory (NAM) based on a recurrent neural network (RNN). This NAM is used for fault diagnosis in fossil electric power plants. By promoting more symmetric decision boundaries, MSMOH increases the accurate convergence of 1024 possible input elements. The results show that MSMOH effectively designs the NAM, leading to better performance than other synthesis algorithms like perceptron, optimal hyperplane (OH), and SMOH. Specifically, MSMOH achieved the highest number of converged input elements and the smallest number of elements converging to spurious memories .
1. Introduction
Pattern classification uses software tools called classifiers to sort data into categories. For these classifiers to function effectively, they must be given carefully chosen input features that allow them to represent and differentiate between distinct classes within the data [1]. Among these tools, support vector machines (SVMs) stand out as a powerful supervised learning technique widely used across many applications due to their robust computational abilities in handling complex classification problems [2]. The operation of SVMs involves mapping input data into a high-dimensional feature space, where it becomes possible to construct a linear decision boundary. The specific characteristics of this boundary contribute to the model’s strong generalization capabilities. The foundational concept of SVMs is to identify an optimal separating hyperplane for cases where data is linearly separable. This approach is then adapted to non-separable data by introducing the concept of the soft margin hyperplane, which tolerates some misclassifications to find the best possible separation. SVMs have demonstrated exceptional performance in several key areas. For instance, in medical imaging they have a strong track record in breast cancer detection from images, where other methods are surpassing in accuracy [3]. For heart beat classification using electrocardiograms, SVMs integrated with artificial neural networks have also improved the performance [4]. In a broader comparison of machine learning algorithms, reference [5] found that SVMs consistently ranked among the best-performing methods. The study evaluated algorithms such as K-nearest neighbor, decision trees, and genetic algorithms. It concluded that both SVMs and long short-term memory networks delivered superior results in terms of both quality and quantity across different types of problems.
Although developed in the 1990s [6], SVMs remain a highly relevant and versatile tool for solving a wide range of modern problems. The continuous growth of artificial intelligence and advances in soft computing have led to new modifications and approaches that further improve SVM performance. Recent innovations have further enhanced the performance of SVMs. For example, a firefly algorithm-optimized SVM has been developed to achieve better classification accuracy than other methods [7]. In other applications, a hybrid model combining SVM with variable model decomposition has proven to be a useful and effective tool for making hydrological predictions in the context of climate change [8]. The integration of SVMs with meta-heuristic algorithms has become a popular method for improving their effectiveness. A notable example is a hybrid grey wolf optimizer combined with an improved sine cosine algorithm. This hybrid model enhances an SVM for classifying electroencephalography signals and achieves a very high classification rate [9]. In [10] a recent method called improved fuzzy least squares twin SVM enhances traditional SVMs for pattern classification. It improves performance by minimizing both structural and empirical risks, which leads to better generalization and faster processing speeds. A novel method called quantum SVM feature selection is introduced in [11] which combines quantum SVMs with multi-objective genetic algorithms. The purpose of this technique is to reduce the amount of data selecting most important features while also improving accuracy and overall performance.
For safety-critical systems like aero engines, chemical plants, and manufacturing, it is essential to ensure high levels of reliability and safety. This is particularly important because of the risk of process abnormalities and component faults. A key objective in these systems is to detect any faults as early as possible [12]. A fault can be defined as a change in a system’s properties or parameters that falls outside of its acceptable or standard operating conditions [13]. The classification of faults often depends on where they occur within a system (e.g., an overheating turbine engine or a cooling pump failure in a power plant). Consequently, a number of different approaches have been created to locate these faults. For instance, in [14] an artificial neural network based on a pattern recognition system is used to successfully detect a stator winding short circuit fault in a permanent magnet synchronous motor. In [15] authors developed a method to identify single fault patterns in power distribution networks with a convolutional neural network and fault pattern-based algorithm.
In this paper, the authors introduce a modified soft margin optimal hyperplane (MSMOH) to improve the linear separation capabilities of SVMs, and initial findings demonstrate that this modification maximizes the margin during hyperplane construction, which in turn reduces the number of support vectors (SVs) in comparison with conventional SMOH. Furthermore, the computational cost is significantly reduced. This ultimately leads to more effective classification for both synthetic data and heart disease diagnosis, where the statistical metrics (mean, standard deviation, and mean absolute generalization gap (MAGG)) during cross-validation indicate that MSMOH generalizes better than the SMOH. In the second stage, neural associative memory (NAM) based on a recurrent neural network (RNN) has been implemented for fault diagnosis in fossil electric power plants. A synthesis approach is developed to determine the weights and connection matrices of the NAM based on RNN using a new MSMOH algorithm which is a modification to the criterion usually utilized to determine the optimal hyperplane (OH) or soft margin optimal hyperplane (SMOH) in the synthesis of SVMs, reference [16]. The RNN is a type of Hopfield network that functions as an associative memory. Its purpose is to store desired patterns as stable memories. However, it is possible that undesirable patterns can appear as spurious memories, the number of which must be as small as possible for better efficacy of the NAM as described in [17,18,19,20]. This allows a stored pattern to be retrieved later, even when the initial input is only a partial or noisy version of that pattern. In practice, these memory patterns are typically represented using either bipolar or binary vectors which correspond to fault patterns that can appear in operation of a fossil electric plant. Once these fault patterns are stored, they can be retrieved when the input pattern contains enough information about that stored pattern. A convergence analysis is carried out using possible input elements to the NAM using the new MSMOH with better results compared to other synthesis approaches such as perceptron, OH, and SMOH algorithms [17,18]. Specifically, MSMOH achieved the highest number of converged input elements which converge to the stored fault pattern and the smallest number of possible input elements that converged to one of spurious memories (only 2).
This paper covers in Section 2 the materials and methods used, while Section 3 provides necessary mathematical background. This includes a detailed explanation of how to construct both the OH and SMOH for SVMs. It also introduces the principles of neural associative memory (NAM), a recurrent neural network model, and its associated training algorithms. In Section 4, MSMOH is introduced, along with the new objective functions developed to acquire corresponding parameters of the model. Section 5 presents classification results which are first presented with synthetic data and then with a heart disease database [21] which had been preprocessed previously where it is compared to conventional methods, where the margin maximization, the number of SVs, and the computational cost of the MSMOH shows better results compared with SMOH. Furthermore, metrics such as precision, sensitivity, specificity, accuracy, receiver operating characteristic (ROC) curve, and area under the curve (AUC) for classification results are obtained. Section 6 presents the NAM design with the MSMOH training for fault diagnosis in fossil electric power plants including convergence analysis, and finally, Section 7 and Section 8 present the discussion and conclusion of this paper.
2. Materials and Methods
This research is structured into two distinct stages. The first stage focuses on demonstrating the advantages of MSMOH in comparison to conventional SMOH. The second stage involves the training of a NAM, based on a recurrent neural network. This model incorporates the MSMOH developed in this study, and the reported results directly reflect this methodological approach.
- 1.
- First Stage: Data Classification and Heart Disease Diagnosis with MSMOH Method.
- Following the description of MSMOH in Section 4, two different datasets are employed to illustrate the superiority of the new method. The first is a two-dimensional synthetic dataset with balanced distribution (symmetric), which allows for a visual comparison of the margin maximization offered by the constructed MSMOH versus SMOH.
- The heart disease database [21], sourced from the UCI repository, constitutes the second dataset with unbalanced data distribution (asymmetric). Prior to classification, this database underwent imputation, codification, and correlation analysis to optimize its representation. Subsequent analysis, encompassing performance metrics such as accuracy, precision, and confusion matrix, among others, are detailed in Section 5.
- 2.
- Second Stage: Fault Pattern Diagnosis.
- Section 6 details the acquisition of fault patterns in fossil electric power plants, along with the procedures employed for codifying the measured patterns.
- Subsequent to the codification of fault patterns into bipolar vectors, a NAM is trained using the novel MSMOH synthesis to facilitate fault pattern diagnosis.
- An analysis of the results obtained from the trained NAM is presented, showcasing its significant advantages when compared to conventional training algorithms such as perceptron, OH, and SMOH.
- Furthermore, the convergence properties—including convergence, non-convergence, and spurious memories—were analyzed and compared for each training algorithm across all 1024 possible element combinations ().
All analyses and methods described in this paper were performed using a Lenovo computer with an Intel(R) i7-10750H CPU, located in Guadalajara, Mexico with MATLAB R2020a installed (“MATLAB” is a registered trademark of the MathWorks, Inc., Natick, MA, USA). The heart disease database preprocessing, however, was completed in a Python 3.12 (“Python” is a registered trademark of the Python Software Foundation) environment using the pandas and scikit-learn [22] open source libraries in Google Colab.
3. Mathematical Preliminaries
This section begins by outlining the fundamental concepts of OH and SMOH, which are the basis of conventional SVM design [6]. It then proceeds to describe NAM equations [17,18,19,23] and its various training algorithms, which include perceptron, OH, and SMOH of SVM.
3.1. Optimal Hyperplane of SVM
SVMs are frequently used for two-class classification with a linear model of the following form:
where are the input vectors, W is a weight vector, and b is the bias term. The model is trained using a training set, O, which consist of M training examples. This set can be formally defined as follows:
where the input pattern for the ith example is represented by , which is a vector with n elements. The corresponding class label for that example is to all M examples in the training set. In a linearly separable case, correct classification of data points are achieved by
These two conditions presented in Equations (3) and (4), which use the weight vector W and the bias term b, can be combined into a single inequality. The combined equality is as follows:
Moreover, the classification of new data points can be obtained through the evaluation of a sign function in the linear model (1):
The signed distance between and the OH is a measure of how far the point is from the decision boundary and on which side it lies. It is calculated using the following equation:
where is the Euclidean norm of the weight vector W, which normalizes the distance. To maximize the distance between classes, the norm of the weight vector, W, must be minimized. This can be achieved using the following function:
Then, obtaining W and b for OH requires minimizing the function (8) considering the constraint in (5) which is equivalent to maximizing the margin, that is the distance between the hyperplane and the nearest data points [6]. The Lagrange multiplier functional method can be used as follows [16,24]:
where the Lagrange multiplier vector is subject to for . To minimize (9) with respect to W, b, and , find the saddle point by minimizing it with respect to W and b and maximizing it with respect to the Lagrange multiplier. As described in [6,25], after getting the respective minimization, (9) can be transformed into the objective function as follows:
with D as a positive semi-definite matrix [6], and , determined by the following equation:
is only determined by (2) and the Lagrange multipliers; hence, minimizing (8), including restrictions (5), is simplified to maximizing using Lagrange multipliers () satisfying the following equation:
Let . It is worth noting that (12) is obtained from the minimization of (9) in terms of b. If a solution to exists, it defines W and b for OH, and W may be expressed as a function of optimum Lagrange multipliers as follows:
The OH has been used in several publications; however, its usefulness is restricted to linearly separable data [1].
3.2. Soft Margin Optimal Hyperplane
SMOH is introduced in [25,26] as an alternative to ease the linearly separable classes criterion. The restrictions are defined as follows:
, where for . is also known as a “slack variable”, and there is one for each training data point. When there is no significant deviation, and the point is inside or in the margin boundary, while for other points, . In Figure 1 the patterns affected with can be observed, where some are inside the margin, others are in the margin, and some are misclassified.
Figure 1.
Soft margin optimal hyperplane with examples of , , and .
The search region for (15) is greater than that for (5), resulting in a better solution for optimization. The values represents deviations from SMOH and are minimized using the following criterion:
In [25], it is additionally suggested to set as “sufficiently small”. In [26], the following statement is presented: “For computational reasons, however, in this work it is considered the case . This scenario represents the smallest that is still computationally simple”. For , the optimization problem may not be convex owing to computing complexity. is the smallest number that guarantees computational feasibility, hence it is chosen for (16). Minimizing the functional in Equation (17) with constraints (15) yields W and b for the SMOH.
where the parameter is a weighting factor that is set based on the specific problem. Its purpose is to balance the importance of the two main components in the equation. To find the solution of , Lagrange multipliers can be used. The solution to this problem is found by maximizing the Lagrange dual function, :
subject to the following:
where D is a positive semi-definite matrix defined by (11). Maximizing (18), under constraints (19) and (20), is a simple quadratic programmatic (QP) [24]. The two dual maximization problems, OH (10) and SMOH (18), are extremely similar, with the main difference being the respective restrictions (13) and (20). After the Lagrange multipliers are obtained, W can be obtained as depicted in Equation (14) considering corresponding restrictions. On the other hand, b can be obtained with the following equation:
where represents the elements considered as SVs, and is considered as an example of the class.
3.3. Neural Associative Memory Based on Recurrent Neural Network
The primary function of a NAM is its ability to store patterns as stable memories. This allows a complete pattern to be retrieved, or recovered, even when the input is an incomplete or partial version of the stored pattern. In practice, these memory patterns are typically represented as bipolar vectors. The specific type of RNN used here is given by [17,18,19,23]:
where is the state vector, signifies the derivative of x with respect to time t, and is the output vector, , where for . is the connection matrix, and is the bias vector, and denotes the activation function, where the following holds:
The initial states of the system in Equation (22) are assumed to be within the range of for all . It is important to note that this system is a variation of the analog Hopfield model, utilizing the saturation activation function, . According to [19] a vector is considered a stable memory vector of the system if and if is an asymptotically stable equilibrium point of the system (22).
Lemma 1.
If and if
then represents a pair of stable memory vectors and an asymptotically stable equilibrium point of (22). The proof of this lemma is available in [19].
Lemma 1 defines as a set of n-dimensional bipolar vectors . For , define . The following synthesis problem addresses the development of (22) for associative memories.
Synthesis problem: Given a set of m bipolar vectors the synthesis problem is to find the system parameters such that the following two conditions are met:
- 1.
- Each of the given vectors becomes a stable memory vector for the system in Equation (22);
- 2.
- The number of undesirable or ”spurious” memory vectors is minimized, and the basin of attraction for each of the desired memory vectors is maximized. This means that even a noisy or incomplete version of a desired vector can still successfully converge to the correct stored memory.
The first item of the synthesis problem can be guaranteed by choosing the such that every satisfies condition (24) of Lemma 1. The second item can be partly ensured by constraining the diagonal elements of the connection matrix. To solve this synthesis problem, A, T, and I from (24) need to be determinate considering and .
Considering this, the given condition in (24) can be equivalently written as follows:
where represents the row of T. denotes the element of I. is the diagonal element of A, and is the entry of .
Training Algorithms for NAM
The NAM can be trained with different algorithms such as perceptron [19], OH [17], and SMOH [18] which have been employed to find the corresponding weights and bias to determine the best and I. These are some of the approaches that could be employed.
- 1.
- Perceptron training algorithmThe perceptron has the following equation:where Z is the perceptron output, is the input vector, is the weight vector, andThe perceptron training finds the weight vector W with the following steps considering and the patterns when and when .
- (a)
- Initialize the weight vector for .
- (b)
- For
- i.
- If and , then update;
- ii.
- If and , then update;
- iii.
- Otherwise, , where for some , and is the perceptron learning rate.
- (c)
- The training stops when no more updates for the weight vector W are needed.
The search of W weight values are presented as follows:such thatfor , andchoose with . For choose if . with and . - 2.
- Optimal hyperplane algorithmThis training is based on the equations described in Section 3.1, where W values are obtained finding the Lagrange multipliers and the b value using the support vectors selected in the optimal hyperplane construction. However, the selection of and I are subject to the following equation:and the vector is defined exactly as in (30). A notable distinction exists between the selection of NAM parameters and perceptron training regarding the determination of T and I. This process requires that with . For choose if . with and .
- 3.
- Soft Margin Training AlgorithmSimilar to the OH training, this training employs the equations presented in Section 3.2 for the search of optimal values to construct the SMOH. In this case, a hyperparameter C is introduced and must be correctly selected for the efficient training of the NAM. Once the weight vector is computed, the following condition must be accomplishedwhere the vector is again considered and defined as in (30), letting the NAM parameters be chosen with the same requirements as in the OH training algorithm where with . For choose if . with and .Despite the selection of the and I parameters being similar to the OH training algorithm, the main difference is that obtaining the W and b parameters is based on the SMOH construction and hinge loss function. Further and deep analysis of the training algorithms for the NAM can be consulted in [17].
4. Proposed Soft Margin Optimal Hyperplane Modification
In this section, a new synthesis method for constructing a soft margin optimal hyperplane, MSOH, is introduced. This novel approach aims to improve upon the traditional methods for building the hyperplane in SVMs. The results of optimization are determined by the criterion used, which must be appropriate for the problem to be solved. Figure 2 shows how affects the function for various positive values of . The effects of show that for criterion (16), when , the effect is similar and independent of values, and the function converges to 1. Then, using (16), the optimization process considers small values (which do not create classification errors) in the same way, indicating that (16) is not an adequate criterion for classification.
Figure 2.
effect on function.
To choose a more suitable optimization criterion for asymmetric data, it is a priority to minimize the effects of large values, as these lead to classification errors. This can be achieved by using a criterion that places a higher penalty on these larger errors, allowing for more fine-tuned adjustments to handle imbalanced datasets. The following criterion can be used for this purpose:
The weight values are now defined to be larger for significant values and smaller for minor ones, as follows:
for and . Replacing (34) in (33), the following expression is obtained:
with . From (35), the respective optimization process prioritizes large values, which produce classification errors.
In Figure 2, it is possible to see the differences in influence for large values of p (or as described in the figure) and also for small increases in when p is large; hence, the proposed criterion (35) is very different from criterion (16) in [25]. In fact for (35) p should be big enough, instead of being small enough as stated in (16). Then, (as suggested in [1]) is the maximum value for criterion (16), and is the minimum one for criterion (35). Additionally, in [26] it is stated that due to computational efficiency, is the only allowable value for (16). Due to similar considerations, for (35). The case is a special instance where the hyperplane is constructed with the same SMOH objective function, as will be described later. The specific value of p depends on the problem being solved.
To define the MSMOH parameters, on the basis of criterion (35), it is required to minimize the following function, named .
with and , where C is arbitrarily defined according to the problem to be solved. In fact, it is a weighting factor case as explained in (17). In order to guarantee convexity, p can be any value larger or equal to 1. Considering that is convex and (15) depends linearly on and , then to solve this minimization problem no negative Lagrange multipliers () can be applied [3], with the respective function formulated as follows:
where are the Lagrange multipliers for (15), and are the ones for . To solve (36), with constrains (15), the Lagrange multipliers can be applied, whose solution corresponds to the respective saddle point of (37), obtained by minimizing with respect to W, b, and and, simultaneously, maximizing with respect to the Lagrange multipliers.
The conditions for the minimum of (37) are as follows:
From (38), W can be obtained as follows:
and from (39) the following constraint is defined:
According to Kühn–Tucker conditions [6], the optimum point for corresponds to ; then, from (40), is formulated as follows:
Substituting (41)–(43) in (37), and then transforming it, the following function is obtained:
depends only on the training examples (2), Lagrange multipliers , regulation parameter C and p being correctly selected. Hence, the minimization of (36) with constraint (15) is transformed to the maximization of , with constraints (13) and (42). The values of the MSMOH parameters can be determined by means of (15), (41), and (43), and the optimal values of the Lagrange multipliers. In particular, the optimal value for b is calculated using the following equation:
where for represents the elements considered as SVs, and is considered as the target value for the class label. The classification function is the same as described in Equation (6).
4.1. First Case of the Proposed Modification, p = 1
Consider the scenario wherein the parameter p approaches the value of 1 from the right-hand side. In order to prevent the final term of Equation (44) from becoming excessively large, it is necessary to consider the following function:
4.2. Second Case of the Proposed Modification, p = 2
4.3. Third Case of the Proposed Modification, p = 3
Considering in Equation (44), then the following holds:
4.4. Forth Case of the Proposed Modification, p = 4
In this case in function (44) resulting in the following:
Although new functions to be maximized under constraints (13) and (42) have been introduced, they cannot be resolved using QP. In instances where , as established in (48) for and (49) for , an alternative method must be employed to solve them. Moreover, the alternative method to solve the objective function of the proposed modification could search for a positive real optimal value of p with the condition of .
5. Data Classification with MSMOH Synthesis for Support Vector Machines
This section details the implementation of the proposed MSMOH synthesis on synthetic data and a preprocessed heart disease database [21] (which underwent missing data imputation and codification). Section 5.1 and Section 5.2.2 compare the results from the new hyperplane synthesis with conventional SMOH. Subsequently, Section 5.2.3 employs cross-validation to illustrate the model’s advantages across varying input data. For this study, the search of the hyperparameter C has been optimized using the widely employed technique of grid search where the range of values in the grid is obtained by forming the search space with a series of logarithmically scaled intervals. Grid search explores all potential hyperparameter combinations inside the proposed space. For this study, the grid size and range are described in Table 1.
Table 1.
Grid Search space.
5.1. Classification of Synthetic Data with MSMOH
In a first stage, the proposed hyperplane construction is used to classify synthetic balanced data with features and classes , as depicted in Figure 3.
Figure 3.
Synthetic data.
The data have been generated in MATLAB where the attributes contain 200 elements with 100 corresponding to class and the other half to class 1. Moreover, the data are normalized with a mean of 0 and a standard deviation of 1, to a common scale to enhance SVM performance by preventing features with larger values from disproportionately influencing the model. For classification problems, the maximization of margin can be calculated with Equation (50) which is important in this section as will be presented in the classification results with the constructed hyperplane.
where is the norm of W.
For simplicity, is used for the MSMOH, as this keeps the formulation as a QP problem. The regularization parameter, C, is set to for both MSMOH and SMOH, a value determined through grid search and fine adjustment. As presented in Table 2, the MSMOH classifies data with superior performance metrics compared to SMOH, specifically for accuracy, sensitivity, and precision. This enables the classifier to accurately detect true positive cases in the case of sensitivity. However, precision shows how frequently the model’s positive predictions are accurate, and the more precise the model is, the fewer false positives it produces. Finally, accuracy quantifies the percentage of accurate predictions the SVM makes out of all model predictions. Moreover, construction of the hyperplane is illustrated in Figure 4a,b where the number of SVs is illustrated with a bold contour, and the constructed hyperplane and the misclassified elements, among others, are described such that it is demonstrated how maximization of the hyperplane with MSMOH synthesis could be accomplished with a fewer number of SVs as presented in Figure 4c where . On the other hand, with the SMOH results, classification performance is good but with a dependency on the use of a great amount of SVs, contrarily to the MSMOH as presented in Table 2.
Table 2.
Classification metrics for MSMOH and SMOH methods.
Figure 4.
(a) Classification of synthetic data with MSMOH; (b) zoom of the classification of margin with MSMOH; (c) the MSMOH constructed and the SVs used for the construction.
Furthermore, maximization of the margin is lower compared to the one obtained with new synthesis giving a value . Classification and maximization of margin with SMOH is presented in Figure 5a, b, and c respectively. Although the MSMOH performance showed a significant improvement in the classification task, using real-world data is necessary to increase the reliability of the SVM model.
Figure 5.
(a) Classification of synthetic data with SMOH; (b) zoom of the classification of margin with SMOH; (c) the SMOH constructed and the SVs used for the construction.
Average computational cost has been calculated using the “tic” and “toc” function of MATLAB for the training of the SVM with MSMOH and SMOH over a different instances. The results are presented in Table 3 where the MSMOH has a faster response in training which reflects a lower computational cost.
Table 3.
Average computational cost for MSMOH and SMOH methods.
5.2. Heart Diseases Diagnosis with MSMOH
Despite the improved classification performance of MSMOH, real-world data is crucial to fully demonstrate its advantages. For this purpose, a database for diagnosing heart diseases has been selected. However, this database contained significant missing data and categorical information. Consequently, it was preprocessed to achieve the best attribute representation for detecting heart disease in patients. The preprocessing steps are detailed in Section 5.2.1. The heart disease diagnosis using SVM-MSMOH is presented in Section 5.2.2, and the same SVM with MSMOH, utilizing cross-validation, is discussed in Section 5.2.3.
5.2.1. UCI Heart Disease Database Preprocessing
This database comprises information on 920 patients, each with 16 diagnostic attributes related to heart problems. Of these, 509 patients are labeled with heart disease and 411 are not. Table 4 presents these data attributes, including the amount of data available for each. Data treatment, considering visualization and implementation, is performed using scikit-learn and pandas libraries. Initial step involved encoding data using label encoding method. This technique transforms categorical data into a numerical format, making it compatible with machine learning methods like SVM. Several encoding techniques were considered, including one-hot encoding (for binary representation), label encoding (assigning unique ordinal numerical values), hashing (mapping categories to numerical values for dimensionality reduction), and target encoding (replacing categories with the mean or mode of the label).
Table 4.
UCI heart disease database attributes.
For this study, label encoding has been specifically selected because the original data includes various attributes that are not suitable for binary representation, unlike what one-hot encoding offers. Examples of attributes where this coding was applied include “Chest pain type,” “dataset” (now referred to as “Origin”), “sex,” and “ST-segment slope type.” Original description of attribute and resulting encoding are illustrated in Table 5 and Table 6. For incomplete data, an imputation strategy has been applied to replace missing values with reasonable estimates derived from the dataset itself. Specifically, continuous variables such as blood pressure, cholesterol, and ST-segment depression have been imputed using the mean, as this approach preserves the central tendency of physiological measurements without introducing significant bias. Conversely, categorical and discrete variables such as the number of colored blood vessels and exercise-induced angina have been imputed using the mode, since replacing missing values with the most frequent category maintains the natural distribution of these attributes.
Table 5.
Data information.
Table 6.
Codified data information.
This combined mean–mode strategy prevents information loss that would result from case deletion, ensures consistency across the dataset, and supports robust model training, thereby enhancing the generalization capacity of the proposed methodology. As a result, the dataset is now complete, as shown in Table 7.
Table 7.
Completed data using imputation technique.
Finally, a correlation analysis has been conducted in MATLAB® to find the attributes that are important and significantly represent the patients characteristics. From this, a correlation threshold was selected to find attributes that could better work for diagnosis of heart diseases. In Figure 6 the obtained correlation results are presented, and the 10 selected attributes are as follows: exang, cp, thalch, oldpeak, slope, Sex, country, Age, chol, and ca.
Figure 6.
Correlation matrix of the heart disease database.
5.2.2. Heart Disease Diagnosis with SVM-MSMOH Classification
Once the database has been preprocessed and is ready for use, the classification task is carried out. In this initial case, all elements of the database are employed to diagnose heart disease. Additionally, more metrics of performance are introduced such as a confusion matrix and F1-score which will help to measure the performance with the imbalanced classes of the database. In this case, for the MSMOH, , and the regularization term C is set to 83,168.5597 for both MSMOH and SMOH; this value was again determined through a grid search (Table 1) and fine adjustment. Performance results of the SVM classification, presented and compared in Table 8, reveal that SMOH outperforms MSMOH in accuracy, sensitivity, and F1-score when using the entire dataset. However, the results also show that MSMOH has better precision and specificity than SMOH. Furthermore, MSMOH offers similar classification performance while using fewer support vectors and a wider margin for hyperplane construction. This suggests that MSMOH is a simpler model. It may also generalize better to new, unseen data. The confusion matrices for MSMOH and SMOH are presented in Figure 7 and Figure 8, respectively.
Table 8.
Heart disease classification metrics with SVM-MSMOH using all data.
Figure 7.
Confusion matrix with SVM-MSMOH classification.
Figure 8.
Confusion matrix with SVM-SMOH classification.
An interpretation of these results suggests that SMOH is marginally more effective in terms of overall accuracy and its ability to correctly identify true positive cases (sensitivity). In contrast, MSMOH demonstrates superior performance in correctly identifying healthy patients (specificity) and in the reliability of its positive predictions (precision). Even though the performance of both classifiers is similar, their ability to generalize to new data has not been entirely tested. To provide a more robust evaluation, the next section explores their classification performance using cross-validation to validate models on unseen data.
5.2.3. Heart Disease Diagnosis with Cross-Validation
Previously, the SVM-SMOH and SVM-MSMOH models are evaluated using a full-sample training approach. To obtain a more robust and reliable estimate of the generalization performance of the models and to assess the advantages of the proposed modification, a k-fold cross-validation scheme is subsequently implemented. Cross-validation is a technique used in machine learning to assess how well a model generalizes to unseen data helping to prevent overfitting, where a model performs well on training data but poorly on new data. The chosen technique is selected due to the moderate size of the dataset.
Cross-validation is performed using five folds, partitioning data into a training set of 735 elements and a validation set of 184, with performance metrics presented in Table 9 and Table 10. Table 9 presents the training results for each fold, detailing metrics such as accuracy, sensitivity, specificity, precision, and F1-score, in addition to the number of SVs and the margin. Moreover, metrics such as the mean and standard deviation are included. In test and training, the standard deviation illustrates how consistent the model is throughout the folds.
Table 9.
Heart disease classification training metrics with SVM-MSMOH in cross-validation with 5 folds.
Table 10.
Heart disease classification test metrics with SVM-MSMOH in cross-validation with 5 folds.
These results show that the training performance is consistent across all five folds. Specifically, the number of SVs and the margin have similar characteristics, which is evident from the mean values calculated for each metric during training. Validation indicates strong generalization performance for the SVM-MSMOH model, particularly with the model from fold 2. This is evident from the balanced metrics: accuracy of , sensitivity of , specificity of , precision of , and an F1-score of as seen in Table 10. This is corroborated from the standard deviation obtained and the confusion matrices for each fold in Figure 9 where (−1) indicates the heart disease presence while (+1) indicates no heart disease presence. Furthermore the ROC curve is illustrated in Figure 10. As previously noted, the second model, illustrated in Figure 10b, yields an AUC of 0.9005, confirming it as the best-performing model.
Figure 9.
Confusion matrix with SVM-MSMOH with cross-validation classification. (a) First validation, (b) second validation, (c) third validation, (d) fourth validation, and (e) fifth validation.
Figure 10.
ROC curve of cross-validation with SVM-MSMOH. (a) First validation, (b) second validation, (c) third validation, (d) fourth validation, and (e) fifth validation.
In contrast, results obtained with the conventional SVM-SMOH are presented in Table 11 where the same fold partition as selected in the previous MSMOH is used for the classification task. Performance indicates significant differences with the proposed modification specifically in the number of SVs needed to construct the optimal hyperplane and a margin maximization which is smaller than the one obtained in the MSMOH with less SVs.
Table 11.
Heart disease classification training metrics with SVM-SMOH in cross-validation with 5 folds.
Despite similarity in training metrics, average performance shows that MSMOH achieved better accuracy () compared to SMOH (). Although SMOH offers better average metrics during training, its testing performance is not entirely consistent.
In contrast, MSMOH provides similar and, in some metrics, superior results compared to SMOH, while using fewer SVs and a wider margin. This is reflected in the model’s better generalization ability, as shown in Table 12, where the average results for each metric indicate that MSMOH outperforms SMOH in many categories.
Table 12.
Heart disease classification test metrics with SVM-SMOH in cross-validation with 5 folds.
In addition, the standard deviation is higher in metrics such as sensitivity, specificity, and precision. Furthermore, the confusion matrices in Figure 11 and the ROC curves in Figure 12 highlight the classification performance and the model’s generalization ability. For comparison, the best SMOH is also obtained from fold 2, but it had a smaller AUC value of 0.8996, as shown in Figure 12b.
Figure 11.
Confusion matrix with SVM-SMOH with cross-validation classification. (a) First validation, (b) second validation, (c) third validation, (d) fourth validation, and (e) fifth validation.
Figure 12.
ROC curve of cross-validation with SVM-SMOH. (a) First validation, (b) second validation, (c) third validation, (d) fourth validation, and (e) fifth validation.
This analysis revealed that SVM-MSMOH consistently achieved balanced and high-quality performance metrics, with model form fold 2 being particularly effective (AUC = 0.9005). In a direct comparison with the SVM-SMOH model, SVM-MSMOH demonstrated crucial advantages: it required fewer SVs, created a wider margin, and showed superior generalization performance on unseen data, even with similar training metrics.
These results, supported with detailed metrics and visualizations, indicate the effectiveness of proposed modification for building a more robust and efficient model. To assure a better comparison between SVM-MSMOH and SVM-SMOH, the Mean Absolute Generalization Gap (MAGG) is computed using Equation (51) to assess the stability and generalization performance. MAGG is defined as the average of the absolute differences between training and test performance for each metric, and the results are presented in Table 13:
where is the value obtained in the m training metric, is the value obtained in the m test metric, and is the metric measured for each model.
Table 13.
SVM-MSMOH and SVM-SMOH (5-fold cross-validation) GAPP comparison.
The SVM-MSMOH model has a smaller difference between training and test performance resulting in a MAGG of . On the other hand, the SVM-SMOH model has a MAGG of . The standard deviation obtained for each metric also helps to analyze the generalization capacity of MSMOH and SMOH. In the case of MSMOH, the standard deviation in the training and test sets is ≤2.5; while in the training of SMOH, the standard deviation accomplishes this condition, in the test, the standard deviation is ≥2.5. This indicate that SVM-MSMOH has a better generalization capacity.
6. Neural Associative Memory with MSMOH Training for Fault Diagnosis in Fossil Electric Power Plants
In this section, the previously presented MSMOH is employed to find optimal values of a NAM used for fault diagnosis. The scheme for fault diagnosis is presented where the scheme has two components: residual generation and fault diagnosis [17]. The scheme is displayed in Figure 13. The first component relies on a comparison between measurements obtained from the plant and corresponding predicted values generated by a neural network predictor. This predictor is developed using neural network models trained with healthy operational data from the plant. Discrepancies between these two sets of values, termed residuals, provide a robust indicator for detecting faults. The calculation of these residuals is as follows:
Variables denote the actual plant measurements, while are the corresponding predicted values. Ideally, under normal plant operation, these residuals should remain consistent regardless of system’s operating state, indicating only the presence of noise and minor disturbance. A fault in the system, however, will cause the residuals to deviate from zero in identifiable ways. For the second component, residuals are converted into bipolar or binary fault patterns through the application of thresholds. These patterns are crucial for training a NAM based on a recurrent neural network designed to perform fault diagnosis. Associative memory itself is trained using the MSMOH algorithm. After the previous stage, residual vectors are generated with ten elements which are evaluated by detection thresholds.
Figure 13.
Fault diagnosis scheme.
Detection thresholds are contained in Table 14.
Table 14.
Detection thresholds.
This evaluation provides a set of residuals encoded as bipolar vectors obtained with Equation (53):
Residuals are encoded in real-time for every detected fault. Subsequently, these encoded patterns undergo analysis to identify and select the most appropriate fault patterns for storage in associative memory. This crucial selection step ensures accurate fault discrimination, reduces incidence of false alarms, and facilitates prompt fault isolation. The procedure is illustrated in Figure 14.
Figure 14.
Fault classification scheme.
Fault patterns are described in Table 15 where each column corresponds, respectively, to the following:
Table 15.
Fault patterns.
- : Normal Operating Conditions;
- : Water Wall Tube Rupture;
- : Superheater Tube Rupture;
- : Superheated Steam Temperature Control Failure;
- : Fouled Regenerative Preheater;
- : Feedwater Pump Variable Speed Drive Operating at Maximum;
- : Stuck Fuel Valve.
6.1. Results on Fault Diagnosis
This section details the training of the NAM (as described in Section 3) using four algorithms: training, OH, SMOH, and proposed MSMOH. The resulting , T, W, I, and B matrices are presented and used in the NAM defined in Equation (22) along with convergence of stored patterns specifically achieved with MSMOH training. A convergence analysis is also conducted to demonstrate the number of elements that converge to the seven stored fault patterns. From a total of 1024 possible combinations, the number of convergent, non-convergent, and spurious memories for each model are shown, demonstrating how many elements converge to 7 store fault patterns.
6.1.1. Results with Perceptron Training Algorithm
For the perceptron training algorithm, the parameters considered are as follows: , , and which have been taken from [23].
6.1.2. Results with OH Training Algorithm
For consistency, parameters for OH, SMOH, and MSMOH training , , and are selected through trial and error. The steps followed to acquire these matrices are described in Section 3.1.
6.1.3. Results with SMOH Training Algorithm
Results obtained with the SMOH training using steps described in Section 3.2 with the same parameters previously described.
6.1.4. Results with MSMOH Training Algorithm
For the NAM training in this section, the same parameters as the OH and SMOH algorithms are used. Additionally, with , Equation (47) was used to obtain the NAM training matrices.
These results indicate that the main difference between the obtained matrices and the training algorithm is the connection matrix T, specifically its diagonal, and the I matrix. After the NAM has been trained, the results are presented in Figure 15, Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21, Figure 22, Figure 23, Figure 24, Figure 25, Figure 26, Figure 27 and Figure 28 which cover both the state evolution of the NAM and the corresponding component outputs. In particular, Figure 15, Figure 17, Figure 19, Figure 21, Figure 23, Figure 25 and Figure 27 show the retrieved pattern response for the seven faults, where subfigure (a) illustrates the state evolution of x, demonstrating a clear convergent behavior toward the equivalent of each stored fault pattern, and subfigure (b) presents the output of y from Equation (22). Complementarily, Figure 16, Figure 18, Figure 20, Figure 22, Figure 24, Figure 26 and Figure 28 depict the outputs of components to associated with each retrieved fault pattern.
Figure 15.
Retrieving of fault pattern . (a) States evolution; (b) fault pattern retrieved.
Figure 16.
Output of NAM obtained for each state x with defined in (22). Figures (a–j) describe the output obtained from to elements of the fault pattern retrieved.
Figure 17.
Retrieving of Fault Pattern 1 . (a) States evolution; (b) fault pattern retrieved.
Figure 18.
Output of NAM obtained for each state x with defined in (22). Figures (a–j) describe the output obtained from to elements of the fault pattern retrieved.
Figure 19.
Retrieving of Fault Pattern 2 . (a) States evolution; (b) fault pattern retrieved.
Figure 20.
Output of NAM obtained for each state x with defined in (22). Figures (a–j) describe the output obtained from to elements of the fault pattern retrieved.
Figure 21.
Retrieving of Fault Pattern 3 . (a) States evolution; (b) fault pattern retrieved.
Figure 22.
Output of NAM obtained for each state x with defined in (22). Figures (a–j) describe the output obtained from to elements of the fault pattern retrieved.
Figure 23.
Retrieving of Fault Pattern 4 . (a) States evolution; (b) fault pattern retrieved.
Figure 24.
Output of NAM obtained for each state x with defined in (22). Figures (a–j) describe the output obtained from to elements of the fault pattern retrieved.
Figure 25.
Retrieving of Fault Pattern 5 . (a) States evolution; (b) fault pattern retrieved.
Figure 26.
Output of NAM obtained for each state x with defined in (22). Figures (a–j) describe the output obtained from to elements of the fault pattern retrieved.
Figure 27.
Retrieving of Fault Pattern 6 . (a) States evolution; (b) fault pattern retrieved.
Figure 28.
Output of NAM obtained for each state x with defined in (22). Figures (a–j) describe the output obtained from to elements of the fault pattern retrieved.
6.2. Convergence Analysis
To increase the reliability of the obtained results and to further show improvement of the MSMOH algorithm, a convergence analysis is conducted. Hopfield networks, such as the NAM used in this study, can generate spurious memories, which are stable memories which are not desired and are not part of the fault patterns used for training. Consequently, testing the NAM with different training algorithms allowed for a through evaluation of its performance with all possible input combinations .
Results for the number of convergent, non-convergent, and spurious elements for each training algorithm are presented in Table 16 where MSMOH has the smallest number of spurious memories and has the highest number of elements that converged to one of stored patterns . This provides strong evidence that MSMOH is superior to the other three methods for retrieving stored fault patterns.
Table 16.
Convergence analysis with all training algorithms.
7. Discussion
This paper introduces a modified optimal hyperplane for SVM, termed MSMOH, which shows significant advantages over the conventional SMOH. These advantages include a reduction in the number of SVs and a wider maximization of the hyperplane’s margin. Classification task have been carried out using synthetic data as a visual tool of hyperplane construction, and real heart disease diagnoses have also been conducted with great results. In Section 5.1, the first approach to the performance of MSMOH classification demonstrated good results, the capacity to reduce the number of SVs, and an increase in the margin, where, in the MSMOH classification, the margin is and the number of SVs is 14 out of 200 possible data inputs. In contrast, SMOH required 171 SVs from the same data input to obtain a margin , while its performance metrics were the worst. Figure 4 and Figure 5 illustrate hyperplane construction for both methods, with their respective subplots (a), (b), and (c) detailing their respective visual representations (SVs, margin, and data points). Furthermore, as shown in Table 3, the MSMOH method demonstrated a significantly lower average computational cost during the training process compared to SMOH.
In Section 5.2.2 Table 8 is presented with the MSMOH for SVM implemented in heart disease classification without considering validation at all, only prediction of all labeled data of the heart disease database. However, this study allows the measurement of the performance with a larger amount of data. In this case, the accuracy obtained with MSMOH is over-passed by that with SMOH, meaning that it has a higher proportion of correct predictions related to the labeled data. On the other hand, specificity and precision are better in MSMOH, where specificity indicates that this model has a better performance to identify negative instances out of all negative instances. F1 score is the result of precision and sensitivity, and despite the differences between the MSMOH and SMOH, the F1 score is slightly lower in MSMOH compared to SMOH. The most important part is the number of SVs needed to construct the hyperplane, with MSMOH using less and generating a wider margin.
In Section 5.2.3 cross-validation is carried out and this allows comparison between training results and test results. In this case, cross-validation helps to acquire a better understanding of the performance of MSMOH and SMOH considering different combinations of folds to train and test the model. MSMOH has a similar average performance in training (Table 9) and testing (Table 10) with a similar number of SVs considered in hyperplane construction, which also gives an average margin in all cases where these results are supported with the average, standard deviation, and MAGG of each metric performance. SMOH illustrates the cross-validation performance in testing (Table 11) and training (Table 12) where the same metrics have been calculated. MAGGs have been calculated to demonstrate the generalization capacity of MSMOH and SMOH, where MSMOH gives a better generalization with MAGG while SMOH gives a MAGG as presented in Table 13.
Moreover, the new MSMOH considering has been employed to train a NAM in Section 6 where results have been compared to other training algorithms where the main problems are related to a high number of spurious memories. However, MSMOH has better performance in comparison with perceptron, OH, and SMOH training due to 1019 possible input elements converging to one of the 7 stored fault patterns and only 5 possible input elements converging to one of the 2 spurious memories as illustrated in Table 16. NAM trained with MSMOH can thus be used for retrieving digit patterns by leveraging its ability to recall memories form a corrupted or incomplete image. It does this by converging to a stored digit pattern that is closest to the input, which provides advantages as presented in this study.
From the perspective of symmetry and asymmetry in classification tasks, the proposed MSMOH formulation aims to construct decision boundaries that maintain a balanced separation between classes, even when the underlying data distribution is asymmetric in terms of class size, feature variance, or SVs distribution. In symmetric scenarios, the hyperplane remains equidistant from the nearest elements of each class, whereas in asymmetric cases, conventional SMOH often shifts the boundary towards the more compact class. By prioritizing the penalization of large misclassification errors, MSOH reduces this bias, achieving a more symmetric margin distribution.
Furthermore, in the context of NAM training, this balanced separation improves the matching rate defined as the accurate convergence of noisy or incomplete input patterns to their corresponding stored patterns while decreasing unmatching occurrences, which include convergence to incorrect patterns or spurious memories. This improvement is particularly relevant in fault diagnosis and disease detection applications, where high matching accuracy under asymmetric and noisy conditions is critical.
Beyond power plants, fault detection is a significant challenge in other areas, such as automation processes and fault-tolerant control systems. For example, a key application is bearing fault diagnosis, where a NAM can be trained to identify and classify defects or malfunctions in rolling element bearings using vibration analysis and other techniques. Ultimately, early detection of faults is crucial for preventing catastrophic failures and minimizing downtime.
8. Conclusions
In conclusion, the results obtained show an improvement in NAM training with the MSMOH applied to fault diagnosis in fossil power plants, with the number of spurious memories being smaller compared to perceptron, OH, and SMOH. This could mean that if more patterns needs to be learn, the MSMOH would reduce the risk of false memories ensuring that the solutions are optimal only for the stored patterns. Furthermore, it is worth mentioning that the obtained results do not consider “optimal restrictions” as described in [17,19] which are limited for perceptron and OH training; this optimal restrictions allows manipulation of the diagonal elements of the connection matrix T when optimal conditions are obtained during NAM training.
These limitations encourage authors to extensively analyze and propose the possible “optimal restrictions” that could be applied for the SMOH and MSMOH in future work including application in other topics (e.g., digit reconstruction, bearing fault diagnosis, and fault tolerant control, among others) where there are more patterns or a larger number of elements are needed to be stored while contributing to advances in non-supervised learning models such as NAM.
On the other hand, the MSMOH algorithms are currently in development to work with alternative methods such as evolutionary algorithms that will allow for searching for optimal values that could improve classification including values of reducing the limitations of case where and the QP solving for the hyperplane construction. Moreover, not only working with linear kernels but also with polynomial and Gaussian ones, including the search of hyperparameters within the hyperplane construction, are avenues for future work. In addition, MSMOH could be compared to broader classification methods and additional performance metrics that allow an extensive validation of MSMOH’s advantages over SMOH.
Author Contributions
Conceptualization, M.A.R.C., J.A.R.-H. and A.Y.A.; methodology, J.A.R.-H. and M.A.R.C.; software, M.A.R.C.; validation, A.Y.A., J.A.R.-H., J.G. and J.C.G.G.; writing—original draft preparation, M.A.R.C. and J.A.R.-H.; writing—review and editing, J.A.R.-H., A.Y.A., M.A.R.C., J.C.G.G. and J.G. All authors have read and agreed to the published version of this manuscript.
Funding
The Centro Universitario de Ciencias Exactas e Ingenierías (CUCEI) of Universidad de Guadalajara (U. de G.), provided funding to publish this paper.
Data Availability Statement
The Heart Disease Database used for this study in Section 5 is public and available in [21]. Moreover, further details of the acquisition of Fossil Electric Power Plants fault pattern used in this study are available in [23].
Acknowledgments
The first author acknowledges the support and facilities provided by the Secretaria de Ciencia, Humanidades, Tecnologia e Innovacion (SECIHTI) with scholarship number 1085717. The authors thank the support and facilities of Universidad Autonoma del Carmen (UNACAR) and U. de G. to establish the academic collaboration network “Sistemas Avanzados, Inteligentes y Bioinspirados Aplicados a la Ingenieria, Tecnologia y Control” via its professors and students.
Conflicts of Interest
The authors declare no conflicts of interest.
Abbreviations
The following abbreviations are used in this manuscript:
| SVM | Support Vector Machine |
| OH | Optimal Hyperplane |
| SMOH | Soft Margin Optimal Hyperplane |
| MSMOH | Modified Soft Margin Optimal Hyperplane |
| NAM | Neural Associative Memory |
| RNN | Recurrent Neural Network |
| SVs | Support Vectors |
| ROC | Receiver Operating Characteristic |
| AUC | Area Under Curve |
| QP | Quadratic Programmatic |
References
- Abe, S. Support Vector Machines for Pattern Classification; Springer: Berlin/Heidelberg, Germany, 2005; Volume 2. [Google Scholar]
- Chandra, M.A.; Bedi, S. Survey on SVM and their application in image classification. Int. J. Inf. Technol. 2021, 13, 1–11. [Google Scholar] [CrossRef] [Scilit]
- Lo, C.S.; Wang, C.M. Support vector machine for breast MR image classification. Comput. Math. Appl. 2012, 64, 1153–1162. [Google Scholar] [CrossRef] [Scilit]
- Homaeinezhad, M.; Tavakkoli, E.; Atyabi, S.; Ghaffari, A.; Ebrahimpour, R. Synthesis of multiple-type classification algorithms for robust heart rhythm type recognition: Neuro-svm-pnn learning machine with virtual QRS image-based geometrical features. Sci. Iran. 2011, 18, 423–431. [Google Scholar] [CrossRef] [Scilit]
- Bansal, M.; Goyal, A.; Choudhary, A. A comparative analysis of K-nearest neighbor, genetic, support vector machine, decision tree, and long short term memory algorithms in machine learning. Decis. Anal. J. 2022, 3, 100071. [Google Scholar] [CrossRef] [Scilit]
- Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef] [Scilit]
- Zhang, H.; Shi, Y.; Yang, X.; Zhou, R. A firefly algorithm modified support vector machine for the credit risk assessment of supply chain finance. Res. Int. Bus. Financ. 2021, 58, 101482. [Google Scholar] [CrossRef] [Scilit]
- Meng, E.; Huang, S.; Huang, Q.; Fang, W.; Wang, H.; Leng, G.; Wang, L.; Liang, H. A hybrid VMD-SVM model for practical streamflow prediction using an innovative input selection framework. Water Resour. Manag. 2021, 35, 1321–1337. [Google Scholar] [CrossRef] [Scilit]
- Divya, P.; Devi, B.A. Hybrid metaheuristic algorithm enhanced support vector machine for epileptic seizure detection. Biomed. Signal Process. Control 2022, 78, 103841. [Google Scholar] [CrossRef] [Scilit]
- Laxmi, S.; Gupta, S.; Kumar, S. Intuitionistic fuzzy least square twin support vector machines for pattern classification. Ann. Oper. Res. 2024, 339, 1329–1378. [Google Scholar] [CrossRef] [Scilit]
- Wang, H. A novel feature selection method based on quantum support vector machine. Phys. Scr. 2024, 99, 056006. [Google Scholar] [CrossRef] [Scilit]
- Gao, Z.; Cecati, C.; Ding, S.X. A Survey of Fault Diagnosis and Fault-Tolerant Techniques—Part I: Fault Diagnosis With Model-Based and Signal-Based Approaches. IEEE Trans. Ind. Electron. 2015, 62, 3757–3767. [Google Scholar] [CrossRef] [Scilit]
- Van Schrick, D. Remarks on terminology in the field of supervision, fault detection and diagnosis. IFAC Proc. Vol. 1997, 30, 959–964. [Google Scholar] [CrossRef] [Scilit]
- Çira, F.; Arkan, M.; Gümüş, B. A new approach to detect stator fault in permanent magnet synchronous motors. In Proceedings of the 2015 IEEE 10th International Symposium on Diagnostics for Electrical Machines, Power Electronics and Drives (SDEMPED), Guarda, Portugal, 1–4 September 2015; pp. 316–321. [Google Scholar] [CrossRef] [Scilit]
- Li, Y.; Zhang, Y.; Liu, W.; Chen, Z.; Li, Y.; Yang, J. A Fault Pattern and Convolutional Neural Network Based Single-phase Earth Fault Identification Method for Distribution Network. In Proceedings of the 2019 IEEE Innovative Smart Grid Technologies—Asia (ISGT Asia), Chengdu, China, 21–24 May 2019; pp. 838–843. [Google Scholar] [CrossRef] [Scilit]
- Vapnik, V. Statistical Learning Theory; John Wiley and Sons: Hoboken, NJ, USA, 1998. [Google Scholar]
- Ruz-Hernandez, J.A.; Suarez, D.A.; Garcia-Hernandez, R.; Sanchez, E.N.; Suarez-Duran, M.U. Optimal training algorithm application to design an associative memory for fault diagnosis at a fossil electric power plant. IFAC Proc. Vol. 2012, 45, 756–762. [Google Scholar] [CrossRef] [Scilit]
- Ruz-Hernandez, J.A.; Sanchez, E.N.; Suarez, D.A. Soft Margin Training for Associative Memories: Application to Fault Diagnosis in Fossil Electric Power Plants. In Soft Computing for Hybrid Intelligent Systems; Springer: Berlin/Heidelberg, Germany, 2008; pp. 205–230. [Google Scholar] [CrossRef] [Scilit]
- Liu, D.; Lu, Z. A new synthesis approach for feedback neural networks based on the perceptron training algorithm. IEEE Trans. Neural Netw. 1997, 8, 1468–1482. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- dos Santos, A.S.; Valle, M.E. Max-C and Min-D Projection Auto-Associative Fuzzy Morphological Memories: Theory and an Application for Face Recognition. Appl. Math. 2023, 3, 989–1018. [Google Scholar] [CrossRef] [Scilit]
- Janosi, A.; Steinbrunn, W.; Pfisterer, M.; Detrano, R. Heart Disease Data Set. 1989. Available online: https://archive.ics.uci.edu/ml/datasets/heart+Disease (accessed on 1 July 2025).
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
- Ruz-Hernandez, J.A.; Sanchez, E.N.; Suarez, D.A. Fault Detection and Diagnosis for Fossil Electric Power Plants via Recurrent Neural Networks. Dyn. Continous Discret. Impuls. Syst. Ser. B 2008, 15, 219. [Google Scholar]
- Aoki, M. Introduction to Optimization Techniques: Fundamentals and Applications of Nonlinear Programming; Macmillan: New York, NY, USA, 1971. [Google Scholar]
- Haykin, S. Neural Networks: A Comprehensive Foundation, 2nd ed.; Prentice Hall: Hoboken, NJ, USA, 1999. [Google Scholar]
- Vapnik, V. The Nature of Statistical Learning Theory; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2000. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).



























