The number of MFCC feature dimensions affects the accuracy of the model and the number of parameters required. Therefore, analysis was conducted to identify a suitable number of MFCC feature dimensions.
4.1. Determination of the Suitable Number of MFCC Feature Dimensions
The effects of the numbers of MFCC feature dimensions and feature frames on the accuracy of the proposed classification architecture were investigated using the GitHub PCG database and PhysioNet/CinC Challenge 2016 database (
Table 6 and
Table 7, respectively). The numbers of MFCC feature dimensions and feature rows were varied from 13 to 20 and from 9 to 16, respectively. Due to the consistent trend of performance across all evaluation metrics (including accuracy, precision, recall, and f1-score), and considering that accuracy is a representative metric of overall classification performance, only accuracy is reported in the main text in order to maintain brevity.
Table 6 and
Table 7 detail the impact of MFCC dimensions (from 13 to 20) and the number of frames (from 9 to 16) on recognition accuracy when using the GitHub database and the physioNet/CinC Challenge 2016 database, respectively.
Each accuracy value in the tables is based on both holdout validation and 10-fold cross-validation. The results presented in
Table 6 indicate that the accuracy of the proposed classification architecture on the GitHub PCG database increased with the number of MFCC feature dimensions and the number of frames. The highest average accuracy for different numbers of MFCC feature dimensions (99.24%) was achieved when the number of frames was 13. Moreover, the highest average accuracy for different numbers of frames (99.3%) was achieved when the number of MFCC feature dimensions was 17. Increasing the numbers of MFCC feature dimensions and feature frames improved the classification accuracy; however, this improvement was not indefinite. The accuracy peaked to 99.6% when the numbers of dimensions and frames were 17 and 13, respectively. Thus, the optimal numbers of MFCC feature dimensions and feature frames for the GitHub PCG database were 17 and 13, respectively.
The results presented in
Table 7 suggest that the accuracy of the proposed classification architecture on the PhysioNet/CinC Challenge 2016 database increased with the number of MFCC feature dimensions and the number of frames. The highest average accuracy for different numbers of MFCC feature dimensions (90.465%) was achieved when the number of frames was 13. Moreover, the highest average accuracy for different numbers of frames (90.431%) was achieved when the number of MFCC feature dimensions was 17. The accuracy peaked to 90.74% when the numbers of dimensions and frames were 17 and 13, respectively. Thus, the optimal numbers of MFCC feature dimensions and feature frames for the PhysioNet/CinC Challenge 2016 database were 17 and 13, respectively.
The stride of convolution layers refers to the interval with which the convolutional filter moves over the input data, and this parameter directly affects the output size of the feature map. Increasing the stride reduces the size of the feature map, thereby reducing the spatial resolution of the output. However, an excessively large stride can result in the loss of useful information. Conversely, a smaller stride improves the spatial resolution, allowing the model to capture finer details. In the proposed WCNN model, the stride of the third convolution layer is set to 2, which reduces the feature map size by half. This reduction not only decreases the computational burden and memory usage but also accelerates the training process and reduces the model size.
Table 8 presents the variations in the number of model parameters with the numbers of MFCC feature dimensions and frames. If the number of MFCC feature dimensions is an odd value, the last part of the input that is smaller than the stride is dropped, leading to different input sizes with the same number of parameters. The number of model parameters was 312,357 when the numbers of MFCC feature dimensions and feature frames were 17 and 13, respectively.
To better understand the developed model’s decision-making process, the Grad-CAM algorithm [
35] was used to generate a heatmap shown in
Figure 8. The brighter areas in the generated heatmap indicate a higher level of attention from the model.
Figure 8 illustrates the visualization results for both correct and incorrect predictions across four disease categories (aortic stenosis, mitral regurgitation, mitral stenosis, mitral valve prolapse) and normal heart sounds. Among these, mitral stenosis and normal achieved 100% prediction accuracy, so no heatmaps for incorrect predictions are included. Moreover, the vertical and horizontal axes in the heatmap represent the numbers of MFCC feature dimensions and feature frames, respectively. The red boxes in
Figure 8 indicates the feature range selected in this study (17 dimensions and 13 frames). For aortic stenosis,
Figure 8a and
Figure 8b show the heatmaps when the predictions are correct and incorrect, respectively. Similarly,
Figure 8c,d display the heatmaps for mitral regurgitation in correct and incorrect predictions. For mitral stenosis,
Figure 8e illustrates the heatmap when predictions are correct, while for mitral valve prolapse,
Figure 8f and
Figure 8g depict the heatmaps for correct and incorrect predictions, respectively. Lastly,
Figure 8h presents the heatmap for normal heart sounds during correct predictions.
The results reveal that, in cases of correct predictions, the proposed model’s attention areas are concentrated within the MFCC range selected for this study (17-dimension, 13-frame). Conversely, when predictions are incorrect, the proposed model’s attention areas deviate from this range. Increasing the numbers of MFCC feature dimensions and frames may lead to a loss of focus because of the inclusion of unnecessary features, which affects the proposed model accuracy. Therefore, selecting the appropriate feature range is crucial for enhancing the proposed model’s performance.
In addition, an independent analysis of the impact of MFCC dimensions and the number of frames on model accuracy was conducted.
Figure 9 and
Figure 10 present box plots of the variations in model accuracy with respect to the numbers of MFCC feature dimensions and feature frames for the GitHub PCG and PhysioNet/CinC Challenge 2016 databases, respectively. In these box plots, the central horizontal line denotes the median (Q2), the box edges correspond to the first (Q1) and third (Q3) quartiles, and the whiskers represent the minimum and maximum values within 1.5 times the interquartile range, while outliers are indicated by individual points. The use of box plots allows visualization of accuracy distributions obtained across multiple folds of cross-validation, thus presenting a range of values rather than a single accuracy result. This approach highlights both the central tendency and the variability of the model performance under different parameter configurations. As depicted in
Figure 9a, when the number of MFCC feature dimensions was set to 17, the model achieved its highest accuracy, and the range from Q1 to Q3 was relatively narrow, indicating stable results. Moreover, as indicated in
Figure 9b, when the number of feature frames was set to 13, the model achieved its highest accuracy (99.6%) and highest average accuracy. The outlier accuracy value of 98.4% in
Figure 9b was caused by the number of MFCC feature dimensions being only 13. This feature dimension setting resulted in the model extracting relatively few features, which affected the stability and accuracy of its results.
The patterns presented in the PhysioNet/CinC Challenge 2016 database (
Figure 10) closely resemble those observed for the GitHub PCG database (
Figure 9). According to
Figure 10a, when the number of MFCC feature dimensions was set to 17, the range from Q1 to Q3 was relatively narrow, and the model achieved its highest average accuracy; thus, the model’s results were relatively stable under this setting.
Figure 10b indicates that when the number of frames was 13, the model achieved its highest accuracy (90.74%) and highest average accuracy; thus, this frame setting resulted in the optimal model accuracy. In summary, the highest model accuracy values for the GitHub PCG database and PhysioNet/CinC Challenge 2016 database (99.6% and 90.74%, respectively) were achieved when the numbers of MFCC feature dimensions and feature frames were set to 17 and 13, respectively, with a total of 312,357 parameters.
The variations in the accuracy and number of parameters of the WCNN-based classification model were investigated with the number of MFCC feature dimensions under the same number of frames (i.e., 13 frames). As presented in
Table 9, when the number of MFCC feature dimensions was set to 13, the number of parameters was 230,437, and the model accuracy was 98.4%. To improve the model accuracy, the number of MFCC feature dimensions was increased to 17, which resulted in the model using 312,357 parameters and achieving an accuracy of 99.6%. However, when the number of MFCC feature dimensions was set to 39 (13 dimensions each for static features, delta features, and delta–delta features), the number of parameters increased to 762,917, but the accuracy slightly decreased to 99.2%. Thus, the addition of dynamic features increased the computational complexity but reduced the model accuracy.
4.2. Experimental Results
To determine the optimal for model training, experiments were conducted with varying batch sizes for both datasets, and the corresponding classification accuracies were evaluated, as summarized in
Table 10. For the GitHub PCG database, batch sizes of 8, 16, and 32 were tested, resulting in accuracies of 99.3%, 99.6%, and 99.4%, respectively. The batch size of 16 yielded the highest accuracy and was therefore selected as the optimal configuration. For the PhysioNet/CinC Challenge 2016 database, batch sizes of 64, 128, and 256 were compared, producing accuracies of 89.62%, 90.7%, and 89.45%, respectively. The batch size of 128 achieved the best performance and was adopted for subsequent experiments. These results indicate that the optimal batch size depends on the dataset scale and complexity, with smaller batches favoring datasets of limited size and larger batches being more effective for larger, more variable datasets.
The accuracy and loss results obtained for the proposed model on the GitHub PCG database are displayed in
Figure 11a and
Figure 11b, respectively.
Figure 11 indicates that the model converged rapidly, and its training parameters stabilized. The accuracy and loss results obtained for the proposed model on the PhysioNet/CinC Challenge 2016 database are depicted in
Figure 12a and
Figure 12b, respectively.
Figure 12 indicates that the training accuracy increased sharply during the first 15 epochs and then gradually stabilized as the model gradually reached the optimal solution. Based on the convergence behavior observed in
Figure 11 and
Figure 12, the model achieved stable accuracy and loss values well before 100 epochs; therefore, 100 epochs were selected as an appropriate balance between convergence and computational efficiency.
Figure 13 and
Figure 14 depict the confusion matrices of the proposed model for testing data from the GitHub PCG and PhysioNet/CinC Challenge 2016 databases, respectively. The proposed model achieved accuracy values of 99% and 93.26% for the aforementioned data, respectively.
To evaluate the importance of the KWC module in enhancing noise robustness, a detailed ablation study was conducted using both clean and noisy datasets. The results are summarized in
Table 11 and
Table 12. As shown in
Table 11, using the GitHub PCG dataset for comparison, the proposed model’s accuracy decreased from 99.6% to 98.9% after removing the KWC module, with the f1-score also dropping from 99.6% to 98.9%. Similarly, other metrics such as precision and recall have declined, with the kappa value dropping from 99.5% to 98.6%. On the PhysioNet/CinC Challenge 2016 dataset, as shown in
Table 12, the removal of the KWC module caused accuracy to decrease from 90.74% to 88.37%, and the f1-score to drop significantly from 84.94% to 69.87%. Precision and recall also saw notable reductions, from 85.85% to 76.74% and 83.46% to 64.71%, respectively. It is worth noting that the addition of the KWC layer results in a more significant improvement in various metrics for the PhysioNet/CinC Challenge 2016 dataset compared to the GitHub PCG dataset.
The PhysioNet/CinC Challenge 2016 dataset is more heavily impacted by noise, and the KWC layer effectively extracts PCG features, thereby enhancing the model’s ability to capture the target signals and reducing the noise’s impact on classification performance. The inclusion or exclusion of the KWC layer does not affect the parameters because the KWC layer is used as a weight, which is ultimately multiplied with the corresponding channels of the feature map. Therefore, even with the addition of the KWC layer, it will not increase the parameters of the model. These results underscore the critical role of the KWC module in enhancing the proposed model’s performance. The experiments conducted on both datasets consistently demonstrate that this module significantly improves classification accuracy, f1-score, and other evaluation metrics. These results align with the theoretical basis that the KWC layer suppresses irrelevant or noisy information by applying adaptive weights to each feature map channel based on their global average activations.
To ensure the stability of the proposed WCNN-based classification model, a comprehensive analysis and comparison of the results from holdout validation and 10-fold cross-validation were conducted. Multiple evaluation metrics, including accuracy, precision, recall, f1-score, and kappa, were used to evaluate the proposed model’s performance from multiple perspectives.
Table 13 presents the proposed model performance results in 10-fold cross-validation for the GitHub PCG database. In the 10-fold cross-validation for this database, the proposed model exhibited average accuracy, precision, recall, f1-score, and kappa values were 99.6%, 99.6%, 99.59%, 99.61%, and 99.5%, respectively.
Table 14 lists the average accuracy of the proposed model for each category in the 10-fold cross-validation with the GitHub PCG database. The overall accuracy for each class was above 99%. The highest accuracy rate of 100% was observed for the “normal” and “mitral stenosis” classes. The aforementioned results proved that the proposed model achieved stable performance and balanced training for each category on the GitHub PCG database.
Table 15 presents the model performance results in 10-fold cross-validation for the PhysioNet/CinC Challenge 2016 database. For this database, the highest model accuracy rate was achieved in the ninth cross-validation, as shown in
Figure 12a. In the 10-fold cross-validation for the aforementioned database, the average accuracy, precision, recall, and f1-score values were 90.74%, 85.85%, 83.46%, and 84.94%, respectively.
Table 16 presents the overall accuracy of the proposed model for each category in the 10-fold cross-validation with the PhysioNet/CinC Challenge 2016 database. The accuracy rates for the normal and abnormal categories were 96.48% and 77%, respectively.
For the PhysioNet/CinC Challenge 2016 database, identifying abnormalities is more challenging than identifying normal scenarios because many PCG records are affected by noise, with considerable variations existing between PCG signals having the same label (
Figure 7). In addition, the ratio of abnormal to normal heart sound data in the aforementioned database is approximately 1:4. To avoid excessive bias toward specific classes in the training set, the original class distribution was preserved across all folds during the training and testing processes in this work. By contrast, Shuvo et al. [
26] achieved an overall accuracy of 86.57% on the same dataset, with class-wise accuracies of 93.27% for normal and 59.06% for abnormal. Compared with these results, the proposed model improved the recognition of abnormal heart sounds to 77%, substantially higher than the 59.06% reported in [
26]. This demonstrates that, despite the data imbalance, the proposed model effectively reduces the gap between imbalanced data classes and improves the classification performance for the minority class. Thus, the classification model proposed in this paper has superior performance and stability to those of existing models in classifying the PhysioNet/CinC Challenge 2016 database.
Models trained on imbalanced datasets may exhibit bias toward the majority class due to uneven gradient accumulation during the optimization process. To mitigate this evaluation bias, AUC-ROC and AP were adopted as complementary metrics to accuracy, enabling a more comprehensive and reliable assessment of the model’s performance, particularly for minority classes. Accordingly, 10-fold cross-validation was conducted on both the GitHub PCG (balanced) and PhysioNet/CinC Challenge 2016 (imbalanced) datasets, with AUC-ROC and AP calculated as the primary evaluation indicators. As shown in
Table 17 and
Table 18, both AUC-ROC and AP were used. On the balanced GitHub PCG dataset, the model achieved near-perfect performance, with average AUC-ROC and AP scores of 0.9998 and 0.9992, respectively, and a negligible difference of 0.0006 between the two metrics. This result aligns with theoretical expectations, where ROC and PR curves typically exhibit similar behavior under balanced class distributions.
In contrast, the imbalanced PhysioNet/CinC 2016 dataset (approximate class ratio of 4:1) yielded an average AUC-ROC of 0.8485 and AP of 0.868, with a modest difference of 0.02, indicating that AP is more sensitive to minority class performance. Despite this gap, the consistently high AP score suggests that the model maintained stable and accurate recognition of the minority class (Abnormal) without requiring any data balancing preprocessing.
According to Mandrekar [
44], AUC values between 0.8 and 0.9 are considered “excellent,” and Maxwell et al. [
45] emphasized that AP values close to 1 indicate strong prediction quality, while those below 0.5 indicate poor minority class performance. The AUC-ROC of 0.8485 and AP of 0.868 achieved by the proposed model on the PhysioNet/CinC 2016 dataset fall within the “excellent” range, demonstrating that even without data balancing, the model can effectively identify minority class instances. This performance highlights the robustness of the proposed architecture and supports the use of AUC-ROC and AP as complementary and reliable metrics for real-world imbalanced classification problems.
4.3. Evaluation of Model Stability Across Balanced and Random Sampling Using 10-Fold Cross Validation
Based on the sampling strategies described in
Section 3.2, the GitHub PCG database was evaluated under both balanced and random sampling conditions within a 10-fold cross-validation framework. The following presents the experimental results obtained under these two settings, with particular emphasis on the model’s stability and classification performance. The experimental results under balanced sampling are presented in
Table 19. As shown, the confusion matrices indicate that most folds achieved error-free classification, with only a very small number of misclassifications in certain folds. The overall average accuracy reached 99.6%, demonstrating the stability of the model. Further examination of the average per-class accuracies presented in
Table 14 shows values of 99.5% for AR, 99% for MR, 100% for MS, 99.5% for MVP, and 100% for N. These results indicate that, under balanced conditions, the model demonstrates exceptionally high recognition capability across all categories.
The results under random sampling are shown in
Table 20 and
Table 21. In this case, the overall average accuracy slightly decreased to 99.4%. While the difference is minor, the confusion matrices reveal that certain folds contained disproportionately large numbers of specific classes. For example, in Fold-2 and Fold-4 the MVP class appeared 28 times in the test set, which reduced its representation in the training set and led to a drop in classification accuracy.
Table 21 reports the average per-class accuracies over all folds: 99.41% for AR, 99.61% for MR, 100% for MS, 98.33% for MVP, and 100% for N. Although the MVP class showed a slight decline compared with balanced sampling, its accuracy remained as high as 98.33%, while all other classes maintained accuracies above 99%. These findings demonstrate that even under imbalanced sampling conditions, the model retained strong classification performance and generalization capability.
These results provide important evidence for applying the proposed approach to another dataset, the PhysioNet/CinC Challenge 2016 database, where the ratio of normal to abnormal samples is approximately 4:1, creating an imbalance. No balancing techniques were applied; instead, the same model and training strategy were used, namely extracting MFCC features as input, employing the WCNN architecture with KWC layers, and training with 10-fold cross-validation using the Adam optimizer. Under these settings, the model maintained strong performance, with reliable recognition of minority classes. To address potential distortion of accuracy due to imbalance, AUC-ROC and AP metrics were additionally reported, providing a more sensitive and reliable evaluation of minority-class performance.
A comparison was also made with the Cardi-Net model proposed in [
21]. According to the results shown in
Figure 7 of [
21], Cardi-Net achieved an overall average accuracy of 98.68% on the GitHub PCG database, with fold-wise accuracies ranging between 95.6% and 99.6%. By contrast, the proposed method achieved an overall average accuracy of 99.6% on the same dataset, with fold-wise accuracies consistently ranging from 98% to 100%, indicating smaller fluctuations. This highlights the superior stability and stronger robustness of the proposed model under imbalanced conditions.
In summary, although class distribution balance can influence model performance to some extent, the architecture and training strategy designed in this study enabled highly stable and accurate classification even under imbalanced scenarios. This robustness ensures reliable application to practical datasets such as PhysioNet, where class imbalance is present. Moreover, the use of AUC-ROC and AP further reflects the model’s capability in minority-class recognition, thereby validating the reliability and generalizability of the proposed approach.
4.4. Model Comparison and Discussion
The proposed lightweight framework achieved accuracy values of 99.6% and 90.74% for the GitHub PCG and PhysioNet/CinC Challenge 2016 databases, respectively. The model achieved the aforementioned performance with approximately 312.357 K parameters and approximately 4.5 M floating-point operations per second (FLOPs). To ensure fair comparison among different models, all comparative evaluations in this study were conducted using the same datasets—GitHub PCG and PhysioNet/CinC Challenge 2016. Accuracy was selected as the primary evaluation metric, representing overall classification performance, and was supplemented by parameters and FLOPs to assess architectural efficiency. Together, these metrics establish a consistent and representative benchmarking framework that ensures the comparability of results.
Table 22 presents a comprehensive comparison between the proposed classification model and various relevant novel models in terms of their parameters, number of FLOPs, classifier, and accuracy. In addition,
Table 23 provides a detailed summary of the data preprocessing strategies and the train/validation/test distributions used in each referenced study.
The paper [
10,
11,
12], and this work all use MFCC as the data preprocessing method. The method in [
10] combines MFCC, DWT, and SVM. It has a window size of 240, with a 33% overlap, and uses a 256-point FFT, ultimately selecting 19 MFCC feature dimensions. The feature vector length for the DWT is 24. Although the use of the DWT increases the complexity of feature extraction, it reduces the number of FFT points, which reduces the computational burden. The model proposed in [
11] combines MFCC with empirical mode decomposition (EMD) and Hilbert–Huang transform. The aforementioned study does not provide specific details regarding how MFCC is used in the model; however, the use of EMD and the Hilbert–Huang transform considerably increases the complexity of preprocessing. The literature [
12] combines MFCC with a 1D CNN-LSTM architecture. This model uses a window size of 2048, with 25% overlap; a 2048-point FFT; and 13 MFCC feature dimensions. This combination effectively leverages the strengths of the CNN and LSTM network, thus making the model suitable for processing complex sound data that include time series. The proposed method in this work combines MFCC with WCNN, using a window size of 1600, with 50% overlap; a 2048-point FFT; and 17 MFCC feature dimensions. Although the EMD-based MFCC processing method used in [
10] improves the effectiveness of feature extraction, its high computational complexity limits its use in real-time applications. By contrast, the 19-dimensional features used in [
9] provide more information; however, as presented in
Table 6 and
Table 7, increasing the number of feature dimensions increases the computational complexity but does not always improve the recognition performance. The model proposed in the present paper, which has a similar MFCC processing configuration to that used in [
12], can achieve high accuracy while avoiding excessive computational burden. Preliminary simulations (
Table 6 and
Table 7) revealed that the model proposed in the present paper achieved its highest accuracy when the number of MFCC feature dimensions was 17.
This work, along with studies [
23,
24,
28,
29], adopts CNN-based architectures. Therefore, an in-depth comparison and analysis were conducted focusing on design choices such as kernel size, stride, max pooling, and dropout. The model developed in [
23] uses the common two-dimensional convolution and pooling strategy, a 3 × 3 kernel size, and a stride of 1. Moreover, it employs 2 × 2 max pooling to gradually reduce the spatial dimensions of the generated feature map. The model developed in [
24] performs 1D convolution operation with a stride of 2 or 3. It also conducts 2 × 2 max pooling and employs two dropout rates (0.15 and 0.30) to enhance the model’s generalization performance and noise resistance. Chen et al. [
28] proposed a CNN model based on VGG16, consisted of three stages of 3 × 3 convolutions followed by max pooling layers, reducing the feature dimensions from 128 × 128 to 16 × 16. The model included seven convolutional layers and four fully connected layers, with a final softmax classifier for binary classification. Nguyen et al. [
29], on the other hand, implemented a lightweight CNN with only three convolutional layers and two fully connected layers. Their model design utilized 3 × 3 convolutions with ReLU activations, batch normalization, and 2 × 2 max pooling, followed by a flattening operation leading to fully connected layers of sizes 100 and 5, with softmax output. The WCNN-based classification model developed in the present study uses 2 × 2 kernels and a stride of 1 in its first two convolution layers, which helps it to process fine input features. Instead of using max pooling to reduce feature dimensions, the proposed model adopts a stride of 2 in its third convolution layer, thereby preserving more of the original information. Moreover, this model incorporates an innovative KWC layer, which enhances the model’s abstraction and representation capabilities through the weighted computation of key features, thereby further optimizing overall model performance. Finally, the dropout function is employed to prevent overfitting in the proposed model.
Using a larger stride in convolution layers can effectively reduce the spatial dimensions of the generated feature map while enabling the capture of more complex features. Compared with convolution, max pooling is a simpler and faster method for reducing dimensions; however, it results in the loss of some detailed information. Model designers must attempt to achieve a suitable balance between retaining crucial information and improving computational efficiency. The KWC layer used in the WCNN-based model developed in this study not only increases the model’s computational efficiency but also improves its ability to recognize key features, which is critical for increasing the accuracy and efficiency of signal classification.
Computational complexity and parameter count are important criteria for evaluating model efficiency in addition to classification accuracy. For clarity, the comparison is presented in stages, beginning with studies that reported only accuracy, followed by those that also provided complexity measures such as parameters and FLOPs. As shown in
Table 22, many earlier works lack full complexity reporting. For example, refs [
8,
10,
11,
13,
15,
16,
17,
19,
25] do not provide model parameters (i.e., the number of parameters in the respective architectures). Regarding the PhysioNet/CinC Challenge 2016 dataset, previous works such as [
19,
22] reported accuracy performances of 88% and 90%, respectively. Singh et al. [
22] proposed the AlexNet architecture, which achieved a 90% accuracy rate, but with an extremely high parameter count of 58 M. Chen et al. [
28] proposed a CNN model combining STFT and Log-MelSpectrogram, which achieved an accuracy of 91.74%—approximately 1% higher than our model. However, their architecture required 17.5 M parameters and 416.9 M FLOPs. According to
Table 23, models reporting higher accuracy on the noisier PhysioNet/CinC Challenge 2016 dataset, such as [
28], adopt filtering and denoising strategies including Butterworth and Savitzky–Golay filters. In contrast, the model proposed in this study achieved 90.74% accuracy without the use of filters, relying solely on MFCC feature extraction and zero-padding segmentation, while requiring only 1.8% of the parameters and 1% of the computational cost compared to [
28].
When comparing the performance of various studies on the GitHub PCG database, the models developed in [
8,
10,
15,
16,
17], which employ machine learning methods, achieved high classification accuracy (99.4%, 97.9%, 99.5%, 95.13%, and 98.53%, respectively); however, the proposed WCNN-based model outperformed these models. Baghel et al. [
24] used data augmentation techniques to increase the quantity of training data for a CNN model and then employed this model to classify signals from the GitHub PCG database. They achieved an accuracy of 98.6% with only 0.28 M parameters. Although the model developed in [
24] uses 11% fewer parameters than that developed in this work, the model developed in [
24] has 98% higher FLOPs and a lower accuracy. The WaveNet model proposed in [
25] achieved an overall accuracy of 97% for detecting HVDs in PCG signals. In [
12], MFCC was used for feature extraction, and a 1D CNN–LSTM model was constructed for heart sound recognition. This model outperformed that proposed in [
25], achieving an accuracy of 99.4% with 0.606 M parameters. Moreover, the authors of [
21] developed the Cardi-Net model for the automatic identification of heart diseases, achieving an accuracy of 98.879% with 9.5 M parameters. The numbers of parameters and FLOPs for this model are 96.7% and 97.6% higher, respectively, than those for the proposed WCNN-based model. In addition, Nguyen et al. [
29] introduced a CNN-based model utilizing Log-MelSpectrogram features, achieving an accuracy of 99.337% on the GitHub PCG dataset using a 1.5-second signal segment. Since this duration is close to the 1.4 second input length used in the present work, the result serves as a meaningful reference for comparison. However, this performance came at the cost of significantly increased model complexity, requiring 1.62 M parameters and over 22.5 M FLOPs—more than five times the size and computational cost of the proposed model. Similarly, Choudhary et al. [
13] proposed a CNN–GRU hybrid architecture and achieved 99.3% accuracy. While the results are promising in terms of classification accuracy, the study did not report the model size or FLOPs, making it difficult to assess its feasibility for deployment on resource-constrained devices.
Two independent models were trained and evaluated separately on the GitHub PCG and PhysioNet/CinC Challenge 2016 databases. These two datasets differ substantially in recording quality, with the former containing relatively clean signals and the latter exhibiting significant noise interference. Consequently, the comparative results between the two datasets demonstrate the proposed model’s robustness under varying noise conditions. To address this problem, paper [
26]’s, paper [
23]’s and paper [
30]’s have evaluated their models on multiple databases. Shuvo et al. [
26] developed a bidirectional-LSTM-based CardioXNet architecture and evaluated it on the GitHub PCG and PhysioNet/CinC Challenge 2016 databases, achieving accuracy values of 99.6% and 86.57%, respectively, with 0.67 M parameters. Similarly, Karhade et al. [
23] used time–frequency images as input for a deep CNN model, employing 1.2 M parameters to achieve accuracy values of 99.48% and 85.16% on the aforementioned two databases, respectively. Compared with the model proposed in [
23], the WCNN-based model developed in the present study achieved higher accuracy on the GitHub PCG and PhysioNet/CinC Challenge 2016 databases while using 74.9% fewer parameters and 99.3% fewer FLOPs. Wang et al. [
30] proposed PCTMF-Net, combining parallel CNNs, transformer modules, and second-order spectral analysis. It achieved 99.36% accuracy on GitHub PCG and 93% on PhysioNet/CinC Challenge 2016 database. On the noisier PhysioNet/CinC 2016 dataset, our model reached 90.74%, about 2.26% lower than [
30]. According to
Table 23, their method includes both digital Butterworth filtering and second-order spectral analysis, which likely enhanced noise suppression. However, their model achieved only 99.36% accuracy on the cleaner GitHub PCG dataset, while the proposed model achieved 99.6%, indicating that excessive filtering may reduce discriminative feature preservation in low-noise settings. Notably, our model achieved strong performance without such filtering, showing better robustness under practical conditions. In terms of complexity, PCTMF-Net uses 2.56 M parameters, over eight times that of our WCNN-based model, highlighting its superior compactness and efficiency. This performance contrast highlights the robustness of the proposed approach, which maintains high accuracy across both clean and noisy datasets while employing a simpler preprocessing pipeline—MFCC feature extraction and zero-padding segmentation—without explicit denoising. The model achieved this efficiency using only 312,357 parameters and 4.47 M FLOPs. Furthermore, according to
Table 23, several studies, such as [
19,
21], employed a 90%/10% training/testing split, consistent with the evaluation procedure in this work. Among them, the proposed model attained the highest accuracy on the GitHub dataset (99.6%). This consistency further validates the model’s competitive performance under standardized experimental conditions.
Table 22 presents the added f1-scores to provide a fairer comparison of model performance. However, not all referenced studies reported f1-scores; specifically, papers [
14,
15,
18,
20,
22,
24,
25,
27] did not include these values. For studies that reported precision together with recall or provided a confusion matrix (e.g., [
20,
22,
25]), we computed the corresponding f1-scores using Equation (7), as presented in
Table 22. It is worth noting that the issue of determining which class is treated as the positive class arises in binary classification. In this case, the f1-score can vary considerably depending on whether the majority or minority class is considered positive. For example, in our experiments using the PhysioNet/CinC Challenge 2016 database, when the majority (normal) class was treated as positive, the precision and recall were 93.99% and 96.48%, respectively, yielding an f1-score of 95.24%. Conversely, when the minority (abnormal) class was treated as positive, the precision and recall were 77.05% and 85.45%, resulting in an f1-score of 81.25%.
In this study, we report the f1-score based on the minority (abnormal) class, with an average f1-score of 84.94% obtained from 10-fold cross-validation. Since most previous works did not specify which class was considered positive, we calculated their f1-scores from their published confusion matrices under the same assumption (i.e., the abnormal class as positive) to ensure a fair comparison. The blue entries in
Table 22 indicate the f1-scores we derived from the corresponding confusion matrices using this consistent evaluation criterion. For multi-class classification, such as the GitHub PCG database, each class is treated as the positive class in turn, and the final f1-score is obtained by averaging across all classes (macro-averaging). Based on the results in
Table 22, our model achieved an f1-score of 99.5% on the GitHub PCG database and 84.94% on the PhysioNet/CinC Challenge 2016 database. Compared with other studies using the same datasets, our proposed model demonstrates comparable or superior performance on both datasets. Specifically, on the GitHub PCG database, our model’s f1-score is similar to those of Khan et al. [
17] (99.4%) and Yaseen et al. [
10] (99.7%), while achieving these results with significantly fewer parameters. On the PhysioNet/CinC Challenge 2016 database, our model outperforms most previous approaches such as Shuvo et al. [
26] (63.29%), Singh et al. [
22] (78.68%), and Wang et al. [
30] (81.83%), highlighting its superior robustness to noise and data variability. In summary,
Table 22 and
Table 23 jointly support a detailed cross-study evaluation, illustrating that the proposed model achieves strong accuracy with minimal resource usage, while maintaining robust performance across datasets with differing levels of noise and class balance.
4.5. Analysis of Signal Quality and Misclassification Patterns
To systematically investigate the impact of noise on feature extraction and classification performance, a quantitative signal quality analysis was conducted. To ensure consistency in frequency resolution, all PCG recordings were resampled to 2000 Hz before frequency analysis. Based on the proposed analysis method, the GitHub PCG dataset showed a high average BER of 707.07, indicating clean and consistent signal quality. In contrast, the PhysioNet/CinC Challenge 2016 dataset exhibited a much lower average BER of 24.96, confirming greater noise contamination. Further analysis revealed that samples not labeled as “unsure” had a slightly higher average BER of 27.85, while the “unsure”-labeled samples showed an extremely low average BER of 2.06, suggesting minimal heart sound energy and severe noise interference. To further validate the correlation between the BER and signal quality, representative samples with varying BER values from both datasets were selected for visual analysis. As shown in
Figure 15,
Figure 16 and
Figure 17, each figure consists of three rows, where the top row presents the raw time-domain signal, the middle row displays the signal filtered within the target frequency band (20–700 Hz), and the bottom row illustrates the noise-band components (0–20 Hz and 700–1000 Hz), which are considered non-informative for heart sound analysis.
Figure 15 presents two samples from the GitHub PCG dataset, with BER values of 228.45 and 2442.15 in
Figure 15a and
Figure 15b, respectively. Both samples exhibit clear periodic patterns and distinct S1/S2 heart sound structures. Even the sample with the lower BER (228.45) still represents a high-quality and easily identifiable signal.
Figure 16 shows two samples from the PhysioNet/CinC Challenge 2016 database that are not labeled as “unsure.” The sample in
Figure 16a has a BER of 0.0878, and its time-domain waveform and frequency spectrum reveal no recognizable heart sound features, reflecting a highly noisy signal. In contrast,
Figure 16b (BER = 128.72) displays clear periodicity and concentrated energy in the relevant frequency bands, indicating better signal quality.
Figure 17 illustrates two samples labeled as “unsure” from the PhysioNet/CinC Challenge 2016 database. The sample in
Figure 17a has a BER of 0.0001 and shows virtually no observable heart sound structure, qualifying it as an extreme noise-dominated and unclassifiable example. Although the sample in
Figure 17b has a slightly higher BER of 66.37, and exhibits some concentrated frequency energy, it still lacks sufficient periodic structure and remains unclassifiable.
Beyond visual inspection, further analysis was conducted to investigate how signal quality, as measured by the BER, relates to actual model performance, particularly in terms of classification errors. Classification performance across different categories was examined through confusion matrix analysis and error-type statistics. In the multi-class classification task (AS, MR, MS, MVP, N), most samples were correctly classified, with true positive counts generally reaching 19 or 20 per class (95–100%). Misclassifications—whether false positives or false negatives—were rare and typically limited to isolated instances, indicating that the model performs robustly in classes with well-defined structures and consistent features.
In the binary classification task (Normal vs. Abnormal), the confusion matrix showed strong recognition for Normal cases (TP = 219, 96.5%), while Abnormal cases had a slightly higher number of false negatives (FN = 14, 22.9%), suggesting that certain abnormal heart sounds were incorrectly identified as normal. To further investigate these errors, BER values were analyzed in relation to classification results. Among samples not labeled as “unsure” in the PhysioNet/CinC Challenge 2016 database, the average BER was 27.85. However, misclassified samples had a notably lower average BER of 19.72, compared to 27.91 for correctly classified samples. This indicates that misclassified signals were generally of lower quality, with greater noise interference and weaker heart sound structures. These findings are consistent with the known characteristics of the dataset and emphasize that environmental noise not only degrades signal quality but also significantly impacts the model’s ability to detect abnormal heart sounds, making it a key factor contributing to classification errors.
The proposed BER method effectively evaluates the signal-to-noise ratio of PCG recordings. As shown in
Figure 15,
Figure 16 and
Figure 17, the GitHub PCG dataset exhibits cleaner signals with higher BER values, whereas the PhysioNet/CinC Challenge 2016 dataset contains more noise and thus lower BERs. Even without applying noise reduction preprocessing, the proposed model achieved 90.74% accuracy on the noisier PhysioNet dataset, demonstrating strong noise resilience. Overall, these results confirm that signal quality—quantified by BER—has a decisive impact on feature extraction and classification accuracy. The performance gap between the two datasets mainly stems from differences in data quality, validating BER as a reliable indicator of signal integrity and model robustness.