4. Results and Discussion
Table 3 presents the results obtained by the DT and RF models using the dataset transformed by the methodology proposed in [
22] and balancing of the target variable using the bootstrapping technique.
The results of both models were balanced for all quality metrics, allowing for highly accurate prediction of the three diseases in the dataset. However, the RF model performs better overall.
Figure 3 shows a comparison of the results obtained using the DT and RF models.
The confusion matrix in
Figure 4 reveals high overall performance in classifying chikungunya, dengue, and Zika using the DT model. The model classifies chikungunya with an accuracy of 88.5%, presenting a very low false-negative rate (0.5%) and no false positives. Dengue had an accuracy of 86.7%, with false negatives (1.7%) and false positives (0.6%). Although Zika showed a slightly lower accuracy of 81.9%, it was still high, with a false negative rate of 4.1% and false positives of 3.0%. Overall, the model was efficient, although it exhibited slight confusion in classifying Zika.
The confusion matrix shown in
Figure 5 reveals that the Random Forest technique offers robust performance in classifying Chikungunya, Dengue, and Zika. The model achieved an accuracy of 88.5% for chikungunya, with no false positives and a very low false-negative rate of 0.5%. For Dengue, the accuracy was 88.0%, with a false-negative rate of 1.0% and no false positives. For Zika, the accuracy was 87.3%, with a false negative rate of 0.4% and false positives of 1.4%. Overall, the model demonstrated high classification ability, although it exhibited slight confusion in identifying Zika compared to the other two classes.
Table 4 shows the results of the models obtained by working with the dataset without applying the methodology proposed by Arrubla et al. [
22] but balanced using the bootstrapping technique.
The results show good performance of the DT model, with a balance in most metrics, although accuracy is the only measure that is below 90%, with 88.8%. However, Random Forest (RF) performs better in the classification of the three diseases, with a balance in all quality metrics, making it a better option to support the classification of these diseases.
Figure 6 presents a comparison of the performance of the two models.
Similarly,
Figure 7 summarises the behaviour of the ten models created using the cross-validation technique in the two experiments. It shows the behaviour of the accuracy and error in each model, highlighting that applying the methodology proposed by Arrubla et al. [
22] allows superior quality metrics to be obtained in the model.
The confusion matrix in
Figure 8 for the decision tree technique indicates mixed performance in classifying Chikungunya, Dengue, and Zika. The model classifies chikungunya with an accuracy of 88.2%, with no false positives and a false negative rate of 0.8%. However, for Dengue, the accuracy was 76.9%, with a remarkably high false-negative rate of 7.3% and a false-positive rate of 4.8%. Zika shows an accuracy of 72.0%, with a false-negative rate of 12.3% and a false-positive rate of 4.7%. Overall, although the decision tree model shows good accuracy for chikungunya, it faces difficulties in classifying Dengue and Zika, evidencing higher confusion between the classes.
The confusion matrix in
Figure 9 for the Random Forest model showed better performance in classifying chikungunya, dengue, and Zika. The model achieved an accuracy of 88.2% for chikungunya, with a false-positive rate of 0.0% and a false-negative rate of 0.8%. For dengue, the accuracy was 83.0%, with 3.7% false negatives and 2.3% false positives. For Zika, the accuracy was 81.4%, with a false-negative rate of 5.3% and a false-positive rate of 2.4%. Overall, the model demonstrated better classification ability with a good balance between classes, although it showed slight confusion in identifying Zika and dengue.
The application of the methodology proposed by Arrubla et al. [
22] significantly improved the performance of the Random Forest and Decision Tree classification models. For Random Forest, the use of the methodology results in a slight improvement in accuracy, especially in the reduction of false positives and negatives, with an accuracy of 88.5% for chikungunya, 88.0% for dengue, and 87.3% for Zika. In comparison, without the methodology, the accuracy was 88.2% for chikungunya, 83.0% for dengue, and 81.4% for Zika, showing a lower discrimination capacity between classes. For decision trees, the proposed methodology also had a positive impact, improving the overall accuracy to 88.5% for chikungunya, 86.7% for dengue, and 81.9% for Zika. Without the methodology, the accuracies were 88.2% for chikungunya, 76.9% for dengue, and 72.0% for Zika, indicating a notable reduction in classification capacity, especially for dengue and Zika.
While it is true that the DT model performs less well compared to RF in both experiments, it is important to mention that it may be more interpretable for the medical community when supporting early decision-making.
Figure 10 shows the model tree, in which the rules generated by the model to perform the respective classifications are shown.
Figure 10 illustrates that headache is the most relevant variable for classifying dengue, while myalgia is key for identifying chikungunya, aligning with PAHO’s 2022 guidelines on the differential symptoms of these diseases. The decision tree classifies cases between chikungunya, dengue, and Zika using symptoms such as headache, myalgia, days of symptoms, IgM, and platelet count. The root node shows that a mild or absent headache is linked to chikungunya, whereas a severe headache strongly indicates dengue. As the tree progresses, additional symptoms, such as myalgia and symptom duration, further refine the classification, with terminal nodes offering pure and definitive predictions for each disease, underscoring the clinical utility of these symptoms.
In contrast to the tree diagram,
Figure 11 shows the importance of the variables generated by the Random Forest (RF) model, where, as in the DT model, headache is the most important variable for classifying diseases, followed by myalgia and arthralgia. These results are in line with the guidelines given by PAHO, which consider these variables as differential in the three diseases.
To validate the predictive performance of the models for each label of the target variable,
Table 5 summarises the quality metrics obtained in the experiment.
From
Table 5, it can be inferred that the models have a high accuracy for all diseases. Specifically, the RF model with the methodology proposed in [
22] achieved the highest accuracy for chikungunya (99.7%), dengue (99.1%), and Zika (98.8%). These results indicate balanced performance across the four quality metrics evaluated, suggesting that the model can correctly predict all labels of the target variable.
Figure 12 also presents the results obtained for the 10 models tested using the DT algorithm and the cross-validation technique. In this figure, the performance measured in terms of accuracy and error is observed, showing that chikungunya is the disease that can be best predicted in all models. However, the accuracy for dengue and Zika in models that do not use the methodology proposed in [
22] presents greater difficulties and errors when recognising these classes. It is highlighted that, by using this methodology, an improvement in the prediction of dengue and Zika is achieved in all the models generated.
On the other hand, when reviewing the behaviour with the RF and cross-validation techniques in
Figure 13, a similar trend to that analysed above is evident, with the difference that its performance is superior in all quality metrics.
The results obtained in this study allow for a highly accurate classification of dengue, Zika, and chikungunya diseases, highlighting the relevance of certain variables in prediction. Consequently, a new experiment was carried out, in which only variables related to signs and symptoms were selected, excluding laboratory results that were not available in the early stages of the disease, as well as variables that were not significant in previous analyses. This new model proposal seeks to align with the medical reality, providing an approach that, based on data obtainable by the physician in the early stages of the disease, effectively supports decision-making in the classification of these pathologies.
Table 6 presents the variables that were selected to create the new model.
The training was carried out under the same conditions as the previous models using stratified cross-validation and the methodology proposed in [
22]. The results are presented in
Table 7.
The results presented in
Table 7 highlight the excellent performance of the RF technique, which achieved a balance of over 99% for all quality metrics. Similarly, the decision tree also showed a solid performance, with an average of 96% across all metrics. These results suggest that the developed models are highly effective and can be adapted for early disease detection, providing valuable support to the medical community for accurate triage of dengue, Zika, and chikungunya. This is especially useful in remote communities where the lack of experienced medical epidemiologists or specialists can make early disease triage difficult.
Table 8 shows the quality metrics of both models and highlights their ability to recognise chikungunya, dengue, and Zika diseases. Although the RF model has superior metrics, suggesting that it might be the preferred option in terms of pure performance, the Decision Tree offers very robust performance and clearer interpretability in the medical domain. This better interpretability makes it a potentially more useful tool in contexts where a detailed understanding of the model’s decisions is critical to support clinical decision-making.
Figure 14 illustrates the decision tree, highlighting that the most significant variable for classifying dengue was headache, followed by abdominal pain, retrocular pain, and arthralgia. These findings are in line with the PAHO guidelines, which identify these symptoms as differential signs, supported by scientific evidence. In addition, patient age emerged as a significant factor in the classification of dengue. For chikungunya, myalgia was observed as a key variable, which is in line with the PAHO indications. However, symptom duration, retroocular pain, and patient age were also identified as important factors in the classification of chikungunya. Finally, in the case of Zika, significant variables for classification include myalgia, abdominal pain, age, and arthralgia. It should be noted that, although relevant in this context, they are not mentioned as distinctive signs or symptoms of Zika in the PAHO 2022 guidelines.
However, the scarcity of specific research on the classification of diseases such as dengue, Zika, and chikungunya limits direct comparisons of results. However, a recent study [
42] addressed this challenge by developing a proposal to classify seven similar diseases, including 137 records of Zika, 127 of dengue, and 140 of chikungunya, in addition to other diseases such as malaria and yellow fever, totalling 1500 records. This proposal compares various algorithms and presents a hybrid technique called HML that combines machine learning techniques with reinforcement learning based on recurrent neural networks (RNNs). The results obtained showed high precision, with an accuracy of 98.7%, precision of 98.7%, recall of 98.4%, and an F1-score of 99.10%.
Despite these promising results, this research does not include confusion matrices that allow the evaluation of the reliability of the classification for each disease individually. When comparing these results with those of our research, it is observed that our models outperform the proposed quality metrics, particularly in terms of accuracy, precision, recall, and F1-score. Furthermore, our research provides a detailed analysis of the confusion matrix level for each class, allowing for a more accurate assessment of the classification capacity of each disease. This highlights not only the effectiveness of our models in differentiating between dengue, Zika, and chikungunya but also the advantage of having detailed metrics to assess and improve classification quality.
The results of this research support the feasibility of a model for early and differential prediction of dengue, Zika, and chikungunya based on signs and symptoms. This model showed high performance with an accuracy of 99.3%, precision of 99.8%, specificity of 99.9%, and F1-Score of 99.9%. Furthermore, its ability to accurately recognise each disease is remarkable, reaching 99.9% for chikungunya, 99.3% for dengue, and 99.3% for Zika.
The use of cross-validation in this study played a crucial role in providing a more accurate estimate of model performance. By employing multiple partitions of the dataset for training and validation, this technique reduces the risk of overfitting and improves the ability of the model to generalise to unseen data. In addition, using cross-validation, more stable and reliable metrics of model performance were obtained, allowing for a more accurate assessment of the model’s ability to predict these diseases.
Bootstrapping was used to balance the classes in model construction. This technique allowed us to work with the unbalanced dataset that made up the dataset, generating multiple samples of equal size to the original dataset and randomly selecting observations with replacements. By applying this technique, we were able to obtain an adequate representation of the training samples, which helped improve the model’s ability to learn, in a balanced way, the characteristics of each disease.
Finally, this study represents a significant advance in the differential prediction of dengue, Zika, and chikungunya using machine learning techniques and the analysis of signs, symptoms, and laboratory variables. The developed model offers robust diagnostic support based on the criteria established in PAHO evidence synthesis (2022), which clearly distinguishes the signs and symptoms of each disease for diagnosis and treatment. With high performance, this model not only demonstrates remarkable accuracy but also has great potential for implementation in clinical settings. Its integration into clinical practice would provide fundamental support to health professionals, facilitate early and accurate diagnoses, and favour timely decision-making that improves patient outcomes.
Moreover, the predictive model developed in this study could be particularly beneficial in regions where dengue, Zika, and chikungunya co-circulate, as early differentiation between these diseases is challenging owing to their similar initial symptoms. This tool could empower healthcare providers to make more informed and rapid decisions regarding patient management, ultimately leading to better care and outcomes.
Although this study presents some limitations regarding the amount of data, especially for chikungunya, which was addressed by specialised computational techniques, it is recognised that the reliability of the model could be improved with a larger volume of data. Despite these limitations, this study establishes a benchmark for future research, since, according to [
20,
21], no comparable studies have been identified in the literature, mainly because of the scarcity of datasets that include records of these viruses.