Development and Utility of Practical Indicators of Critical Outcomes in Dengue Patients Presenting to Hospital: A Retrospective Cross-Sectional Study

Global travel and climate change have drastically increased the number of countries with endemic or epidemic dengue. The largest dengue outbreak in Taiwan, with 43,419 cases and 228 deaths, occurred in 2015. Practical and cost-effective tools for early prediction of clinical outcomes in dengue patients, especially the elderly, are limited. This study identified the clinical profile and prognostic indicators of critical outcomes in dengue patients on the basis of clinical parameters and comorbidities. A retrospective cross-sectional study was conducted in a tertiary hospital from 1 July 2015 to 30 November 2015. Patients diagnosed with dengue were enrolled, and the initial clinical presentations, diagnostic laboratory data, details of the underlying comorbidities, and initial management recommendations based on 2009 World Health Organization (WHO) guidelines were used to evaluate prognostic indicators of critical outcomes in dengue patients. Dengue patients from another regional hospital were used to evaluate accuracy. A group B (4 points) classification, temperature < 38.5 °C (1 point), lower diastolic blood pressure (1 point), prolonged activated partial thromboplastin time (aPTT) (2 points), and elevated liver enzymes (1 point) were included in the scoring system. The area under the receiver operating characteristic curve of the clinical model was 0.933 (95% confidence interval [CI]: 0.905–0.960). The tool had good predictive value and clinical applicability for identifying patients with critical outcomes.


Introduction
Dengue is the most rapidly spreading mosquito-borne viral disease in the world, with approximately 50 million dengue infections occurring annually and 4 billion people at risk of dengue [1]. The number of countries with endemic or epidemic dengue has increased drastically in recent years as a consequence of the spread of dengue viruses due to frequent global travel and climate change [2]. There have been cases of dengue fever in southern Taiwan since 1987 [3]. After a decade with few dengue cases, the largest outbreak occurred in 2015. This outbreak, caused by dengue serotype 2, led to a total of 43,419 cases and 228 deaths [4]. The highest prevalence rate was observed among patients aged over 70 years [5], which was in contrast to the predominantly younger age of dengue patients in other Southeast Asian countries [6][7][8].
Dengue is a dynamic systemic disease with a wide spectrum of clinical presentations. Most patients recover after a self-limiting nonsevere clinical course; however, a small proportion of infections progress to severe disease, mostly characterized by plasma leakage with or without hemorrhage [9]. Early recognition and appropriate treatment of dengue may prevent patients from developing more severe clinical conditions and reduce the case fatality rate [10]. To help clinicians decide when a patient should be monitored and how intensively a patient should be treated, the WHO released the 2009 classification system, which divides patients with symptomatic dengue virus infections into patients with or without warning signs and patients with severe dengue. Accordingly, case management focuses on three categories (groups A, B, and C). However, as indicated in the management recommendations, even dengue patients without warning signs who are discharged from the hospital can develop critical outcomes. The sensitivity of the WHO 2009 classification criteria for identifying patients who need intensive care at the first presentation has been shown to be 52% [11]. The sensitivity is even lower in older patients, resulting in substantial challenges regarding the achievement of an early clinical diagnosis in older patients [12]. Management at the primary and secondary care levels, where patients are first seen and evaluated, is critical for determining the clinical outcome of dengue. A well-managed frontline response not only reduces the number of unnecessary hospital admissions but also saves the lives of dengue patients. Simple but effective tools can help identify those at risk of developing critical outcomes who need intensive care.
Information on the practicality and cost-effectiveness of tools for the early prediction of critical outcomes in dengue patients, especially older patients, has been limited. Previous studies have identified the risk factors associated with intensive care unit (ICU) admission or mortality due to dengue [11,[13][14][15][16]. Most of these studies were limited by small sample sizes. A number of identified risk factors for critical outcomes in dengue patients only develop as the disease progresses to the critical phase and are not observable in the early stage. In addition, patients with critical dengue experience a cytokine storm with high levels of circulating proinflammatory cytokines, leading to endothelial activation and vascular leakage with hemorrhage and shock [17,18]. These cytokine levels, however, can only be measured in research laboratories and not in clinical settings.
Recognizing the importance of the early identification of dengue patients at risk for critical outcomes, we developed and externally evaluated a simple scoring tool that can be easily, inexpensively, and usefully applied in clinical settings to identify dengue patients in the early phase of the disease who are at greater risk of critical outcomes.

Study Design and Setting
According to the Communicable Disease Control Act in Taiwan, dengue fever is classified as a category 2 notifiable communicable disease that should be reported to the Taiwan Center for Disease Control (CDC) within 24 h. Confirmed dengue patients need to meet the clinical (any of the two inclusive symptoms including headache/back pain/muscle pain/arthralgia/bone headache, rash, leukopenia, nausea/vomiting, positive tourniquet test, or any of the warning signs), laboratory (positive dengue virus isolated from blood, positive molecular nucleotides from clinical specimens, positive nonstructural protein (NS1) in serology, positive IgM or IgG in serology in the acute dengue stage, positive IgM or IgG conversion or fourfold elevation in paired serology), and epidemiological (positive dengue patients are nearby or travel history to dengue-affected areas) criteria [19,20].
This retrospective cross-sectional study was conducted using hospital-based clinical data from 1 July 2015 to 30 November 2015 associated with the dengue outbreak of 2015. Clinical data on dengue adult patients (≥18 years) were collected from dengue cases reported to Taiwan CDC in National Cheng Kung University Hospital, a 1000-bed tertiary care teaching and major dengue-designated treatment hospital in Tainan, southern Taiwan. All the enrolled patients for analysis included those with dengue diagnosis and presentation that met the Taiwan CDC's notifiable infectious disease criteria. Patients visiting either the emergency department (ED) or outpatient department (OPD) were reviewed.
The clinical information included initial clinical symptoms, temperature, respiratory rate, pulse, and blood pressure, and the laboratory data we used were either in the initial blood sample taken within 2 h after the ED triage or during the OPD visit. The diagnosis of dengue was confirmed by serum NS1 detection (Standard Diagnostics, Inc., Kyonggi-do, Republic of Korea) [21], enzyme-linked immunosorbent assay (ELISA)-based detection of specific IgM antibodies, or real-time polymerase chain reaction (RT-PCR) (Lightmix ® kit, TIB MOLBIOL GmbH, Berlin, Germany) based on the Taiwan CDC's criteria. Patient data, including demographic characteristics, triage parameters, vital signs, clinical symptoms and signs, laboratory data, diagnoses, and outcomes (i.e., critical or noncritical status), were collected from electronic medical records. Symptoms such as abdominal pain and vomiting several times in one day or that lasted for more than one day were recorded in the medical records. We analyzed the information that was presented in the first three days for further analysis.
Adult dengue patients diagnosed in another major designated dengue treatment hospital outfitted with 400 beds, 20 of which were ICU beds, in another dengue-endemic region during 2011-2015 were used for external utility evaluation of the scoring system. All criteria used for the external utility evaluation were the same as those used in the study hospital. Figure 1 shows the flowchart of the study. A total of 4229 symptomatic laboratoryconfirmed dengue patients were enrolled initially. However, 3528 dengue patients were excluded from further analysis, including 270 dengue patients initially diagnosed with severe dengue and 3258 dengue patients who lacked complete vital signs and clinical data, e.g., activated partial thromboplastin time (aPTT) and alanine transaminase (ALT) and aspartate transaminase (AST) or platelet levels. A total of 701 dengue patients were classified on the basis of presentation at the first hospital visit as groups A (may be sent home) and B (referred for in-hospital care) according to the 2009 WHO criteria. Group A was defined as patients who did not have warning signs and who were able to tolerate adequate volumes of oral fluids and pass urine at least once every 6 h. Group B was defined as patients with any of the following features: (1) coexisting conditions, such as pregnancy, infancy, old age, diabetes mellitus (DM), renal failure, or chronic hemolytic diseases; (2) social circumstances, such as living alone or living far from the hospital; or (3) existing warning signs (abdominal pain or tenderness, persistent vomiting, clinical fluid accumulation, mucosal bleeding, lethargy/restlessness, liver enlargement (>2 cm), or elevated hematocrit level). Group C was defined as patients with severe plasma leakage with shock and/or fluid accumulation with respiratory distress, severe bleeding, or severe organ impairment. Considering the population and comorbidity distributions in Taiwan, the following underlying comorbidities and conditions were selected as important parameters for further model analysis: characteristics such as age 65 years or older; conditions such as DM, chronic kidney disease (CKD), or end-stage renal disease (ESRD), liver cirrhosis, chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), and neoplasms [22]; and clinical symptoms such as high or low temperature, tachypnea, tachycardia, and bleeding tendency as reflected in a prolonged aPTT, PT, and severe thrombocytopenia. Shock was defined as a systolic pressure <90 mmHg with a diastolic blood pressure (DBP) <60 mmHg and organ dysfunction. Tachycardia was defined as a heart rate >120/min. Tachypnea was defined as a respiratory rate >20/min. High fever was defined as a tympanic temperature ≥38.5 °C. Severe thrombocytopenia was defined as a platelet count <100,000/µL. A prolonged aPTT was defined as international normalized ratio (INR) ≥1.2 and ≥50 s. Acute elevated liver enzymes were defined as an AST or ALT level ≥200 IU/mL. The International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) ICD-10 codes used for chronic diseases were as follows: DM-E08-E10; asthma or COPD-J41, J43, J44, J45, J47; malignant neoplasms-C00-C90, D00-D49; CKD or ESRD-N059, N18; and chronic liver disease and cirrhosis-K740, K741, K742, K743, K744, K746, and CHF-I50. Dengue patients with critical outcomes were defined as those who progressed to death regardless of the presence of any warning signs or underlying diseases, and severe dengue was reported and confirmed by the Taiwan CDC. Severe dengue was defined by any of the following symptoms during the disease course: severe plasma leakage leading to shock or fluid accumulation with respiratory distress; severe bleeding; or Considering the population and comorbidity distributions in Taiwan, the following underlying comorbidities and conditions were selected as important parameters for further model analysis: characteristics such as age 65 years or older; conditions such as DM, chronic kidney disease (CKD), or end-stage renal disease (ESRD), liver cirrhosis, chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), and neoplasms [22]; and clinical symptoms such as high or low temperature, tachypnea, tachycardia, and bleeding tendency as reflected in a prolonged aPTT, PT, and severe thrombocytopenia. Shock was defined as a systolic pressure < 90 mmHg with a diastolic blood pressure (DBP) < 60 mmHg and organ dysfunction. Tachycardia was defined as a heart rate > 120/min. Tachypnea was defined as a respiratory rate > 20/min. High fever was defined as a tympanic temperature ≥ 38.5 • C. Severe thrombocytopenia was defined as a platelet count < 100,000/µL. A prolonged aPTT was defined as international normalized ratio (INR) ≥ 1.2 and ≥50 s. Acute elevated liver enzymes were defined as an AST or ALT level ≥ 200 IU/mL. The International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) ICD-10 codes used for chronic diseases were as follows: DM-E08-E10; asthma or COPD-J41, J43, J44, J45, J47; malignant neoplasms-C00-C90, D00-D49; CKD or ESRD-N059, N18; and chronic liver disease and cirrhosis-K740, K741, K742, K743, K744, K746, and CHF-I50. Dengue patients with critical outcomes were defined as those who progressed to death regardless of the presence of any warning signs or underlying diseases, and severe dengue was reported and confirmed by the Taiwan CDC. Severe dengue was defined by any of the following symptoms during the disease course: severe plasma leakage leading to shock or fluid accumulation with respiratory distress; severe bleeding; or severe organ impairment such as elevated transaminase levels ≥ 1000 IU/L, impaired consciousness, or heart impairment. Critical outcomes were determined by a consensus committee organized by the Taiwan CDC that included infectious disease and critical care specialists. The clinical epidemiologist analyzed the anonymous data and was unaware of the data collection process.

Model Analysis for the Dengue Severity Score
An efficient scoring system was created to identify critical outcomes in dengue patients based on basic clinical and laboratory features, including only those variables that are relevant and readily available when patients arrive at most hospitals. Previous literature reviews determined potential factors, e.g., vital signs and important laboratory data. Later, candidate factors were selected by multivariable binary logistic regression. The significant coefficients of parameters were transformed into a dengue severity score by inverse odds ratios [23]; this score was constructed by dividing each coefficient with the smallest coefficient (0.49) of the multivariate logistic regression models with backward stepwise selection. The predictor with the smallest coefficient is intuitive, considering the 1-point score mathematically presented. The severity scores were rounded up or down to the nearest 0.5.

Data Analysis
Potential indicators of critical outcomes in dengue patients were tested with a nonparametric test for trend. The Pearson's chi-square test, the Mantel-Haenszel chi-square test, and logistic regression models were used to compare categorical and continuous variables between groups, yielding odds ratios (ORs) with 95% confidence intervals (CIs). The Mann-Whitney U test was used to compare continuous variables when the variable was either ordinal or continuous but not normally distributed. For all tests, a two-sided p ≤ 0.05 was considered statistically significant. Clinical vital signs, important and common laboratory data, and previously mentioned important outcome factors, such as thrombocytopenia (<100,000/µL), aPTT, were enrolled for the primary factor survey [24,25]. The predictive ability was analyzed by multivariable binary logistic regression and is presented as the coefficients and ORs. Potential factors for determining severity scores were derived by transforming the coefficients of the parameters and used to estimate the strength of the association [20,22,[26][27][28]. Moreover, scores for predicting critical outcomes in dengue patients were calculated with receiver operating characteristic curve (ROC) analysis.
The optimal cutoff value was obtained by ROC curve analysis, and its sensitivity and specificity for predicting critical outcomes in dengue patients were measured. The area under the curve (AUC) defined how well the model could discriminate between critical and noncritical patients with dengue. In addition, dengue patients were dichotomized by the cutoff value obtained by generating an ROC curve and maximizing the Youden index (Youden index: J = max [sensitivity + specificity] − 1) [29]. The essential indicators, sensitivity, specificity, and likelihood ratios were analyzed to measure the diagnostic testing accuracy. SAS (Statistical Analysis Software 9.4, SAS Institute Inc., Cary, NC, USA) was used for all the data analyses.

Ethics Statement
The present study was approved by the Research Ethics Committee of the study medical centers. All the study protocols were performed in accordance with the principles of the Helsinki Declaration. Both the study protocol and the data were approved by the Institutional Review Board (IRB), National Health Research Institutes (EC1060705-E), National Cheng Kung University Hospital (A-ER-104-223) and Kaohsiung Municipal Siaogang Hospital (KMUHIRB-E(1)20200052). To protect personal privacy, patient information was collected and anonymized, and the patients were de-identified prior to analysis; therefore, the requirement for informed consent was waived by the IRB.

Results
During the dengue epidemic period, a total of 4229 dengue-confirmed adult patients visited the ED and OPD of the study hospital and were included in this study (Table S1). Table 1 demonstrates the significant differences in demographics between the dengue patients from the ED and OPD. Among the ED patients with confirmed dengue, 33.2% were elderly adults aged ≥65 years (Range: 18-103 years). Among the enrolled dengue patients with underlying comorbidities, the most common underlying diseases were DM (83, 12.4% in the ED; 8, 18.8% in the OPD), followed by neoplasms (60, 7.5% in the ED; 3, 9.3% in the OPD), CKD/ESRD (42, 6.4% in the ED; 5, 15.6% in the OPD), and liver cirrhosis (4, 0.6% in the ED; 0, 0% in the OPD). Compared with the patients who visited the OPD, patients who presented at the ED were relatively older and had a higher likelihood of comorbidities. A total of 4229 dengue patients were classified based on presentations at the first hospital visit as group A (may be sent home) of 1396 dengue patients and group B (referred for in-hospital care) of 2833 dengue patients according to the 2009 WHO criteria.

Major Parameters for Predicting Critical Outcomes in Dengue Patients
The dengue patients' clinical conditions based on the WHO classification system are shown in Figure 1. Of the 4229 dengue patients, 3528 were excluded from prognostic tool development, 270 were initially diagnosed with severe dengue, and 3258 lacked complete vital signs and clinical data. The median duration of illness for these included and excluded participants visiting ED or OPD was one day (Table S2). Out of the 701 study subjects, 263 were diagnosed without warning signs and 438 were diagnosed with warning signs. We identified these patients according to the WHO management recommendations: 156 patients were classified into group A and 545 were classified into group B. In the course of treatment for dengue, 47 patients were later confirmed to have critical outcomes (Figure 1). A total of 701 cases with complete clinical and laboratory data were analyzed ( Table 1).
The results of the chi-square tests or Fisher's exact tests for the comparisons of demographic variables, clinical features, and laboratory data between the patients with and without critical outcomes in the study hospital who had complete clinical information are summarized in Table 2. The highest OR was 76.92 (95% CI: 18.45-322.58) for patients aged 65 years and older who had any warning signs (nausea, vomiting, hematuria, epistaxis, tarry stool, vaginal bleeding, abdominal pain, weakness, and poor appetite) or underlying diseases associated with poor dengue outcomes, including DM, CKD, ESRD, liver cirrhosis, COPD, CHF, and neoplasms [22,30,31]. Multiple parameters were integrated and analyzed by multivariate logistic regression analysis with backward selection, and the results are shown in Table 3. The smallest coefficient of the multivariate logistic regression models was 0.49 for patients with tachycardia. The estimated severity scores ranged from 1 to 4, and the sum of the total scores was 9. Significant parameters were used to construct possible scoring models. On the basis of the ORs and 95% CIs, dengue with critical outcomes was associated with patients with underlying disease (OR = 60.23, 95% CI: 14.11-257.05), a tympanic temperature lower than 38.5 • C (OR =2.68, 95% CI: 1.18-6.08), lower DBP (OR = 3.14, 95% CI: 1.01-9.76), aPTT ≥ 50 s (aOR = 5.76, 95% CI: 2.30-14.42), and elevated liver enzymes (OR = 3.16, 95% CI: 1.10-9.07). The sensitivity, specificity, positive likelihood ratio (LR+), and negative likelihood ratio (LR−) from the ROC curve analysis are shown in Table 4. The sensitivities for predicting critical outcomes in dengue patients with scores of ≥1 and ≥9 were 100.00% and 4.26%, respectively. The LR+ values of 21.28, 23.19, and 97.40 for scores of 6, 7, and 8 show that the identification of patients with critical outcomes with those scores is robust. Alternatively, the LR− values of 0.03, 0.06, and 0.06 for scores of 2, 3, and 4 indicate that the ability of those scores to rule out dengue patients with critical outcomes is excellent. ROC curve analysis was used to evaluate model performance ( Figure 2). The AUC is equivalent to the probability that a randomly selected dengue patient with critical outcomes has a higher score than a randomly selected dengue patient without critical outcomes. This prognostic tool showed an AUC of 0.933 (95% CI: 0.905-0.960). The values of Youden's index (sensitivity + specificity − 1) were also calculated to facilitate the selection of an appropriate cutoff value. The optimal probability cutoff is the maximum value of Youden's index. The data show that the highest Youden's index value of 0.725 was obtained with a cutoff score of 4. Overall, these findings confirmed the excellent performance of this model with regard to the identification of patients developing critical outcomes based on this dataset.
ROC curve analysis was used to evaluate model performance (Figure 2). The AUC is equivalent to the probability that a randomly selected dengue patient with critical outcomes has a higher score than a randomly selected dengue patient without critical outcomes. This prognostic tool showed an AUC of 0.933 (95% CI: 0.905-0.960). The values of Youden's index (sensitivity + specificity −1) were also calculated to facilitate the selection of an appropriate cutoff value. The optimal probability cutoff is the maximum value of Youden's index. The data show that the highest Youden's index value of 0.725 was obtained with a cutoff score of 4. Overall, these findings confirmed the excellent performance of this model with regard to the identification of patients developing critical outcomes based on this dataset.

Utility Evaluation of the Indicators from the External Data
To further evaluate the utility of the indicators for identifying critical outcomes in dengue patients who presented dengue-associated symptoms/signs in the first three days, with the best cutoff value of 4, we enrolled 787 patients from another designated dengue hospital in Kaohsiung. The enrolled cases included dengue patients diagnosed from 2011 to 2015. A total of 65 out of 787 patients had critical outcome scores ≥4 ( Table 5). The results

Utility Evaluation of the Indicators from the External Data
To further evaluate the utility of the indicators for identifying critical outcomes in dengue patients who presented dengue-associated symptoms/signs in the first three days, with the best cutoff value of 4, we enrolled 787 patients from another designated dengue hospital in Kaohsiung. The enrolled cases included dengue patients diagnosed from 2011 to 2015. A total of 65 out of 787 patients had critical outcome scores ≥4 ( Table 5). The results for the dengue patients showed a high sensitivity (100%) and acceptable specificity (69%). The AUC of the ROC curve of 0.8 (0.768-0.832) with a Youden's index value of 0.693 was within the acceptable range. Table 5. Accuracy evaluation of the dengue critical scores from the external hospital.

Discussion Estimation
In this study, we developed a new scoring system to predict critical outcomes in dengue patients who presented dengue-associated symptoms/signs in the first three days by integrating clinical presentations; age; chronic comorbidities, such as DM, CKD, chronic heart failure, and neoplasms; and abnormal laboratory findings. Given the importance of early identification and treatment of dengue with critical outcomes, the scoring algorithm proposed in this study can be considered clinically useful, especially during dengue outbreaks.
The surge in dengue patients resulted in an overload of the healthcare burden, which reveals the difficult problems inherent in early diagnosis of severe dengue and hospital beds allocation in the healthcare system. Our prediction tool provides primary physicians with a method of identifying patients likely to progress to dengue with critical outcomes early in the clinical course. Users can rapidly determine which patients with dengue infections may require further monitoring or intensive care simply using their age, warning signs, vital signs, and simple laboratory data, including aPTT and liver function results, to calculate their score. Dengue patients with severe disease can be treated successfully if they are diagnosed very early [10]. The development of a prognostic tool to detect the likelihood of progression to dengue with critical outcomes is warranted to reduce the morbidity and mortality rates associated with dengue [32]. The use of this scoring system could decrease unnecessary blood tests and reduce medical expenses in clinical settings. Early, rapid, and affordable prediction tools for dengue with critical outcomes may also limit unnecessary hospital admission, enable appropriate hospital bed allocation, and in turn decrease the burden on healthcare systems during dengue outbreaks. The present scoring system had a relatively low specificity for utility evaluation in the external data. Further studies that enroll more dengue patients are warranted to improve the performance of the scoring system.
This study generated regression models with backward selection to integrate the 2009 WHO dengue case classification system with laboratory data [10]. For a given cutoff value, a critical or noncritical classification can be made for each individual by comparing the measurement to the cutoff value, as was shown previously in a multiple regression model that was constructed and validated for the clinical diagnosis of dengue in the pediatric population [33]. In addition to sensitivity and specificity, our study provided another statistical tool, likelihood ratios (LRs), to understand diagnostic tests in clinical practice and to determine how much the utilization of a particular test will alter the probability. A positive likelihood ratio, or LR+, is the ratio between the probability of true positives and the probability of false positives. For example, an LR+ of 97.4 for a score of 8 means that the likelihood of dengue patients developing critical outcomes, known to be positive, is 97.4 compared to those who will not develop critical outcomes.
The AUC indicates the accuracy of the scoring system. The AUC shows how well the model can discriminate between critical and noncritical patients; AUCs between 0.90 and 1 indicate excellent performance. The AUC of 0.933 in our study suggests that the score we used in our study should be an effective measure of accuracy to discriminate dengue patients with critical and noncritical outcomes. The optimal Youden's index value (0.725) is more intuitive for interpretation in the clinical setting than the metrics provided in previous studies. The cutoff value of 4 yielded the highest sensitivity (95.7%) and specificity (76.8%), with an LR+ of 4.1, which showed that the score had a strong discriminatory ability for predicting critical outcomes in dengue patients.
Compared to other predictive scores for critical dengue, this scoring system has several advantages. First, our data included the initial clinical presentations, inexpensive diagnostic laboratory data, and details of the underlying comorbidities. The broad scope of the factors included in this scoring system makes it better able than the well-known WHO classification system to predict dengue with critical outcomes. Although the WHO classification system considers underlying diseases and symptoms and suggests different management strategies, it does not mention detailed definitions of the underlying diseases. Second, our scoring system was developed on the basis of a population that included patients with mild cases who visited the OPD, patients with moderate cases who needed advanced care, and patients with severe cases who were admitted to ICUs. Approximately one-fifth of the total number of patients with dengue and half of the patients with severe dengue during this outbreak in Tainan city were enrolled in our study [22]. The broad coverage of the study population suggests that it is representative of the target population. Third, our study recruited a wide spectrum of patients with different risk factors for critical dengue. Approximately 40% of the enrolled population was of advanced age. These patients had relatively more comorbidities, such as diabetes, CKD, heart diseases, malignancies, and COPD, and the distributions of these comorbidities were similar to the comorbidity profiles in many middle-and high-income countries. Therefore, this scoring system should be generalizable for general use in most dengue-endemic or dengue-epidemic areas. The complexity of patients' underlying conditions highlights the importance of the sensitivity of the scoring system for predicting the development of dengue with critical outcomes in this population. Early recognition of unstable vital signs, such as lower temperature, insufficient intravascular volume (low diastolic BP), and comorbidities, enables healthcare providers to promote the initiation of advanced care plans to improve clinical outcomes.
There are several limitations that need to be addressed. First, although nearly 80% of the screened and eligible patients were excluded due to missing data, the patients with missing data had mild clinical symptoms initially compared to the enrolled patients (Table S2). Excluding the patients with missing data would not interfere with the prognostic indicator analysis. The study hospital is designated as one of the major medical care centers for dengue fever by the Ministry of Health and Welfare, Taiwan. Nearly all dengue and severe dengue patients were included for analysis in our study during the study period. The study group should have enough representatives, and an evidence-based evaluation of the results should not be affected by missing data. Furthermore, the dengue patients diagnosed (regardless of whether they were admitted) at this hospital received OPD followups in the same hospital. Dengue patients from other hospitals with critical conditions were also referred to the study hospital for further treatment. A retrospective cross-sectional study design was adopted that only measured the correlation of various indicators rather than causality. As a result, outcome misclassification was minimal. Second, the enrolled patients visited healthcare facilities in the first three days after disease onset. The initial presentations may have lacked sufficient information to predict critical outcomes in dengue patients. For these patients, optimal supportive care and follow-up were required. Our scoring system provided guidance regarding the need for monitoring. If a patient has a high score, admission to the ICU should be arranged immediately. Third, the majority of the patients included in the development and usefulness evaluation were infected with dengue virus (DENV) serotype 2 [34]. Whether this scoring system would be equally effective for patients infected or reinfected with other subtypes of dengue viruses, such as DENV3 and DENV4, needs to be confirmed. Fourth, the aim of this study was early identification of dengue patients at risk for critical outcomes. Our data only included clinical presentations in the first three days. It is warranted to evaluate whether the scoring system can be used for any day after the onset of the symptoms. Fifth, we could not differentiate whether the dengue patients had a primary or secondary infection because the serial paired characterization of types of antibody response is not routinely performed during the outbreak in Taiwan. However, few cases were infected in Tainan city with dengue infection before the severe epidemics in 2015, and thus the patients in this study were more likely to have primary infections [30,31]. Finally, the scoring system was developed in Taiwan, a country with universal healthcare and convenient access to healthcare facilities. Whether the scoring system is applicable to patients from different dengue-endemic countries still needs confirmation.

Conclusions
In conclusion, this scoring system based on clinical characteristics and inexpensive laboratory test results has good accuracy for predicting critical outcomes in dengue patients, with an AUC of 0.933. Our model had good predictive ability and was externally validated. More data are needed for reliability and validation testing.
Supplementary Materials: The following supporting information can be downloaded at: https:// www.mdpi.com/article/10.3390/tropicalmed8040188/s1. Table S1. Demographic characteristics of dengue patients in a teaching hospital during the 2015 dengue outbreak in Tainan, Taiwan; Table  S2. Comparison of the demographic characteristics of included and excluded dengue patients in a teaching hospital during the 2015 dengue outbreak in Tainan, Taiwan.