Rapid Triage of Children with Suspected COVID-19 Using Laboratory-Based Machine-Learning Algorithms

In order to limit the spread of the novel betacoronavirus (SARS-CoV-2), it is necessary to detect positive cases as soon as possible and isolate them. For this purpose, machine-learning algorithms, as a field of artificial intelligence, have been recognized as a promising tool. The aim of this study was to assess the utility of the most common machine-learning algorithms in the rapid triage of children with suspected COVID-19 using easily accessible and inexpensive laboratory parameters. A cross-sectional study was conducted on 566 children treated for respiratory diseases: 280 children with PCR-confirmed SARS-CoV-2 infection and 286 children with respiratory symptoms who were SARS-CoV-2 PCR-negative (control group). Six machine-learning algorithms, based on the blood laboratory data, were tested: random forest, support vector machine, linear discriminant analysis, artificial neural network, k-nearest neighbors, and decision tree. The training set was validated through stratified cross-validation, while the performance of each algorithm was confirmed by an independent test set. Random forest and support vector machine models demonstrated the highest accuracy of 85% and 82.1%, respectively. The models demonstrated better sensitivity than specificity and better negative predictive value than positive predictive value. The F1 score was higher for the random forest than for the support vector machine model, 85.2% and 82.3%, respectively. This study might have significant clinical applications, helping healthcare providers identify children with COVID-19 in the early stage, prior to PCR and/or antigen testing. Additionally, machine-learning algorithms could improve overall testing efficiency with no extra costs for the healthcare facility.


Introduction
The coronavirus disease , caused by a novel strain of betacoronavirus (SARS-CoV-2), has marked the past two years with over 6.7 million deaths worldwide. New cases are still being diagnosed but with predominantly mild clinical manifestations. COVID-19 primarily occurs in the adult population, but children play a significant role in the spread of the disease. Children with COVID-19 usually only have a few mild symptoms or no symptoms at all, which is why they remain unrecognized. Additionally, newborns, infants, and toddlers cannot wear protective masks in an appropriate way, and moreover, they cannot clearly describe their health condition. Regardless of the disease's severity and the amount of viral load, it is important to consider that pediatric patients may contribute to the transmission chain. For all of the above reasons, the pediatric population should receive special attention during the current pandemic [1][2][3].
Even though viral pneumonia has been recognized as the main clinical presentation of this disease, representing the main cause of its severity and mortality, SARS-CoV-2 infection Therefore, the aim of this study was to assess the utility of the most common machinelearning algorithms in the rapid triage of children with suspected COVID-19 using easily accessible and less expensive laboratory parameters.

Materials and Methods
This cross-sectional study included 280 children with PCR-confirmed SARS-CoV-2 infection (COVID group), treated at the Institute for Children and Youth Health Care of Vojvodina, Novi Sad, Serbia, in the period from March 2020 to December 2022. The control group (non-COVID group) consisted of 286 children with respiratory symptoms who were SARS-CoV-2 PCR-negative. The exclusion criteria were chronic diseases, malignancies, hematological diseases, and missing data ( Figure 1). The detection of SARS-CoV-2 antigen in nasopharyngeal swab samples of children was performed by the qPCR technique at the Institute of Public Health of Vojvodina, Novi Sad, Serbia. Children were divided into six age groups according to chronological age: newborn (0-28 days), infant (1-12 months), toddler (1-3 years), preschool (4-6 years), school (7-14 years), and adolescent (15-18 years).
Viruses 2023, 15, x FOR PEER REVIEW 3 of 11 Therefore, the aim of this study was to assess the utility of the most common machine-learning algorithms in the rapid triage of children with suspected COVID-19 using easily accessible and less expensive laboratory parameters.

Materials and Methods
This cross-sectional study included 280 children with PCR-confirmed SARS-CoV-2 infection (COVID group), treated at the Institute for Children and Youth Health Care of Vojvodina, Novi Sad, Serbia, in the period from March 2020 to December 2022. The control group (non-COVID group) consisted of 286 children with respiratory symptoms who were SARS-CoV-2 PCR-negative. The exclusion criteria were chronic diseases, malignancies, hematological diseases, and missing data ( Figure 1). The detection of SARS-CoV-2 antigen in nasopharyngeal swab samples of children was performed by the qPCR technique at the Institute of Public Health of Vojvodina, Novi Sad, Serbia. Children were divided into six age groups according to chronological age: newborn (0-28 days), infant (1-12 months), toddler (1-3 years), preschool (4-6 years), school (7-14 years), and adolescent (15-18 years).

Data Acquisition
Data on parameters of the complete blood count and baseline biochemical parameters, including aspartate aminotransferase (AST), alanine aminotransferase (ALT), gamma-glutamyl transferase (GGT), lactate dehydrogenase (LDH), and C-reactive protein (CRP), were collected on the day of admission. All data were obtained through the institute's laboratory information system using the structured query language (SQL) code as a searching tool. The blood samples were collected using 0.5 mL violet-topped microtubes with ethylenediaminetetraacetic acid dipotassium salt dehydrate (K2EDTA) as a

Data Acquisition
Data on parameters of the complete blood count and baseline biochemical parameters, including aspartate aminotransferase (AST), alanine aminotransferase (ALT), gammaglutamyl transferase (GGT), lactate dehydrogenase (LDH), and C-reactive protein (CRP), were collected on the day of admission. All data were obtained through the institute's laboratory information system using the structured query language (SQL) code as a searching tool. The blood samples were collected using 0.5 mL violet-topped microtubes with ethylenediaminetetraacetic acid dipotassium salt dehydrate (K2EDTA) as a blood clotting inhibitor (Becton Dickinson, Franklin Lakes, NJ, USA). The values were determined from the hematology analyzer Advia 2120 (Siemens Healthcare, Erlangen, Germany) and chemistry analyzer DxC 700 AU (Beckman Coulter, Brea, CA, USA).

Data Preprocessing
First, all patients with missing data were excluded from the study, following one of the above-mentioned eligibility criteria. Second, the outliers were identified as data points located outside the whiskers of the box plot and excluded from further analysis. Third, Spearman's correlation was used to screen out highly correlated laboratory parameters in order to minimize the number of input parameters (the threshold value was set to 0.4). A correlation heatmap was used to visualize the strength of relationships between the parameters ( Figure S1). Fourth, min-max normalization was applied to transform each parameter into the range [0, 1] in order to treat them with equal weight without distorting the general distribution in the source data.

Baseline Statistical Analyses
Statistical analyses (descriptive and inferential) were performed using open-source software, JASP version 0.16.4.0 (Department of Psychological Methods, University of Amsterdam, Amsterdam, The Netherlands). The significance level for the calculated differences was set at 0.05. For continuous random variables, the normality of distribution was estimated using the Shapiro-Wilk test. Between-group differences were analyzed using the Mann-Whitney U-test. Univariate logistic regression analysis was performed to determine the parameters, which could predict COVID-19 occurrence.

Machine-Learning Algorithms
The following machine-learning algorithms were tested in this study: random forest (RF), support vector machine (SVM), linear discriminant analysis (LDA), artificial neural network (ANN), k-nearest neighbors (KNN), and decision tree (DT). All these algorithms belong to supervised learning, and their goal is classification. The data set was divided into two subsets: the training set and the test set, with an 80:20 split. The training set was validated through stratified cross-validation, where each tuning cycle involved a different non-overlapping holdout data set. The performance of each algorithm was confirmed by additional, independent data set-the test set. The final evaluation of the model included the calculation of accuracy, sensitivity, specificity, positive predictive value, and negative predictive value from the confusion matrix, i.e., the table of predicted and actual values of a classifier. The values were expressed as percentages. Discrimination between groups (COVID and non-COVID) by machine-learning algorithms was presented using receiver operating characteristics (ROC) curves.

Ethical Approval
The study was approved by the Ethics Committee of the Institute for Children and Youth Healthcare of Vojvodina (22 July 2022; No. 3280-2).

Results
Following the eligibility criteria, in the period from March 2020 to December 2022, a total of 566 children treated at the Institute for Children and Youth Health Care of Vojvodina, Novi Sad, Serbia, were included in the study. The median age of the COVID-19 group was 4.2 years with a female share of 46.4%, while the median age of the non-COVID group was 3.8 years with a female share of 47.9%. Children were divided into six age groups according to chronological age: newborn (7.6%), infant (15.9%), toddler (24.9%), preschool (18.6%), school (17.8%), and adolescent (15.2%).

Clinical Laboratory Features
The initial data included 22 clinical laboratory parameters, a complete blood count, and baseline biochemical parameters. After screening out highly correlated instances (Spearman's rank correlation coefficient over 0.4), a total of 14 parameters were included in further analysis: white blood cells (WBC), red blood cells (RBC), mean corpuscular volume (MCV), mean corpuscular hemoglobin concentration (MCHC), platelets (PLT), mean platelet volume (MPV), plateletcrit (PCT), platelet distribution width (PDW), absolute lymphocyte count (LYM#), absolute eosinophil count (EOS#), AST, GGT, LDH, and CRP (Table 1). Univariate logistic regression analysis was employed to examine the association of individual laboratory parameters with the presence of SARS-CoV-2 infection in children.
The following parameters demonstrated significant diagnostic properties as independent predictors: WBC, MCHC, MPV, and PDW (Table 1). PCR-SARS-CoV-2-negative children had higher values of WBC and MCHC, while children with COVID-19 had higher values of MPV and PDW.

Machine-Learning Algorithm Performances
A comparison of six investigated machine-learning algorithms, based on the standard evaluation metrics, was carried out with a reduced number of instances (Table 2). The RF and SVM models demonstrated the highest accuracy of 85% and 82.1%, respectively, while all the other algorithms classified instances with an accuracy lower than 80%. The RF and SVM models demonstrated better sensitivity than specificity and better negative predictive value than positive predictive value. The F1 score, which combines positive predictive value (precision) and sensitivity (recall) using their harmonic means, was higher for the RF than for the SVM model, 85.2% and 82.3%, respectively. After evaluating the performance of the best model (in our study, this was the RF model), the feature importance was compared based on its increase in node purity, i.e., its mean decrease in accuracy. The most prominent instances (node purity over 0.01) were shown to be MPV, WBC, MCHC, PDW, and LYM#. Discrimination between groups (COVID and non-COVID) by machine-learning algorithms was presented using receiver operating characteristics (ROC) curves ( Figure 2). The RF and SVM models demonstrated the highest accuracy of 85% and 82.1%, respectively, while all the other algorithms classified instances with an accuracy lower than 80%. The RF and SVM models demonstrated better sensitivity than specificity and better negative predictive value than positive predictive value. The F1 score, which combines positive predictive value (precision) and sensitivity (recall) using their harmonic means, was higher for the RF than for the SVM model, 85.2% and 82.3%, respectively. After evaluating the performance of the best model (in our study, this was the RF model), the feature importance was compared based on its increase in node purity, i.e., its mean decrease in accuracy. The most prominent instances (node purity over 0.01) were shown to be MPV, WBC, MCHC, PDW, and LYM#. Discrimination between groups (COVID and non-COVID) by machine-learning algorithms was presented using receiver operating characteristics (ROC) curves ( Figure 2).

Discussion
In order to limit the spread of the SARS-CoV-2 virus, it is necessary to detect positive cases as soon as possible and isolate them. However, the small number of available qPCR tests, their high price, and the relatively high percentage of false negative results of these tests brought about the need for additional diagnostic tools [18]. Machine-learning algorithms for automatic disease detection have been increasingly applied in different areas of medicine. Machine learning is a field of artificial intelligence, which provides systems with the ability to automatically learn from experience. The main purpose of these models is to find appropriate patterns in the data, i.e., to produce statistically reliable and reproducible results [19,20].
In this study, the authors evaluated the performance of the six most common machine-learning algorithms: RF, SVM, LDA, ANN, KNN, and DT. The RF and SVM models outperformed the others with an accuracy of 85% and 82.1%, respectively. Both algorithms fall under supervised machine learning. The task of supervised machine-learning algorithms is to "learn" the prediction function h(x) based on a given training data set, so that h(x) is an optimal approximation of the target classes, in this case COVID and non-COVID [20,21].
The SVM machine-learning technique solves the problems of non-linear classification and regression using convex quadratic programming methods. This model only uses instances from the training set that contribute most to the optimal solution of the quad-

Discussion
In order to limit the spread of the SARS-CoV-2 virus, it is necessary to detect positive cases as soon as possible and isolate them. However, the small number of available qPCR tests, their high price, and the relatively high percentage of false negative results of these tests brought about the need for additional diagnostic tools [18]. Machine-learning algorithms for automatic disease detection have been increasingly applied in different areas of medicine. Machine learning is a field of artificial intelligence, which provides systems with the ability to automatically learn from experience. The main purpose of these models is to find appropriate patterns in the data, i.e., to produce statistically reliable and reproducible results [19,20].
In this study, the authors evaluated the performance of the six most common machinelearning algorithms: RF, SVM, LDA, ANN, KNN, and DT. The RF and SVM models outperformed the others with an accuracy of 85% and 82.1%, respectively. Both algorithms fall under supervised machine learning. The task of supervised machine-learning algorithms is to "learn" the prediction function h(x) based on a given training data set, so that h(x) is an optimal approximation of the target classes, in this case COVID and non-COVID [20,21].
The SVM machine-learning technique solves the problems of non-linear classification and regression using convex quadratic programming methods. This model only uses instances from the training set that contribute most to the optimal solution of the quadratic programming problem, forming the so-called support vectors. SVM is a very popular and reliable prediction method. During the COVID-19 pandemic, its application has been confirmed for diagnostic purposes [22][23][24], mortality risk assessment [25,26], detecting undertriage in telephone triage [27], etc.
Unlike SVM, which belongs to the category of "individual" algorithms in supervised machine learning, RF belongs to the class of ensemble methods, which combine the results of several individual methods in a certain way. This approach aims to obtain better prediction results than any of the individual methods. For RF construction, an ensemble is formed consisting of several hundred to several thousand DTs. The advantages of DTs, compared to other machine-learning methods, are their simplicity of implementation and the comprehensibility of the procedure. There are rules by which trees are quickly formed, and the output can be easily interpreted. In addition, DTs allow attributes to have missing values, which is not the case with SVM. However, one of the disadvantages of the DT method is its instability. A small change in the input training data can lead to a significant change in the topology of the tree. Instability occurs due to many possible splits, which often have approximately the same importance (competitor splits). Therefore, a small change in the data can lead to a completely different partition, which further introduces changes to all the branches of the tree below it. RF overcomes these limitations by aggregating the prediction results of hundreds of individual trees [21]. Therefore, the RF model has been widely used during the COVID-19 pandemic for disease diagnosis [28,29], predicting patient outcomes [30][31][32], recommending hospitalization [33], processing of healthcare and travel data to identify COVID-infected people [34], etc.
The evaluation metrics for the RF and SVM models based on clinical laboratory data reported in other studies were similar to this study. Our RF model demonstrated an accuracy of 85%, while Çubukçu et al. [35] reported an accuracy of 85.2% in their RF model using complete blood count parameters and clinical chemistry parameters as input variables, but in an adult population. Our SVM model demonstrated an accuracy of 82.1%, thus outperforming a model proposed by Thimoteo et al. [36], who reported an accuracy of 73.7% in their SVM model, including complete blood count parameters only, as well as an adult population.
The model presented in this study cannot outperform the PCR method, which is considered the diagnostic gold standard, with an average efficiency of over 96% [37]. Conversely, the suggested model outperformed the immunochromatography method used in the rapid SARS-CoV-2 antigen tests. In the beginning of the pandemic, only a few antigen tests received emergency use authorization (EUA) from regulatory authorities, indicating their acceptable performance [38]. Subsequently, some of them were reported to have a sensitivity of no more than 30% [39]. Since these tests were widely used for diagnosing active infections, their performance has significantly improved over time. One of the most commonly used rapid antigen tests at our institute has an overall sensitivity of 79.6% [40]. Our RF and SVM models demonstrated a sensitivity of 86% and 84.8%, respectively.
Including additional clinical biochemical analyses in a machine-learning algorithm, such as ferritin [41], fibrinogen, D-dimer [42], procalcitonin, interleukin-6 [43], etc., may increase the accuracy, sensitivity, and specificity of the model. However, minimal blood sample volume is imperative in pediatric health research. For example, a complete blood count analysis can be performed using a total blood volume of 25 µL. Blood volume overdraws in pediatric laboratory medicine should always be taken into consideration from a legal and ethical perspective [44].
Children experience milder symptoms during the course of the SARS-CoV-2 infection in comparison to adult individuals. Age-related differences may reflect disease severity.
These age-related differences include differences in immunity, differences in binding affinity of the SARS-CoV-2 target receptors, etc. [45,46]. In our study, children were divided into six age groups according to chronological age: newborn (0-28 days), infant (1-12 months), toddler (1-3 years), preschool (4-6 years), school (7-14 years), and adolescent (15-18 years). According to the systematic review conducted by Carobene et al. [47], only half of the PubMed and Scopus publications on the application of artificial intelligence in COVID-19 diagnostics take demographic data, such as gender and age, into consideration. The reference ranges in pediatric laboratory medicine are strictly defined and age-dependent. Therefore, it is mandatory to compare individuals within the same age group. This is often overlooked by many researchers, which produces misleading conclusions. Another important consideration and strength of the study is the period in which the children were included, i.e., March 2020-December 2022. This period covers several waves of the pandemic, both before and after the vaccination program had started.
Data on laboratory-based machine-learning approaches for the detection and triage of children with COVID-19 are scarce. Previous studies were mainly focused either on the adult population or, in the case of pediatric population, on radiological rather than laboratory findings, and on outcome rather than detection. With that being considered, this study is a unique contribution to pediatric laboratory medicine.
Multiplex PCR and rapid panel antigen tests (lateral immunochromatography) are used in the diagnosis of viral infections in pediatrics. However, the healthcare system faces a constant challenge in finding an additional diagnostic modality, which is fast, reliable, and cheap, such as AI algorithms. Nevertheless, the potential drawbacks of machine learning in personalized laboratory medicine should be taken into consideration. Diagnostics should always strongly rely on human skills, including physical examination, critical perception of medical history, etc. Machine-learning algorithms provide multi-dimensional biomedical data, which should not be overrated and should only be observed as supporting information for making a final diagnosis [48].
Analyzing the cost-benefit properties of the algorithms proposed in this study, it can be concluded that the healthcare system can benefit from these algorithms in terms of both time and money. The turnaround time for SARS-CoV-2 qPCR tests can vary depending on several factors, including testing capacity and demand, laboratory workload and staffing, supply chain issues, transportation, logistics, etc. The average turnaround time for SARS-CoV-2 qPCR tests ranges from a few hours to a couple of days [49]. On the contrary, laboratory tests used as input data in the proposed algorithm can be performed within a few minutes, and the algorithm can deliver results within a few seconds. Taking finances into consideration, the proposed algorithms have an advantage over the SARS-CoV-2 qPCR test. The price for a single SARS-CoV-2 qPCR test may vary depending on several factors, such as the country and healthcare system. However, the average price ranges from around USD 50 to USD 200 per test. On the other hand, the baseline laboratory parameters used in this study can be obtained from automated clinical chemistry analyzers for no more than USD 50 [50,51]. Moreover, all the parameters used in this study are part of routine testing upon admission to our institute. Therefore, the proposed algorithm entails no additional laboratory-related cost. Furthermore, the statistical package used in this study is an open-source program without the need for license-related costs. Based on the above-mentioned claims, it can be inferred that the implementation of the proposed machine-learning algorithms could improve overall testing efficiency with no extra costs for the healthcare facility.
The use of machine learning in COVID diagnostics has several clinical implications, which can greatly impact the detection and management of the disease, such as early detection and diagnosis, improved accuracy and efficiency, risk stratification and prognosis, personalized treatment plans, monitoring disease trends and outbreaks, etc. It is important to note that while machine learning holds great promise, it should be integrated into clinical practice with caution. Rigorous validation, ethical considerations, and ongoing monitoring are necessary to ensure the reliability, safety, and ethical use of machine-learning models in COVID diagnostics [52,53].
There are certain limitations to the approach proposed in this study and some practical considerations for future research to be considered. First, this is a single-center study, which only includes a limited number of children. Second, the pediatric patients in this study are all European. Performing multi-institutional and multi-national studies could evaluate whether the proposed models could perform well in other human races. Third, all children with underlying conditions were excluded from the proposed study, making its clinical applicability for children with co-infections limited. Fourth, only blood counts and baseline biochemical parameters were included in the study, as they are easily and immediately obtained at admission. However, other non-laboratory parameters could also be included, resulting in a potentially better diagnostic model. Fifth, the control group was heterogeneous, without a specific pathogen classification. They were all labeled only as non-COVID, i.e., SARS-CoV-2-negative.

Conclusions
Herein, the six most common machine-learning algorithms for the rapid triage of children with suspected COVID-19 were presented and validated. The RF and SVM models outperformed the others with fairly high accuracy. Thereby, a set of clinical laboratory features with markedly high prediction potential was identified, including MPV, WBC, MCHC, PDW, and LYM#. The results of this study might have significant clinical applications, helping healthcare providers identify children with COVID-19 in the early stages, prior to PCR and/or antigen testing. Additionally, machine-learning algorithms could improve overall testing efficiency with no extra costs for the healthcare facility. However, the potential drawbacks of machine learning in personalized laboratory medicine should be taken into consideration.