Predicting Rheumatoid Arthritis Development Using Hand Ultrasound and Machine Learning—A Two-Year Follow-Up Cohort Study

Background: The early diagnosis and treatment of rheumatoid arthritis (RA) are essential to prevent joint damage and enhance patient outcomes. Diagnosing RA in its early stages is challenging due to the nonspecific and variable clinical signs and symptoms. Our study aimed to identify the most predictive features of hand ultrasound (US) for RA development and assess the performance of machine learning models in diagnosing preclinical RA. Methods: We conducted a prospective cohort study with 326 adults who had experienced hand joint pain for less than 12 months and no clinical arthritis. We assessed the participants clinically and via hand US at baseline and followed them for 24 months. Clinical progression to RA was defined according to the ACR/EULAR criteria. Regression modeling and machine learning approaches were used to analyze the predictive US features. Results: Of the 326 participants (45.10 ± 11.37 years/83% female), 123 (37.7%) developed clinical RA during follow-up. At baseline, 84.6% of the progressors had US synovitis, whereas 16.3% of the non-progressors did (p < 0.0001). Only 5.7% of the progressors had positive PD. Multivariate analysis revealed that the radiocarpal synovial thickness (OR = 39.8), PIP/MCP synovitis (OR = 68 and 39), and wrist effusion (OR = 12.56) on US significantly increased the odds of developing RA. ML confirmed these US features, along with the RF and anti-CCP levels, as the most important predictors of RA. Conclusions: Hand US can identify preclinical synovitis and determine the RA risk. The radiocarpal synovial thickness, PIP/MCP synovitis, wrist effusion, and RF and anti-CCP levels are associated with RA development.


Introduction
Rheumatoid arthritis (RA) is a chronic inflammatory condition characterized by joint inflammation and progressive joint damage, primarily affecting the hands and feet [1,2].The timely diagnosis and effective management of RA are critical to prevent irreversible joint destruction and functional impairment [3][4][5][6].However, early detection poses a challenge due to the often subtle and nonspecific initial symptoms [7,8].There is a pressing need for accurate diagnostic methods capable of identifying RA in its early stages, particularly among individuals with undifferentiated arthritis [9][10][11][12].
Imaging modalities such as ultrasound (US) [13,14] and magnetic resonance imaging (MRI) have emerged as valuable tools for the detection of early inflammatory changes in the body, even before the onset of clinical synovitis [15,16].Imaging modalities such as US and MRI [17] have emerged as valuable tools for detecting early inflammatory changes before the onset of clinical synovitis [18][19][20].High-frequency ultrasound has shown promise as a cost-effective, non-invasive, and widely available method for the detection of mild synovitis and in predicting the progression to clinical arthritis [9,19,21].However, previous research on the effectiveness of US in early disease detection has been limited by small sample sizes and variability [17,[22][23][24].
This study aimed to evaluate the prognostic utility of hand ultrasound (US) in the early diagnosis of RA among 326 individuals presenting with nontraumatic hand joint pain who initially did not meet the established criteria for RA.Leveraging supervised machine learning techniques, we analyzed the US findings alongside other clinical and serological data to identify predictive features for RA development over an extended observation period.The insights garnered from this research have the potential to inform the development of diagnostic and prognostic models integrating ultrasound assessment for risk stratification in individuals with undifferentiated arthritis.The early identification of high-risk individuals enables the targeted implementation of disease-modifying treatments, thereby optimizing patient outcomes.

Study Design
We carried out a single-center, prospective cohort study at the rheumatology clinic of Dr. Shariati Hospital in Tehran, Iran, from 2018 to 2021, to assess the diagnostic effectiveness of hand ultrasonography in the early detection of RA among patients with nontraumatic hand joint pain.The primary goal was to evaluate how well ultrasound imaging could identify patients who would develop clinically evident RA within a two-year follow-up period.A power analysis, assuming a 20% difference in RA incidence between the exposed and unexposed groups, with a significance level of 0.05 and 80% power, guided our target enrollment of 300 patients.

Participants and Clinical Assessment
Eligible participants were individuals aged 18 years or older, presenting to the clinic with nontraumatic hand joint pain with a duration of less than 12 months and no prior diagnosis of rheumatic disease [25].Exclusion criteria included symptoms attributable solely to degenerative joint disease, a definite RA diagnosis at initial presentation, previous arthritis, treatment with disease-modifying antirheumatic drugs (DMARDs) or glucocorticoids, systemic infections or autoimmune diseases, lymphoproliferative disorders, a history of cancer or recent radiotherapy, and refusal to participate or attend follow-up visits.
Participants provided written informed consent; completed a baseline questionnaire detailing their symptoms and family and medical history; and underwent clinical examinations and laboratory tests.Tests included the erythrocyte sedimentation rate (ESR), C-reactive protein (CRP), IgM-RF, and anti-cyclic citrullinated peptide (anti-CCP) antibodies, using standardized equipment (HITACHI 912 autoanalyzer; Stat Fax 4200 ELISA Microplate Reader, Tokyo, Japan), as per the manufacturer guidelines.
The primary outcome measure, the development of rheumatoid arthritis (RA) within a 24-month period, was determined using the 2010 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) classification criteria.The evaluation of RA development was performed by a rheumatologist who was blinded to the patients' ultrasonography results [25].Detailed information about diagnostic criteria and abbreviations can be found in the Supplementary File S1.

Ultrasonography Examination
All patients underwent hand ultrasonography within one week of clinical assessment, performed by a radiologist with over four years of experience in musculoskeletal ultrasonography, who was unaware of the patients' clinical and laboratory data.Using a MyLabX8 Platform (Esaote, Genoa, Italy) with a multifrequency (LA523, 2-9 MHz) linear array probe, the radiologist scanned various hand joints, including the bilateral proximal interphalangeal (PIP), metacarpophalangeal (MCP), and distal interphalangeal (DIP) joints; the interphalangeal joint (IP) of the thumb; the first carpometacarpal (CMC) joint; and the wrist joints.Each scan lasted about 20 min, following a standardized OMERACT protocol, involving a midline 12 o'clock longitudinal scan perpendicular to the joint surface [5,26].
The radiologist utilized both grayscale (GS) and power Doppler (PD) imaging to evaluate synovitis, effusion, erosion, and cartilage damage, applying a semiquantitative scoring system (0-3) [25,26].Synovitis was identified by a GS grade above 1 or any PD activity.The radiologist recorded the GS and PD scores, along with each patient's exposure status, in a digital database.The PD settings were optimized for sensitivity and artifact reduction, with adjustments made to the pulse repetition frequency and the positioning of the color box to accurately capture blood flow signals.
The ultrasonography equipment was regularly calibrated to ensure accuracy, and patient images were stored on a local PACS server.After data collection, all images were reviewed collectively to discuss the findings (Figures 1-4).
The primary outcome measure, the development of rheumatoid arthritis (RA) within a 24-month period, was determined using the 2010 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) classification criteria.The evaluation of RA development was performed by a rheumatologist who was blinded to the patients' ultrasonography results [25].Detailed information about diagnostic criteria and abbreviations can be found in the Supplementary File S1.

Ultrasonography Examination
All patients underwent hand ultrasonography within one week of clinical assessment, performed by a radiologist with over four years of experience in musculoskeletal ultrasonography, who was unaware of the patients' clinical and laboratory data.Using a MyLabX8 Platform (Esaote, Genoa, Italy) with a multifrequency (LA523, 2-9 MHz) linear array probe, the radiologist scanned various hand joints, including the bilateral proximal interphalangeal (PIP), metacarpophalangeal (MCP), and distal interphalangeal (DIP) joints; the interphalangeal joint (IP) of the thumb; the first carpometacarpal (CMC) joint; and the wrist joints.Each scan lasted about 20 min, following a standardized OMERACT protocol, involving a midline 12 o'clock longitudinal scan perpendicular to the joint surface [5,26].
The radiologist utilized both grayscale (GS) and power Doppler (PD) imaging to evaluate synovitis, effusion, erosion, and cartilage damage, applying a semiquantitative scoring system (0-3) [25,26].Synovitis was identified by a GS grade above 1 or any PD activity.The radiologist recorded the GS and PD scores, along with each patient's exposure status, in a digital database.The PD settings were optimized for sensitivity and artifact reduction, with adjustments made to the pulse repetition frequency and the positioning of the color box to accurately capture blood flow signals.
The ultrasonography equipment was regularly calibrated to ensure accuracy, and patient images were stored on a local PACS server.After data collection, all images were reviewed collectively to discuss the findings (Figures 1-4).

Follow-Up
We monitored the patients for two years after the initial assessment.We conducted phone interviews with patients who did not wish to visit the clinic for their 6-, 12-, and 18-month evaluations.We inquired about their symptoms, treatments, and outcomes and

Follow-Up
We monitored the patients for two years after the initial assessment.We conducted phone interviews with patients who did not wish to visit the clinic for their 6-, 12-, and 18-month evaluations.We inquired about their symptoms, treatments, and outcomes and

Follow-Up
We monitored the patients for two years after the initial assessment.We conducted phone interviews with patients who did not wish to visit the clinic for their 6-, 12-, and 18-month evaluations.We inquired about their symptoms, treatments, and outcomes and advised them to visit the outpatient clinic for further examination by a rheumatologist if they showed signs of clinical synovitis.At the 24-month visit, we determined if the patients had rheumatoid arthritis or non-inflammatory pain based on the rheumatologist's decision using the 2010 ACR/EULAR criteria (score of more than 6/10) [25].

Data Preprocessing and Model Development
The GS and PD scores for each assessed joint were used directly as input features for the machine learning models, without additional preprocessing.We employed several supervised learning algorithms, including decision trees, support vector machines (linear, quadratic, and Gaussian kernel), k-nearest neighbors, AdaBoost, neural networks, and random forests, to identify ultrasound characteristics predictive of preclinical RA.
The selection of the final model was guided by the documented performance in medical image classification tasks.After careful consideration, the random forest (RF) algorithm, an ensemble method that amalgamates multiple decision trees to enhance the prediction accuracy and mitigate overfitting, was deemed most suitable [27][28][29][30].
The hyperparameters, including the number of trees (set to 80), were fine-tuned using a grid search approach to optimize the model performance.The performance of the RF model was assessed using k-fold cross-validation (k = 5) and a comprehensive set of metrics, including sensitivity, specificity, precision, the F1-score, and the Matthews correlation coefficient (MCC).This approach ensured robust evaluation, particularly considering the class imbalance present in the dataset [31].

Statistical Analysis
Statistical analysis was performed using SPSS version 22. Quantitative data were presented as means and standard deviations, while qualitative data were expressed as frequencies and percentages.We assessed group differences using independent-samples t-tests for continuous data and chi-square or Fisher's exact tests for categorical data.Our dataset contained minimal missing values, less than 5% across all variables, which were handled through pairwise deletion.The hypothesis tests conducted were exploratory, aimed at generating rather than confirming hypotheses; therefore, no adjustments for multiple comparisons were made.Logistic regression analyses were used to investigate potential predictors of RA development, with results presented as odds ratios and 95% confidence intervals.The agreement between the initial physician assessments and final diagnoses was measured using Cohen's kappa coefficient.The diagnostic accuracy of ultrasonography was evaluated through the sensitivity, specificity, and positive and negative predictive value, each reported with 95% confidence intervals.We assumed no unmeasured confounding for the validity of our results, addressing potential confounders via regression-based covariate adjustment.Statistical significance was determined at a p-value of less than 0.05.

Study Cohort and RA Development
Of the 503 patients screened for eligibility, 326 (65% of those screened) were included in the final analysis, with a follow-up period of 24 months.The cohort predominantly comprised females (83%), with an average age of 45.10 ± 11.37 years.During the follow-up, 123 patients (37.7% of the cohort) were diagnosed with clinical RA according to the 2010 ACR-EULAR criteria.

Baseline Characteristics
The demographic and laboratory characteristics at baseline are detailed in Table 1.The patients who developed RA (RA group) were significantly older on average (47.10 years, 95% CI: 45.32-48.88)compared to those who did not develop RA (non-RA group) (43.90 years, 95% CI: 42.49-45.31;p = 0.019).Significant differences were also observed in the baseline laboratory markers between the groups, with the RA group showing higher mean values of the white blood cell count (WBC; p = 0.012), erythrocyte sedimentation rate (ESR; p = 0.003), rheumatoid factor (RF; p < 0.0001), and anti-citrullinated protein antibodies (anti-CCP; p < 0.0001).

Baseline US Findings
Ultrasonography at baseline revealed the significantly higher prevalence of synovitis in any hand joint in the RA group compared to the non-RA group (84.6% vs. 16.3%;p < 0.0001).The RA group also exhibited the higher prevalence of positive PD signals (5.7% vs. 0.5%; p = 0.003).There were no significant differences between the groups in terms of carpal bone erosion, wrist ganglion cysts, bone hypertrophy, and IP joint erosion.

Machine Learning Model Performance
Applying various machine learning models to predict RA based on the baseline variables highlighted the random forest model, which comprised 80 trees, as the most effective, demonstrating an F1-score of 82.1.This model also showed high precision (82.11%) and sensitivity (82.09%).The performance overview of the different models is detailed in Table 3, with the ROC and precision-recall curves for the random forest model depicted in Figure 5. Applying various machine learning models to predict RA based on the baseline variables highlighted the random forest model, which comprised 80 trees, as the most effective, demonstrating an F1-score of 82.1.This model also showed high precision (82.11%) and sensitivity (82.09%).The performance overview of the different models is detailed in Table 3, with the ROC and precision-recall curves for the random forest model depicted in Figure 5.

Feature Importance in Prediction Models
The results of this analysis indicated that radiocarpal synovial thickening, MCP synovitis, PIP synovitis, RF, and anti-CCP were the most influential features, with relative importance scores of 0.23, 0.18, 0.16, 0.13, and 0.11, respectively.For a visual representation of these feature importance scores, please refer to Figure 6.

Feature Importance in Prediction Models
The results of this analysis indicated that radiocarpal synovial thickening, MCP synovitis, PIP synovitis, RF, and anti-CCP were the most influential features, with relative importance scores of 0.23, 0.18, 0.16, 0.13, and 0.11, respectively.For a visual representation of these feature importance scores, please refer to Figure 6.

Impact of Ultrasonographic Features on Model Performance
Incorporating US features significantly enhanced the performance metrics across all models, particularly for the random forest model, which saw its F1-score increase from 77.8 to 82.1, precision from 77.7 to 82.11, and sensitivity from 77.9 to 82.09.Detailed improvements are documented in Tables 3-5.

Impact of Ultrasonographic Features on Model Performance
Incorporating US features significantly enhanced the performance metrics across all models, particularly for the random forest model, which saw its F1-score increase from 77.8 to 82.1, precision from 77.7 to 82.11, and sensitivity from 77.9 to 82.09.Detailed improvements are documented in Tables 3-5.

Discussion
This study aimed to evaluate the prognostic value of hand US in the early detection of RA in patients with nontraumatic hand joint pain.We identified specific US features, including radiocarpal synovial thickening, PIP and MCP synovitis, and wrist effusion, as strong predictors of RA development when combined with elevated RF and anti-CCP titers.At the conclusion of the follow-up, 37.7% of the patients had developed clinical arthritis, with 84.6% showing ultrasound synovitis at baseline, indicated by a grade of synovial hypertrophy in GS greater than 1 or the presence of PD abnormalities.Of these, 48.8% had synovitis in only one joint, while 35.9% had it in two or more joints.Clinically, patients who progressed to RA were more likely to exhibit elevated WBC counts, ESR levels, and RF and-CCP titers.US synovitis strongly correlated with the development of clinical RA at the patient level.However, positive PD abnormalities were rare (2.5%) and did not predict future clinical RA.These results align with previous research indicating that US synovitis is a significant marker in pre-RA patients and can predict the onset of clinical arthritis in those at risk [4,5,11,32,33].However, the prevalence and predictors of RA development may vary depending on the US method, scanning protocol, population characteristics, assessment of effusion and erosion, or examination of other modes, such as PD or GS.
In a cohort study by Nam et al., patients experiencing arthralgia, irrespective of their antibody status, were followed for one year.Baseline ultrasound synovitis was defined as GS ≥ 2 and/or PD ≥ 1.By the end of the study, 16% of participants developed arthritis, 59% of whom had positive US findings at the start.The wrists (26%) and MTP joints (11%) were most frequently affected.Independent predictors of inflammatory arthritis development included age, US synovitis, a positive PD signal, and morning stiffness.Interestingly, US effectively ruled out IA in patients without US synovitis [5].Van Beers-Tas et al. observed that among 163 seropositive individuals, 31% developed clinical arthritis after a median of 12 months, with 86% meeting the 2010 ACR/EULAR criteria for RA.Their analysis suggested that, excluding the MTP joints, the US synovial thickness correlated with both the occurrence and timing of clinical arthritis at the patient level.Positive PD indicators, as in our findings, were infrequent among those at risk and were not predictive [11].Naredo et al. conducted a prospective study on anti-CCP+ patients without clinical arthritis, assessing GS, PD, and erosions across 32 joints, including the wrists, MCPs, PIPs, and MTPs.Eightysix percent of those who developed clinical arthritis had at least one US abnormality at baseline, compared to 67% of non-progressors.All US findings predicted IA progression, with the highest risk associated with PD abnormalities [34].Van Steenbergen et al. studied 192 arthralgia patients with positive autoantibody titers, finding that 23% progressed to RA after an average of 11 months.Their study indicated a trend towards US abnormalities in ≥1 joints at baseline being predictive of arthritis development.However, contrary to our findings, they demonstrated that US predicted clinical synovitis at the joint level rather than the patient level [33].
The collective evidence presented in this study suggests that the utilization of US in the assessment and stratification of the RA risk appears to be of substantial value during the preclinical stages of the disease, surpassing the sole reliance on clinical examination and autoantibody status.However, it is crucial to acknowledge that the observed inconsistencies among the findings may be attributed to the variability in the scanning protocol employed, which includes the application of US examinations solely to symptomatic joints or to all joints, encompassing the MTP joints and larger joints.Additionally, another potential explanation for the discrepancies observed in the results could be the existence of technical disparities between US machines, as these variations have a notable impact on the detection of PD signals [11].
Previous research indicates that clinicians' requirements for US scanning vary by indication, with a higher number of joints assessed for diagnostic purposes to detect inflammation and initiate treatment promptly in cases of widespread pain.For ongoing synovial activity, the choice of joints to scan often depends on the presence of pain or the tendency of certain joints to be more frequently affected.Studies also reveal that rheumatologists in university settings typically perform US on fewer than 10 joints, constrained by the available time [35].Although simplified US joint count scoring systems have been evaluated for early RA and found feasible in assessing joint inflammation, selecting the optimal subset of asymptomatic joints for clinical use remains challenging.Employing a multiple-joint scoring system is more time-consuming, but focusing on the joints commonly affected by RA could save time and possibly enhance the sensitivity [33].To refine joint selection in US protocols, we analyzed all hand joints using US and employed regression models and machine learning techniques to identify the most significant US predictors of RA progression [11].
The results from a multivariate regression model and machine learning analysis indicate that specific US findings significantly influence the likelihood of RA progression in the U.S. population.Notably, radiocarpal synovial thickening, PIP and MCP synovitis, and wrist effusion at baseline, as identified by grayscale US, are strong independent predictors of RA in at-risk patients.These US variables, coupled with high RF and anti-CCP titers in patients presenting with arthralgia but not fulfilling the RA clinical criteria, are highly valuable for risk stratification in clinical settings.Additionally, the study underscores the potential of AI, particularly the random forest algorithm, in enhancing CAD systems, showing superior performance over other models.
Based on the findings, a simplified US protocol targeting the key joints and scores may enhance the efficiency and feasibility of screening and diagnosing pre-rheumatoid arthritis (pre-RA) patients.Additionally, incorporating machine learning models could improve the accuracy and objectivity in predicting RA development by using ultrasound data along with other clinical and laboratory variables.However, further research is needed to validate and optimize these US protocols and machine learning models across different settings and populations.
Another noteworthy observation is that wrist joint synovial thickening and synovitis in the PIP and DIP joints can manifest in the early stages of OA, as evidenced by prior studies [36].Recent imaging techniques have revealed that synovial inflammation is common in both the early and late phases of OA [36], particularly affecting the DIP, PIP, and the base of the thumb [37].In our study, the presence of US-positive findings in these joints at baseline, without the subsequent development of RA during follow-up, may have represented early manifestations of OA.However, additional research with longer follow-up is necessary to confirm this.This suggests that US may not always be specific enough to distinguish between RA and OA, particularly when clinical and laboratory indicators are unclear.Thus, the careful interpretation of US results is essential to avoid diagnostic errors.

Study Strengths and Limitations
This study exhibits several notable strengths.Firstly, its prospective design, coupled with a substantial sample size and an extended 24-month follow-up period, lends credibility to the findings.Additionally, the implementation of a standardized US scanning protocol enhanced the reliability and reproducibility of the data collection process.Furthermore, the innovative application of machine learning techniques facilitated the identification of the most predictive US features, underscoring the study's methodological rigor.
However, it is essential to acknowledge certain limitations.As a single-center study, the generalizability of the findings may be restricted, necessitating further validation across diverse healthcare settings and populations.Despite concerted efforts to adjust for key variables, the potential for residual confounding cannot be entirely ruled out.Moreover, while the US protocol employed in this study was comprehensive, there may be alternative joint areas or scoring systems that could yield additional predictive value, warranting further exploration.
Consequently, future research endeavors spanning multiple centers, diverse populations, and the exploration of alternative US protocols or scoring systems are recommended.Such initiatives would not only validate the present findings but also refine and enhance the predictive capabilities of the proposed approach.By addressing these limitations, the study's impact could be amplified, contributing to the development of more robust and generalizable predictive models for the early detection of rheumatoid arthritis.

Figure 2 .
Figure 2. Examples of joint gray scale ultrasound findings.(A) Wrist joint showing synovial hypertrophy (indicated by double-headed blue arrows) and joint capsule distention (double-headed green arrows).(B) Proximal interphalangeal (PIP) joint showing synovial hypertrophy (indicated by double-headed blue arrows) and joint capsule distention (double-headed green arrows).Osteophytes are marked by red arrows.

Figure 4 .
Figure 4. Ultrasound findings of the MCP joint.(A) Grade II joint capsule distension is observed without significant hypervascularity.The double-headed blue arrows indicate synovial thickness.(B) Moderate joint effusion and joint capsule thickening.The double-headed white arrows indicate joint capsule thickness, yellow arrows show joint effusion, and the green rectangle encompasses Color Doppler imaging to show vascularity.(C) A normal MCP joint with standard synovial thickness, as shown by the double-headed blue arrows.The red double-headed arrow represents the normal MCP joint anatomy.

Figure 3 .Figure 3 .
Figure 3. Examples of joint Doppler ultrasound findings.(A) Radiocarpal joint showing synovial thickening, marked as "1" within the green rectangle, and increased vascularity on Doppler ultrasound, indicated by blue and red areas within the green rectangle.(B) Proximal interphalangeal (PIP) joint with synovitis and hypervascularity on Doppler ultrasound, indicated by blue and red areas within the green rectangle.

Figure 4 .
Figure 4. Ultrasound findings of the MCP joint.(A) Grade II joint capsule distension is observed without significant hypervascularity.The double-headed blue arrows indicate synovial thickness.(B) Moderate joint effusion and joint capsule thickening.The double-headed white arrows indicate joint capsule thickness, yellow arrows show joint effusion, and the green rectangle encompasses Color Doppler imaging to show vascularity.(C) A normal MCP joint with standard synovial thickness, as shown by the double-headed blue arrows.The red double-headed arrow represents the normal MCP joint anatomy.

Figure 4 .
Figure 4. Ultrasound findings of the MCP joint.(A) Grade II joint capsule distension is observed without significant hypervascularity.The double-headed blue arrows indicate synovial thickness.(B) Moderate joint effusion and joint capsule thickening.The double-headed white arrows indicate joint capsule thickness, yellow arrows show joint effusion, and the green rectangle encompasses Color Doppler imaging to show vascularity.(C) A normal MCP joint with standard synovial thickness, as shown by the double-headed blue arrows.The red double-headed arrow represents the normal MCP joint anatomy.

Figure 6 .
Figure 6.The predictive value of the US and laboratory variables according to the random forest algorithm.

Figure 6 .
Figure 6.The predictive value of the US and laboratory variables according to the random forest algorithm.

Table 1 .
Baseline demographic and laboratory characteristics.

Table 2 .
Independent predictive variables associated with RA progression for 24-month follow-up.

Table 3 .
The results of the evaluation metrics for machine learning methods including tree, linear SVM, quadratic SVM, SVM with Gaussian kernel, KNN, AdaBoost, random forest, and NN for US and laboratory features.

Table 3 .
The results of the evaluation metrics for machine learning methods including tree, linear SVM, quadratic SVM, SVM with Gaussian kernel, KNN, AdaBoost, random forest, and NN for US and laboratory features.

Table 4 .
The results of the evaluation metrics for machine learning methods including tree, linear SVM, quadratic SVM, SVM with Gaussian kernel, KNN, AdaBoost, random forest, and NN for US features.

Table 5 .
The results of the evaluation metrics for machine learning methods including tree, linear SVM, quadratic SVM, SVM with Gaussian kernel, KNN, AdaBoost, random forest, and NN, for laboratory features.

Table 4 .
The results of the evaluation metrics for machine learning methods including tree, linear SVM, quadratic SVM, SVM with Gaussian kernel, KNN, AdaBoost, random forest, and NN for US features.

Table 5 .
The results of the evaluation metrics for machine learning methods including tree, linear SVM, quadratic SVM, SVM with Gaussian kernel, KNN, AdaBoost, random forest, and NN, for laboratory features.