Application of Machine Learning Predictive Models for Early Detection of Glaucoma Using Real World Data

Abstract

Early detection of glaucoma is critically important for the prevention of irreversible blindness. We developed a predictive analytic framework through temporal data carpentry and applications of a suite of machine learning and logistic regression methods for the early prediction of glaucoma using electronic health records (EHR) from over 650 hospitals and clinics across the USA. Four different machine-learning classification methods were applied using the whole dataset for predictive analysis. The accuracy, sensitivity, specificity, and f1 score were calculated using five-fold cross-validation to train and refine the models. The XGBoost, multi-layer perceptron (MLP), and random forest (RF) performed comparably well based on the area under the receiver operating characteristics curve (AUC) score of 0.81 for predicting glaucoma one year before the onset of the disease compared to the logistic regression (LR) score of 0.73. This study suggests that the ML methods can capture potential pre-glaucoma patients in advance before the occurrence of clinical symptoms from their history of EHR encounters, thus possibly leading to earlier intervention and preventive treatment.

1. Introduction

Glaucoma is the second-leading cause of irreversible blindness worldwide [1]. About 70 million people worldwide have glaucoma, and about 4.4 million are blind from optic nerve damage due to undiagnosed glaucoma globally [1,2]. In the USA, the American Glaucoma Society has reported that 2.7 million Americans have glaucoma, but only half of these people know that they have it [3]. In many cases, the patient’s intraocular pressure and glaucoma development are undetectable due to the painless and diabolical progression of the disease [4]. Furthermore, glaucoma negatively impacts the quality of life and creates a substantial economic burden overall [1]. In the United States, glaucoma treatment and healthcare costs have been estimated at $2.5 billion per year [5].

Consequently, vision scientists have reported that early detection is key to reducing the burden of glaucoma on the healthcare system [6]. Early detection of glaucoma allows for earlier treatment, possibly avoiding expensive surgical procedures or irreversible blindness. Glaucoma is a heterogeneous condition; patients may have different symptoms and different progression rates. In addition, the current glaucoma growth rate and its economic burdens are unsustainable [7]. As a result, it warrants a systematic evaluation for glaucoma risk assessment and early prediction for better management of glaucoma.

Several risk factors have been reported for the development of glaucoma, including elevated intraocular pressure (IOP) [8], the use of certain medications, older age, a greater optic nerve cup-to-disc ratio, and family history [9,10]. Though it has been demonstrated that lowering IOP reduces the rate of glaucomatous damage and consequent vision loss, current glaucoma predictions are targeted at lowering IOP as a modifiable risk factor. However, in many cases, higher-than-average IOP does not cause neurodegenerative damage or vision loss [11]. IOP alone cannot predict glaucoma development, but comorbidity analysis can be an effective ancillary tool for glaucoma detection in the early stages of the disease [12]. Further, Joanne et al. have reported that systemic disease conditions such as diabetes are associated with a 2.8-fold increased incidence of glaucoma development compared to non-diabetic cohorts [13]. Furthermore, Deokule et al. have reported that vascular conditions, such as coronary artery diseases, indicate glaucoma development and can serve as a prognostic marker [14]. In addition, several population-based cross-sectional studies, such as the Egna-Neumarkt Glaucoma Study, the Rotterdam Eye Study, and the Blue Mountain Eye Studies, have demonstrated an association between elevated blood pressure and glaucoma [14,15,16].

Conversely, medical treatment of systemic diseases—diabetes and hypertension—may have a multifactorial effect on glaucoma onset that has not always been examined in prospective or cross-sectional studies. It is noteworthy to mention that most of these studies were knowledge-based or hypothesis-driven, which suggests that a modest number of pre-set variables were included in their predictive analysis. With the extensive adaptation of electronic health records (EHR) and their supporting secondary use in research [17], we now have the opportunity to apply data-driven retrospective approaches to develop predictive models through understanding comorbidities that impact the onset of glaucoma and their related medications. Moreover, EHR data contains valuable information that can be used for many purposes, including decision-making support [18], prognosis markers for medical concepts [19], and risk assessment [20].

In recent years, machine learning (ML) methods have been successfully applied in many data analytics areas, including clinical data analysis, image classification, diagnosis, and decision support [21]. Bertsimas et al. combined unsupervised and supervised ML methods and predicted near-future healthcare costs for patients [22]. Furthermore, using EHR records, ML methods have been applied to augment risk prediction for glaucoma risk factors, cataract surgery complications, and diabetic retinopathy [23,24,25,26]. Studies have examined the relationship between risk factors and postoperative visual acuity [27,28]. Almazroa et al. have reported automated glaucoma detection using fundus images [29].

Additionally, a systematic review article has reported various ML methods used for risk assessment, progression prediction, and the diagnosis of glaucoma using image techniques [26]. Image processing methods can report lesions that have already occurred at the molecular level. In a recent study, Braganca et al. applied deep learning methods and demonstrated about 90% accuracy in detecting glaucoma from fundus images captured using a smartphone and handheld ophthalmoscope [30]. In contrast, the use of EHR records predicts the onset of glaucoma based on the history of comorbidities. More recently, Baxter et al. reported machine learning-based predictive modeling of glaucoma surgical intervention using EHR records and demonstrated that the prediction of the need for surgical intervention for glaucoma management six months in advance has a certain degree of success [31].

Very few studies have been conducted to investigate the early predictive model for glaucoma onset using EHR data obtained from multiple clinical centers. In the future, a predictive model must focus on a large cohort of a study population representing heterogeneity and diversity, thus bringing clinical value to ophthalmologists.

2. Materials and Methods

2.1. Data Source and Sample

This data was extracted from an extensive EMR database (Cerner Health Facts^®), which is comprised of deidentified EHR data from over 650 hospitals and clinics across the USA and represents more than 80 million unique patients. For this study, we retrieved deidentified patient clinical data from 2001–2015, as described earlier [32]. Though the EMR database contains many attributes, we used systemic diseases, medication, and demographic information. Furthermore, the timestamped record of each event provides a temporal relationship between clinical encounters and patterns. The institutional review board (IRB #2006793) approved this study protocol at the University of Missouri. The methods reported in this research can be applied to other registry data collected from other EHR vendors.

2.2. Data Extraction and Feature Representation

The next step in the process included checking the EMR database for the availability of glaucoma patient records using the International Classification of Disease (ICD), ICD9 365, from 2001 to 2015. The inclusion and exclusion criteria for the glaucoma cohort are that patients should have at least two eye-related visits before the diagnosis of glaucoma. We then collected non-glaucoma patient cohorts as a control dataset. Non-glaucoma patients were determined based on eye clinic visits, excluding those not diagnosed with glaucoma code 365. Finally, we used glaucoma and non-glaucoma patient cohorts and retrieved all EHR records associated with those patients’ records, including demographic information, systemic diseases, and laboratory results. All these ICD9 codes related to various diagnoses were aggregated into the superclass three-digit level as one attribute. Similarly, the medication codes were grouped into generic names.

2.3. Data Preprocessing and Temporal Data Modeling

Preprocessing the data involved various filtering, transformation, binarization, and joining techniques to create a data table for the study. We then applied further filtering to exclude patients with no attributes. The final dataset has a total of 830,125 unique patient records, with 135,045 glaucoma patients (labeled as a glaucoma dataset) and 695,080 non-glaucoma patients (marked as a non-glaucoma dataset). These data were organized into temporal order using timestamp information. Each patient record should have at least two variables in a 15-year-wide longitudinal dataset as an inclusion criterion. We subsampled a balanced dataset containing 33,611 unique patient records (16,805 glaucoma patients and 16,806 non-glaucoma patients) as input data for early prediction analysis. All 16,805 glaucoma patients were first diagnosed with the disease during a yearlong period from 2014 to 2015. For early predictive analysis, the input glaucoma dataset contains records up to one year before the diagnosis of glaucoma. There were 32 attributes in the final dataset from demographics, systemic diseases, and medications for analysis (Appendix A). Finally, one-hot encoding converted the categorical variables (race, gender) of n possible values into n binary variables, as shown in Figure 1.

2.4. Machine Learning Models

The data was analyzed using a machine learning library in the Python programming language. First, we randomly split the data into 80% training and 20% testing data for analysis. ML analysis performance was assessed using 5-fold cross-validation, which provides a robust and reliable measurement of model performance [31]. We used the listed parameters for each ML model, as shown in

Table 1. Parameters used in different machine learning methods.

Learning Model	Loss Function	Parameter Estimation	Complexity Reduction
Logistic Regression	Log loss	Gradient descent	L2 regularization
XGBoost classifier	Squared error	Booster parameters	Regression
Random Forest	Square loss: (Y-Y) 2	CART	Move down tree based on x predict value at the leaf
MLP Classifier	Activation: relu	Solver: adam	Learning rate init = 0.001

, for the entire dataset. We trained a series of four algorithms, including logistic regression (LR), random forest (RF), XGBoost (XGB), and multi-layer perceptron (MLP). The functionality of logistic regression, a parametric-based model, and the other models, such as RF and XGB, is non-parametric. We used a loss function, parameter estimation, and complexity reduction for each ML parameter listed in Table 1. These ML methods have been used in several predictive analyses [33,34,35] with a large dataset.

2.5. Model Performance and Hyperparameter Tuning

There are many hyperparameter tuning methods available. We used GridSearchCV from the Scikit-Learn library and selected the best-tuned parameter for model prediction (for MLP: “activation”: “relu”, “alpha”: 0.05), “hidden_layer_sizes”: (10, 30, 10), “learning_rate”: “adaptive”, “solver”: “Adam”) based on the accuracy score and maximizing the overall AUC using 5-fold cross-validation. We also used standard metrics such as the sensitivity, specificity, and F1 score of each model for performance interpretation. The confusion matrix plot was analyzed based on true positive (TP), false positive (FP), true negative (TN), and false-negative (FN) outcomes. For accuracy, the percentage of samples correctly classified as positive is divided by the number of positive examples. We measured the area under the receiver operating characteristic curve and model calibration using the training and testing datasets. We trained several models and tested them with the entire dataset. Each ML model was separately trained with the same set of input data.

3. Results

The overall prevalence of glaucoma estimated from the dataset was about 16%. Among the different race groups, the incidence of glaucoma was higher in African Americans (23.2%), Asians (16.8%), and East Indians (16.2%) compared to other races (i.e., Hispanics (11.0%). When comparing gender, a higher prevalence of glaucoma (16.6%) was observed among the female population than the male population (15.7%) in this dataset. The four machine learning models’ power of prediction is shown in

Table 2. Accuracy metric for evaluating classification models. The highest accuracies (in bold) are from XGBoost and MLP.

Model	Prediction	Precision	Recall	F1	Accuracy
LR	0	0.7830	0.6341	0.7007	0.7281
	1	0.6904	0.8228	0.7508
RF	0	0.7911	0.8427	0.8161	0.8105
	1	0.8308	0.7763	0.8024
XGB	0	0.8033	0.8386	0.8026	0.8161
	1	0.8302	0.7936	0.8115
MLP	0	0.8086	0.8259	0.8171	0.8146
	1	0.8210	0.8033	0.8121

. The sensitivity for different classifiers of the data showed the highest with MLP (81%), XGBoost (81%), and random forest (81%), followed by logistic regression (73%). We further computed the precision of different models. The MLP has the highest precision (82%), followed by the random forest (83%) and XGBoost (83%). The recall (glaucoma cases) was highest for logistic regression (82%), and XGBoost (83%). Similarly, the actual positive prediction rate is also higher with MLP, Random Forest, and XGBoost than with other models’ performances.

The accuracy of model prediction was plotted on the AUC plot, which showed a similar trend. We also performed the ROC of all these models built by different ML algorithms (Figure 2), and the performance of these models was evaluated by examining the true value (the actual glaucoma label) versus the predicted value (the predicted glaucoma label). The plot of TPF (sensitivity) versus FPF (1-specificity) across varying cut-offs generates a curve in the unit square, which shows the tradeoff between the true positive fraction (TPF) and a false-positive fraction (FPF). The ROC curve lying on the diagonal line (black dots) reflects a diagnostic test performance that is no better than the chance level. The ROC curve slope is equal to the ratio of the two density functions describing the separator variable’s distribution at a given point in glaucoma versus the non-glaucoma likelihood ratio. The ROC curve also shows the models’ discriminatory ability and displays how useful the classification model is at distinguishing between glaucoma and non-glaucoma. Further, the error rate (unpredicting the actual label by these models) may be due to the EHR data’s inherent ambiguity or the fact that the heterogeneity of all variability is not saturated in the present dataset. Further, the type 1 error rate is high in LR, and the type II error rate is high in the RF model compared to all other models (

Table 3. Elements of the confusion metric for evaluating classification models’ performance assessments for four machine learning models.

Model	TP	FP	FN	TN	Accuracy
LR	2140	1235	593	2754	0.7281
RF	2840	530	750	2602	0.8096
XGB	2826	544	692	2660	0.8161
MLP	2784	587	659	2692	0.8146

).

We performed a crude and adjusted odds ratio analysis to understand which features could serve as keys for the prediction models. We compared it with the top 5 important variables used for model classification.

Odds ratio analysis has been widely used in clinical research and can directly make inferences from the results. For example, the crude odds ratio between glaucoma and cataract was 1.19, with a 95% confidence interval of 1.13–1.25. The adjusted odds ratio between glaucoma and cataract was 1.36, with a 95% confidence interval of 1.28–1.43. Further, the estimated adjusted odds ratio between glaucoma and obesity was 1.14, with a 95% confidence interval of 1.07–1.21. Similarly, the adjusted odds ratios between glaucoma and atherosclerosis were 1.2 (95% CI: 1.07, 1.34), suggesting that atherosclerosis is also associated with glaucoma compared to non-glaucoma groups (

Table 4. Adjusted odds ratio analysis—1-year data presented only with OR 1.14 and above.

	Crude OR (95%CI)	Adj. OR (95%CI)	P (Wald’s Test)	P (LR-Test)
Cataract	1.19 (1.13,1.25)	1.36 (1.28,1.43)	<0.001	<0.001
Atherosclerosis	0.73 (0.66,0.81)	1.2 (1.07,1.34)	0.001	0.001
Type 2 diabetics	0.8 (0.77,0.84)	1.14 (1.08,1.21)	<0.001	<0.001
Obesity	0.89 (0.84,0.94)	1.14 (1.07,1.21)	<0.001	<0.001
Lacrimal disorder	1.28 (1.17,1.4)	1.14 (1.03,1.26)	0.008	0.008

).

Relevant attributes from RF and LR suggest that many variables are associated with the glaucoma dataset. Notably, the collection of all the results from different methods indicates that cataracts, obesity, and atherosclerosis are associated with an increased risk of developing glaucoma compared to non-glaucoma patient groups. As reported in the literature, hypertension, obesity, type 2 diabetes, and African American races are known risk factors for the onset of glaucoma. Studies have reported many risk factors for the onset of glaucoma [36,37,38]. We identified a new connection between atherosclerosis and glaucoma onset in the present study. The present study shows an association between atherosclerosis and glaucoma. Further study is needed to understand the underlining effect of atherosclerosis on the onset of glaucoma.

4. Discussion

We compared the performance of predictive models, such as RF, LR, XGB, and MLP, using a real-world, large-scale EHR dataset. We observed that the performance of the predicted variables varies from model to model, as it has been reported that no unique algorithm or model works best for every problem. Sramka et al. used a support-vector machine model for the clinical intraocular lens calculator [39], while Lin et al. used the random forest method to predict myopia onset [40]. Further, the logistic regression prediction model has been widely used for decision-making and feature selection [41,42]. Baxter et al. have focused on using deep learning methods to improve the selection of patients for the glaucoma surgical intervention [30]. The present study evaluated the performance of four ML models and found that the RF, XGB, and MLP models produced higher prediction accuracy.

We know that machine learning generally requires a large amount of data, which is a strength of our study. When an outcome depends on a few variables, it is possible to correlate/associate the variables with the disease’s phenotype. However, when the result depends on multiple factors with different temporal and spatial weights, it becomes difficult to pinpoint the variables and their dependencies on the prediction. In addition, some ML algorithms perform better than other algorithms with the same set of variables in a dataset41. Within the same input dataset, the current study also shows that XGBoost, random forest, and MLP prediction accuracy outperform the LR model.

The classification results are shown in Table 2, and the highest accuracy rate was reported in the RF and XGBoost models at 81% compared to the logistic regression classification. We believe that the reason for the higher accuracy in XGBoost and RF is that they use an ensemble learning approach to discover more predictable variables from the medical dataset. The advantage of using ensemble methods in these algorithms is that they allow for a training model with quantitative and qualitative inputs. Furthermore, without prior domain knowledge, the MLP network model predicts well for generalized data. Additionally, when making predictions, the Random Forest and XGBoost had the majority votes in the terminal vote. On the other hand, it has been reported that logistic regression models perform poorly with binary responses because they estimate whether there is an increase or decrease in the predicted log odds of the response variables while holding all other variables constant. That could be a reason for the lower prediction accuracy rate of 73% with LR in the present study. In the early prediction of glaucoma or any other medical or health-related monitoring, false positive diagnoses can be more beneficial for a patient than false negative predictions. For example, telling a patient that they have a higher chance of developing glaucoma and recommending biannual checkups or close monitoring would be more beneficial for the individual than a false negative report where the patient would miss an early warning of glaucoma onset that would develop into glaucoma in the near future.

Overall, the research suggests that the higher accuracy rates obtained from the RF, XGB, and MLP models using the balanced dataset can be considered for predicting glaucoma in a real-world setting. With the present dataset, the prediction accuracy for glaucoma onset was about 81%, which is reasonably significant for glaucoma detection using only EHR data and without observing/analyzing traditional fundus images or optical coherence tomography (OCT) results. Though early diagnosis of glaucoma using ocular coherence tomography (OCT) has been considered significant, it also comes with an inherent error rate of ~20%. Hampton et al. have reported that, in general, only up to 82% of glaucoma diagnoses can be determined using the information provided by medical history and OCT image data [43].

Studies have reported many risk factors for the development and progression of glaucoma. Though age has been reported as a well-known common risk factor for glaucoma [36], understanding exactly how other preventable risk factors increase the risk of glaucoma is necessary to tackle this growing problem. Because several risk factors are implicit with aging, it becomes challenging to draw a line between the aging process and disease progression [37]. Since the present study used age-matched glaucoma and non-glaucoma groups, we did not incur any inferences based on age. Consistent with the existing literature, the present study identified significant risk factors or comorbidities for the development of glaucoma, including cataracts. This analysis also revealed additional risk factors, such as lacrimal disorders, with an OR of about 1.14. While cataracts have long been associated with glaucoma formation, other factors such as primary and chronic angle closure, lacrimal disorders, and obesity are unknown risk factors associated with the development of glaucoma. These findings suggest that this information can be used to inform risk stratification to mitigate glaucoma onset as needed. Further, an independent study is needed to understand the role of lacrimal disorders and obesity in the development of glaucoma onset.

4.1. Interpretation and Clinical Relevance

Only a few common variables were observed between the top three models, suggesting that different algorithms predict different variables as predictors for the model. In the RF model, high BMI, cataracts, lipid metabolism, and abnormal glucose levels are significant predominant predictors of glaucoma, whereas in the LR model, gender, gentamicin, and obesity have higher coefficients with glaucoma (Appendix B). The OR analysis shows that cataracts, atherosclerosis, and Type 2 diabetes led to a higher risk of developing glaucoma. Many studies have reported that African-American race is a significant risk factor for glaucoma. Analysis of the risk factors in the underlying genetic makeup of African Americans, as Blachon et al. have reported, shows that genetic polymorphisms in the mitochondrial cytochrome c oxidase subunit 1, which plays a role in trabecular meshwork function, are higher among African Americans, which may have a role in the increased prevalence of primary open angle glaucoma [44]. Furthermore, obesity, hypertension, cataracts, atherosclerosis, and Type 2 diabetes could be good predictors for the development of glaucoma before its clinical onset. Zhao et al. previously conducted a meta-analysis to better understand the relationship between hypertension and POAG, using IOP as endpoints. They identified 60 studies, including seven longitudinal cohort analyses, from various parts of the world. Virtually all studies showed a positive association between hypertension and POAG. However, when the RF model was compared to the logistic regression model in the current study, hypertension was found to be the most significant risk factor for glaucoma. The RF model predicted hypertension as a top-five critical glaucoma risk factor with the highest accuracy (Appendix B).

Observing only a few common variables between the models suggests that each model uses a different set of variables for prediction. It is critical to understand the clinical relevance of the variables and their uses in the clinical application as an evolution measure of ML-predicted features. Integrating the clinically relevant predictive model into the EHR data would enable healthcare providers to target suitable patients for proper early intervention. The application of an evidence-driven ML predictive model in the clinical setting could be a significant step forward in preventing glaucoma-related vision loss by identifying patients’ susceptibility to the onset of glaucoma and allowing for early intervention.

4.2. Limitations of Predictive Analysis

There are many advantages to using EHR data, as it can contain a longitudinal medical history of patients, including symptoms, laboratory findings, procedures, prescriptions, and test results that can be easily used on a computerized platform [45,46]. The secondary use of EHR data is faster and can collect more measurements from a larger population. However, the longitudinal dataset has significant advantages over the aggregated dataset for the early predictive model. Adapting any ML model in the health care system without further clinical trial examination, on the other hand, can be difficult. We must understand the combined effect of different variables, such as the confounding effect of risk variables, and how long it may take to develop glaucoma after exposure to such variables.

Further, many challenges are associated with leveraging EHR data for predictive analysis because of its complex nature. Extracting information from these heterogeneous datasets requires correct data collection techniques, proper data preprocessing, reasonable dimension reduction, and suitable machine learning algorithms. In addition, building a longitudinal dataset to capture a patient’s history’s temporal events is challenging.

Further, implementing data-driven analysis into the day-to-day workflow is challenging because of the different regulatory systems in various institutions, including insurance, hospital managers, security, and IT teams. In addition, longitudinal data collection requires many patients’ records to capture all possible heterogeneity in the population. Though the ethics and regulations may improve over time, the EHR data also has inherent limitations, including missing, incomplete, and incorrect data. With the sparse and natural missing data of electronic health records, a longitudinal dataset analysis approach works well and yields significantly better predictions than traditional methods. However, more work needs to be done for further validation, and improvements to these methods must be made before implementation in a clinical setting.

5. Conclusions

We have adopted longitudinal data for early prediction of glaucoma using an extensive EHR database. Patients can be screened based on the comorbidities encountered in their electronic health records by ML methods to prevent irreversible blindness from glaucoma. This EHR dataset used in the present study contains real-world heterogeneous data collected from patients’ medical records and demonstrates the prediction of early detection of glaucoma. Furthermore, the ML performance, including random forest, XGB, and MLP, can predict glaucoma versus non-glaucoma conditions with nearly 81% accuracy. The use of real-world data and ML methods can benefit glaucoma specialists and ophthalmologists in identifying glaucoma-susceptible patients far in advance compared to current modalities. With further investigation, we hope that this temporal data mining approach could be employed to provide a powerful tool for delivering early predictive value for glaucoma onset.