A Machine-Learning Model Based on Morphogeometric Parameters for RETICS Disease Classification and GUI Development

Featured Application

This work presents a Graphics User Interface that applies two automated learning models based on machine-procured independent variables to assist ophthalmology professionals in keratoconus disease diagnosis and classification.

Abstract

This work pursues two objectives: defining a new concept of risk probability associated with suffering early-stage keratoconus, classifying disease severity according to the RETICS (Thematic Network for Co-Operative Research in Health) scale. It recruited 169 individuals, 62 healthy and 107 keratoconus diseased, grouped according to the RETICS classification: 44 grade I; 18 grade II; 15 grade III; 15 grade IV; 15 grade V. Different demographic, optical, pachymetric and eometrical parameters were measured. The collected data were used for training two machine-learning models: a multivariate logistic regression model for early keratoconus detection and an ordinal logistic regression model for RETICS grade assessments. The early keratoconus detection model showed very good sensitivity, specificity and area under ROC curve, with around 95% for training and 85% for validation. The variables that made the most significant contributions were gender, coma-like, central thickness, high-order aberrations and temporal thickness. The RETICS grade assessment also showed high-performance figures, albeit lower, with a global accuracy of 0.698 and a 95% confidence interval of 0.623–0.766. The most significant variables were CDVA, central thickness and temporal thickness. The developed web application allows the fast, objective and quantitative assessment of keratoconus in early diagnosis and RETICS grading terms.

1. Introduction

Corneal tomography is a validated technology for evaluating the changes occurring in the corneal morphology of keratoconus (KC) disease [1], which allows for the control of the geometric decompensation driven by the asymmetry present while this disease progresses [2,3]. However, no agreement has been reached about the relative importance of the indices and technologies to be used to detect which patients may suffer post-surgical corneal iatrogenic ectasia when evaluating patients’ suitability for refractive surgery [4,5,6,7,8,9,10].

Some studies have evaluated the objective efficiency of different indices for early stages of this disease in an individualized way. These studies are generally related to values of curvature, elevation and pachymetry [11], and to patterns based on the combination of indices that come from the same [12,13,14,15,16] or different technologies [7,9,17,18,19,20]. The evaluation of these patterns is based on subjective analysis [7,21], which is critical to gain a better discrimination capability according to clinical experience. Nevertheless, there are still significant discrepancies in terms of the relative value of the results obtained in their evaluation of the risk probability associated with disease development.

In KC, geometric decompensation occurs and causes disease progression [11], which means increased high-order optical aberrations and high irregular astigmatism values [22], whose main refractive sign is the impossible complete optical compensation of their ametropia by spherical-cylindrical lenses. Consequently, corrected visual acuity will diminish compared to individuals with no corneal pathology [23]. The scientific literature contains many classifications for the degree of KC severity [24,25,26,27,28,29,30,31]. However in clinical practice, it is difficult to handle the many indices on which these classifications are based for proper optical-geometrical evaluations of disease progression. From an optical point of view, patients show a deteriorated spectacle-corrected visual acuity during disease development, insofar as their visual performance worsens as the degree of KC severity progresses. Following this criterion, a classification of disease stages has been developed [23,32]: the so-called RETICS grading. This grading takes into account four geometrical parameters (Internal Astigmatism, RMS Coma-Like, Q_8mm and Pachimetry) and a functional one (Corrected Distance Visual Acuity (CDVA)) to establish five KC degrees: I to V.

This study develops and validates a Graphics User Interface (GUI) that combines two automatic learning models based on a set of independent variables with two aims: defining a new concept of risk probability associated with the development of early-stage KC and classifying disease severity according to the RETICS (Thematic Network for Co-Operative Research in Health) scale to assist ophthalmology professionals in disease management.

2. Materials and Methods

2.1. Patients

This research work was conceived as an observational comparative study. It comprised 169 eyes of 169 subjects divided into two groups. To avoid potential biases, those cases showing any other ocular comorbidity that could affect the present study parameters, who had undergone any ocular surgical procedure, or had worn contact lenses in the 4 weeks prior to the topographical evaluation, were excluded from both groups.

The first group, called the “control” group, comprised 62 healthy eyes of 62 patients (48.4% males, 51.6% females) whose ages ranged from seven to 60. The cases included in the control group were randomly selected from the refractive surgery candidates, and the data used for this study were acquired during their pre-surgical appointments, always with the same experienced technician.

The second group was formed by 107 KC candidates (63.2% males, 36.8% females) aged from 15 to 98. They were classified into five subgroups in accordance with the RETICS grading system [32].

The procedure followed for KC group diagnosis and classification was based on state-of-the-art clinical and topographical evaluations (Figure 1), including uncorrected distance visual acuity (UDVA), CDVA, manifest refraction (sphere and cylinder), slit-lamp biomicroscopy, Goldmann tonometry, fundus evaluation and ultrasonic pachymetry [33]. In all cases, pre-surgical evidence for KC was assessed: asymmetric bowtie pattern with or without skewed axes, localized stromal thickness reduction, conical protuberance at the apex, Fleischer ring, Vogt striae or anterior stromal scar.

All the evaluations were made at Vissum Corporation Alicante (a centre affiliated with the Miguel Hernández University of Elche, Elche, Spain), and now form part of the official “Iberia” database of KC cases created for the National Network for Clinical Research in Ophthalmology RETICS-OFTARED. Patients were adequately informed about the study and agreed to freely participate in it. The study was also ratified by the hospital’s Ethical Committee for Clinical Research according to the ethical guidelines in the Declaration of Helsinki (7th revision, October 2013, Fortaleza, Brazil).

2.2. Methods

Each case selected to form part of this research was examined using a Sirius System^® tomographer (Costruzione Strumenti Oftalmici, Florence, Italy), following the specifications of a validated procedure previously created by our research group, which is clearly explained in former research works [2,34]. This procedure is effective when used for both the diagnosis and characterization of KC [35,36], and comprises two phases: 3D virtual modeling, followed by a morpho-geometric analysis.

The product of this procedure is a patient-specific 3D custom corneal model, which can be studied to determine several morpho-geometric parameters that have already been described and used in a previous study [37]. Of them all, anterior corneal surface area (anterior surface area), posterior corneal surface area (posterior surface area) and total corneal volume (total volume) were selected to be used, along with demographic, pachymetry and clinical parameters (Figure 1).

2.3. Statistical Analysis

The quantitative variables were summarized using mean±standard deviation (SD), median and interquartile range (25th and 75th percentiles). The Shapiro–Wilk test was employed to assess if the quantitative variables followed normal distribution. The Student’s t-test was run to compare the normally distributed variables between two groups, while an ANOVA was used when there were three groups or more. For the non-normally distributed variables, the Mann–Whitney test and the Kruskal–Wallis test were, respectively, carried out. Differences in the qualitative variables among independent groups where compared by the χ² test. The predictive score model for early KC detection was defined using multivariate logistic regression with the control and RETICS grade I groups. Seventeen variables were included in the model: two demographic (age and gender), seven optical (CDVA, coma-like, Q_8mm, spherical-like, RMS total, high-order, astigmatism), five pachymetry (central thickness, temporal, nasal, superior, inferior) and three morpho-geometric (total volume, anterior surface area, posterior surface area). Model discriminative efficiency was evaluated by receiver operating characteristic (ROC) curves, when area under curve (AUC), sensitivity and specificity were taken as performance indicators. An internal cross-validation procedure was followed using bootstrap aggregating (bagging) [38]. This procedure works as follows: it first generates a new dataset of equal size by sampling with replacement from the original dataset. The model is then trained with these data. Finally, this model is used to make predictions on those cases not used during training. This procedure is repeated 100 times to obtain a set of quality parameters that can be averaged, and confidence intervals that can be calculated. On average, 63.2% of the original data were used in all these 100 training steps. The remaining 36.8% were used for validation. Ordinal multivariate logistic regression was utilized to determine the RETICS grade with the same predictor variables. In this case, the confusion matrix was employed to estimate model performance by means of sensitivity, specificity and balanced accuracy per group. Data were evaluated by R Statistics v3.6.1 (R Foundation for Statistical Computing, Vienna, Austria) [39].

Packages “tidyr”, “dplyr”, “dlookr” and “smbinning” were used for data loading, exploration and transformation. Packages “corrplot”, “yarrr” and “FactoMiner” were employed for data visualization. Packages “pROC”, “ROCR” and “Epi” were utilized for the ROC curve analysis and representation. Logistic regression models were trained with the “glm” function from the base package. Ordered logistic regression models were trained with the “polr” function from the MASS package. Confusion matrices were analyzed by the “caret” package. Optimal sample size calculations were made by the “rcompanion” and “pmsampsize” packages.

The statistical power analysis was conducted with simulation using the Wald test to estimate the power for each covariate according to sample size, as described in the literature [40,41].

A web application was developed using Shiny v1.3.2 (RStudio Inc., Boston, MA, USA) [42] and the ShinyAuthr v0.0.99 authentication module (Paul Campbell, Paris, France) [43]. This application was deployed in a private secure institutional network (because patients’ clinical data were used, and the application was, therefore, accessible only from controlled computers to minimize the security risks linked to using patients’ data).

Application landing page, shown in Figure 2, is a login form that adds a secured authentication layer. No registering possibility was included, and new users can only be directly added by the administrator. After logging in, users view a form with text boxes corresponding to all the model predictors that are filled by default with sample values from a healthy individual. Users can type in new values and, after pressing the “get score” button values, they are passed to trained models and predictions are made (Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8), including the early detection KC score (known as “keratoscore”), the RETICS grade prediction and a brief graphical description of the hypothetical cornea, with a schema indicating the representation of the different parameters.

In summary, the GUI application was developed using a responsive design, which makes it accessible from any network-connected device because it does not require any software or driver being installed, apart from an up-to-date web browser.

3. Results

Sixty-two healthy eyes (36.7%) (control group, Ctrl) and 107 KC eyes (63.3%) (KC group, KC) were herein considered. Several subgroups were established in the KC group depending on the disease stage according to the RETICS grading system: grade I (44 eyes, 41.1%), grade II (18 eyes, 16.9%), grade III (15 eyes, 14.0%), grade IV (15 eyes, 14.0%) and grade Plus-V (15 eyes, 14.0%). The main demographic information in the control and KC subgroups are displayed and summarized in

Table 1. Demographic information for healthy individuals (Ctrl) and the RETICS-classified keratoconus patients, graded between I and V.

	Ctrl	I	II	III	IV	V	p
Number of cases	62	44	18	15	15	15
Age in years (mean ± SD)	37.5 ± 14.4	41.2 ± 16.6	47.4 ± 22.5	40.3 ± 12.6	34.0 ± 14.3	36.3 ± 18.0	0.161
Female/Male	32/30	12/32	6/12	7/8	6/9	4/11	0.146

. No significant differences were found in age and gender terms among all the groups.

Table 2. Descriptive analysis for the quantitative variables and p-values for normality and differences between RETICS groups. CDVA: Corrected Distance Visual Acuity; Q: asphericity; RMS, root mean square

Variables	Mean	SD	Median	Range	Normality	Ctrl vs. RETICS I	Ctrl–RETICS I–II–III–IV–V
Demographic
Age	39	16	38	7–98	<0.001	0.665	0.344
Optical
CDVA	0.77	0.32	0.96	0.05–1.20	<0.001	<0.001	<0.001
Coma-like	1.85	2.15	1.01	0.08–13.0	<0.001	<0.001	<0.001
Q8mm	−0.63	0.70	−0.45	−2.80–2.82	< 0.001	0.025	<0.001
Spherical-like	0.70	0.93	0.44	0.15–7.20	<0.001	<0.001	<0.001
RMS total	3.22	3.09	2.38	0.33–15.6	<0.001	<0.001	<0.001
High-order	2.02	2.33	1.12	0.24–13.8	<0.001	<0.001	<0.001
Astigmatism	2.31	2.25	1.57	0.04–11.22	<0.001	<0.001	<0.001
Pachymetry
Central thickness	499	62	508	285–633	<0.001	<0.001	<0.001
Temporal	545	50	546	385–645	0.073	<0.001	<0.001
Nasal	579	48	579	451–692	0.835	<0.001	<0.001
Superior	591	50	590	408–695	0.062	<0.001	<0.001
Inferior	559	57	563	332–762	<0.001	<0.001	<0.001
Morpho-Geometric
Total volume	24.7	1.9	24.4	19.8–29.1	0.335	<0.001	<0.001
Anterior surface area	43.35	0.59	43.24	42.49–47.44	<0.001	<0.001	<0.001
Posterior surface area	44.70	0.93	44.53	43.53–51.14	<0.001	<0.001	<0.001

summarizes the descriptive analysis outcomes obtained for all the quantitative variable analyses in the control vs. RETICS I and the control vs. KC groups. The descriptive analysis indicated that most variables did not follow normal distribution. Statistically significant p-values were found for all the variables between the control group (Ctrl) and the RETICS grade I group, except for age (p = 0.665). When testing for differences within all the groups (Ctrl and RETICS grade I to V), every p-value was significant, except for age (p = 0.344).

The multivariate logistic regression model results are summarized in

Table 3. Summary of the multivariate logistic regression model for the Ctrl vs. RETICS I patients. CDVA: Corrected Distance Visual Acuity; Q: asphericity; RMS, root mean square

		Std.	Z	p		95% CI
Variables	Coefficient	Error	Value	Value	OR	Lower	Upper
Demographic
Age	0.135	0.078	1.727	0.084	1.145	1.021	1.416
Gender	5.267	2.520	2.090	0.037	193.907	4.154	2.14 × 105
Clinical
CDVA	−6.345	4.613	−1.376	0.169	0.002	0.001	1.821
Coma-like	4.072	1.917	2.124	0.034	58.687	3.615	1.16 × 104
Q8mm	6.027	4.618	1.305	0.192	414.459	0.070	3.02 × 107
Spherical-like	1.673	1.259	1.329	0.119	5.329	0.671	1.08 × 102
RMS total	2.482	1.591	1.560	0.119	11.969	1.071	8.65 × 102
High-order	5.534	2.484	2.227	0.026	253.143	5.830	3.12 × 105
Astigmatism	−1.054	1.218	−0.866	0.387	0.348	0.014	2.963
Pachymetry
Central thickness	-0.242	0.116	−2.083	0.037	0.785	0.577	0.941
Temporal	0.028	0.012	2.248	0.025	1.028	1.009	1.064
Nasal	0.007	0.011	0.641	0.522	1.007	0.987	1.034
Superior	0.092	0.058	1.587	0.113	1.096	0.995	1.263
Inferior	−0.051	0.034	−1.479	0.139	0.951	0.870	0.995
Morpho-Geometric
Total volume	0.067	0.547	0.124	0.901	1.070	0.349	3.908
Anterior surface area	−0.326	0.532	−0.614	0.539	0.722	0.218	2.105
Posterior surface area	−1.189	0.779	−1.525	0.127	0.305	0.044	1.199
Constant	135.027	61.595	2.192	0.028	-	-	-

with the coefficients for each variable. The variables that made a statistically significant contribution in the model are shown below: gender, coma-like, central thickness, high-order and temporal.

The ROC curve in Figure 9 shows an optimal cut-off point of 0.475 with a training AUC of 0.990, a sensitivity of 0.977 and a specificity of 0.919, corresponding to 59 true negative cases, 41 true positive cases, three false-positive cases and three false-negative cases. The bootstrapped validation values corresponding to the 95% CI are 0.843 ± 0.058 for AUC, 0.844 ± 0.095 for sensitivity and 0.838 ± 0.081 for specificity.

Figure 10 shows the distribution of the calculated scores in both groups, where the vast majority of the control cases obtained a score between 0 and 0.1, and between 0.9 and 1 for RETICS I.

The parameters for the multivariate ordinal logistic regression model are summarized in

Table 4. Summary of the ordinal logistic regression model for the Ctrl and RETICS grade I-V patients. CDVA: Corrected Distance Visual Acuity; Q: asphericity; RMS, root mean square

		Std.	t	p		95% CI
Variables	Coefficient	Error	Value	Value	OR	Lower	Upper
Demographic
Age	0.016	0.011	1.475	0.140	1.016	0.995	1.037
Gender	−0.073	0.361	−0.203	0.839	0.929	0.458	1.885
Clinical
CDVA	−5.495	0.865	−6.355	0.001	0.004	0.001	0.022
Coma-like	0.106	0.348	0.304	0.761	1.111	0.562	2.197
Q8mm	0.184	0.359	0.512	0.609	1.201	0.595	2.423
Spherical-like	−0.063	0.250	−0.250	0.803	0.940	0.575	1.535
RMS total	−0.422	0.298	1.416	0.157	1.526	0.850	2.737
High-order	0.621	0.345	1.797	0.072	1.860	0.945	3.660
Astigmatism	−0.250	0.239	−1.046	0.296	0.779	0.488	1.244
Pachymetry
Central thickness	−0.024	0.007	−3.298	0.001	0.977	0.963	0.991
Temporal	0.006	0.002	2.662	0.008	1.006	1.002	1.010
Nasal	−0.002	0.003	−0.771	0.441	0.998	0.992	1.004
Superior	0.006	0.007	0.933	0.351	1.006	0.993	1.019
Inferior	−0.002	0.005	−0.522	0.601	0.998	0.989	1.007
Morpho-Geometric
Volume	0.071	0.141	0.500	0.617	1.073	0.814	1.414
Anterior area	−0.204	0.133	−1.533	0.125	0.816	0.623	1.058
Posterior area	0.046	0.129	0.361	0.718	1.048	0.814	1.348
Intercepts
Ctrl vs. RETICS I	−16.769	0.017	−98.428	<0.001	-	-	-
RETICS I vs. II	−13.926	0.396	−35.205	<0.001	-	-	-
RETICS II vs. III	−12.270	0.522	−23.530	<0.001	-	-	-
RETICS III vs. IV	−9.998	0.765	−13.072	<0.001	-	-	-
RETICS IV vs. V	−6.691	1.057	−6.330	<0.001	-	-	-

. In this case, the variables that made a statistically significant contribution to the model were: CDVA, central thickness and temporal.

The confusion matrix for the training dataset is shown in

Table 5. The ordinal logistic regression confusion matrix showing sensitivity, specificity and balanced accuracy for each group. The total sum of cells by rows shows the total number of true cases present, while each column represents how many cases the model classified in that category.

	Predicted Value
True Value	Ctrl	I	II	III	IV	V
Ctrl	55	7	0	0	0	0
I	12	26	4	2	0	0
II	2	9	6	3	1	0
III	0	3	3	6	3	0
IV	0	1	0	1	12	1
V	0	0	0	1	1	13
Sensitivity	0.887	0.591	0.333	0.400	0.800	0.867
Specificity	0.869	0.840	0.954	0.955	0.968	0.994
Balanced accuracy	0.878	0.715	0.643	0.677	0.884	0.930

, with an overall accuracy of 0.698 at a 95% CI between 0.623 and 0.766. Sensitivity, specificity and balanced accuracy are also shown for each group.

Figure 11 depicts power according to the simulated sample size for both models, including only the variables with maximum power over 0.9 at some point within the range, plus Age and Gender. The remaining variables and their respective maxima for both the binary logistic regression and ordinal logistic regression models were Coma-Like Deviation (0.61 and 0.62), Q_8mm (0.55 and 0.57), Spherical-Like Deviation (0.49 and 0.55), Nasal Thickness (0.81 and 0.83), Superior Thickness (0.88 and 0.89), Inferior Thickness (0.72 and 0.77), Volume (0.69 and 0.71), Anterior Surface (0.83 and 0.89) and Posterior Surface (0.58 and 0.60). For both models, the variables of High-Order Aberration, CDVA, Central Thickness, Total RMS and Temporal Thickness have power values over 0.80 for the sample sizes exceeding 150, and the powers for the ordinal logistic model at small sample sizes (below 100) are somewhat lower than for the binary logistic regression models.

Graphics User Interface

Figure 3 is a screenshot corresponding to one healthy individual (Ctrl). It includes a 3D image of a characteristic cornea showing a schematic representation of how different predictors are calculated based on physical measurements.

Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 are five screenshots corresponding to five representatives of the five RETICS grades I–V. The score rapidly rises when passing from Ctrl to grade I and remains at 100% for the other grades, which is consistent with the high sensitivity and specificity in the model. Each prediction includes a 3D image of a characteristic cornea from the corresponding RETICS group, along with a schematic representation of how different predictors are calculated based on physical measurements.

The four examples in Figure 12 indicate the difficulty of detecting early-stage KC. Indeed, some cases are wrongly classified. We must bear in mind that this early KC detection model was trained using diagnostics made by ophthalmological professionals as a “gold standard”, and some inevitable undetermined amount of subjective information was taken for granted. During the fitting process, the model attempted to find some generalization to bind predictors and prediction with the best possible performance, but some samples might not match any kind of generalization given by training data’s subjective nature. Therefore, it is reasonable to expect some lack of accuracy, which does not necessarily mean failure in the model’s fitting capability. Our model quantitatively confirmed the difficulty of discerning both control and grade I groups as 17 of 106 cases (16%) and obtained a score between 0.1 and 0.9 which is, therefore, in the aforementioned “halfway” situation, and thus confirmed this tool’s utility.

4. Discussion

The range of techniques that allow the characterization and evaluation of the degree of KC severity vary, but the ultimate trends use machine-learning [44] and neural networks [45].

This study defines two automated learning models based on a set of independent variables (demographic, optical, pachymetric, morpho-geometric) to characterize the optical-geometrical cornea structure in different KC phases. More specifically, it considered a probability model of the risk associated with suffering early-stage KC, as well as another model to classify the degree of KC severity depending on patients’ visual limitation levels.

This approach offers some very useful advantages. First, it summarizes information that derives from many parameters of different natures (qualitative or quantitative, measured in distinct units) in a single number that can be easily read and understood, which minimizes the risk of some key pieces of information going unnoticed. Otherwise, this risk is fairly high, as common analytical reports usually contain many printed pages of different parameters, which often include no associated normality range and must be read in a matter of seconds. Second, it evaluates the combined action of these different parameters, which could imply a high value in a particular key parameter, clearly indicative of disease being present, but might be less evident when the increase in many different key parameters is slight. In this situation, a score can assist health professionals in their decision-making process as it provides an objective and quantitative scale that takes into account the joint action of a set of diverse parameters [3].

A multivariate logistic regression model was fitted using healthy (Ctrl) and RETICS grade I individuals, which always made predictions ranging between 0 and 1 when applied to new data. This prediction came closer to 0 when input data were similar to those that characterized healthy controls, and close to 1 if they were similar to grade I KC patients. All those individuals with an intermediate score were considered to be more or less halfway between both groups, i.e., they showed some kind of abnormality vs. healthy controls, which indicates KC-developing patients, but had not yet reached the cut-off point to be diagnosed as diseased. Therefore, this multivariate logistic regression output could be considered to be proportionally associated with the probability of suffering KC. A higher model output indicated a stronger probability. Even though this relationship is not linear, such scores are frequently used as predicting tools to assess patient status and prognosis in different fields [3,46,47,48].

For initial KC development, the analyzed model-based studies generally showed a good discrimination between normal and early KC eyes. However, one of the main challenges faced by ophthalmologists today is that no consensus has been reached by experts about the characterization patterns of early corneal ectasia [7,49]. This is mainly the result of the diversity of designations used to refer to subtler KC manifestations. It is also due to the fact that the wide variety of indices used for detection are technology-specific, which does not make them easily interchangeable [19,49]. As it is difficult to directly compare these studies, we resorted to using authors’ own writings to compare them to our work.

Several studies have reported similar results to ours about models based on Scheimpflug metrics [5,6,7,20]. Hwang et al. [7] proposed a first model based on the combination of five metrics (AUC: 0.86, Sensitivity: 83%, Specificity: 83%). Similar results have been obtained by other authors [5,6,20] in the model development phase and based on the exclusive metrics of the same Scheimpflug technology. In contrast, our study obtained higher performance values for model development (AUC: 0.95) and similar ones in the validation phase (AUC: 0.85), and was composed of metrics that are not exclusive to the same Scheimpflug technology.

Smadja et al. [50] used a machine-learning algorithm based on decision trees to analyze 55 parameters deriving from anterior and posterior corneal measurements. They found that the most discriminant variables related to posterior surface asymmetry and thickness spatial distribution achieved 93.6% sensitivity and 97.2% specificity when discriminating between normal and forme fruste KC. However, some authors [51] considered that including eyes from patients who had already been diagnosed with KC in one eye was inappropriate because it biases the sample if we contemplate that the genetic determinants for KC appearing are already present in them.

In our study, the performance measurements of the early detection model in AUC, specificity and sensitivity terms indicated very high performance with the training dataset, with all three values reaching around 0.95, and very few false positives and false negatives. These figures are significantly lower for the validation procedure, which dropped to about 0.85, but were well over 80% in all cases. These findings indicate good model-validated performance, but also suggest the presence of some overfitting. This is otherwise reasonable if we take into account the relatively few training cases (62 healthy individuals and 44 RETICS grade I patients), which were significantly lower than the training values.

Other authors have proposed using a multivariate system based on combining two different technologies. Saad et al. [9] combined two technologies to propose a model based on 54 variables and six discriminant functions, and reported 93% sensitivity and 92% specificity in the model development phase. It was validated in a later study with 92% sensitivity and 96% specificity [52]. Other studies have suggested combining several different technologies [53,54,55], but the authors defined a more advanced form of KC when they included patients with manifest inferior steepening [49,56].

The only reference found in the scientific literature with a classification system that uses visual acuity as a parameter is that by Wisse et al. [3], who established a scoring system that relied on five parameters (age, quality of vision, uncorrected distance visual acuity, refraction difference, maximum keratometry difference). However, this score did not classify disease grades, but disease progression. Consequently, its aim was to determine if crosslinking treatment would be necessary or not.

Our work presents and validates a probability model of the risk associated with suffering KC. Our research group is unaware of any previous study that combined demographic, optical, pachymetric and morpho-geometric variables successfully and in real-time to detect early KC.

Finally regarding the clinical KC development phase, experts’ criteria have converged to diagnose this disease as the degree of severity of progressing KC [24]. Although there are several classifications that characterize clinical KC, to the authors’ knowledge, there is no disease severity classifier based on patients’ visual limitation. Our classifier is based on an ordinary logistic regression model that combines 17 variables and presents an overall accuracy of 0.698; that is, our model correctly classifies almost 70% of patients. The performance indicators for ordinal logistic regression are fairly lower, particularly for the RETICS grade I, II and III patients, with balanced accuracies ranging between 0.623 and 0.766. Grade II patients present the worst accuracy, with nine in every 18 patients being wrongly classified as grade I. Once again, a small sample size is the most probable explanation for such behavior, along with its non-homogeneity (controls and grade I patients are much more abundant that grade II–V patients). It could also indicate that these three groups are not clearly differentiated. Therefore, all the conclusions drawn from the ordinal logistic regression model must be considered very carefully. Even when the results are reasonably good, the training process should be repeated with a bigger and more homogeneous sample. It would also be desirable to validate our results with other ethnicities and populations.

Another limitation of our study is the proven dependence that clinical metrics has on the technology employed to measure it [19], which means that our results are only valid for those eyes tested with a Sirius tomographer (CSO, Florence, Italy).

5. Conclusions

A web application was developed and deployed that combines two machine-learning models to support ophthalmologic professionals: a multivariate logistic regression model for early KC predictions and an ordinal logistic regression model to assign diagnosis grades on the RETICS scale. This application has a responsive design, and it allows any sort of device to be used (computer, tablet or smartphone). It also incorporates security measurements (authentication layer and accession from intranet only). The early KC prediction model shows high-performance indicators, even though some overfitting appears, while the RETICS grading prediction model’s accuracy is remarkably lower, particularly for grade I, II and III patients. In both cases, repeating the training process with a bigger sample should be considered. This falls in line with recently published recommendations for sample size calculations by multivariate prediction models [57] which, for this case, proposes an optimal sample size of 374 individuals, with a minimum of 125. An optimal figure could not be reached, given our biobank’s limited database size. Moreover, even though multivariate and ordinal logistic regressions are state-of-the-art and widely used techniques for modeling biomedical research data, many other powerful artificial intelligence techniques are available (particularly deep learning techniques), and their use is strongly advised for improving the quality of results.