ES-RED (Early Seizure Recurrence in the Emergency Department) Calculator: A Triage Tool for Seizure Patients

Seizure is a common neurological presentation in patients visiting the emergency department (ED) that requires time for evaluation and observation. Timely decision and disposition standards for seizure patients need to be established to prevent overcrowding in the ED and achieve patients’ safety. Here, we conducted a retrospective cohort study to predict early seizure recurrence in the ED (ES-RED). We randomly assigned 688 patients to the derivation and validation cohorts (2:1 ratio). Prediction equations extracted routine clinical and laboratory information from EDs using logistic regression (Model 1) and machine learning (Model 2) methods. The prediction equations showed good predictive performance, the area under the receiver operating characteristics curve showing 0.808 in Model 1 [95% confidential interval (CI): 0.761–0.853] and 0.805 in Model 2 [95% CI: 0.747–0.857] in the derivation cohort. In the external validation, the models showed strong prediction performance of 0.739 [95% CI: 0.640–0.824] in Model 1 and 0.738 [95% CI: 0.645–0.819] in Model 2. Intriguingly, the lowest quartile group showed no ES-RED after 6 h. The ES-RED calculator, our proposed prediction equation, would provide strong evidence for safe and appropriate disposition of adult resolved seizure patients from EDs, reducing overcrowding and delays and improving patient safety.


Introduction
Overcrowding and prolonged waiting time in the emergency department (ED) affect the safety and satisfaction of patients, especially critically ill patients. Consequently, efforts have been made to overcome these problems, such as creating a severity triage tool, using a standard working form, and relocating human resources [1][2][3][4][5]. In addition, repeated visits to the ED are one of the overcrowding-causing factors [6]. Therefore, it is essential to establish safe disposition standards for each disease.
Early seizure recurrence in the ED (ES-RED) within 24 h occurs in 13-18% of patients presenting with resolved seizures, with 85% of them occurring within six hours; therefore, a 6-24 h observation period is recommended for patients with seizures visiting a hospital [7][8][9]. An observation time of > 6 h, sometimes > 24 h due to the lack of convincing criteria for disposition of seizure patients, and the subsequent overcrowding in the ED is associated with patient safety [1,2,10]. Therefore, predicting whether and when ES-RED occurs in patients presenting with resolved seizures would enable timely decisions for their proper and safe disposition. This strategy could help relieve overcrowding and prevent delays in the ED.
A comprehensive analysis for predicting ES-RED and guidance for safe disposition of patients are lacking. Previous studies have focused on the individual risk factors of ES-RED as predictors and have demonstrated that several clinical factors such as age, sex, seizure characteristics, and alcohol consumption or laboratory findings such as venous blood gas, glucose levels, and sodium levels are associated with the ES-RED [7,8,11]. However, such research did not provide a pragmatic measure for the prediction of ES-RED. Therefore, this study aimed to propose a model for predicting ES-RED using routinely evaluated basic clinical information, imaging findings, and laboratory findings to facilitate timely decision making for the safe disposition from the ED of adult patients with resolved seizures in the ED.

Study Design and Population
This retrospective observational cohort study analyzed the electronic medical records of adult patients who presented with seizure as a chief complaint at an ED of a tertiary referral medical institution, from 1 March 2016 to 30 June 2019. The inclusion criteria were (1) age 18 years or older and (2) presenting with a seizure. Exclusion criteria were (1) status epilepticus finally diagnosed by epileptologists, (2) seizures occurred more than 24 h before visiting the hospital, (3) refused to be examined and treated in the ED, (4) transferred to other hospitals within four hours, or (5) had a suspected seizure mimic. A seizure mimic was diagnosed through detailed history taking, laboratory and electrophysiological studies in the ED, or during follow-up visits at the outpatient clinic of the neurology department, which included convulsive syncope, hyperventilation syndrome, altered mental state induced by drug intoxication, and psychogenic non-epileptogenic seizure.
Basic patient information was collected, such as gender, age, history of medical illnesses (including neurological illnesses), recent alcohol-drinking habits, sleep condition, and routine laboratory test results. When available, brain imaging and electroencephalography results were collected. We defined ES-RED as the seizure recurrence before discharge or within 24 h of visit [7][8][9]. In the case of discharge within 24 h, the patient and caregiver were recommended to revisit the ED if the seizure recurs. An acute symptomatic cause was defined as an acute brain insult temporally related to a seizure occurring within seven days, metabolic derangements detected during ED visits, or drug-induced seizures [12][13][14]. We divided the enrolled patients into 'ES-RED' and 'no-ES-RED' groups. This study was approved by the Institutional Review Board of Ajou University Hospital (AJIRB-MED-MDB-19-467). The requirement for informed consent was waived due to the study's retrospective nature.

Development of Prediction Models
The enrolled patients (n = 688) were randomly assigned to either the derivation or validation cohort (2:1 ratio). To generate a prediction model for ES-RED, baseline demographics, clinical characteristics, seizure characteristics and triggers, vital signs, neurological exam at presentation, and laboratory and imaging findings were analyzed within the derivation cohort. Then, the prediction models that were generated in the derivation cohort were directly applied to the validation cohort to estimate the predictive performance. Two different models were developed in the study: Model 1 used conventional logistic regression analysis for selecting variables, whereas Model 2 was based on a machine learning technique, the least absolute shrinkage and selection operator (LASSO).

Statistical Analysis
Variables are expressed as numbers (percentage) and median values (interquartile range (IQR)). Categorical and continuous variables from the ES-RED and no-ES-RED groups in the derivation cohort were compared using the Chi-squared test, Fisher's exact test, or Mann-Whitney U test [15,16]. The normality of the distribution was assessed using the Shapiro-Wilk test [17,18].

Model 1
Logistic regression analyses were performed within the derivation cohort to predict ES-RED. First, statistically significant variables from univariate logistic regression analyses (p < 0.05) were included in the multivariate logistic regression analysis. Then, clinically relevant and statistically feasible variables (p < 0.2) were selected again from the multivariable logistic regression analysis to generate the final beta estimates of the regression equation. The beta estimates were calculated using the backward stepwise logistic regression analysis with intercepts.

Model 2
Model 2 was generated within the derivation cohort using the LASSO machine learning technique. The rationale behind using the LASSO technique was to select the variables out of a large number of relevant variables used in our study. Variables that were statistically significant in the univariate analyses were included in the LASSO analysis. The penalty-tuning parameter (lambda) was estimated using ten-fold cross-validation. The optimal lambda was determined within one standard error of the minimal lambda. The variables selected using the optimal lambda were incorporated into the backward stepwise multivariate logistic regression analysis, as in Model 1, to calculate the beta estimates.
In each prediction model, the values of the generated prediction equations were compared between the derivation and validation cohorts. The median values were compared using the Mann-Whitney U test, and variances were compared using Levene's test. Receiver operating characteristic (ROC) curve analyses were performed within the derivation and validation cohorts. The area under the ROC curve (AUC), sensitivity, specificity, and accuracy were calculated to measure the predictive performances. The values from the generated equations were further divided into quartiles (Q1-Q4), and the association between the quartiles and the rate of ES-RED was analyzed. The association between the quartiles from each prediction model and ES-RED timing was also analyzed using the Kaplan-Meier curve. Statistical analyses were performed using SPSS 25.0 for Windows (SPSS Inc., Chicago, IL, USA) and R version 3.6.3 (R Foundation for Statistical Computing, Vienna, Austria).

Development of Prediction Models in the Derivation Cohort
In the derivation cohort, we determined independent risk factors for ES-RED using univariate and multivariate logistic regression analyses for Model 1 ( Table 2). In the univariate logistic regression, age; taking two or more ASMs; two or more seizures within 24 h before the ED visit; initial GCS score; initial SBP; levels of hemoglobin, serum glucose, albumin, uric acid, potassium, chloride, and lactic acid; and presence of acute or remote symptomatic causes of seizures were significantly associated with ES-RED. After incorporating these variables into the multivariate logistic regression, taking two or more ASMs; two or more seizures within 24 h before the ED visit; initial SBP (in mmHg); hemoglobin level (in g/dL); and serum glucose (in mg/dL), uric acid (in mg/dL), potassium (in mmol/L), and lactate levels (in mmol/dL) were finally selected for generating the following prediction equation (Equation (1);    For Model 2, the LASSO machine learning technique was used, and lambda was selected within one standard error of the minimal lambda. Age; taking two or more ASMs; two or more seizures within 24 h before the ED visit; initial SBP; GCS score on arrival; and hemoglobin, serum glucose, uric acid, and lactic acid levels were selected and incorporated into the final variables composing the following Equation (2) (Tables 3 and S1):  (Figure 1a). Sensitivity, specificity, and accuracy were calculated for each equation. In the derivation cohort, Equations (1) and (2) had 77.3% and 76.0% sensitivity, 74.5% and 75.1% specificity, and 75.0% and 75.2% accuracy, respectively ( Table 4). The derivation cohort subjects were divided into quartiles according to the equation outputs, and the ES-RED risk in each quartile was analyzed. The frequency of ES-RED was significantly different among quartile groups, with the higher quartile showing a higher ES-RED frequency; ES-RED rates in Q4 were > 40% in both equations (Equation (1)     AUC, area under the receiver operating characteristic curve; CI, confidence interval.  The cumulative incidence-analyzed using the Kaplan-Meier curve-from both equations showed that Q4 was associated with significantly higher ES-RED rates over time (p < 0.001; Figure 1c,d). Previous studies reported that most ES-REDs occurred within 6 h in the ED and stays more than 6 h contributed to overcrowding in the ED [11,13]. After that, we focused on ES-RED after 6 h in the ED. Most ES-REDs (89.3%) occurred within 6 h, similar to the previous report. Those who experienced ES-RED after 6 h were predominantly observed (75%) in the fourth quartile.

Validation of Prediction Equations
We applied the prediction equations directly to the validation cohort to estimate the predictive performances. In the validation cohort, there was no statistically significant difference in other variables except for the more focal features (22.5% vs. 12.4%, p = 0.003), two or more seizures within 24 h before presentation (26.9% vs. 18.9%, p = 0.016), and the slightly higher potassium level (4.1 mmol/L (IQR, 3.8-4.3 mmol/L) vs. 4.0 mmol/L (IQR, 3.73-4.20 mmol/L), p = 0.006) than the derivation cohort (Table S2).
The values from each prediction equation were calculated and compared between the derivation and the validation cohorts. There were no significant differences between the cohorts with regard to the median values and interquartile ranges (  (1) and (2) (2); Figure 2b). Most ES-RED occurred within 6 h, which is consistent with the derivation cohort. Interestingly, only the highest quartile (Q4) showed ES-RED in the validation cohort after 6 h (Figure 2c,d).
Subsequently, patients who experienced ES-RED after 6 h were analyzed. In the total cohort (derivation + validation cohorts), 11 patients with ES-RED after 6 h were identified. None of these 11 patients belonged to Q1 in either Equation (1) or (2), while 9 (81.8%) of these patients belonged to Q4 in Equations (1) and (2), thereby suggesting a disposition criterion based on the generated prediction equations.

Discussion
In this single-center, retrospective cohort study of adult patients presenting with resolved seizures, we generated prediction models after exploring the factors associated with ES-RED. We found that clinical and laboratory parameters can successfully predict ES-RED, thereby developing two prediction models. Between the two equations, due to Equation (1) being simpler, having a slightly better predictive performance, and requiring fewer variables than Equation (2), we propose using Equation (1) as an ES-RED calculator to predict ES-RED.

Discussion
In this single-center, retrospective cohort study of adult patients presenting with resolved seizures, we generated prediction models after exploring the factors associated with ES-RED. We found that clinical and laboratory parameters can successfully predict ES-RED, thereby developing two prediction models. Between the two equations, due to Equation (1) being simpler, having a slightly better predictive performance, and requiring fewer variables than Equation (2), we propose using Equation (1) as an ES-RED calculator to predict ES-RED.
Just three studies have investigated the risk factors for ES-RED in adult patients with resolved seizures, to the best of our knowledge. These studies reported that alcoholism, history of seizure, age, gender, number of seizures before hospitalization, and levels of pH, bicarbonate, base excess, lactic acid, sodium, and calcium in venous blood tests were associated with early recurrence of seizures [7,8,11]. However, these findings have limited utility in the clinical field because they did not provide a measure to predict ES-RED. Variables in our prediction equations are consistent with previous studies of risk factors for anytime seizure recurrence. Gultekingil et al., reported younger age, taking multiple ASMs, multiple seizure events within 24 h, and abnormal neurological examination or neuroimaging findings regarding the risk factors for seizure recurrence in the pediatric observation unit of the ED [19]. Kim et al., reported the number of seizures, neurological disorders, and an abnormal EEG finding as significant predictors of seizure recurrence after a single seizure [20]. A review by Rizvi et al., reported that older or younger age, female gender, partial seizure, multiple seizure events, remote symptomatic etiology, and abnormal neurological examination were risk factors for seizure recurrence after the first seizure event [21]. The prediction equations presented in the current study include variables presented in previous studies, such as age, multiple seizures before visiting the ED, abnormal GCS score, glucose level, and serum lactate level. On the other hand, our equations included variables not mentioned previously, namely SBP, hemoglobin level, serum potassium level, and uric acid level.
In our study, uric acid showed a negative correlation with ES-RED. Although the exact underlying mechanism is unclear, some studies have investigated uric acid's role in seizure disorders. Wang et al., showed a U-shaped association between serum uric acid levels and post-stroke epilepsy [22]. Our institute also reported that low uric acid levels help distinguish refractory status epilepticus from responsive status epilepticus patients [23]. The inverse correlation between uric acid and ES-RED in the current and previous studies suggests a potential beneficial effect of uric acid on various seizure disorders. Further research is needed on uric acid's role in preventing seizures.
Another notable finding in this study is that neuroimaging and electroencephalography findings were not independently associated with ES-RED. A study reported an increase in hospital stay by approximately three hours for the acquisition of electroencephalography and neuroimaging study [24]. The findings in our study suggest that waiting for several hours in the ED to take electroencephalography and neuroimaging tests is unnecessary unless essential.
Our study showed that patients with the lowest quartile (less than −2.65 in Equation (1) and less than −1.09 in Equation (2)) in ES-RED prediction equations had no recurrence after six hours. These findings could help determine the monitoring duration or disposition of seizure patients with low values in our equations. In addition, patients in the highest quartile (more than −1.06 in Equation (1) or more than 0.51 in Equation (2)) comprised > 80% of patients who suffered from ES-RED after six hours. This finding could help provide evidence for early admission of such patients with high values in Equations (1) and (2) to the observation zone for neurological monitoring. Otherwise, information on the risk of seizure recurrence can be given to the patients and caregivers. These results could have a positive impact on reducing overcrowding in the ED.
The present study has limitations. First, our study is a single-center retrospective observational cohort study. However, the strength is that we included more subjects than in previous studies and proposed the predictive equation using routinely evaluated clinical information and laboratory findings in the ED. Second, our study site was a tertiary referral medical center, and patients with minor symptoms may have been transported to other hospitals. Finally, the ES-RED calculator may seem more complicated than the other scoring systems because not all input values are integers. However, most hospitals use computerized systems and can automatically link laboratory values to ES-RED calculators. This automated system would be easier to apply in a real-world clinical situation because the physician would only need to fill out simple clinical information. In addition, it could be applied as a decision-making system by using artificial intelligence through machine learning techniques.
In summary, our study identifies predictive factors for ES-RED and proposes the ES-RED calculator, a prediction equation. Overcrowding and delays in the ED are important issues, and seizure is a commonly reported neurologic symptom in the ED, which requires seizure patients to stay in the ED for a long time. Our identified factors and proposed ES-RED calculator could help reduce overcrowding and delay in the ED through early, safe, appropriate, and convincing disposition of adult resolved seizure patients.

Supplementary Materials:
The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/jcm11133598/s1, Figure S1: Flow chart of patient selection, Table S1: LASSO analyses of variables associated with ES-RED in the derivation cohort, Table S2: Comparison of variables between the derivation and validation cohorts.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Ajou University Hospital (AJIRB-MED-MDB-19-467).

Informed Consent Statement:
The requirement for informed consent was waived by the Institutional Review Board of Ajou University Hospital due to the study's retrospective nature.

Data Availability Statement:
The data that support the findings of this study are available upon reasonable request to the corresponding author. Direct correspondence regarding this article to Jun Young Choi.

Conflicts of Interest:
The authors declare that there is no conflict of interests regarding the publication of this article. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.