An Open-Access Dataset of Hospitalized Cardiac-Arrest Patients: Machine-Learning-Based Predictions Using Clinical Documentation

Abstract

Cardiac arrest is a sudden loss of heart function with serious consequences. In developing countries, healthcare professionals use clinical documentation to track patient information. These data are used to predict the development of cardiac arrest. We published a dataset through open access to advance the research domain. While using this dataset, our work revolved around generating and utilizing synthetic data by harnessing the potential of synthetic data vaults. We conducted a series of experiments by employing state-of-the-art machine-learning techniques. These experiments aimed to assess the performance of our developed predictive model in identifying the likelihood of developing cardiac arrest. This approach was effective in identifying the risk of cardiac arrest in in-patients, even in the absence of electronic medical recording systems. The study evaluated 112 patients who had been transferred from the emergency treatment unit to the cardiac medical ward. The developed model achieved 96% accuracy in predicting the risk of developing cardiac arrest. In conclusion, our study showcased the potential of leveraging clinical documentation and synthetic data to create robust predictive models for cardiac arrest. The outcome of this effort could provide valuable insights and tools for healthcare professionals to preemptively address this critical medical condition.

1. Introduction

Dysfunction in the heart’s conduction system frequently leads to cardiac arrest, which results in the heart’s inability to pump blood efficiently. Ventricular fibrillation is the primary cause of cardiac arrest in 65–80% of cases [1]. Numerous heart-related conditions can lead to cardiovascular complications, including coronary artery disease, cardiomyopathy, inherited conditions, congenital heart disease, heart valve disease, acute myocarditis, and conduction disorders, such as long QT syndrome. However, many patients may not recognize or ignore symptoms, with chest pain being the most common indicator of potential cardiac issues.

Ischemic coronary illness causes over 70% of cardiac arrests and is the leading cause of cardiac dysfunction. Risk factors include hypertension, hyperlipidemia, diabetes, smoking, age, and a family history of coronary disease [2]. In Sri Lanka, cardiovascular diseases (CVD) accounted for a high mortality rate of 534 deaths per 100,000 [3]. The risk of cardiac arrest within the first 24 h after a heart attack was 20–30%, and the survival rate was around 25%, even with proper medical treatment.

Timely help and treatment are crucial for survival in sudden cardiac arrest cases. Early warning systems (EWSs) could help identify patients at an increased risk of deterioration and subsequent death due to cardiac arrest [4]. Patients admitted to the intensive care unit (ICU) due to sudden cardiac arrest have frequently exhibited symptoms of clinical deterioration hours prior to the event, which could have been detected early using an early warning system (EWS). A study carried out by Marinkovic [5] demonstrated that patients who had higher EWS scores before having a cardiac arrest experienced the worst outcomes. This highlighted the need for an EWS for the early detection and treatment of cardiac arrest in order to reduce the mortality rate of this condition. Therefore, an EWS was designed based on patients’ vital signs to aid this process [6].

According to previous studies, single parameter and aggregate weighted track and trigger systems (AWTTS) were used by multiple EWSs [7]. Many EWSs have been developed using statistical methods and linking observations. Some have been developed based on clinical consensus, and there has been no common ground of agreement regarding whether one EWS’s performance was better than another EWS’s [8].

In machine-learning and deep-learning domains, the deep-learning–based early warning system (DEWS) model was stated as the first model, which was developed using a deep-learning technique to predict cardiac arrest [9]. When exploring further, it was observed that machine learning and AI technology have played a pivotal role in the early prediction of cardiac arrest [9,10,11,12,13,14,15,16,17,18,19,20].

In low- to middle-income countries (LMIC), The effectiveness of using an EWS developed in high-income countries (HIC) may be compromised, as the algorithms may not be sufficiently sensitive and specific to identify patients at risk of deterioration [21]. Furthermore, most of the risk algorithms have been based on European populations, and it was unclear whether they would be valid for South Asian populations [22]. Although all the previously mentioned machine-learning models utilized electronic medical records, it is essential to note that LMIC countries like Sri Lanka remain unfamiliar with electronic medical records (EMR) beyond clinical documentation, making it challenging to find data and models that can meet their specific requirements.

Therefore, the research aim of this study was to develop a model that considered these unique needs and limitations by making several important contributions:

• Publishing an open-access bed-head-ticket dataset.
• Introducing a machine-learning model that could predict the risk of fatal cardiac arrest and showed improved results.
• Analyzing the dataset with machine-learning models to compare the usability of the dataset.

Repository links

• Dataset (Zenodo) https://zenodo.org/record/7603772
• Model (GitHub) https://github.com/LahiruRajapaksha/cardiac-arrest-prediction-using-bed-head-tickets.git

2. Materials and Methods

2.1. Methodology

To begin, Figure 1 illustrates a detailed workflow that explains different stages and procedures undertaken during our study. The research was designed as a retrospective cohort study, focusing on patients admitted to the cardiac ward between 13 August 2018 and 6 February 2020, at the Teaching Hospital, Karapitiya (THK), Galle, Sri Lanka. The study population included patients who had been transferred to the cardiac ward from the emergency treatment unit (ETU), excluding pediatric patients. A total of 112 patients, aged 15–89 years, were included in the study, with 82 male patients and 30 female patients. The bed head ticket (BHT) dataset is available at Zenodo (https://zenodo.org/record/7603772), licensed as Creative Commons Attribution 4.0 International (CC BY 4.0) [23].

Ethical clearance was obtained from the Ethics Review Committee of the Faculty of Medicine, University of Ruhuna, Galle, Sri Lanka, and the study adhered to the relevant guidelines and regulations established by the committee. Permission to access data was granted by the Director of the Teaching Hospital, Karapitiya. As this was a retrospective study, obtaining informed consent from all subjects was not applicable.

Data collection involved extracting information from the BHTs in the hospital’s record room. Due to the absence of electronic clinical data, each BHT was manually examined to collect the necessary data. The BHTs contained information on the patient’s health status, clinical history, management actions, investigations, treatments, progress, and diagnosis.

The extracted BHT data were categorized into five main categories: demographic features, examinations, lab reports, patient histories, and outcomes (Figure 2). The extracted features, observed at a rate of over 60% (Figure 3) within the study group, were selected for model development.

2.2. Data Pre-Processing

As illustrated in Figure 4, the extracted clinical data were primarily divided into two categories: time-series data and non-time-series data. The time-series data consisted of patient information recorded in relation to time, while non-time-series data encompassed the remaining data. The clinical dataset contained 21 features: age, heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure (DBP), respiratory rate (RR), body temperature (BT), blood oxygen saturation level (SpO₂), level of consciousness, troponin level, total cholesterol level, fasting blood sugar (FBS), serum electrolytes (sodium, potassium, chloride), urea level, creatinine level, triage score (risk score upon admission), alcohol consumption, smoking status, family history of ischemic heart diseases (FHIHD), and hospital outcome.

Of the 21 features, 19 were selected for inclusion, while 3 were excluded from the final feature list. Troponin level, total cholesterol level, and fasting blood sugar level, which could be used to detect co-morbidities in patients, were excluded due to insufficient records in the BHTs for the majority of patients. Data pre-processing consisted of two steps. First, 19 common features were selected, including the patient hospital outcome (target variable). Second, missing value imputation was performed for the selected features. In cases of missing data for a particular feature in a patient, the data value obtained most recently to that missing value was considered, or, if not available, the median value of the data was used.

When considering time-series data (SBP, DBP, HR, RR, BT, SpO₂), patients were monitored hourly throughout their hospital stay. In the selected dataset, patients were monitored for at least 1 h and a maximum of 266 h (11 days). A suitable time step (time window) for observation was chosen within this range. The time step was determined based on the average observation time of a patient (52 h). This 52-hour observational time window was considered the prediction window of the model. For the non-time-series data, results from the patient’s lab tests on the hospital admission date were also considered.

2.3. Data Division and Preparation

After pre-processing the dataset, we partitioned the dataset into three distinct subsets at a ratio of 8:1:1, namely the training set, testing set, and validation set. Then, the training dataset was further enhanced through the infusion of synthetic data generated by the SDV (Figure 5). The goal of the SDV was to build generative models of relational databases [24]. By incorporating its capability of creating artificial instances that could mimic the statistical properties of the original dataset, we expanded our training data, fostering greater diversity and aiding the model’s ability to capture complex patterns.

When generating the synthetic data for experiments 2–4, our consistent aim was to achieve a balanced ratio of survived-patient data to dead-patient data at 1:1, ensuring equal representation (

Table 1. Summary of the number of patient records and sequences contained in training datasets. Except for experiment 01, we maintained the survived/dead-patient ratio as 1:1.

	Training Dataset 01		Training Dataset 02		Training Dataset 03		Training Dataset 04
	No. of Sequences	No. of Records	No. of Sequences	No. of Records	No. of Sequences	No. of Records	No. of Sequences	No. of Records
Real Survived-Patient Data	4287	74	4287	74	0	0	4287	74
Synthetic Survived-Patient Data	0	0	0	0	4870	74	4870	74
Real Dead-Patient Data	743	15	743	15	0	0	743	15
Synthetic Dead-Patient Data	0	0	1768	59	4702	74	5259	133
Total	5030	89	6798	148	9572	148	15,159	296

).

2.4. Model Architecture

Integrating artificial intelligence into clinical practice has led to numerous studies focused on predicting adverse events, such as cardiac arrest, before they occur. Evidence from these studies suggested that deep-learning models were more efficient at identifying high-risk patients than existing EWSs [25]. The Recurrent neural network (RNN) models were particularly well-suited for handling temporal sequence data. Among the RNN variants, long short-term memory (LSTM) demonstrated remarkable performance among various sequence-based tasks [26]. Recent studies have also shown that deep-learning models employing RNN architectures with LSTM outperformed clinical prediction models developed using logistic regression [27,28]. Consequently, we chose the LSTM structure to model the temporal relationships within the data extracted from the BHTs.

2.4.1. LSTM Model

The deep-learning cardiac arrest prediction model (DLCAPM) consisted of a LSTM model that was designed to handle time-series data, followed by a dense layer. During the model’s development, we employed a sigmoid activation function for the dense layer, the Adam optimizer with default parameters, and binary-cross entropy as the loss function. The complete dataset was divided into 20% for validation and testing and 80% for training. Inputs to the LSTM model included SBP, DBP, HR, RR, BT, and SpO₂ levels. The input data fed into the LSTM comprised a three-dimensional array as no. of patients × time step × no. of input features (5030, 6798, 9572, and 15,159 patient record sequences, respectively), as illustrated in Figure 6. The first phase of the model handled temporal data, while the second phase combined the results of the time-series data with the non-time-series data. The combined data were then input into the decision-tree model to predict the final outcome based on the model’s results.

2.4.2. Decision-Tree Model

In the medical domain, the clinical practice involves continuous decision-making, where optimal decision-making strives to maximize effectiveness and minimize loss [29]. Clinical decision analysis (CDA) highlighted the significant role of decision trees in this process [30]. Among the five methodologies employed for decision-making, designing a decision tree was considered one [31]. Due to their reliability, effectiveness, and high accuracy in decision-making, decision trees have been widely utilized in various medical decision-making studies [32]. These factors led us to select the decision tree as the model to handle non-time-series data, including the LSTM outcome. The LSTM model was designed to generate a risk score based on time-series data acquired within a time window of 52 h.

2.4.3. Latent Vector Space

Latent vector space was employed in machine learning to analyze data that could be mapped to a latent space, where similar data points were in close proximity. In simpler terms, it could be described as a representation of compressed data. This latent space representation retained all the essential information required to represent the original data points, thereby facilitating data analysis. Latent space representations have been utilized to transform more complex forms of raw data, such as images and videos, into simpler representations, a concept implemented in representation learning.

In the context of the LSTM model, the latent vector space served as an input parameter for the decision-tree model. The decision-tree inputs were a combination of non-time-series data, including demographic information, lab results, triage score, Glasgow-coma-scale (GCS) values, and patient histories, as well as the latent output of the LSTM model (Figure 7). This approach allowed for a more comprehensive analysis by integrating both time-series and non-time-series data, enhancing the model’s decision-making capabilities.

2.4.4. Handling the Data-Imbalance Problem

Class imbalance has been a common challenge in real-world data, particularly in medical fields, making it difficult to optimize machine-learning algorithm performance. In this study, there were 93 majority-class (survived) and 19 minority-class (dead) patients, yielding a 1:5 ratio.

To address the class imbalance, the synthetic minority over-sampling technique (SMOTE) was used first to over-sample the minority class, generating synthetic data and achieving a 1:1 class ratio. However, as was recently demonstrated by Van den Goorbergh, SMOTE did not improve discrimination but, instead, led to significantly miscalibrated models [33]. Due to this reason, we moved forward with a SDV to address the data-imbalance problem.

3. Results

In this result section, we present the results of a series of comprehensive experiments conducted to assess the performance of the deep-learning cardiac arrest prediction model. To enhance the robustness of our experiments, we incorporated an SDV to generate synthetic data, permitting us to thoroughly assess the predictive capabilities of the model. (

Table 2. Experiments performed by incorporating synthetic data in order to evaluate the results of the combined model. RS = Real Survived, RD = Real Dead, SS = Synthetic Survived, SD = Synthetic Dead. The survived/dead ratio of the patients was maintained at 1:1.

Experiment No.	RS	RD	SS	SD	Accuracy
01	74	15	-	-	0.954
02	74	15	-	59	0.954
03	-	-	74	74	0.964
04	74	15	74	133	0.967

).

After executing the designed experiments, we were able to achieve peak accuracy through experiment 4, where the experiment encompassed a dataset of 296 patient records. The hyper-parameters were optimized according to

Table 3. Hyper-parameter values.

No. of Epochs	Learning Rate	Batch Size	LSTM Nodes	Optimizer
100	0.001	10	2	Adam

.

The creatinine levels were the most critical predictor, followed by the sodium and blood-urea levels. Studies have suggested that these markers were related to renal function, which was linked to cardiovascular diseases [34]. Additionally, potassium levels, FHIHD, and age played key roles in classifying cardiac patients, supporting the association between cardiovascular diseases and decreased potassium levels [35].

FHIHD was recognized as a well-established risk factor for cardiovascular diseases [36]. In the study’s findings, the risk factor FHIHD emerged as one of the most reliable predictive features for cardiac arrest. (Figure 8) displays the probabilistic prediction window for six randomly selected patients (three from each of the two classes). The LSTM model was proficient in providing predictions from admission up to 52 h later. In other words, the model’s prediction window spanned 52 h. The model demonstrated a prediction accuracy of 96% with a confidence interval from 95.01% to 95.85%.

3.1. Comparison with Existing Models

We assessed the performance of the developed deep-learning cardiac arrest prediction model (DLCAPM) by comparing it with well-established machine-learning algorithms, such as logistic regression, random forest, naïve Bayes, and support vector machine (SVM). The evaluation used all the features incorporated in the DLCAPM model.

Table 4. Performance comparison with existing machine-learning models (The Bolded entries of LSTM & decision tree were the two models used in the deep-learning cardiac arrest prediction model (DLCAPM)).

Model	Accuracy	Sensitivity	Specificity	PPV	NPV	F-Score
LSTM	0.96	0.95	0.93	0.98	0.81	0.86
Decision Tree	0.76	0.69	0.81	0.72	0.79	0.80
SVM	0.89	0.84	0.82	0.81	0.82	0.84
Logistic Regression	0.88	0.93	0.81	0.92	0.81	0.87
Random Forest	0.88	0.89	0.90	0.87	0.81	0.91
Naïve Bayes	0.85	0.89	0.80	0.82	0.88	0.91

presents the performance metrics, including accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F-score, for each of the compared models. The results demonstrated that the LSTM component of the DLCAPM model exhibited superior performance in comparison to the selected models.

3.2. Correlation Analysis

The correlation analysis evaluated the relationships among the model’s input features. Pearson’s and Spearman’s coefficients have commonly been used: The former was for normally distributed variables, and the latter for skewed or ordinal variables. A coefficient near ±1 indicated a strong correlation, either positive or negative [37]. In this study, Spearman’s rank correlation was used due to the Gaussian distribution, and Figure 9 shows the heat map of the correlational coefficients. A high positive correlation existed between systolic and diastolic blood pressure values.

3.3. Characteristics of the Study Population

In the cohort study, patient data were analyzed to assess the observed characteristics of the study population, as shown in

Table 5. Characteristics of the study population.

Characteristic	Data
Study period	13 August 2018–6 February 2020
Hospital	Teaching Hospital Karapitiya, Galle, Sri Lanka
Total patients	112
Input vectors	19
Age group	59–76 years
Male (%)	73%
Symptoms before admission	Chest pain on the left side (1/2 h before the admission), Tightening of the chest, Vomiting, Sweating, Nausea, Cough, Fever
Patients with FHIHD (%)	37%
Consume alcohol, Male (%)	69%
Smoking, Male (%)	63%
Smoking & use alcohol, Male (%)	26%

. From these observations, we could infer that males may be more susceptible to cardiovascular diseases and cardiac arrests. This study also examined the impact of various risk factors that could potentially contribute to developing CVD. Among these risk factors, alcohol consumption, smoking, and an FHIHD were identified as the most significant contributors. Of the total population (males and females), 37% of individuals reported having an FHIHD. Notably, none of the female patients were documented to consume alcohol or smoke. Among the male patients, 73% (81 patients) were found to engage in at least one of these behaviors, with 69% (55 patients) consuming alcohol, 63% (51 patients) smoking, and 26% (21 patients) engaging in both alcohol consumption and smoking.

4. Discussion

The model’s performance was assessed using two optimization algorithms, Adam and Admax. Multiple iterations were performed, and hyper-parameter combinations were used to identify the four most optimal configurations, which yielded the highest accuracy.

Table 6. Performance of the model on four experimental datasets. Note that we tested the LSTM and the decision-tree models separately.

Metric	LSTM				Decision Tree
	Experiment Number				Experiment Number
	01	02	03	04	01	02	03	04
Accuracy	0.93	0.85	0.94	0.96	0.80	0.83	0.80	0.76
Precision	0.90	0.88	0.90	0.98	0.68	0.70	0.66	0.72
Recall	0.91	0.85	0.91	0.95	0.65	0.68	0.63	0.69
F-Score	0.81	0.86	0.81	0.86	0.82	0.85	0.82	0.80
AUC score	0.97	0.95	0.97	0.98	0.79	0.83	0.79	0.75

presents the evaluation metrics for each of these four selected runs. Among these experiments, the best results were achieved in experiment 04 for training the model.

In a study related to ours, researchers had utilized a dataset comprised of 15 electronic-medical-record (EMR) data parameters [38]. Interestingly, eight of these parameters, namely age, gender, diastolic blood pressure (DBP), systolic blood pressure (SBP), body temperature, respiratory rate (RR), blood pressure (BP), and creatinine, were also a part of our study. However, their research had a more extensive dataset of 34,452 patients, whereas our study had a limited sample of 112 patients. Due to the limited size of our dataset and the class-imbalance issue, we employed the synthetic data vault to generate a training dataset, while the other previous study had utilized the SMOTE technique. Nonetheless, considering the datasets of both studies, it might be beneficial to explore cross-validation approaches for the models.

Table 7. Evaluation metrics of LSTM and decision-tree models of the deep-learning cardiac arrest prediction model.

Statistic	Value		95% CI
	LSTM Model	Decision-Tree Model	LSTM Model	Decision-Tree Model
Sensitivity	95.83%	69.57%	95.37% to 96.25%	47.08% to 86.79%
Specificity	93.42%	81.82%	92.07% to 94.61%	64.54% to 93.02%
Positive Predictive Value	98.71%	72.73%	98.44% to 98.93%	55.19% to 85.24%
Negative Predictive Value	81.04%	79.41%	79.37% to 82.60%	67.07% to 87.96%

shows the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy measures for the outperformed LSTM and the decision-tree models in experiment 04, respectively.

The receiver-operating characteristic (ROC) curve is a graphical representation that demonstrates the diagnostic ability of binary classifiers by plotting sensitivity against specificity. A better-performing classifier would have a curve closer to the top-left corner. To compare classifier performances, a common approach has been to calculate the area under the ROC curve. Figure 10a presents the ROC curve for the LSTM model.

However, the visual interpretations and comparisons of ROC curves could be misleading for imbalanced datasets. To address this issue, precision–recall curves were utilized. Figure 10b illustrates the precision–recall curve for the LSTM model, and the supplementary figure shows the curve for the decision-tree classifier model.

The confusion matrix, which is crucial for statistical classifications in machine learning, is a table describing a classification model’s performance and identifying class confusion. Figure 11a,b displays the confusion matrices for the LSTM and decision-tree models, respectively.

4.1. Comparison with Existing EWSs

DLCAPM was evaluated against existing cardiac arrest early warning scores, including the modified early warning score (MEWS), the cardiac arrest risk triage score (CART), and the National Early Warning Score (NEWS), based on research by [10,16,39,40]. MDCalc (https://www.mdcalc.com/ accessed on 26 August 2023), which has been referred to as the most broadly used clinical decision tool by medical professionals, was used to calculate risk scores for CART, MEWS, and NEWS, using collected data (RR, SpO₂, BT, SBP, DBP, HR, age, triage score).

Table 8. Performance comparison of DLCAPM (deep-learning cardiac arrest prediction model, a combination of LSTM and decision-tree models) against CART (cardiac arrest risk triage score), MEWS (modified early warning score), and NEWS (National Early Warning Score) models. Bolded entries are the results of the DLCAPM.

Model	Accuracy	Sensitivity	Specificity	PPV	NPV	F-Score
LSTM	0.96	0.95	0.93	0.98	0.81	0.86
Decision Tree	0.76	0.69	0.81	0.72	0.79	0.80
CART	0.60	0.50	0.75	0.75	0.75	0.60
MEWS	0.80	0.93	0.40	0.82	0.4	0.50
NEWS	0.80	0.84	0.66	0.94	0.66	0.66

displays the results for each score.

4.2. Limitations

The record room at the Teaching Hospital, Karapitiya (THK) utilized Microsoft Excel to store limited data from bed head tickets, making it necessary to manually review each patient’s record in order to obtain the required information for this study, which was time-consuming. The small sample size, the potential patient heterogeneity, and the focus on a specific patient population within a single hospital unit limited the research’s generalizability, potentially rendering it inapplicable nationwide. These limitations stemmed from the legacy methods of maintaining patient data and the difficulties in retrieval.

Due to limited resources, only 112 patient records were extracted, which prevented reaching the desired sample size for the model. To address this limitation, a SDV was employed to increase the minority sample size. Furthermore, the study faced constraints due to the scarcity of local research on the development of cardiac arrest EWSs in Sri Lanka [21,41]. While the FHIHD feature provided a high-level understanding of a patient’s family history and whether they had any heart-related conditions, it only offered a limited view of the patient’s generational tree. Recent studies have revealed that sudden cardiac deaths have occurred in completely asymptomatic individuals who carried pathogenic variants in genes linked to sudden cardiac death [42]. However, this study did not collect any genetic data, which was another limitation.

5. Conclusions

An efficient deep-learning cardiac risk prediction model was developed using clinical features based on the BHTs of THK cardiac patients. This simple model used accessible patient data and could offer bedside support for healthcare workers and assist the decision-making process. An open-access dataset was published to encourage further research.

Early and accurate predictions can aid in timely interventions and prevent cardiac events. Despite the model’s high accuracy, addressing data limitations could further improve the model. Additional research is needed to explore its applicability in other healthcare settings and its integration with existing patient monitoring tools for enhanced predictions.

6. Future Work

To enhance the development of more generalized models, we aim to conduct future studies with overlapping datasets, as we have a substantial amount of published clinical data that share similar information. Furthermore, we plan to complement our existing datasets by including this dataset in cross-validations with diverse geographical populations.