You are currently viewing a new version of our website. To view the old version click .
MDPIMDPI
Search all articles
Healthcare
Download PDF
  • Article
  • Open Access

22 June 2023

A Comparison of the Charlson and Elixhauser Methods for Predicting Nursing Indicators in Gastrectomy with Gastric Cancer Patients

and
1
Department of Nursing, Chungbuk National University, Cheongju 28644, Republic of Korea
2
Asan Medical Center, University of Ulsan College of Medicine, Seoul 05505, Republic of Korea
*
Author to whom correspondence should be addressed.
Healthcare2023, 11(13), 1830;https://doi.org/10.3390/healthcare11131830
This article belongs to the Special Issue Nursing Contributions to Improve Healthcare Outcomes
Version Notes
Order Reprints
Review Reports

Abstract

Background: Comorbidity indices such as Charlson’s (CCI) and Elixhauser’s (ECI) are used to adjust the patient’s care, depending on the severity of their condition. However, no study has compared these indices’ ability to predict nursing-sensitive outcomes (NSOs). We compared the performance of CCI and ECI in predicting NSOs in gastric cancer patients’ gastrectomy. Methods: Gastric cancer patients with gastrectomy, aged 19 years or older and admitted between 2015 and 2016, were selected from the Korea Insurance Review and Assessment Service database. We examined the relationships between NSOs and CCI or ECI while adjusting patient and hospital characteristics with logistic regression. Results: The ECI item model was the best in view of the C-statistic and Akaike Information Criterion for total NSO, physiologic/metabolic derangement, and deep vein thrombosis, while the Charlson item model was the best for upper gastrointestinal tract bleeding. For the C-statistic, the ECI item model was the best for in-hospital mortality, CNS complications, shock/cardiac arrest, urinary tract infection, pulmonary failure, and wound infection, while the CCI item model was the best for hospital-acquired pneumonia and pressure ulcers. Conclusions: In predicting 8 of 11 NSOs, the ECI item model outperformed the others. For other NSOs, the best model varies between the ECI item and CCI item model.

1. Introduction

With the increase in the size of the aging population, the incidence of cancer is also increasing. Stomach cancer is one of the most frequently occurring cancers in South Korea, and cases of gastric cancer numbered 26,662 in 2020 [1]. The major reasons for the higher number of gastric cancer cases are high-salt food, spicy and charred meat, smoking, drinking, and Helicobacter pylori infection [2]. The main treatment for gastric cancer without metastasis is gastrectomy [3]. The incidence of gastrectomy is the highest among gastrointestinal (GI) operations for cancer in Korea [4].
A reliable measure for surgical quality is always necessary. Direct measures of surgical outcomes are the best for measuring quality [5]. In the same context, measuring nursing outcomes are the best for measuring nursing quality [6]. Nursing-sensitive outcomes (NSOs) serve as valuable tools for monitoring and improving nursing quality. They are indicators of the patient’s health status that result directly from the care provided by nurses [7]. Those for adults include in-hospital death, central nervous system (CNS) complications, shock or cardiac arrest, hospital-acquired pneumonia, upper gastrointestinal tract bleeding, pressure ulcers, deep vein thrombosis, urinary tract infection, wound infection, pulmonary failure, sepsis, and physiologic/metabolic derangement [8,9]. Nonetheless, it is necessary to adjust these indicators based on patient acuity to effectively compare variations in nursing quality among different hospitals [10].
The quality and safety of healthcare, including nursing care, is becoming more and more important, despite of finite resources. In accordance with this, skill-mixing and nursing modes need to be redesigned. In this regard, more research is focused on addressing the impact of nursing components such as nurse staffing, nurse characteristics, clinical nursing support, and the nursing practice environment on the quality and safety of patient care [11]. The impact of hospital structures and the processes of nursing care on outcomes can be examined with adjustments in patient factors [12]. Therefore, an appropriate risk adjustment model is essential in nursing quality studies [5]. Administrative databases are large, and they are easy to access for research into care quality after multiple surgical procedures and nursing care [13,14,15,16].
Comorbidity indices such as Charlson’s (CCI) and Elixhauser’s (ECI) from an administrative database are widely used to adjust the patient’s risk level in health services research [17]. In 1987, Charlson et al. developed a method for patient risk adjustment using administrative data, which includes 19 clinical diagnoses (due to their relevance to 1-year mortality). The index is the sum of the weighted score for these diagnoses. Several studies have validated this index [18,19,20]; since then, it has been the one most widely used to assess comorbidity. After publication of ICD-10 codes, the Charlson’s method was also translated from ICD-9 to ICD-10 codes [21].
Another popular method for patients’ risk adjustment is that of Elixhauser et al., created in 1998 [22]. It includes 30 comorbidity measures, so is more comprehensive. The Elixhauser comorbidity method can be used to predict in-hospital mortality, hospitalization length, and hospital charges. Complications during hospitalization should be distinguished from pre-existing diagnoses before admission when using this index.
There are several studies that compare the performance of the Charlson and Elixhauser methods in various population [23,24,25,26,27,28]. The Elixhauser method seems better than the Charlson method for risk adjustment because it has a higher C-statistic than the Charlson method. Lieffers et al. [23] studied these methods with colorectal cancer patients for survival prediction, Quail et al. [24] with a general population, diabetic patients, and osteoporosis patients for mortality and hospitalization prediction, and Chang et al. [25] and Lee et al. [26] for oral cancer survival. Tsai et al. [27] showed that in patients with schizophrenic disorder, for mortality prediction, ECI was superior, and Baron et al. [28] reported that ECI was better than CCI in predicting minor complications, airway complications, hemorrhagic anemia, myocardial infarction, cardiac arrest, urinary tract infection, and pulmonary embolism after anterior lumbar interbody fusion operations.
However, no study has compared these indices’ ability to predict nursing-sensitive outcomes (NSOs) in patients with gastric resections. Consequently, it is not known how accurately these methods can adjust for NSOs in patients with gastric resections. We used the Korean Health Insurance Review and Assessment Service (Korea HIRA) database to investigate the incidences of comorbid conditions, and to compare the predictive performance of CCI and ECI for the NSOs of gastric cancer patients with gastrectomy.

Objective

The objective of this study was to compare the performance of two common indices in predicting the NSOs of patients with gastric resections for gastric cancer.
The more specific objectives are
  • To identify incidences of comorbid conditions using the Korean HIRA reimbursement data of gastric resection patients with gastric cancer;
  • To compare the predictive performance of CCI and ECI for the NSOs of gastric cancer patients with gastrectomy.

2. Materials and Methods

The methodology employed in this study is akin to the authors’ prior research that utilized a database from the identical institution [29]. However, the aim and the operative procedure under consideration—a comparison between the Charlson and Elixhauser indices concerning gastrectomy for primary gastric cancer—differ from the authors’ previous investigation.

2.1. Database

We utilized the Korea HIRA database, encompassing reimbursement claims for inpatient and outpatient services from 2015 to 2016. All hospitals in Korea, including tertiary and general ones, must submit the reimbursement claims of their patients to HIRA electronically. This database contains patient demographics (sex and age), diagnoses up to 30, all types of operations and procedures provided, and the details of patient care (including the usage of an emergency room or intensive care unit). Diagnoses are coded by the International Classification of Diseases, tenth revision (ICD-10), while procedure codes are from the Korean claim code of HIRA.

2.2. Participants

2.2.1. Inclusion Criteria

Using electronic reimbursement claims data, we identified all patients aged 19 years or older who had undergone gastrectomy for gastric cancer (identified by ICD-10 code C16, which refers to malignant neoplasm of the stomach) between January 2015 and December 2016. The patient information we used was anonymized.

2.2.2. Exclusion Criteria

Patients without nationwide individual identification numbers were excluded from the analysis, as their previous inpatient and outpatient claim data could not be tracked, and their comorbidity scores (measured by Charlson’s and Elixhauser’s comorbidity indices) could not be calculated. In addition, only primary cancers were included in the analysis, while metastatic cancers and those originating from adjacent organs, such as the colon, pancreas, or gallbladder, were excluded.

2.3. Patient and Hospital Characteristics

The patient characteristics used in the analysis were sex, age, and medical security type. The medical security type can be used as an indicator of socioeconomic status (SES); health insurance represents a relatively higher SES, and medical aid represents a lower SES.
The characteristics of clinical practice were the route of admission, the use of ICU, the type of gastrectomy (subtotal or total), and lymph node dissection. Route of admission is a measure of relative urgency and is categorized as outpatient or emergency room.
The hospital characteristics included hospital location and type. The hospital location was categorized as Seoul (the capital of Republic of Korea), metropolitan, and city. The hospital type was classified as tertiary hospital, general hospital, or hospital.

2.4. Measures

2.4.1. Comorbidity Methods

We compared the two comorbidity indices, Charlson (CCI) and Elixhauser (ECI). The Charlson method includes 17 diagnostic categories, while the Elixhauser method includes 30 diagnostic categories. Comorbidities for the Charlson’s and Elixhauser’s indices were identified by the ICD-10 code. Each patient’s comorbidities were identified by the ICD-10 code within six months before admission, using outpatient and inpatient service records. However, newly diagnosed comorbidities identified during the gastrectomy admission were not included, because those cannot be distinguished from the complications of inpatient treatment. Therefore, CCI and ECI were calculated with the identified pre-existing comorbidities at the time of hospitalization for the gastrectomy.

2.4.2. Nursing-Sensitive Outcomes

Eleven NSOs were selected, including in-hospital death, CNS complications, shock/cardiac arrest, hospital-acquired pneumonia, upper GI tract bleeding, pressure ulcers, deep vein thrombosis, urinary tract infection, wound infection, pulmonary failure, and physiologic/metabolic derangement. The definition of each NSO was based on Needleman et al. [30], who used ICD-9 codes, and Wilson et al. [7], who used ICD-10 codes and Australian DRG. ICD-10 codes and Korean diagnosis-related groups (DRG) to match each NSO were determined with the consultation of HIRA’s medical recorder and researchers. The dividend, divisor, and exclusion criteria for each NSO were defined as the above, and the 11 NSO incidences were calculated using HIRA’s inpatient reimbursement data.

2.5. Statistical Analysis

Data analysis was performed using SAS® 9.4 (Cary, NC, USA) software. Descriptive statistics, including mean, standard deviation (SD), frequency, and proportion, were used to summarize patient and hospital characteristics and NSO status. The admission case was considered the unit of elementary event. The differences in proportions and means by patient and hospital characteristics were tested using the chi-squared test and Student’s t-test. Univariate and multiple logistic regression were used to examine the relationship between CCI or ECI and NSOs, adjusted with patient and hospital characteristics. The base model included admission route (outpatient or emergency room as an urgency indicator), gastrectomy type (total or subtotal), lymph node dissection, hospital location, and hospital type. Item models are models in which CCI or ECI disease items are added to the base model as independent variables. Continuous models are models in which CCI or ECI scores are added as quantitative independent variables. Categorical models are models in which categorized CCI (Charlson’s score <3 or ≥3) or ECI (Elixhauser’s score < 0, 0, 1–4, ≥5) are added to the base model. Two-sided tests were used for all statistical tests, and statistical significance was deemed with p < 0.05. The Akaike information criterion (AIC) and C-statistic were the indicators used to assess the performance and discriminative ability of the methods over the base model, which includes sex, age, medical security type, admission route, gastrectomy type, lymph node dissection, hospital location, and hospital type. The AIC was calculated as 2k–2ln(L), where k is the number of parameters of the model, and L is the maximum likelihood. A smaller AIC value corresponds with the better data explanation ability of the model. The C-statistic shows the discrimination ability, with values of 0.5 (no ability), 0.7 to 0.8 (acceptable), 0.8 to 0.9 (excellent), 0.9 to 1.0 (outstanding), and 1 (perfect discrimination).

3. Results

3.1. Patient and Hospital Characteristics

The study included 33,323 inpatient cases of gastrectomy for gastric cancer in Korea between 2015 and 2016, among which 3545 (10.64%) had at least one NSO indicator. Patients with at least one NSO were significantly different from those without any NSO indicator in terms of sex, age, medical security type, admission route, ICU experience, gastrectomy type, hospital location, hospital type, Charlson’s comorbidity score, and Elixhauser’s comorbidity score (p < 0.001). Patients with at least one NSO had a longer average length of hospital stay (27.92 days) compared to those without any NSOs (23.23 days) (p < 0.001). Lymph node dissection did not show a significant difference in NSO rate (p = 0.247; Table 1).
Table 1. Distribution of hospital and patient characteristics.
The mean age of patients with at least one NSO indicator was 65.99 years, which was higher than that of patients without any NSO indicator (61.46 years) (p < 0.0001). Males had a higher incidence rate of NSO (11.33%) compared to females (9.25%) (p < 0.001). Patients with medical aid had a higher NSO rate (19.51%) than patients with health insurance (p < 0.001). The NSO rate was higher for patients admitted via emergency room than those admitted via outpatient department (p < 0.001). Patients who underwent total gastrectomy had a higher NSO rate than those who underwent subtotal gastrectomy (p < 0.001). The NSO rate was higher in patients who had experienced ICU care and had higher Charlson’s or Elixhauser’s comorbidity scores (p < 0.001). Patients admitted to hospitals in large metropolitan areas or tertiary hospitals had a lower NSO rate compared to those admitted to hospitals in city areas or non-tertiary hospitals (p < 0.001).

3.2. Charlson and Elixhauser Comorbidities in Patients

The comorbidity rates of peptic ulcer, diabetes mellitus and hypertension were more than 5% in patients with gastrectomy. Comorbidities such as AIDS/HIV, lymphoma, obesity, and drug abuse were less than 0.1%.
Patients with any NSO indicator had significantly different Charlson’s comorbidities than their counterparts, including myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, chronic pulmonary disease, rheumatic disease, peptic ulcer disease, mild and moderate liver disease, diabetes mellitus without and with end-organ damage, hemiplegia, renal disease, any malignancy, and metastatic solid tumor (p < 0.05). There was no significant difference in NSO rate by AIDS/HIV (p = 0.7301).
Patients with any NSO indicator had significantly different Elixhauser’s comorbidities than their counterparts, including congestive heart failure, cardiac arrhythmias, valvular disease, pulmonary circulation disorders, peripheral vascular disorders, hypertension, paralysis, neurodegenerative disorders, chronic pulmonary disease, diabetes mellitus both uncomplicated and complicated, hypothyroidism, renal failure, liver disease, peptic ulcer disease excluding bleeding, metastatic cancer, solid tumor without metastasis, rheumatoid arthritis/collagen vascular diseases, coagulopathy, weight loss, fluid and electrolyte disorders, blood loss anemia, deficiency anemia, alcohol abuse, drug abuse, psychoses, and depression (p < 0.05). There was no significant difference in NSO rate by AIDS/HIV, lymphoma, and obesity (p > 0.05; Table 2).
Table 2. Distribution of Charlson and Elixhauser comorbidities in patients.

3.3. Adjusted Odds Ratio of Nursing Indicators Using Charlson and Elixhauser Comorbidities

Table 3 shows the non-adjusted and adjusted odds ratios for the NSO indicators. Congestive heart failure, myocardial infarction, cerebrovascular disease, hemiplegia, dementia, peripheral vascular disease, rheumatic disease, chronic pulmonary disease, mild and moderate liver disease, peptic ulcer disease, diabetes mellitus with or without end-organ damage, renal disease, metastatic solid tumor, and any malignancy in Charlson’s comorbidities were significant factors after adjustment with patient- and hospital-related characteristics (p < 0.05). Those diseases significantly increased the risk of NSO incidence. Additionally, Charlson’s comorbidity score (≥3) had an odds ratio of 1.99, compared to the Charlson’s comorbidity score, which was less than 3 (Table 3).
Table 3. Adjusted odds ratio of nursing indicators using Charlson and Elixhauser comorbidities.
Elixhauser’s comorbidities with the adjustment of control variables were significant factors for cardiac arrhythmias, congestive heart failure, paralysis, hypertension, pulmonary circulation disorders, chronic pulmonary disease, peripheral vascular disorders, neurodegenerative disorders, renal failure, diabetes mellitus both uncomplicated and complicated, peptic ulcer disease without bleeding, hypothyroidism, liver disease, solid tumor without metastasis, metastatic cancer, rheumatoid arthritis/collagen vascular diseases, weight loss, coagulopathy, fluid and electrolyte disorders, alcohol abuse, deficiency anemia, depression, and psychoses (p < 0.05). Those diseases significantly increased the risk of NSO incidence. However, valvular disease, blood loss anemia, obesity, lymphoma, and drug abuse were not significant factors (p > 0.05). Additionally, Elixhauser’s comorbidity score (0) had an odds ratio of 0.52, compared to the Elixhauser’s comorbidity score, which was less than 0 (Table 3).

3.4. Comparison of Charlson and Elixhauser Comorbidities with Respect to Nursing Indicator

Table 4 shows the AIC and C-statistics for the Charlson and Elixhauser comorbidity methods adjusted with sex, age, medical security type, admission route, gastrectomy type, lymph node dissection, hospital location, and hospital type. The Elixhauser item model was the best comorbidity risk adjustment method with the lowest AIC, highest C-statistic and receiver operating characteristic (ROC) curve for total NSOs (AIC 20518.49, C-statistics 0.729, Figure 1), deep vein thrombosis (AIC 2539.31, C-statistics 0.741), and physiologic/metabolic derangement (AIC 9405.67, C-statistics 0.733). The Charlson item model was the best comorbidity risk adjustment method with the lowest AIC and highest C-statistic for upper GI tract bleeding (AIC 4753.04, C-statistics 0.793). The Elixhauser item model was the best comorbidity risk adjustment method with the highest C-statistic for in-hospital mortality (C-statistics 0.863), CNS complications (C-statistics 0.804), shock/cardiac arrest (C-statistics 0.779), urinary tract infection (C-statistics 0.746), pulmonary failure (C-statistics 0.785), and wound infection (C-statistics 0.799). The Charlson item model was the best comorbidity risk adjustment method with the highest C-statistic for hospital-acquired pneumonia (C-statistics 0.747) and pressure ulcers (C-statistics 0.759). When the comorbidities were summed as a weighted single score, the Elixhauser item model performed better than the Charlson item model (Table 4).
Table 4. Comparison of Charlson and Elixhauser comorbidities with respect to nursing indicators.
Figure 1. ROC curve of nursing-sensitive outcomes by model.
As the emphasis on improving surgical and nursing quality intensifies, evaluating the preoperative risks of surgical candidates remains crucial. The patient’s particular comorbidity burden is one of the most crucial factors to consider during this assessment. While individual comorbidities have been linked to an increased risk of several postoperative adverse events such as NSOs, the CCI or the ECI is typically used to assess a patient’s comorbidity burden [31]. Despite being frequently used, there is no literature available on whether these two measures can predict NSOs in gastrectomy with gastric cancer.
We were able to validate each of the two methods for this patient population, as overall C-statistics were more than 0.72. A C-statistic approximating 0.75 is considered acceptable for discrimination, and validates methods for ongoing use [16]. This result showed the excellent performance of the model for in-hospital mortality and CNS complications, with C-statistics of 0.80–0.90, and acceptable performance regarding other NSOs, with C-statistics of 0.70–0.80 [28]. However, Baron et al. [28] showed the poor performance (C-statistics < 0.70) of CCI for cardiac arrest, pneumonia, and surgical site infection, as well as age, sex, race, and primary payer, not including clinical and hospital characteristics such as route of admission, the use of intensive care unit, type of gastrectomy, hospital location, and hospital type. We used these clinical and hospital characteristics in addition to ECI or CCI, so the C-statistics exceeded 0.70. Meanwhile, the C-statistic was 0.695 in deep vein thrombosis without ECI or CCI. Therefore, it is necessary to use ECI or CCI to accurately predict NSOs in addition to clinical and hospital characteristics, since the clinical and hospital characteristics were associated with occurrence of NSO.
This study found the Elixhauser model was a better predictor of total NSOs, and eight of the eleven NSOs, than the CCI model. This study found the ECI statistically superior to the CCI in predicting in-hospital mortality, CNS complications, shock/cardiac arrest, deep vein thrombosis, physiologic/metabolic derangement, urinary tract infection, pulmonary failure, and wound infection during the inpatient postoperative period. In other words, the ECI model also predicted infection-related complications more effectively. The reason for this may be the inclusion of compromised immune function in the ECI, but not in the CCI [28]. Meanwhile, the CCI model significantly outperformed the ECI in the prediction of three of these NSOs: hospital-acquired pneumonia, upper GI tract bleeding, and pressure ulcers. Additionally, models using Elixhauser and Charlson items were better at risk adjustment than models with CCI or ECI as a continuous or categorized variable.
These results are similar to those of previous studies on anterior cervical discectomy and fusion by Ranson et al. [31]. The studies in patients with anterior cervical discectomy and fusion showed that the CCI model was a poor predictor of any complication after operation, and the ECI model was an excellent method for predicting deep vein thrombosis, in-hospital mortality, wound infection, septic shock, and pulmonary embolism (compared to CCI). Meanwhile, the CCI model was better in the prediction of hospital-acquired pneumonia than ECI [31]. Baron et al. [28] reported that the ECI model was a better method than CCI for predicting cardiac arrest, pulmonary embolism, and wound infection. Grendar et al. [16] showed that the CCI model was a poor predictor of in-hospital mortality in gastric resection. In addition, the CCI model was an acceptable predictor of any major complication, but a poor predictor of any minor complication. The Elixhauser model was slightly better, producing acceptable predictions of major complications, but poor predictions of minor complications. These results suggest that both models have some values in predicting postoperative complications [31]. However, the selection of the adjustment model could vary between the ECI item and CCI item models for specific nursing indicators, based on the predictive performance. Therefore, it is better to use the ECI item model for predicting in-hospital mortality, CNS complications, shock/cardiac arrest, deep vein thrombosis, physiologic/metabolic derangement, urinary tract infection, pulmonary failure, and wound infection, and to use the CCI item model for predicting hospital-acquired pneumonia, upper GI tract bleeding, and pressure ulcers in inpatients with gastrectomy. This multiple logistic model may be incorporated into electronic nursing recording (ENR), and automatically warn of the risk of NSO.
Meanwhile, the C-statistics of CCI and ECI models for in-hospital mortality were the highest, at 0.853 and 0.863, respectively. These results reflect that CCI and ECI were developed to predict 1-year mortality and in-hospital mortality. To improve the prediction power of each indicator, we need to develop an adjustment method by indicator, such as the Braden scale, or the surgical pressure ulcer risk scale for pressure ulcers [32,33]. We could not adjust using Braden scale for pressure ulcers, since the measurement of the Braden scale was not included in the reimbursement data, although a nurse may have measured the Braden scale of a given patient at the time of their hospital admission [34]. To measure nursing quality effectively, the policy should include the Braden scale or a score of surgical site infection [35] in the administrative data.
The superior performance of the ECI can likely be attributed to its better characterization of the underlying comorbidity burden, achieved by adding more comorbidities not included in the CCI [36]. Some of the patient factors included in the ECI but excluded from the CCI have been identified as strong predictors of NSOs for gastric resection, such as pulmonary circulation disorders, fluid and electrolyte disorders, and coagulopathy. The range of odds ratios of comorbidities in CCI was from 1.12 (metastatic solid tumor) to 2.67 (congestive heart failure). However, the range of odds ratios of comorbidities in ECI was from 1.31 (peptic ulcer disease excluding bleeding) to 10.15 (pulmonary circulation disorder). There are five comorbidities in ECI with odds ratios more than 2.67: pulmonary circulation disorder, paralysis, neurodegenerative disorders, coagulopathy, and fluid and electrolyte disorders. Patients with severe coagulopathy and fluid and electrolyte disorders [16] undergoing gastric resection are at greater risk of in-hospital mortality. Therefore, nurses should frequently monitor patients’ conditions and provide nursing care to prevent the occurrence of NSOs in inpatients with pulmonary circulation disorder, paralysis, neurodegenerative disorders, coagulopathy, and fluid and electrolyte disorders.
In this study, the ORs of AIDS/HIV and drug abuse could not be obtained because there were so few patients with AIDS/HIV (n = 1), and drug abuse (n = 2). The prevalence of AIDS/HIV and drug abuse was very low in Korea. In 2021, in Korea, the total number of patients with AIDS/HIV was 15,196 [37], and the number of patients with drug abuse was 12,613 [38]. It is necessary to select comorbidities by considering their prevalence in the country concerned. A significant association between four Elixhauser-listed comorbidities (vascular disease, lymphoma, obesity, blood loss anemia) and nursing-sensitive outcomes was not found. The prevalence of comorbidities for patients with gastrectomy in this study was from 0% (1 case) to 8.78%. They do not seem to be significant for those diseases with a low volume of patients, such as lymphoma (n = 24) and obesity (n = 22). The obesity prevalence in our study was only 0.07%, which was much lower than the 38.3% in the Korean adult population in 2020 [39]. Our dataset was obtained from the reimbursement database of HIRA in the years 2015~2016. There is a possibility the comorbidity rate is different at the current time, but most hospitals in Korea did not report obesity in the insurance claim if the patients were not treated for obesity at the time of admission. However, patients at the extremes of BMI (underweight or morbidly obese) seem to have the highest postoperative morbidity and mortality risk [40]. It is necessary to record BMI in administrative databases to screen patients at a high risk of developing complications.
Several potential limitations must be considered when interpreting the results of this study. Firstly, large administrative databases may underreport chronic medical conditions. Secondly, the use of ICD-10 codes to identify patient comorbidities may result in coding errors that cannot be fully eliminated. Additionally, as a retrospective study, it may be subject to biases and confounding factors. Moreover, the CCI and ECI models used in this study may not be generalizable to populations receiving different treatments.

4. Conclusions

The ECI was found to be more effective than the CCI in predicting eight out of eleven NSOs. Since accurate prediction and prevention of NSOs can enhance patient outcomes and nursing quality, using the ECI (with an item model) to forecast inpatient complications after elective gastrectomy should be taken into consideration. Identifying patients at high risk of developing complications in hospital in the early stages may facilitate the development of intervention strategies to reduce modifiable risk factors. Additionally, the methods demonstrated in this study are utilized by HIRA and other investigators to monitor levels of nursing quality. This monitoring process aims to improve nursing quality and provide patients with information regarding the quality of nursing care in Korean hospitals. Subsequent research should evaluate the performance of ECI in predicting different types of NSOs arising from various operations.

Author Contributions

Conceptualization, C.-G.K. and K.-S.B.; methodology, C.-G.K. and K.-S.B.; software, K.-S.B.; validation, C.-G.K. and K.-S.B.; formal analysis, C.-G.K.; investigation, C.-G.K.; resources, K.-S.B.; data curation, C.-G.K.; writing—original draft preparation, C.-G.K.; writing—review and editing, K.-S.B.; supervision, K.-S.B.; project administration, K.-S.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Asan Medical Center (AMC) institutional review board (IRB) approved this study (Approval No. 2018-0245) in accordance with international ethics standards.

Data Availability Statement

Authors analyzed the data within the database system of the Korea Health Insurance Review and Assessment Service (a government agency), which cannot be transferred to outside the agency.

Acknowledgments

This article was previously presented as a poster titled “Comparison of Charlson and Elixhauser Method for Predicting Nursing Indicators in Gastrectomy with Gastric Cancer Patients” at the ESMO Congress 2022, 9–13 September 2022, in Paris, France. We thank Joon Seo Lim from the Scientific Publications Team at Asan Medical Center for his editorial assistance in preparing the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Korea Central Cancer Registry, National Cancer Center. Annual Report of Cancer Statistics in Korea in 2020. Ministry of Health and Welfare. 2022. Available online: https://ncc.re.kr/cancerStatsList.ncc?searchKey=total&searchValue=&pageNum=1 (accessed on 28 February 2023).
  2. Noh, S.H.; Yonsei Stomach Cancer Center. The Cure Manual of Stomach Cancer; Health Chosun: Seoul, Republic of Korea, 2016. [Google Scholar]
  3. Guideline Committee of the Korean Gastric Cancer Association (KGCA), Development Working Group & Review Pane. Korean Practice Guideline for Gastric Cancer 2018: An Evidence-based, Multi-disciplinary Approach. J. Gastric. Cancer 2019, 19, 1–48. [Google Scholar] [CrossRef]
  4. National Health Insurance Service. Main Surgery Statistical Yearbook for 2019. National Health Insurance Service. 2020. Available online: https://www.nhis.or.kr/nhis/together/wbhaec06800m01.do?mode=download&articleNo=10803657&attachNo=312917 (accessed on 28 February 2023).
  5. Dimick, J.B. How Should We Risk-Adjust Hospital Outcome Comparisons? Comment on “Predicting In-Hospital Mortality in Patients Undergoing Complex Gastrointestinal Surgery”. Arch. Surg. 2012, 147, 135–136. [Google Scholar] [CrossRef]
  6. Moritz, P. Innovative nursing practice models and patient outcomes. Nurs. Outlook 1991, 39, 111–114. [Google Scholar]
  7. Wilson, S.; Bremner, A.P.; Hauck, Y.; Finn, J. Evaluation of paediatric nursing-sensitive outcomes in an Australian population using linked administrative hospital data. BMC Health Serv. Res. 2013, 13, 396. [Google Scholar] [CrossRef] [PubMed]
  8. Schreuders, L.W.; Bremner, A.P.; Geelhoed, E.; Finn, J. Using linked hospitalisation data to detect nursing-sensitive outcomes: A retrospective cohort study. Int. J. Nurs. Stud. 2014, 51, 470–478. [Google Scholar] [CrossRef] [PubMed]
  9. Twigg, D.E.; Gelder, L.; Myers, H. The impact of understaffed shifts on nurse-sensitive outcomes. J. Adv. Nurs. 2015, 71, 1564–1572. [Google Scholar] [CrossRef] [PubMed]
  10. White, P.; McGillis Hall, L.; Lalonde, M. Adverse patient outcomes. In Nursing Outcomes: The State of the Science; Doran, M.D., Ed.; Jones & Bartlett Learning: Sudbury, MA, USA, 2011; pp. 241–284. [Google Scholar]
  11. Havaei, F.; MacPhee, M.; Dahinten, V.S. The effect of nursing care delivery models on quality and safety outcomes of care: A cross-sectional survey study of medical-surgical nurses. J. Adv. Nurs. 2019, 75, 2144–2155. [Google Scholar] [CrossRef]
  12. Tourangeau, A.E.; Doran, D.M.; McGillis, H.L.; O’Brien, P.L.; Pringle, D.; Tu, J.V.; Cranley, L.A. Impact of hospital nursing care on 30-day mortality for acute medical patients. J. Adv. Nurs. 2007, 57, 32–44. [Google Scholar] [CrossRef]
  13. Myers, R.P.; Quan, H.; Hubbard, J.N.; Shaheen, A.A.; Kaplan, G.G. Predicting in-hospital mortality in patients with cirrhosis: Results differ across risk adjustment methods. Hepatology 2009, 49, 568–577. [Google Scholar] [CrossRef]
  14. Davila, J.A.; El-Serag, H.B.G.I. GI epidemiology: Databases for epidemiological studies. Aliment. Pharmacol. Ther. 2007, 25, 169–176. [Google Scholar] [CrossRef]
  15. Aiken, L.H.; Sloane, D.M.; Bruyneel, L.; Van den Heede, K.; Griffiths, P.; Busse, R.; Diomidous, M.; Kinnunen, J.; Kózka, M.; Lesaffre, E.; et al. RN4CAST consortium. Nurse staffing and education and hospital mortality in nine European countries: A retrospective observational study. Lancet 2014, 383, 1824–1830. [Google Scholar] [CrossRef]
  16. Grendar, J.; Shaheen, A.A.; Myers, R.P.; Parker, R.; Vollmer, C.M., Jr.; Ball, C.G.; Quan, M.L.; Kaplan, G.G.; Al-Manasra, T.; Dixon, E. Predicting in-hospital mortality in patients undergoing complex gastrointestinal surgery: Determining the optimal risk adjustment method. Arch. Surg. 2012, 147, 126–135. [Google Scholar] [CrossRef]
  17. Austin, S.R.; Wong, Y.N.; Uzzo, R.G.; Beck, J.R.; Egleston, B.L. Why summary comorbidity measures such as the Charlson comorbidity index and Elixhauser score work. Med. Care 2015, 53, e65–e72. [Google Scholar] [CrossRef]
  18. Zhang, J.X.; Iwashyna, T.J.; Christakis, N.A. The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims. Med. Care 1999, 37, 1128–1139. [Google Scholar] [CrossRef]
  19. Moro-Sibilot, D.; Aubert, A.; Diab, S.; Lantuejoul, S.; Fourneret, P.; Brambilla, E.; Brambilla, C.; Brichon, P.Y. Comorbidities and Charlson score in resected stage I nonsmall cell lung cancer. Eur. Respir. J. 2005, 26, 480–486. [Google Scholar] [CrossRef]
  20. Kastner, C.; Armitage, J.; Kimble, A. The Charlson comorbidity score: A superior comorbidity assessment tool for the prostate cancer multidisciplinary meeting. Prostate Cancer Prostatic Dis. 2006, 9, 270–274. [Google Scholar] [CrossRef]
  21. Quan, H.; Sundararajan, V.; Halfon, P.; Fong, A.; Burnand, B.; Luthi, J.C.; Saunders, L.D.; Beck, C.A.; Feasby, T.E.; Ghali, W.A. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med. Care 2005, 43, 1130–1139. [Google Scholar] [CrossRef]
  22. Elixhauser, A.; Steiner, C.; Harris, D.R.; Coffey, R.M. Comorbidity measures for use with administrative data. Med. Care 1998, 36, 8–27. [Google Scholar] [CrossRef]
  23. Lieffers, J.R.; Baracos, V.E.; Winget, M.; Fassbender, K. A comparison of Charlson and Elixhauser comorbidity measures to predict colorectal cancer survival using administrative health data. Cancer 2011, 117, 1957–1965. [Google Scholar] [CrossRef]
  24. Quail, J.M.; Lix, L.M.; Osman, B.A.; Teare, G.F. Comparing comorbidity measures for predicting mortality and hospitalization in three population-based cohorts. BMC Health Serv. Res. 2011, 11, 146. [Google Scholar] [CrossRef]
  25. Chang, H.J.; Chen, P.C.; Yang, C.C.; Su, Y.C.; Lee, C.C. Comparison of Elixhauser and Charlson Methods for Predicting Oral Cancer Survival. Medicine 2016, 95, e2861. [Google Scholar] [CrossRef] [PubMed]
  26. Lee, C.C.; Ho, H.C.; Su, Y.C.; Chen, P.C.; Yu, C.H.; Yang, C.C. Comparison of different comorbidity measures for oral cancer patients with surgical intervention: A longitudinal study from a single cancer center. Auris Nasus Larynx 2016, 43, 322–329. [Google Scholar] [CrossRef] [PubMed]
  27. Tsai, K.Y.; Hsieh, K.Y.; Ou, S.Y.; Chou, F.H.; Chou, Y.M. Comparison of Elixhauser and Charlson Methods for Discriminative Performance in Mortality Risk in Patients with Schizophrenic Disorders. Int. J. Environ. Res. Public Health 2020, 17, 2450. [Google Scholar] [CrossRef] [PubMed]
  28. Baron, R.B.; Neifert, S.N.; Ranson, W.A.; Schupper, A.J.; Gal, J.S.; Cho, S.K.; Caridi, J.M. A Comparison of the Elixhauser and Charlson Comorbidity Indices: Predicting In-Hospital Complications Following Anterior Lumbar Interbody Fusions. World Neurosurg. 2020, 144, e353–e360. [Google Scholar] [CrossRef]
  29. Kim, C.G.; Bae, K.S. Relationship between nurse staffing level and adult nursing-sensitive outcomes in tertiary hospitals of Korea: Retrospective observational study. Int. J. Nurs. Stud. 2018, 80, 155–164. [Google Scholar] [CrossRef]
  30. Needleman, J.; Buerhaus, P.; Mattke, S.; Stewart, M.; Zelevinsky, K. Nurse-staffing levels and the quality of care in hospitals. N. Engl. J. Med. 2002, 346, 1715–1722. [Google Scholar] [CrossRef]
  31. Ranson, W.A.; Neifert, S.N.; Cheung, Z.B.; Mikhail, C.M.; Caridi, J.M.; Cho, S.K. Predicting In-Hospital Complications After Anterior Cervical Discectomy and Fusion: A Comparison of the Elixhauser and Charlson Comorbidity Indices. World Neurosurg. 2020, 134, e487–e496. [Google Scholar] [CrossRef]
  32. Huang, C.; Ma, Y.; Wang, C.; Jiang, M.; Foon, L.Y.; Lv, L.; Han, L. Predictive validity of the braden scale for pressure injury risk assessment in adults: A systematic review and meta-analysis. Nurs. Open. 2021, 8, 2194–2207. [Google Scholar] [CrossRef]
  33. Aloweni, F.; Ang, S.Y.; Fook-Chong, S.; Agus, N.; Yong, P.; Goh, M.M.; Tucker-Kellogg, L.; Soh, R.C. A prediction tool for hospital-acquired pressure ulcers among surgical patients: Surgical pressure ulcer risk score. Int. Wound J. 2019, 16, 164–175. [Google Scholar] [CrossRef]
  34. Korea Institute for Healthcare Accreditation. 3rd Accreditation Standards for Acute Hospital; Korea Institute for Healthcare Accreditation: Seoul, Republic of Korea, 2018. [Google Scholar]
  35. Tu, R.H.; Huang, C.M.; Lin, J.X.; Chen, Q.Y.; Zheng, C.H.; Li, P.; Xie, J.W.; Wang, J.B.; Lu, J.; Cao, L.L.; et al. A scoring system to predict the risk of organ/space surgical site infections after laparoscopic gastrectomy for gastric cancer based on a large-scale retrospective study. Surg. Endosc. 2016, 30, 3026–3034. [Google Scholar] [CrossRef]
  36. Southern, D.A.; Quan, H.; Ghali, W.A. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med. Care 2004, 42, 355–360. [Google Scholar] [CrossRef]
  37. Korea Disease Control and Prevention Agency. 2021 Management of HIV/AUDS. 2022. Available online: https://www.kdca.go.kr/contents.es?mid=a20301070504 (accessed on 28 February 2023).
  38. The Korean Academy of Science and Technology. The Current Status of Drug Abuse and Policies to Solve the Problems; KAST Research Report; The Korean Academy of Science and Technology: Seongnam, Republic of Korea, 2019. [Google Scholar]
  39. Statistics Korea Government. E-national Indicator: Prevalence of Obesity. 2022. Available online: https://www.index.go.kr/unity/potal/main/EachDtlPageDetail.do?idx_cd=2705 (accessed on 28 February 2023).
  40. Valentijn, T.M.; Galal, W.; Tjeertes, E.K.; Hoeks, S.E.; Verhagen, H.J.; Stolker, R.J. The obesity paradox in the surgical population. Surgeon 2013, 11, 169–176. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Download PDF
Effect of Scapular Fixation on Lateral Movement and Scapular Rotation during Glenohumeral Lateral Distraction Mobilisation
Previous
Nurses and Physicians’ Perceptions Regarding the Role of Oncology Clinical Nurse Specialists in an Exploratory Qualitative Study
Next