Next Article in Journal
Effects and Mechanisms of Calcium Ion Addition on Lead Removal from Water by Eichhornia crassipes
Previous Article in Journal
Canadian Arctic Contaminants and Their Effects on the Maternal Brain and Behaviour: A Scoping Review of the Animal Literature
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 17
Issue 3
10.3390/ijerph17030927
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assess the Performance and Cost-Effectiveness of LACE and HOSPITAL Re-Admission Prediction Models as a Risk Management Tool for Home Care Patients: An Evaluation Study of a Medical Center Affiliated Home Care Unit in Taiwan

by
Mei-Chin Su
1,2,
Yi-Jen Wang
3,4,
Tzeng-Ji Chen
2,3,5,
Shiao-Hui Chiu
1,
Hsiao-Ting Chang
3,5,
Mei-Shu Huang
1,
Li-Hui Hu
1,
Chu-Chuan Li
1,
Su-Ju Yang
1,
Jau-Ching Wu
5,6 and
Yu-Chun Chen
2,3,5,*
1
Department of Nursing, Taipei Veterans General Hospital, Taipei 11217, Taiwan
2
Institute of Hospital and Health Care Administration, National Yang-Ming University, Taipei 11221, Taiwan
3
Department of Family Medicine, Taipei Veterans General Hospital, Taipei 11217, Taiwan
4
Department of Primary Care and Public Health, Imperial College London, London W6 8RP, UK
5
School of Medicine, National Yang-Ming University, Taipei 11221, Taiwan
6
Department of Pediatric Neurosurgery, Neurological Institute, Taipei Veterans General Hospital, Taipei 11217, Taiwan
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(3), 927; https://doi.org/10.3390/ijerph17030927
Submission received: 13 January 2020 / Revised: 27 January 2020 / Accepted: 28 January 2020 / Published: 2 February 2020
(This article belongs to the Section Public Health Statistics and Risk Assessment)
Browse Figures
Versions Notes

Abstract

:
The LACE index and HOSPITAL score models are the two most commonly used prediction models identifying patients at high risk of readmission with limited information for home care patients. This study compares the effectiveness of these two models in predicting 30-day readmission following acute hospitalization of such patients in Taiwan. A cohort of 57 home care patients were enrolled and followed-up for one year. We compared calibration, discrimination (area under the receiver operating curve, AUC), and net reclassification improvement (NRI) to identify patients at risk of 30-day readmission for both models. Moreover, the cost-effectiveness of the models was evaluated using microsimulation analysis. A total of 22 readmissions occurred after 87 acute hospitalizations during the study period (readmission rate = 25.2%). While the LACE score had poor discrimination (AUC = 0.598, 95% confidence interval (CI) = 0.488–0.702), the HOSPITAL score achieved helpful discrimination (AUC = 0.691, 95% CI = 0.582–0.785). Moreover, the HOSPITAL score had improved the risk prediction in 38.3% of the patients, compared with the LACE index (NRI = 0.383, 95% CI = 0.068–0.697, p = 0.017). Both prediction models effectively reduced readmission rates compared to an attending physician’s model (readmission rate reduction: LACE, 39.2%; HOSPITAL, 43.4%; physician, 10.1%; p < 0.001). The HOSPITAL score provides a better prediction of readmission and has potential as a risk management tool for home care patients.

1. Introduction

Prediction of readmission has received a lot of research attention recently for its important role in quality improvement and evolving payment reforms [1,2,3]. Previous studies suggest that readmissions occurred frequently in 15–25% of patients and are associated with excess harm and numerous burdens to the health system [3,4]. The LACE index and the HOSPITAL score are two of most commonly used prediction models that proactively stratify and identify patients at a high risk of readmission [5,6,7,8,9]. The simplicity of the LACE index and HOSPITAL score distinguishes them from other models, and they have been widely adopted in current clinical information systems as a result [9]. However, quite a few studies found that either the LACE index or HOSPITAL score, originally developed for medical and surgical patients, demonstrates inconsistent model performance when used in different settings [10,11,12,13].
Prediction of readmission is critically important to reduce readmissions for home health care patients. The readmission rate is high, around 30–100% among such patients, and greatly impacts the whole health system [3,14]. Those home care patients usually have a certain degree of disability and are quite vulnerable after leaving the hospital; such a significant jump between the hospital and home requires preventive interventions with more insight into patient health at home. An ideal readmission prediction model helps to stratify patients with a high risk of readmission, targets the delivery of resource-intensive interventions to the patients with the greatest need, and improves outcomes [15].
There is limited information on using the LACE index and HOSPITAL score models as risk assessment tools against readmission for home care patients. As there is no single prediction model that works well in all scenarios, most readmission prediction models require validation and evaluation using real world data in different settings [7,9]. The goal of this study is to evaluate how well the LACE index and HOSPITAL score models predict whether home care patients will be readmitted after their hospitalization. Moreover, a microsimulation model is performed to analyze and compare the cost and effectiveness of readmission prediction models using real-world data.

2. Materials and Methods

2.1. Setting, Data Source, and Ethical Concerns

This retrospective cohort study was conducted in a medical center affiliated with a home care unit providing a large service of more than 5000 home visits for 400 disabled patients per year in Taipei, Taiwan. Since several factors increase the likelihood of readmission, the current setting of this study helped to minimize influences resulting from system-level factors and to focus on patient-level factors. The complexity of transition between hospital and home care service is largely simplified by a team effort (patient transferal for hospitalization by home-visit physician, intra-hospital discharge planning, and post-discharge instruction by home care nurses), which minimizes the bias resulting from external factors. Moreover, the high availability of Taiwan’s health system further reduces any bias introduced regarding the concern of accessibility barriers for disabled patients.
Demographic information and relevant variables were extracted from the hospital’s electronic medical system. This study’s protocol was approved by the Institutional Review Board of Taipei Veterans General Hospital (IRB#2018-08-016AC). The Review Board waived the requirement of written informed consent from each patient who was involved in our study because the data we analyzed were de-identified secondary data, which were published for research purposes. No identifiable private information or human biospecimens were involved.

2.2. Study Cohort, Enrolled Hospitalizations, and 30-Day Readmissions

Patients who were registered and used home care services in this home care unit before 31 December 2016 constituted the study population. In Taiwan, patients who are wheelchair- or bed-bound for more than half of the day or who require considerable assistance and frequent medical care are eligible for home care services covered by Taiwan’s National Health Insurance.
We enrolled all acute hospitalizations between 1 January 2017 and 31 December 2017 as the study population. We excluded acute hospitalization which met the following conditions: end of the study, transfer to a chronic care facility (e.g., nursing home, respiratory care ward), transfer to other home care facilities, or death. Multiple eligible hospitalizations made by a single patient was counted accordingly (Figure 1).
The outcomes of discharge of enrolled acute hospitalizations were determined by 30-day readmissions. Hospitalizations were designated as “hospitalization with 30-day readmissions” if patients were readmitted within 30 days after their discharge, whereas hospitalizations were regarded as “hospitalization with successful discharge” if they were free from readmission. Readmission rates were calculated as the number of “hospitalization with 30-day readmissions” divided by the number of hospitalizations.

2.3. The LACE Index and HOSPITAL Score Models for Prediction of Readmission

To compare the performance of the LACE index and HOSPITAL score models, both scores were calculated for each hospitalization (Figure 1).
The LACE indexes were calculated by points assigned for the patient’s length of stay, the acuity of the patient’s admission, the degree of comorbid illness or illnesses (as measured by the Charlson comorbidity index), and the number of times patients had been to the emergency department in the last six months [16].
The HOSPITAL scores were calculated by points assigned for blood hemoglobin level at discharge, blood sodium level at discharge, having a procedure during the hospital stay, index admission type, number of hospital admissions in the previous year, and length of stay of more than 5 days. We did not include the factor “discharge from an oncology service” since such groups rarely used oncology-related services because of limited benefits. For patients with multiple blood test results during hospitalization, the last one closest to the discharge date was used [17].
Since there was no single best measurement for the performance of a prediction model, we compared the LACE and HOSPITAL models with the following measurements: discrimination, calibration, sensitivity, specificity, and net reclassification improvement (NRI) [18]. A microsimulation model, shown below in Figure 2, simulated and further evaluated the cost-effectiveness of each model before being applied to the real world. Discrimination is the ability of a predictive model to separate hospitalizations into high risk of 30-day readmission or successful discharge. Measured as an area under the receiver-operating curve (AUC), it is suggested that an AUC less than 0.60 reflects poor discrimination; 0.60 to 0.75, possibly helpful discrimination; and more than 0.75, clearly useful [19]. Calibration, however, as a measure reflects the extent to which a model correctly estimates the absolute risk (i.e., the difference between predicted and observed risk at different levels).
NRI is a new and popular method for assessing how much better a new model is at predicting risk than another model; it is called net reclassification. NRI is usually expressed as the percentage of a new model reclassifying patients into risk categories compared with a previous model [20].

2.4. Microsimulation Model

Microsimulation is a technique used to model complex real-life events by simulating the impact of policy change on the individual units (micro units) that make up the system where the events occur [21,22]. We constructed a microsimulation model to evaluate the cost-effectiveness of the LACE index and HOSPITAL score when applied to hospitalizations of the current study cohort. To estimate the effect of prediction in the real world, we incorporated a third model, attending physician, as a 30-day readmission prediction made by home-care attending physicians as a ground truth. According to a previous study, the sensitivity and specificity of an attending physician were 0.23 (95% CI = 0.13–0.37) and 0.84 (95% CI = 0.75–0.90), respectively [23]. The fourth model, all intervention (no prediction, null model), was introduced by assuming preventive efforts were performed for all acute hospitalizations without any prediction.
We assumed the home care unit should make a preventive intervention against readmission for every acute hospitalization when predicted with a high risk for 30-day admission after discharge to home. The cost of interventions thus equals the number of hospitalizations predicted as high risk for 30-day admission multiplied by the unit cost of the intervention. A 30-day readmission reduction rate is calculated by the difference between observed and expected readmission rates. Naturally, a better model has a lower cost of interventions and a higher readmission reduction rate. However, the cost of physicians (i.e., extra payment for physicians doing prediction work) and prediction model deployment (i.e., maintenance cost for machines and technicians) are not counted in the simulation model.

2.5. Error Analysis and Statistical Analysis

Error analysis was performed to examine the measurement error of the data. The first author (M.-C.S.) rechecked the data and verified with an electronic health record. Statistical analyses were performed using IBM SPSS Statistics, version 24, (IBM Corp., Armonk, NY, USA), MedCalc, version 19 (MedCalc Software, Ostend, Belgium), and R, version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria). Microsimulation was performed using TreeAge Pro 2019, R2 (TreeAge Software, Williamstown, MA, USA). Demographic information, admission history, and parameters for LACE and HOSPITAL scores were obtained from the electronic medical record. Receiver operator characteristic (ROC) curves were constructed for the LACE model and the HOSPITAL model and Youden’s index was used to determine the cut-off point of both models. We compared calibration with the Hosmer–Lemeshow goodness-of-fit test and discrimination with an area under the receiver operating curve (AUC). The reclassification with net reclassification improvement (NRI) was used to identify patients at risk of 30-day readmission for both models. A two-tailed level of 0.05 was considered statistically significant.

2.6. Experimentation of Microsimulation Model

A microsimulation analysis was performed to assess the cost-effectiveness of readmission prediction models. Performance measures (sensitivities and specificities) and readmission rates calculated from the real-world data were used as variables in the microsimulation model (Appendix A, Table A1 and Table A2). We further assumed preventive efforts intervened for every high-risk hospitalization (i.e., acute hospitalizations predicted as a high likelihood of readmission) with a success rate of 50%. The current model only included the direct costs of preventive intervention against readmission for acute hospitalization for those predicted as at high risk for readmission, thus the cost of a physician (e.g., extra payment) and maintenance of a computer system were not counted. Cost of the all intervention model (i.e., no prediction model) was used as a reference. Cost-effectiveness pair with scatter plot was used to evaluate these four readmission prediction models.

3. Results

3.1. Study Population and 30-Day Readmission Rates

A total of 87 acute hospitalizations of a home care patient cohort of a medical center affiliated home care unit in 2017 was included in the current study. Generally, the study population was quite old, at an average age of 87.4 years. The readmission rate was 25.3%, which means more than a quarter of patients were readmitted within 30 days after their discharge. There was no age and gender difference between hospitalizations with 30-day readmission and with successful discharge. While HOSPITAL scores of hospitalizations with 30-day readmission were significantly higher than hospitalizations with successful discharge, there was no significant difference for the LACE index between these two outcomes of hospitalization (Table 1). The error analysis showed that the issue of measurement error is minimal in the current study.

3.2. Discrimination and Calibration of the LACE and HOSPITAL Models

HOSPITAL scores achieved helpful discrimination (AUC = 0.691, 95% CI = 0.582–0.785), while LACE scores had poor discrimination (AUC = 0.598, 95% CI = 0.488–0.702) (Figure 3, Table 2). The findings suggest that HOSPITAL scores have a better ability to identify high risk readmission hospitalization. Both LACE or HOSPITAL scores show agreement between observed readmission and predictions (Hosmer–Lemeshow goodness-of-fit test, both p > 0.05). There is a high agreement between LACE index and HOSPITAL score (number of agreements vs. disagreements: 69 vs. 18; McNemar test, p = 1.000) which suggests that the results of both models are similar.
The LACE index and HOSPITAL score models show similar performance in terms of accuracy, sensitivity, and specificity (Table 3). However, the HOSPITAL score improved the risk prediction in 38.3% of the patients, compared with the LACE index (NRI = 0.383, 95% CI = 0.068-0.697, p = 0.017), which clearly suggests the HOSPITAL score model is 38.3% more likely to identify hospitalization with high risk of 30-day readmission than the LACE index.

3.3. Cost of Intervention and Readmission Rate Reduction of the LACE Index and HOSPITAL Score Models

Both the LACE index and HOSPITAL score models are useful as risk assessment tools against readmission among home care patients, although the HOSPITAL score performs slightly better than the LACE index model. These two models effectively reduce the readmission rate compared to attending physicians, with the cost of more preventive efforts. There was a clear 30-day readmission rate reduction of 40% for both the LACE index and HOSPITAL score models. Such findings suggest that the LACE index and HOSPITAL score models significantly reduce readmission rates much more than attending physicians (readmission rate reduction of the LACE index model vs. attending physician model, 39.2% vs. 10.1%; the HOSPITAL score model vs. attending physician model, 43.4% vs. 10.1%; both p < 0.001). Moreover, the HOSPITAL score model further reduced the readmission rate by 4% compared to the LACE index (readmission rate reduction of the LACE index vs. the HOSPITAL score, 39.2% vs. 43.4%, p < 0.05 when sample size >1100) (Table 3).
However, the simulated results clearly show that the all intervention model (no prediction) is the costliest model, followed by the HOSPITAL score and LACE index models, respectively. In contrast, the attending physician model has the least cost. The cost of preventive efforts of the LACE index and HOSPITAL score models was at least 3.5 times larger than that of the attending physician model (LACE index vs. attending physician, 66.8% vs. 18.6%, 3.58-fold; HOSPITAL score vs. attending physician, 72.0% vs. 18.6%, 3.87-fold) (Table 3).
Both the LACE index and HOSPITAL score models are more efficient than attending physicians and all intervention (no prediction, null model). When all costs of preventive intervention and 30-day readmission rate reduction are considered together, the all intervention model is naturally the least efficient one since preventive interventions are performed on all hospitalizations; thus, it has the highest cost of interventions and the highest 30-day readmission rate reduction (Figure 4). Taking the all intervention model as a reference, both the LACE index and HOSPITAL score models reduced 30-day readmission rates with a lesser cost of intervention (Figure 4).

4. Discussion

The current study assesses the performance of the LACE index and HOSPITAL score models, the two most commonly used prediction models to identify patients at high risk of hospital readmission; moreover, we use real-world data from a cohort of home health care patients to evaluate the real-world effectiveness of these two models as a risk assessment tool prior to execution. Such real-world evidence is important for home care providers and for the evaluation framework that generates real-world evidence which is much more critical for policy makers.
Prediction of readmission is critically important to reduce readmissions for home health care patients, those who are usually old and with certain degrees of disabilities. The results show that the HOSPITAL score was superior to the LACE index in terms of discrimination (the ability to identify hospitalization with a higher probability for readmission). While both the LACE index and HOSPITAL score models effectively reduced readmission rates at 40% compared to attending physicians, the HOSPITAL score model provided slightly better performance than the LACE index. The current study offers clear real-word evidence of the usefulness of the prediction model as a risk assessment tool against readmission for home care patients. Home care providers are encouraged to consider adopting the HOSPITAL score model to optimize their care plans.
Our results further extend the current knowledge not covered in previous studies to home care patients. Both the LACE index and HOSPITAL score demonstrate convincing clinical values in reducing readmission even though the discrimination power of these models seems suboptimal. This finding is similar to other works summarized in Appendix B. We reviewed 23 previous works and found that the discrimination power of either the LACE index or HOSPITAL score only reached ‘poor’ to ‘possibly helpful’ level but was still useful in most of the scenarios (AUC of LACE index ranges 0.55–0.82 in Table A3 vs. AUC of HOSPITAL score ranges 0.60–0.77 in Table A4). Moreover, while LACE index and HOSPITAL scores previously focused on the relative health population, the current results add new information about a novel use of prediction models as risk assessment tools in an old (mean age is 87.4 years old) disabled home care population with high comorbidities (average comorbidities index = 1.7) and high mortalities (mortality rate >20% per year).
The results clearly demonstrate that the readmission prediction model is a useful risk assessment tool for home care patients. Our results reveal a 40% reduction in the readmission rate by proactively targeting preventive intervention toward high-risk hospitalizations of home care patients. Previous studies also showed that a great portion of 30-day readmissions are avoidable through a series of resource-intensive interventions, including post-acute support, post-discharge monitoring [24], telehealth rehabilitation programs [25], and home care case management [26,27]. A readmission prediction model is of course the very first step to stratify patients and optimize resource utilization for home care patients. Moreover, a growing body of evidence suggests an earlier use of readmission prediction models prior to discharge. Readmission prediction made during hospitalization would help the home care team to prepare coaching sessions to care givers, transitional care intervention programs [28,29], and to arrange supporting services including social workers, in-home rehabilitation, and in-home nutrition consultation, before the patients are discharged to their homes and has the potential to prevent more readmissions [30,31].
The current study presents a comprehensive framework to evaluate the real-world effectiveness of prediction models. An evaluation of the performance of predictive models is never an easy task. Traditionally, a dozen performance measures such as sensitivities, specificities, a confusion matrix, ROC curves, AUC (also known as c-statistics), etc. have been proposed; however, none of these measures sufficiently reflect the effectiveness of models, so that we cannot compare models using any of these measures alone [19]. To address this issue, the net reclassification index (NRI) is a novel method trying to provide a single-number summary for assessing how much better a new model is at predicting risk than another model [20]. However, the NRI is not without problems and may be misleading in certain conditions [32]. We thus built a microsimulation model based on real-world data to better estimate the cost and effectiveness in the real-world [21]. Our analytic framework, combining traditional measures, NRI, and a microsimulation model, would be particularly useful in implementing prediction models in clinical practice. For example, prediction model developed in a large research sites could be adopted and applied to other clinical settings by incorporating this framework [33,34].
Both the LACE index and HOSPITAL score prediction models show a satisfactory reduction in readmission rates; however, there is still room for improvement for these models. While readmission is the common endpoint resulting from multiple factors [35,36], some previous studies suggested a limited use of the LACE index and HOSPITAL score models because these two models include only a few variables which lead to inconsistent performance across different settings [7,12,37,38]. Moreover, the current study found only a portion of variables have predictive power for readmission among home care patients (one out four variables in the LACE index, three out of six variables in the HOSPITAL score, respectively). Therefore, some studies suggested introducing more variables or using state-of-the-art methods such as machine learning and deep learning to gain a better predictive power [39,40,41]. However, using the real-world data, we found that both the LACE index and HOSPITAL score models already perform far better than physicians and might be good enough in readmission rate reduction for home care patients. The predictive power of a model is not everything; our results suggest real-world evidence generated from simulation using real-world data would help decision-makers in choosing a prediction model with more clinical usefulness [18,42].
Prediction models should augment home care team’s decision making, not replace them. Ethical considerations should never be overlooked when putting prediction models into practice. Previous studies on anchoring effects suggested that the process of human decision making is easily influenced by an initial piece of information (LACE index or HOSPITAL score in the current study), which is hard to avoid [43]. Moreover, a certain portion (10% to 20%) of patients with high risk for readmission would be falsely stratified as low risk using either the LACE index or HOSPITAL score model. As a result, we suggest that these prediction models should never be used alone. Furthermore, special attention and auditing should be made to patients stratified as low risk.
Future studies will have to look at ways to develop a more accurate and generalized prediction model for home care patients. Many parameters are not fitted well for home care patients and Taiwan’s health system. For example, the length of stay (L) in Taiwan is usually much longer than other health systems, home care patients are often admitted via the emergency department but the outpatient department (acuity of admission (A)), and those patients are rarely admitted to the oncology ward for cancer treatment (discharge from an oncology service (O)). Moreover, the LACE index and the HOSPITAL score take different parameters into consideration, and it is possible that these two models may be appropriate for home care patients that are of different conditions. As a result, a new model combining the LACE and HOSPITAL models with re-calibration according to Taiwan’s health system may increase the predictive power for home care patients. Furthermore, pooling validating results from other sites externally would be important before extrapolating the current results.
There are some limitations in the current study. First, generalizability is the fundamental issue of prediction models. Although the current study clearly demonstrated the feasibility of using the LACE index and HOSPITAL score models as risk assessment tools for patients in a large home care facility in Taiwan, more data from home care facilities of different scales are needed to ensure the result is applicable to every home care facility. However, the current study presents a comprehensive framework so every home care facility can validate the result using their own data and make the result more applicable. Secondly, system level factors for readmission are not considered in the current study. Although both the LACE index and HOSPITAL score models worked well in the current setting, the results may deviate when there is system change in insurance policy, coverage of home care service, accessibility of hospital services, etc. Therefore, these prediction models as risk assessment tools should be validated and calibrated periodically. Thirdly, many parameters of both models are not fitted well for home care patients and Taiwan’s health system. This decreases the predictive power of these prediction models. A revised prediction model based on the LACE index and HOSPITAL score models might have better performance. Fourthly, the current prediction model is a simplification of real-life scenarios and the inability to fully model the complex nature of readmission. As in our results, the strength of the readmission rate reduction is dependent on the choice of model structure. Rigorous model validation and assessment of the performance and outcomes can help assess the robustness of the model findings and address this limitation.

5. Conclusions

While both the LACE index and HOSPITAL score models effectively reduce the readmission rate, the HOSPITAL score model shows a slight advantage over the LACE index model as a risk assessment tool for home care patients. Rigorous validation using a simulation model based on local real-world data is warranted before generalization and execution of the current result.

Author Contributions

Conceptualization, M.-C.S., C.-C.L., L.-H.H., S.-J.Y., M.-S.H., S.-H.C., H.-T.C., T.-J.C. and J.-C.W.; data curation, M.-C.S., C.-C.L., L.-H.H., S.-J.Y., M.-S.H. and Y.-C.C.; formal analysis, M.-C.S. and Y.-C.C.; funding acquisition, Y.-C.C.; investigation, M.-C.S.; methodology, Y.-C.C.; project administration, S.-H.C., H.-T.C. and T.-J.C.; resources, J.-C.W.; supervision, H.-T.C.; validation, Y.-J.W.; writing—original draft, M.-C.S., Y.-J.W. and Y.-C.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by an intramural grant of Taipei Veterans General Hospital (V108C-037, V109C-006). The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

Table
Table A1. Summary of parameters used in the microsimulation model to evaluate the 30-day readmission rate reduction and cost of preventive intervention by the LACE index, HOSPITAL score, and attending physician prediction models.
Table A1. Summary of parameters used in the microsimulation model to evaluate the 30-day readmission rate reduction and cost of preventive intervention by the LACE index, HOSPITAL score, and attending physician prediction models.
30-Day Readmission Prediction ModelsSensitivity (%)(95% CI)Specificity(95% CI)
LACE index81.8(59.7–94.8)44.6(32.3–57.5)
HOSPITAL score90.9(70.8–98.9)41.5(29.4–54.4)
Attending physician 123(13–37)84(75–90)
1 Parameters used for the attending physician model were extracted from [23].
Table
Table A2. Summary of variables and key parameters used in the microsimulation model.
Table A2. Summary of variables and key parameters used in the microsimulation model.
VariablesType, ValueLower LimitModeUpper Limit
Readmission rateConstant = 0.25---
Successful rate of preventive interventionConstant = 0.5---
LACE index—SensitivityTriangular distribution0.5970.8180.948
HOSPITAL score—SensitivityTriangular distribution0.7080.9090.989
Attending physician—SensitivityTriangular distribution0.130.230.37
LACE index—SpecificityTriangular distribution0.3230.4460.575
HOSPITAL score—SpecificityTriangular distribution0.2940.4150.544
Attending physician—SpecificityTriangular distribution0.750.840.90

Appendix B

Table
Table A3. Summary of previous studies which used the LACE index predictive model for readmission of different populations. Ordered according to year of publication.
Table A3. Summary of previous studies which used the LACE index predictive model for readmission of different populations. Ordered according to year of publication.
No.Studies Used LACE IndexPopulation (n)Area under Curve (AUC)(95% CI) 1Discrimination Power 2
1.Cotter et al., (2012) [12]UK population, n = 5070.57NAPoor
2.Van Walraven et al., (2012) [44] 3Medical records, n = 500,0000.771(0.767–0.775)Clearly useful
3.Wang et al., (2014) [45]Heart failure patients, n = 2530.664(0.575–0.752)Possibly helpful
4.Low et al., (2015) [46]Singapore population, n = 58620.628(0.602–0.653)Possibly helpful
5.Yazdan-Ashoori et al., (2016) [47]Heart failure patients, n = 3780.58(0.57–0.61)Poor
6.Damery et al., (2017) [48]Inpatient health records, n = 71070.773(0.768–0.779)Clearly useful
7.Low et al., (2017) [49]Singapore, elder inpatients, n = 17,0060.595(0.581–0.608)Poor
8.Robinson et al., (2017) [13]Inpatient health records, n = 4320.58(0.48–0.68)Poor
9.Baig et al., (2018) [50]New Zealand population, n = 213,4400.752NAClearly useful
10.Hakim et al., (2018) [51]COPD 4 patients, n = 26620.63(0.62–0.65)Possibly helpful
11.Miller et al., (2018) [37]Inpatients, n = 3590.620(0.521–0.718)Possibly helpful
12.Saluk et al., (2018) [52]Radical cystectomy patients, n = 34700.581(0.556-0.606)Poor
13.Caplan et al., (2019) [53] 3Brain tumor population, n = 3520.58NAPoor
14.Caplan et al., (2019) [54] 3Craniotomy patients, n = 2380.69NAPossibly helpful
15.Ibrahim et al., (2019) [55]Heart failure patients, n = 7300.551(0.503–0.598)Poor
16.Robinson et al., (2019) [56]Inpatients, n = 19160.598(0.58–0.64)Poor
17.Shaffer et al., (2019) [57]Surgical patients, n = 192,6700.82NAPossibly helpful
1 CI: confidence interval, NA: not available. 2 Discrimination power: AUC < 0.60, poor discrimination; AUC = 0.60 to 0.75, possibly helpful discrimination; AUC > 0.75, clearly useful discrimination [19]. 3 A modified version of LACE index, LACE+ index, is used. 4 COPD: Chronic Obstructive Pulmonary Disease.
Table
Table A4. Summary of previous studies used HOSPITAL score predictive model for readmission for different populations. Ordered according to year of publication.
Table A4. Summary of previous studies used HOSPITAL score predictive model for readmission for different populations. Ordered according to year of publication.
No.Studies Used HOSPITAL ScorePopulation (n)Area under Curve (AUC)(95% CI) 1Discrimination Power 2
1.Aubert et al., (2016) [10]Switzerland, inpatients, n = 3460.70(0.62–0.79)Possibly helpful
2.Donze et al., (2016) [58]Inpatients, n = 117,0650.72(0.72–0.72)Possibly helpful
3.Kim et al., (2016) [59]Inpatients, n = 42080.65NAPossibly helpful
4.Robinson (2016) [60]Inpatients, n = 9980.77(0.73–0.81)Clearly useful
5.Aubert et al., (2017) [61]Inpatients, n = 117,0650.69(0.68–0.69)Possibly helpful
6.Burke et al., (2017) [8]Inpatients, n = 91810.68(0.67–0.71)Possibly helpful
7.Robinson et al., (2017) [13]Inpatient health records, n = 4320.75(0.67–0.83)Possibly helpful
8.Ibrahim et al., (2019) [55]Heart failure inpatients, n = 7300.595(0.549–0.641)Poor
9.Robinson et al., (2019) [56]Inpatients, n = 19160.675(0.65–0.70)Possibly helpful
1 CI: confidence interval, NA: not available. 2 Discrimination power: AUC < 0.60, poor discrimination; AUC = 0.60 to 0.75, possibly helpful discrimination; AUC > 0.75, clearly useful discrimination [19].

References

  1. Wadhera, R.K.; Joynt Maddox, K.E.; Wasfy, J.H.; Haneuse, S.; Shen, C.; Yeh, R.W. Association of the Hospital Readmissions Reduction Program with Mortality Among Medicare Beneficiaries Hospitalized for Heart Failure, Acute Myocardial Infarction, and Pneumonia. JAMA 2018, 320, 2542–2552. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Wang, J.; Liebel, D.V.; Yu, F.; Caprio, T.V.; Shang, J. Inverse Dose-Response Relationship Between Home Health Care Services and Rehospitalization in Older Adults. J. Am. Med. Dir. Assoc. 2019, 20, 736–742. [Google Scholar] [CrossRef] [PubMed]
  3. Meadow, A.; Sangl, J. Hospital Readmissions in Medicare Home Healthcare: What Are the Leading Risk Indicators? Home Health. Now. 2019, 37, 213–221. [Google Scholar] [CrossRef] [PubMed]
  4. Pedersen, M.K.; Nielsen, G.L.; Uhrenfeldt, L.; Lundbye-Christensen, S. Risk Assessment of Acute, All-Cause 30-Day Readmission in Patients Aged 65+: A Nationwide, Register-Based Cohort Study. J. Gen. Intern. Med. 2019, 34, 226–234. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Mehta, H.B.; Sieloff, E.; Veeranki, S.P.; Sura, S.D.; Riall, T.S.; Senagore, A.J.; Goodwin, J.S. Risk Prediction Models for Hospital Readmission in Surgery: A Systematic Review. J. Am. Coll. Surg. 2017, 225, e113. [Google Scholar] [CrossRef] [Green Version]
  6. Brooke, B.S.; Goodney, P.P.; Kraiss, L.W.; Gottlieb, D.J.; Samore, M.H.; Finlayson, S.R. Readmission destination and risk of mortality after major surgery: An observational cohort study. Lancet 2015, 386, 884–895. [Google Scholar] [CrossRef] [Green Version]
  7. Kansagara, D.; Englander, H.; Salanitro, A.; Kagen, D.; Theobald, C.; Freeman, M.; Kripalani, S. Risk Prediction Models for Hospital Readmission A Systematic Review. JAMA-J. Am. Med. Assoc. 2011, 306, 1688–1698. [Google Scholar] [CrossRef] [Green Version]
  8. Burke, R.E.; Schnipper, J.L.; Williams, M.V.; Robinson, E.J.; Vasilevskis, E.E.; Kripalani, S.; Metlay, J.P.; Fletcher, G.S.; Auerbach, A.D.; Donze, J.D. The HOSPITAL Score Predicts Potentially PrevenTable 30-Day Readmissions in Conditions Targeted by the Hospital Readmissions Reduction Program. Med. Care 2017, 55, 285–290. [Google Scholar] [CrossRef] [Green Version]
  9. Zhou, H.; Della, P.R.; Roberts, P.; Goh, L.; Dhaliwal, S.S. Utility of models to predict 28-day or 30-day unplanned hospital readmissions: An updated systematic review. BMJ Open 2016, 6, e011060. [Google Scholar] [CrossRef]
  10. Aubert, C.E.; Folly, A.; Mancinetti, M.; Hayoz, D.; Donze, J.D. Prospective validation and adaptation of the HOSPITAL score to predict high risk of unplanned readmission of medical patients. Swiss Med. Wkly. 2016, 146. [Google Scholar] [CrossRef] [Green Version]
  11. Cooksley, T.; Nanayakkara, P.W.B.; Nickel, C.H.; Subbe, C.P.; Kellett, J.; Kidney, R.; Merten, H.; Van Galen, L.; Henriksen, D.P.; Lassen, A.T.; et al. Readmissions of medical patients: An external validation of two existing prediction scores. QJM-Int. J. Med. 2016, 109, 245–248. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Cotter, P.E.; Bhalla, V.K.; Wallis, S.J.; Biram, R.W.S. Predicting readmissions: Poor performance of the LACE index in an older UK population. Age Ageing 2012, 41, 784–789. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Robinson, R.; Hudali, T. The HOSPITAL score and LACE index as predictors of 30 day readmission in a retrospective study at a university-affiliated community hospital. PeerJ 2017, 5. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Pesko, M.F.; Gerber, L.M.; Peng, T.R.; Press, M.J. Home Health Care: Nurse-Physician Communication, Patient Severity, and Hospital Readmission. Health Serv. Res. 2018, 53, 1008–1024. [Google Scholar] [CrossRef]
  15. Simpson, M.; Macias Tejada, J.; Driscoll, A.; Singh, M.; Klein, M.; Malone, M. The Bundled Hospital Elder Life Program-HELP and HELP in Home Care-and Its Association with Clinical Outcomes Among Older Adults Discharged to Home Healthcare. J. Am. Geriatr. Soc. 2019, 67, 1730–1736. [Google Scholar] [CrossRef] [PubMed]
  16. Van Walraven, C.; Dhalla, I.A.; Bell, C.; Etchells, E.; Stiell, I.G.; Zarnke, K.; Austin, P.C.; Forster, A.J. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. Can. Med. Assoc. J. 2010, 182, 551–557. [Google Scholar] [CrossRef] [Green Version]
  17. Donze, J.; Aujesky, D.; Williams, D.; Schnipper, J.L. Potentially Avoidable 30-Day Hospital Readmissions in Medical Patients Derivation and Validation of a Prediction Model. JAMA Intern. Med. 2013, 173, 632–638. [Google Scholar] [CrossRef]
  18. Steyerberg, E.W.; Vickers, A.J.; Cook, N.R.; Gerds, T.; Gonen, M.; Obuchowski, N.; Pencina, M.J.; Kattan, M.W. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology 2010, 21, 128–138. [Google Scholar] [CrossRef] [Green Version]
  19. Alba, A.C.; Agoritsas, T.; Walsh, M.; Hanna, S.; Iorio, A.; Devereaux, P.J.; McGinn, T.; Guyatt, G. Discrimination and Calibration of Clinical Prediction Models Users’ Guides to the Medical Literature. JAMA-J. Am. Med. Assoc. 2017, 318, 1377–1384. [Google Scholar] [CrossRef]
  20. Kerr, K.F.; Wang, Z.; Janes, H.; McClelland, R.L.; Psaty, B.M.; Pepe, M.S. Net reclassification indices for evaluating risk prediction instruments: A critical review. Epidemiology 2014, 25, 114–121. [Google Scholar] [CrossRef] [Green Version]
  21. Krijkamp, E.M.; Alarid-Escudero, F.; Enns, E.A.; Jalal, H.J.; Hunink, M.G.M.; Pechlivanoglou, P. Microsimulation Modeling for Health Decision Sciences Using R: A Tutorial. Med. Decis. Making 2018, 38, 400–422. [Google Scholar] [CrossRef] [PubMed]
  22. Pencina, K.M.; D’Agostino, R.B.; Vasan, R.S.; Pencina, M.J. Microsimulation model to predict incremental value of biomarkers added to prognostic models. J. Am. Med. Inf. Assoc. 2018, 25, 1382–1385. [Google Scholar] [CrossRef] [PubMed]
  23. Allaudeen, N.; Schnipper, J.L.; Orav, E.J.; Wachter, R.M.; Vidyarthi, A.R. Inability of providers to predict unplanned readmissions. J. Gen. Intern. Med. 2011, 26, 771–776. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Srivastava, A.; Do, J.M.; Sales, V.L.; Ly, S.; Joseph, J. Impact of patient-centred home telehealth programme on outcomes in heart failure. J. Telemed. Telecare 2019, 25, 425–430. [Google Scholar] [CrossRef]
  25. Bhatt, S.P.; Patel, S.B.; Anderson, E.M.; Baugh, D.; Givens, T.; Schumann, C.; Sanders, J.G.; Windham, S.T.; Cutter, G.R.; Dransfield, M.T. Video Telehealth Pulmonary Rehabilitation Intervention in Chronic Obstructive Pulmonary Disease Reduces 30-Day Readmissions. Am. J. Respir. Crit. Care Med. 2019, 200, 511–513. [Google Scholar] [CrossRef]
  26. Cunha Ferre, M.F.; Gallo Acosta, C.M.; Dawidowski, A.R.; Senillosa, M.B.; Scozzafava, S.M.; Saimovici, J.M. 72-hour hospital readmission of older people after hospital discharge with home care services. Home Health Care Serv. 2019, 38, 153–161. [Google Scholar] [CrossRef]
  27. Dowding, D.W.; Russell, D.; Onorato, N.; Merrill, J.A. Technology Solutions to Support Care Continuity in Home Care: A Focus Group Study. J. Healthc. Qual. 2018, 40, 236–246. [Google Scholar] [CrossRef]
  28. Parry, C.; Min, S.J.; Chugh, A.; Chalmers, S.; Coleman, E.A. Further application of the care transitions intervention: Results of a randomized controlled trial conducted in a fee-for-service setting. Home Health Care Serv. 2009, 28, 84–99. [Google Scholar] [CrossRef]
  29. Coleman, E.A.; Parry, C.; Chalmers, S.; Min, S.J. The care transitions intervention: Results of a randomized controlled trial. Arch. Intern. Med. 2006, 166, 1822–1828. [Google Scholar] [CrossRef] [Green Version]
  30. Ballester, N.; Parikh, P.J.; Donlin, M.; May, E.K.; Simon, S.R. An early warning tool for predicting at admission the discharge disposition of a hospitalized patient. Am. J. Manag. Care 2018, 24, e325–e331. [Google Scholar]
  31. Tabak, Y.P.; Sun, X.W.; Nunez, C.M.; Gupta, V.; Johannes, R.S. Predicting Readmission at Early Hospitalization Using Electronic Clinical Data an Early Readmission Risk Score. Med. Care 2017, 55, 267–275. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Pepe, M.S.; Fan, J.; Feng, Z.; Gerds, T.; Hilden, J. The Net Reclassification Index (NRI): A Misleading Measure of Prediction Improvement Even with Independent Test Data Sets. Stat. Biosci. 2015, 7, 282–295. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Xu, L.; Zhao, H.; Qiu, J.; Zhu, W.; Lei, H.; Cai, Z.; Lin, W.H.; Huang, W.; Zhang, H.; Zhang, Y.T. The Different Effects of BMI and WC on Organ Damage in Patients from a Cardiac Rehabilitation Program after Acute Coronary Syndrome. Biomed. Res. Int. 2015, 2015, 942695. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Xu, L.; Huang, X.; Ma, J.; Huang, J.; Fan, Y.; Li, H.; Qiu, J.; Zhang, H.; Huang, W. Value of three-dimensional strain parameters for predicting left ventricular remodeling after ST-elevation myocardial infarction. Int. J. Cardiovasc. Imaging 2017, 33, 663–673. [Google Scholar] [CrossRef] [PubMed]
  35. Flaks-Manov, N.; Topaz, M.; Hoshen, M.; Balicer, R.D.; Shadmi, E. Identifying patients at highest-risk: The best timing to apply a readmission predictive model. BMC Med. Inform. Decis. Mak. 2019, 19, 9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Greenwood, K.L.; LaMori, J.C.; Smith, B.; Doshi, D.; Davis, C. Impact of Behavioral Health Screening on Proactive Identification of Patients at Risk for Hospital Readmission. Popul. Health Manag. 2019, 22, 196–204. [Google Scholar] [CrossRef]
  37. Miller, W.D.; Nguyen, K.; Vangala, S.; Dowling, E. Clinicians can independently predict 30-day hospital readmissions as well as the LACE index. BMC Health Serv. Res. 2018, 18, 32. [Google Scholar] [CrossRef] [Green Version]
  38. Steyerberg, E.W.; Uno, H.; Ioannidis, J.P.A.; van Calster, B.; Ukaegbu, C.; Dhingra, T.; Syngal, S.; Kastrinos, F. Poor performance of clinical prediction models: The harm of commonly applied methods. J. Clin. Epidemiol. 2018, 98, 133–143. [Google Scholar] [CrossRef]
  39. Eckert, C.; Nieves-Robbins, N.; Spieker, E.; Louwers, T.; Hazel, D.; Marquardt, J.; Solveson, K.; Zahidl, A.; Ahmadl, M.; Barnhill, R.; et al. Development and Prospective Validation of a Machine Learning-Based Risk of Readmission Model in a Large Military Hospital. Appl. Clin. Inform. 2019, 10, 316–325. [Google Scholar] [CrossRef]
  40. Morgan, D.J.; Bame, B.; Zimand, P.; Dooley, P.; Thom, K.A.; Harris, A.D.; Bentzen, S.; Ettinger, W.; Garrett-Ray, S.D.; Tracy, J.K.; et al. Assessment of Machine Learning vs Standard Prediction Rules for Predicting Hospital Readmissions. JAMA Netw. Open 2019, 2, e190348. [Google Scholar] [CrossRef] [Green Version]
  41. Min, X.; Yu, B.; Wang, F. Predictive Modeling of the Hospital Readmission Risk from Patients’ Claims Data Using Machine Learning: A Case Study on COPD. Sci. Rep. 2019, 9, 2362. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Walsh, C.G.; Sharma, K.; Hripcsak, G. Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk. J. Biomed. Inform. 2017, 76, 9–18. [Google Scholar] [CrossRef] [PubMed]
  43. Lieder, F.; Griffiths, T.L.; M Huys, Q.J.; Goodman, N.D. The anchoring bias reflects rational use of cognitive resources. Psychon. Bull. Rev. 2018, 25, 322–349. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Van Walraven, C.; Wong, J.; Forster, A.J. LACE+ index: Extension of a validated index to predict early death or urgent readmission after hospital discharge using administrative data. Open Med 2012, 6, e80–e90. [Google Scholar]
  45. Wang, H.; Robinson, R.D.; Johnson, C.; Zenarosa, N.R.; Jayswal, R.D.; Keithley, J.; Delaney, K.A. Using the LACE index to predict hospital readmissions in congestive heart failure patients. BMC Cardiovasc. Disord. 2014, 14, 97. [Google Scholar] [CrossRef] [Green Version]
  46. Low, L.L.; Lee, K.H.; Hock Ong, M.E.; Wang, S.; Tan, S.Y.; Thumboo, J.; Liu, N. Predicting 30-Day Readmissions: Performance of the LACE Index Compared with a Regression Model among General Medicine Patients in Singapore. Biomed. Res. Int. 2015, 2015, 169870. [Google Scholar] [CrossRef] [Green Version]
  47. Yazdan-Ashoori, P.; Lee, S.F.; Ibrahim, Q.; Van Spall, H.G. Utility of the LACE index at the bedside in predicting 30-day readmission or death in patients hospitalized with heart failure. Am. Heart J. 2016, 179, 51–58. [Google Scholar] [CrossRef]
  48. Damery, S.; Combes, G. Evaluating the predictive strength of the LACE index in identifying patients at high risk of hospital readmission following an inpatient episode: A retrospective cohort study. BMJ Open 2017, 7, e016921. [Google Scholar] [CrossRef] [Green Version]
  49. Low, L.L.; Liu, N.; Ong, M.E.H.; Ng, E.Y.; Ho, A.F.W.; Thumboo, J.; Lee, K.H. Performance of the LACE index to identify elderly patients at high risk for hospital readmission in Singapore. Medicine (Baltimore) 2017, 96, e6728. [Google Scholar] [CrossRef]
  50. Baig, M.; Zhang, E.; Robinson, R.; Ullah, E.; Whitakker, R. Evaluation of Patients at Risk of Hospital Readmission (PARR) and LACE Risk Score for New Zealand Context. Stud. Health Technol. Inform. 2018, 252, 21–26. [Google Scholar]
  51. Hakim, M.A.; Garden, F.L.; Jennings, M.D.; Dobler, C.C. Performance of the LACE index to predict 30-day hospital readmissions in patients with chronic obstructive pulmonary disease. Clin. Epidemiol. 2018, 10, 51–59. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  52. Saluk, J.L.; Blackwell, R.H.; Gange, W.S.; Zapf, M.A.C.; Kothari, A.N.; Kuo, P.C.; Quek, M.L.; Flanigan, R.C.; Gupta, G.N. The LACE Score as a Tool to Identify Radical Cystectomy Patients at Increased Risk of 90-Day Readmission and Mortality. Curr. Urol. 2018, 12, 20–26. [Google Scholar] [CrossRef] [PubMed]
  53. Caplan, I.F.; Sullivan, P.Z.; Kung, D.; O’Rourke, D.M.; Choudhri, O.; Glauser, G.; Osiemo, B.; Goodrich, S.; McClintock, S.D.; Malhotra, N.R. LACE+ Index as Predictor of 30-Day Readmission in Brain Tumor Population. World Neurosurg. 2019, 127, e443. [Google Scholar] [CrossRef] [PubMed]
  54. Caplan, I.F.; Zadnik Sullivan, P.; Glauser, G.; Choudhri, O.; Kung, D.; O’Rourke, D.M.; Osiemo, B.; Goodrich, S.; McClintock, S.D.; Malhotra, N.R. The LACE+ index fails to predict 30–90 day readmission for supratentorial craniotomy patients: A retrospective series of 238 surgical procedures. Clin. Neurol. Neurosurg. 2019, 182, 79–83. [Google Scholar] [CrossRef] [PubMed]
  55. Ibrahim, A.M.; Koester, C.; Al-Akchar, M.; Tandan, N.; Regmi, M.; Bhattarai, M.; Al-Bast, B.; Kulkarni, A.; Robinson, R. HOSPITAL Score, LACE Index and LACE+ Index as predictors of 30-day readmission in patients with heart failure. BMJ Evid. Based Med. 2019. [Google Scholar] [CrossRef]
  56. Robinson, R.; Bhattarai, M.; Hudali, T.; Vogler, C. Predictors of 30-day hospital readmission: The direct comparison of number of discharge medications to the HOSPITAL score and LACE index. Future Healthc. J. 2019, 6, 209–214. [Google Scholar] [CrossRef] [Green Version]
  57. Shaffer, B.K.; Cui, Y.; Wanderer, J.P. Validation of the LACE readmission and mortality prediction model in a large surgical cohort: Comparison of performance at preoperative assessment and discharge time points. J. Clin. Anesth 2019, 58, 22–26. [Google Scholar] [CrossRef]
  58. Donze, J.D.; Williams, M.V.; Robinson, E.J.; Zimlichman, E.; Aujesky, D.; Vasilevskis, E.E.; Kripalani, S.; Metlay, J.P.; Wallington, T.; Fletcher, G.S.; et al. International Validity of the HOSPITAL Score to Predict 30-Day Potentially Avoidable Hospital Readmissions. JAMA Intern. Med. 2016, 176, 496–502. [Google Scholar] [CrossRef]
  59. Kim, L.D.; Kou, L.; Messinger-Rapport, B.J.; Rothberg, M.B. Validation of the HOSPITAL Score for 30-Day All-Cause Readmissions of Patients Discharged to Skilled Nursing Facilities. J. Am. Med. Dir. Assoc. 2016, 17. [Google Scholar] [CrossRef]
  60. Robinson, R. The HOSPITAL score as a predictor of 30 day readmission in a retrospective study at a university affiliated community hospital. PeerJ 2016, 4. [Google Scholar] [CrossRef] [Green Version]
  61. Aubert, C.E.; Schnipper, J.L.; Williams, M.V.; Robinson, E.J.; Zimlichman, E.; Vasilevskis, E.E.; Kripalani, S.; Metlay, J.P.; Wallington, T.; Fletcher, G.S.; et al. Simplification of the HOSPITAL score for predicting 30-day readmissions. BMJ Qual. Saf. 2017, 26, 799–805. [Google Scholar] [CrossRef]
Ijerph 17 00927 g001 550
Figure 1. Flowchart of data processing for outcomes of acute hospitalization of a disabled cohort at a medical center affiliated home care unit in Taiwan, 2017 (n = 107).
Figure 1. Flowchart of data processing for outcomes of acute hospitalization of a disabled cohort at a medical center affiliated home care unit in Taiwan, 2017 (n = 107).
Ijerph 17 00927 g001
Ijerph 17 00927 g002 550
Figure 2. Microsimulation model to compare cost-effectiveness of four readmission prediction models when applied to acute hospitalizations of a disabled cohort of a medical center affiliated home care unit in Taiwan, 2017. (n = 87).
Figure 2. Microsimulation model to compare cost-effectiveness of four readmission prediction models when applied to acute hospitalizations of a disabled cohort of a medical center affiliated home care unit in Taiwan, 2017. (n = 87).
Ijerph 17 00927 g002
Ijerph 17 00927 g003 550
Figure 3. Receiver operating characteristic (ROC) curves of the LACE index and HOSPITAL score prediction models for 30-day readmission after acute hospitalization of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
Figure 3. Receiver operating characteristic (ROC) curves of the LACE index and HOSPITAL score prediction models for 30-day readmission after acute hospitalization of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
Ijerph 17 00927 g003
Ijerph 17 00927 g004 550
Figure 4. Cost-effectiveness scatter plot of cost of preventive intervention against readmission by 30-day readmission rate reduction of the LACE index, HOSPITAL score, and attending physician models compared to the all intervention model (no prediction, a null model). A simulated result of acute hospitalizations of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87). The higher 30-day readmission rate reduction implies greater effectiveness in reducing readmission, while the higher cost of preventive interventions implies more expense. The dotted diagonal line refers to equal cost and readmission rate reduction. Different data points denote different prediction models. Dots below the dotted diagonal line (right lower area) are deemed more efficient than dots above the line (left upper area).
Figure 4. Cost-effectiveness scatter plot of cost of preventive intervention against readmission by 30-day readmission rate reduction of the LACE index, HOSPITAL score, and attending physician models compared to the all intervention model (no prediction, a null model). A simulated result of acute hospitalizations of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87). The higher 30-day readmission rate reduction implies greater effectiveness in reducing readmission, while the higher cost of preventive interventions implies more expense. The dotted diagonal line refers to equal cost and readmission rate reduction. Different data points denote different prediction models. Dots below the dotted diagonal line (right lower area) are deemed more efficient than dots above the line (left upper area).
Ijerph 17 00927 g004
Table
Table 1. Demographic information and parameters used in the LACE and HOSPITAL prediction models by outcomes of acute hospitalization of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
Table 1. Demographic information and parameters used in the LACE and HOSPITAL prediction models by outcomes of acute hospitalization of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
Demographics and ParametersAll Acute Hospitalizations
(n = 87)		Acute Hospitalization Followed by 30-Day Readmission
(n = 22)		Acute Hospitalization with Successful Discharge
(n = 65)		--
Mean(SD 1)Mean(SD 1)Mean(SD 1)p-ValueSig. 2
Personal demographics
Gender------0.055-
Female, no. (%)40(46.0)14(63.6)26(40.0)--
Male, no. (%)47(54.0)8(36.4)39(60.0)--
Age87.4(12.4)89.0(8.1)86.9(13.6)0.482-
LACE prediction model
Length of stay (L)5.6(0.7)5.6(0.6)2.6(0.6)0.654-
Acuity of admission (A)2.7(0.9)2.9(0.6)2.6(0.6)0.21-
Comorbidities (C)1.7(0.9)1.6(0.9)1.7(0.9)0.864-
Emergency department visits (E)1.9(1.3)2.5(1.1)1.7(1.1)0.017*
LACE index11.8(2.6)12.5(1.8)11.5(1.8)0.053-
HOSPITAL prediction model 3
Hemoglobin level (H)0.7(0.5)0.9(0.4)0.6(0.5)0.009**
Sodium level (S)0.3(0.4)0.1(0.3)0.3(0.5)0.009**
Procedure during the index admission (P)0.3(0.5)0.3(0.5)0.3(0.5)0.967-
Index type of admission (IT)0.9(0.3)0.9(0.3)0.8(0.4)0.465-
Number of admissions (A)1.7(1.2)2.3(1.2)1.5(1.2)0.007**
Length of stay (L)1.9(0.4)2.0(0.0)1.9(0.4)0.083-
HOSPITAL score5.8(1.6)6.6(1.4)5.5(1.6)0.003**
1 SD: standard deviation; 2 Sig.: significance level: * p < 0.05; ** p < 0.01; 3 The study population rarely uses oncology services and thus the parameter ‘discharge from an oncology service (O)’ is not included.
Table
Table 2. Area under curve (AUC), sensitivity, and specificity of the LACE index and HOSPITAL score prediction models for 30-day readmission after acute hospitalizations of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
Table 2. Area under curve (AUC), sensitivity, and specificity of the LACE index and HOSPITAL score prediction models for 30-day readmission after acute hospitalizations of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
30-Day Readmission Prediction ModelsArea under Curve (AUC)(95% C.I.)p-ValueSig. 1-
LACE index0.598(0.474–0.722)0.170--
HOSPITAL score0.691(0.573–0.808)0.008**-
30-Day Readmission Prediction ModelsAccuracy (%)Sensitivity (%)(95% CI)Specificity (%)(95% CI)
LACE index54.581.8(59.7–94.8)44.6(32.3–57.5)
HOSPITAL score54.090.9(70.8–98.9)41.5(29.4–54.4)
1 Sig.: significance level: ** p < 0.01.
Table
Table 3. Thirty-day readmission rate reduction and cost of preventive intervention by the LACE index and HOSPITAL score prediction models and all intervention model compared to the attending physician model. Simulated results of acute hospitalizations of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
Table 3. Thirty-day readmission rate reduction and cost of preventive intervention by the LACE index and HOSPITAL score prediction models and all intervention model compared to the attending physician model. Simulated results of acute hospitalizations of a home care patient cohort of a medical center affiliated home care unit in Taiwan, 2017 (n = 87).
30-Day Readmission Prediction Models 130-Day Readmission Rate Reduction (%)(IQR 3)Cost of Preventive Intervention (%) 4(IQR 2)
LACE index39.2(39.1–39.4)66.8(61.0–72.6)
HOSPITAL score43.4(43.3–43.5)72.0(63.1–79.4)
Attending physician10.1(9.8–10.3)18.6(17.2–19.9)
All intervention50.0(50.0–50.0)100 (reference)-
1 Preventive intervention was performed depending on the likelihood of 30-day readmission made by each prediction model. LACE index: LACE index ≥11; HOSPITAL score: HOSPITAL score ≥5; attending physician: simulated prediction made by attending physicians [23]; all intervention: preventive efforts are performed for all acute hospitalizations without any prediction. 2 IQR: interquartile range. 3 The current model only includes the direct costs of preventive intervention against readmission for acute hospitalization for those predicted as at high risk for readmission, thus the cost of a physician (e.g., extra payment) and maintenance of a computer system are not counted. Cost of the all intervention model (i.e., no prediction model) is used as a reference.

Share and Cite

MDPI and ACS Style

Su, M.-C.; Wang, Y.-J.; Chen, T.-J.; Chiu, S.-H.; Chang, H.-T.; Huang, M.-S.; Hu, L.-H.; Li, C.-C.; Yang, S.-J.; Wu, J.-C.; et al. Assess the Performance and Cost-Effectiveness of LACE and HOSPITAL Re-Admission Prediction Models as a Risk Management Tool for Home Care Patients: An Evaluation Study of a Medical Center Affiliated Home Care Unit in Taiwan. Int. J. Environ. Res. Public Health 2020, 17, 927. https://doi.org/10.3390/ijerph17030927

AMA Style

Su M-C, Wang Y-J, Chen T-J, Chiu S-H, Chang H-T, Huang M-S, Hu L-H, Li C-C, Yang S-J, Wu J-C, et al. Assess the Performance and Cost-Effectiveness of LACE and HOSPITAL Re-Admission Prediction Models as a Risk Management Tool for Home Care Patients: An Evaluation Study of a Medical Center Affiliated Home Care Unit in Taiwan. International Journal of Environmental Research and Public Health. 2020; 17(3):927. https://doi.org/10.3390/ijerph17030927

Chicago/Turabian Style

Su, Mei-Chin, Yi-Jen Wang, Tzeng-Ji Chen, Shiao-Hui Chiu, Hsiao-Ting Chang, Mei-Shu Huang, Li-Hui Hu, Chu-Chuan Li, Su-Ju Yang, Jau-Ching Wu, and et al. 2020. "Assess the Performance and Cost-Effectiveness of LACE and HOSPITAL Re-Admission Prediction Models as a Risk Management Tool for Home Care Patients: An Evaluation Study of a Medical Center Affiliated Home Care Unit in Taiwan" International Journal of Environmental Research and Public Health 17, no. 3: 927. https://doi.org/10.3390/ijerph17030927

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop