Analysis of the Larissa Heart Failure Risk Score: Predictive Value in 9207 Patients Hospitalized for Heart Failure from a Single Center

Early risk stratification is of outmost clinical importance in hospitalized patients with heart failure (HHF). We examined the predictive value of the Larissa Heart Failure Risk Score (LHFRS) in a large population of HHF patients from the Cleveland Clinic. A total of 13,309 admissions for heart failure (HF) from 9207 unique patients were extracted from the Cleveland Clinic’s electronic health record system. For each admission, components of the 3-variable simple LHFRS were obtained, including hypertension history, myocardial infarction history, and red blood cell distribution width (RDW) ≥ 15%. The primary outcome was a HF readmission and/or all-cause mortality at one year, and the secondary outcome was all-cause mortality at one year of discharge. For both outcomes, all variables were statistically significant, and the Kaplan–Meier curves were well-separated and in a consistent order (Log-rank test p-value < 0.001). Higher LHFRS values were found to be strongly related to patients experiencing an event, showing a clear association of LHFRS with this study outcomes. The bootstrapped-validated area under the curve (AUC) for the logistic regression model for each outcome revealed a C-index of 0.64 both for the primary and secondary outcomes, respectively. LHFRS is a simple risk model and can be utilized as a basis for risk stratification in patients hospitalized for HF.


Introduction
Heart failure (HF) is a pandemic characterized by high prevalence and incidence [1].At least 15 million patients in Europe suffer from HF, according to the European Society of Cardiology (ESC) [2].In the United States, data from the ARIC Community Surveillance (2005-2013) revealed a significant number of annual first episodes of acute decompensated HF in the range of 25.7-34.8per 1.000 person years for patients aged ≥ 75 years [3].Although a number of life-saving therapeutic strategies have been implemented in the last two decades, both pharmaceutical (ACE-inhibitors, ARB, ARNI, beta-blockers, mineralocorticoid antagonists) and interventional (primary percutaneous coronary intervention, cardiac resynchronization therapy, implantable cardioverter-defibrillators, and left ventricular assist devices), recent data from the ESC-HFA Heart Failure Long-Term Registry (ESC-HF-LT) revealed 1-year all-cause mortality rates of 23.6% for hospitalized HF (HHF) and 6.4% for chronic HF (CHF) [4].
Timely and precise risk stratification of HF patients is of outmost importance, especially in those acutely hospitalized, where the clinical decision is often challenging [5].Consequently, several predictive scores have been evolved and validated in HHF [6][7][8][9].Nonetheless, in the majority of these prognostic models, numerous parameters (multivariable risk scores) are incorporated, making their calculation difficult and frequently requiring the utilization of sophisticated calculators [10].Accordingly, we validated the predictive value of the very simply calculated Larissa Heart Failure Risk Score (LHFRS) [11,12] in a large population of patients hospitalized for HF from a single health system.

Patients
A total of 13,309 hospital admissions for HF, corresponding to 9207 patients, were extracted from the Cleveland Clinic Health System electronic health record system (HER), which satisfied the inclusion criteria: (1) Occurrence in Ohio (from December 2009 through May 2015) with HF as the primary reason for hospital admission.(2) Admission hemoglobin value ≥ 10 g/dL.

The Larissa Heart Failure Risk Score
The derivation of LHFRS has been previously described in detail.[11] Variables included are HTN history, MI history, and RDW at admission.Absence of HTN history, presence of MI history, and admission RDW value ≥15% are assigned 2, 1, and 1 points, respectively.Patients with LHFRS = 0 have the best score, whereas those with LHFRS = 4 have the worst score.

Definitions
HTN history was defined as having a diagnosis of HTN on the problem list during admission; MI history was defined as having an MI diagnosis any time prior to admission; and RDW values that were measured on the day of admission were used.Regarding all-cause mortality, the date of death was used.This consisted of either: (a) the date of death if the patient died in the hospital and thus it was recorded; or (b) the date of death according to the Ohio Death Index, which is the state's record of people who died in Ohio or were Ohio residents and died outside of Ohio.The ICD codes used for the present analysis are depicted in Table 1.

Outcomes
The primary outcome of the present analysis was either a HF readmission or all-cause mortality within 1-year of discharge.The secondary outcome was all-cause mortality within 1-year of discharge.The first occurrence of readmission or death after discharge was taken to be the event time.Patients were followed through November 2017, though censoring did not occur during the time-horizon of interest.Therefore, a binary outcome variable was defined for analysis, indicating whether each event occurred within 1-year of discharge.

Statistical Analysis
Continuous variables are presented as mean and standard deviation, whereas categorical variables are counts and percentages (%).Statistical tests were performed to test for differences between the outcome groups for the primary and secondary outcomes.p-values were considered statistically significant at the 0.05 level.To examine the discriminatory ability of the LHFRS stratification, Kaplan-Meier curves were estimated, and log-rank tests were performed to assess differences in survival curves.The Akaike information criterion (AIC) was used to estimate the likelihood of the model to predict/estimate the future values.Finally, the LHFRS score was directly used in the calculation of the concordance probability (C-index, area under the curve [AUC]).The R statistical software (R Core Team (2018).R: A language and environment for statistical computing.R Foundation for Statistical Computing, Vienna, Austria) were used for data analysis.This study complies with the Declaration of Helsinki, and an Institutional Review Board (IRB) approval was obtained.

Results
Of the 9207 patients in the analysis, 49.5% were male, 67.9% were Caucasian, and 28.4% were African-American.Table 2 displays the univariate summaries of each risk-factor and each demographic attribute at the admission level.All p-values appear to show statistically significant differences among the levels of each outcome.
Figure 1A shows the Kaplan-Meier curves with 95% confidence bands for the primary and secondary outcomes stratified by the computed LHFRS.Empirically, the curves overall appear to be well-separated and in a fairly consistent order.The log-rank test for both outcomes produced a p-value < 0.001.Figure 1A shows the Kaplan-Meier curves with 95% confidence bands for the primary and secondary outcomes stratified by the computed LHFRS.Empirically, the curves overall appear to be well-separated and in a fairly consistent order.The log-rank test for both outcomes produced a p-value < 0.001.
Figure 1B reiterates the discriminatory ability of the LHFRS by showing the proportion of observed admissions resulting in primary/secondary outcomes at 1-year.It appears the overall trend indicates a consistent relationship between the LHFRS and the risk of the outcome(s).The bootstrapped-validated AUC for the logistic regression model for each outcome using restricted cubic splines for RDW revealed a C-index of 0.64 both for the primary and secondary outcomes, respectively.Figure 2 shows the non-linear effect found by the model on each outcome, stratified by combinations of HTN and MI groups.The effects are consistent with the original paper [11] in that patients with a history of MI and no history of HTN are at the highest risk for either event.
We also examined the importance of RDW, HTN, and MI on the primary and sec- Figure 1B reiterates the discriminatory ability of the LHFRS by showing the proportion of observed admissions resulting in primary/secondary outcomes at 1-year.It appears the overall trend indicates a consistent relationship between the LHFRS and the risk of the outcome(s).The bootstrapped-validated AUC for the logistic regression model for each outcome using restricted cubic splines for RDW revealed a C-index of 0.64 both for the primary and secondary outcomes, respectively.
Figure 2 shows the non-linear effect found by the model on each outcome, stratified by combinations of HTN and MI groups.The effects are consistent with the original paper [11] in that patients with a history of MI and no history of HTN are at the highest risk for either event.We also examined the importance of RDW, HTN, and MI on the primary and secondary outcomes.Logistic regression models were built while modeling the effect of RDW with a three-knot restricted cubic spline.For both outcomes, all variables were found to be statistically significant, including the non-linear effect of RDW on each outcome.The importance of each variable was then ranked by the amount of increase in AIC upon the removal of each variable from the full model.Figure 3 shows the ranked variable importance for each outcome.In both outcomes, RDW was by far the most influential variable in predicting the outcome, followed by the history of HTN and the history of MI.Since all 'Importance' measures are positive, this indicates that all variables are influential in explaining the outcomes.The next step was to investigate the sensitivity and specificity of the LHFRS using a less granular threshold (≤2 versus >2).LHFRS values above 2 were accompanied by low sensitivity and high specificity, as depicted in Table 3.The next step was to investigate the sensitivity and specificity of the LHFRS using a less granular threshold (≤2 versus >2).LHFRS values above 2 were accompanied by low sensitivity and high specificity, as depicted in Table 3.

Discussion
The present analysis was an external validation of the discriminatory ability of LHFRS in a large population of patients with HHF.Although the performance of LHFRS for the primary endpoint (HF rehospitalizations and/or all-cause mortality) and for the secondary endpoint (all-cause mortality) within 1 year from the initial hospitalization was found to be moderate, the model can be used as a reliable tool for the early detection of highrisk patients as those with a high LHFRS (i.e., 4, n = 1044) exhibited significantly worse outcomes in comparison to those with a low LHFRS (i.e., 0, n = 2430).This score consists of three variables (RDW, HTN history, and MI history) that have established prognostic value.RDW is a simple parameter of the complete blood count test (CBC) that expresses the variability of the size of red blood cells (RBCs) and has been traditionally used for the classification of different types of anemia [13].However, over the last few years, there has been evidence that increased RDW is associated with a poor prognosis in patients with coronary artery disease, stroke, diabetes, and HHF [14].In our study, patients with increased RDW (≥15%) manifested significantly higher event rates compared with those with RDW values < 15%.Interestingly, as shown in Figure 2, a further increase in RDW beyond 15% is also associated with an incremental risk of adverse events.
Low blood pressure is a negative risk factor, whereas HTN may serve as a protective mechanism in HHF patients, contrary to what is observed in the normal population [15].The concept of "reactive hypertension", which proposes the occurrence of a functional cardiac reserve during acute physiologic stress, may be a logical explanation.Patients with impaired functional cardiac reserve and lower blood pressure, even in acute situations (i.e., admission for HF), demonstrate a worse prognosis in comparison to those with higher values of blood pressure [16].Gheorghiade et al. reported that low blood pressure (<120 mm Hg) at admission was an independent risk factor for an unfavorable prognosis in the participants of the OPTIMIZE-HF registry [17].In our study, patients with no history of HTN were at highest risk for 1-year adverse outcomes.
The history of MI is also a risk factor for a poor prognosis [18].The strong association between HF and MI was demonstrated in the Valsartan in Acute Myocardial Infarction Trial (VALIANT) registry, which reported a denovo HF rate of 10.3% in 11,040 stable MI patients during the median 25-month follow-up [19,20].Regarding our analysis, patients with a history of MI exhibited a higher risk for the primary and secondary endpoints in comparison to those without a history of MI.However, MI history was found to be the least influential variable of LHFRS in predicting the outcomes.This may be attributed to the Cleveland Clinic's successful quality of care performance, as depicted by the low mortality rates among patients who had percutaneous coronary intervention (PCI) procedures and/or presented with acute MI, the short door-to-balloon time, and the 100% administration of all guideline-recommended categories of medical therapy before and after PCI [21].Thus, it is possible that the performance of LHFRS was "underestimated" in this patient population with unique characteristics and quite different from the central Greece patient cohort used to derive the LHFRS.Furthermore, the modest performance of the LHFRS can partially be explained by the fact that ~45% of patients have an LHFRS equal to 1 or 2; as shown in Figure 1B, these risk groups are nearly indistinguishable.
In the current study, a history of HTN was observed in 46.6% of patients, which is higher than that found in the ADHF/NT-proBNP score (42.9%) [22], comparable to that reported in the ESCAPE study (47%) [23] and ELAN-SCORE (51%) [24], and lower than that presented in the ESC-HF-LT registry (65.6%) [4] and the OPTIMIZE-HF score (72%) [25].Moreover, in our study, the percentage of MI history was 20.6%, similar to that observed in the OPTIMIZE-HF study (22%) [25] and lower compared to the EFFECT study (37.9%) [26] and the ESCAPE study (44.4%) [23].The baseline characteristics of the population included in the current study are different from the population examined in our previous report, as far as the race and the percentage of HTN history [11].Interestingly, our study primary (all-cause mortality and/or HF rehospitalization) and secondary (all-cause mortality) endpoints were found to be higher and similar compared to the ESC-HF-LT registry (53.2% vs. 36% and 27.5% vs. 23.6%,respectively) [4].Both study endpoints are generally considered robust and well accepted [27].
An interesting review and meta-analysis by Ouwerkerk et al. [28] showed that the mean C-statistic of the risk prediction models in HF was 0.66 ± 0.0005 in general, 0.63 ± 0.001, and 0.71 ± 0.001 with respect to the endpoints of mortality or rehospitalization (combined) and mortality (independently), respectively, which is similar to the current analysis of the LHFRS performance.An external validation of the SENIORS elderly HF risk model at the RICA registry revealed C-statistics of 0.60 and 0.66 for the primary (all-cause mortality or cardiovascular hospitalization) and secondary (all-cause mortality) endpoints, respectively [29].The validation of the EFFECT score and of the more recent AHEAD Score exhibited a C-index of 0.76 and 0.631 for the endpoint of 1-year all-cause mortality, respectively [26,30].
The role of risk scores in HHF is of substantial value as they can be utilized as prognostic tools and assist in clinical decision-making, especially in conditions where the decision must be taken promptly and accurately.However, the majority of the risk scores consist of multiple variables and require the utilization of sophisticated calculators, making their adoption challenging in daily clinical practice [10].For example, the risk models derived from the OPTIMIZE-HF study, the EFFECT study, and the ESCAPE study include 12, 10, and 8 variables, respectively [23,25,26].Our score is comprised of just three variables, namely two questions from the patient's medical history and one marker from the CBC.All of these parameters are easily obtainable, widely available, and low-cost.Therefore, the LHFRS can be practically implemented even by the emergency department.Moreover, a persistent association between the LHFRS and the risk of adverse events was reported in the present analysis, and score values > 2 demonstrated a good specificity for 1-year adverse outcomes.Hence, the utilization of LHFRS may facilitate clinical decision-making (i.e., more aggressive decongestive strategies in "very high-risk patients" with LHFRS ≥ 3), triage care plans (i.e., admission to the Intensive Care Unit) and close follow-up.Furthermore, a distinct characteristic of LHFRS is its potential to discriminate between the long-term risks of patients with HHF.Typically, HHF prognostic models are developed to predict short-term outcomes [6,31] based on the fact that outcomes during hospitalization or up to 6 months after discharge may seem more relevant to acute pathologies such as HHF syndromes.Nevertheless, long-term outcomes are clinically relevant (i.e., the Seattle HF Score) [32].For instance, the administration of inotropes in an acute setting may have desirable short-term but neutral or even harmful long-term effects [33,34].Taken all together, LHFRS exhibited an acceptable discriminatory ability in the present work, despite the fact that it is the most parsimonious long-term risk model in HHF (Figure 4) [35].

Limitations
This study has several limitations that need to be addressed.First, it is a retrospective study, and hence the results depend on the accuracy of the data collection.Second, the single-center nature of the present work in a high-volume tertiary center may limit its generalizability in diverse clinical settings.This is especially true regarding the predictive value of one of the constituents of the LHFRS, namely the MI history.However, most likely, MI treatment is less efficient in "real-world" centers, increasing the predictive value of the MI history and consequently of the LHFRS in this setting.Thirdly, patients with Hg values < 10 g/dL were excluded, and, therefore, this score was not tested in severely anemic patients.However, anemia, which may influence RDW values, was present in only 18% of the HHF patients in Get With the Guidelines-Heart Failure [36].
The prognosis determined at admission may be affected by several post-discharge factors, including medical interventions, revascularization, cardiac resynchronization therapy, and implantable cardioverter defibrillator implantation.Assigning medical history by manual chart review is timely but accurate.Assigning medical history from electronic databases are challenging and dependent upon discrete searchable variables or complex algorithms to extract specific medical histories.Various algorithms were utilized to develop methods to determine medical history, including HTN and MI.Lastly, a number of predictive models have been developed for HF patients based on artificial intelligence over the last few years [37][38][39].However, these models have important limitations, such as the inability to account for time-to-event variations as well as the "interpretability issue" (i.e., poor explanation of the predictions) [40].

Future Perspectives
The LHFRS has demonstrated the ability to predict risk in hospitalized HF patients for subsequent events up to 1 year post-discharge, irrespective of the ejection fraction.An elevated LHFRS at the time of HF hospitalization might prospectively identify patients with advanced HF, a population with an increased risk of adverse outcomes [41].Long-term outcomes (1 year post-discharge) may be clinically important, especially with respect to identifying advanced HF patients that are potential candidates for left ventricular assist devices (LVAD) and/or heart transplantation.Furthermore, the combination of the LHFRS with novel markers of prognosis in HF, such as spot urinary sodium, may be a fruitful area of research [42].

Conclusions
LHFRS is easily calculated and can be utilized as a basis for risk prediction in patients with HHF.An elevated LHFRS of ≥3 is associated with a 1-year mortality rate over 35%.Further studies investigating the role of this simple prognostic tool in HF patients are needed.

Figure 1 .
Figure 1.(A) Kaplan-Meier curves for the primary and secondary outcomes stratified by the computed Larissa Heart Failure Risk Score (LHFRS); (B) Percent of patients experiencing each outcome stratified by the Larissa Heart Failure Risk Score (LHFRS) classification.

Figure 1 .
Figure 1.(A) Kaplan-Meier curves for the primary and secondary outcomes stratified by the computed Larissa Heart Failure Risk Score (LHFRS); (B) Percent of patients experiencing each outcome stratified by the Larissa Heart Failure Risk Score (LHFRS) classification.

Figure 2 .
Figure 2. Three-knot cubic spline of red blood cell distribution width (RDW) stratified by hypertension (HTN) and myocardial infarction (MI) groups for the primary and secondary outcomes.

Figure 3 .
Figure 3. Relative variable importance rankings on the primary and secondary outcomes measured

Figure 2 .
Figure 2. Three-knot cubic spline of red blood cell distribution width (RDW) stratified by hypertension (HTN) and myocardial infarction (MI) groups for the primary and secondary outcomes.

Figure 2 .
Figure 2. Three-knot cubic spline of red blood cell distribution width (RDW) stratified by hypertension (HTN) and myocardial infarction (MI) groups for the primary and secondary outcomes.

Figure 3 .
Figure 3. Relative variable importance rankings on the primary and secondary outcomes measured by the increase in Akaike information criterion (AIC) upon removal of each variable from the full model.

Figure 3 .
Figure 3. Relative variable importance rankings on the primary and secondary outcomes measured by the increase in Akaike information criterion (AIC) upon removal of each variable from the full model.

Figure 4 .
Figure 4. Most prognostic risk models consist of several variables and are complex.The LHFRS is the most parsimonious risk model in HF and includes only three variables (i.e., hypertension history, myocardial infarction history, and red cell distribution width).Its estimation is very simple and does not require the use of sophisticated calculators.The increase in the LHFRS (i.e., from 0 to 4) is related to the incremental rise of unfavorable event risk.Abbreviations: LHFRS, Larissa heart failure risk score; HTN, hypertension; MI, myocardial infarction; RDW, red blood cell distribution width.Adapted with permission from Kitai T. et al. (2020), Copyright © 2023 Elsevier B.V., Ref. [35].

Figure 4 .
Figure 4. Most prognostic risk models consist of several variables and are complex.The LHFRS is the

Table 1 .
ICD codes and descriptions of the variables used in the analysis.
Hypertensive heart and chronic kidney disease with heart failure and stage 5 chronic kidney disease, or end-stage renal disease
Hypertensive heart and chronic kidney disease, malignant, with heart failure and chronic kidney disease stage I through stage IV, or unspecified 404.03 Hypertensive heart and chronic kidney disease, malignant, with heart failure, and with chronic kidney disease stage V or end-stage renal disease 404.11Benign hypertensive heart and renal disease with congestive heart failure
Acute myocardial infarction of the other anterior wall, subsequent episode of care 410.32 Acute myocardial infarction of the inferoposterior wall, subsequent episode of care 410.42 Acute myocardial infarction of the other inferior wall, subsequent episode of care 410.02Acute myocardial infarction of the anterolateral wall, subsequent episode of care 410.52 Acute myocardial infarction of the other lateral wall, subsequent episode of care

Table 2 .
A univariate summary of risk-factors in addition to demographic variables at the admission level.Counts (percentages) were computed for categorical variables and the mean (SD) for continuous ones.For categorical variables, a Chi-square test of independence was performed.For numeric variables, a Wilcoxson rank-sum test was performed.p-values testing differences among outcome groups are also displayed.

Table 3 .
Examining the sensitivity/specificity of using a less granular threshold of the LHFRS (<=2 vs. >2) for: (a) all-cause death or rehospitalization for heart failure; (b) all-cause death.
Institutional Review Board Statement: Institutional Review Board (IRB) approval was obtained.Institution name: Cleveland Clinic, Study Number: 15-1361.Patient consent was waived due to the observational nature of this study.