Next Article in Journal
Analysis of Urban Forest Healing Program Expected Values, Needs, and Preferred Components in Urban Forest Visitors with Diseases: A Pilot Survey
Next Article in Special Issue
How Many Hours of Device Wear Time Are Required to Accurately Measure Physical Activity Post Stroke?
Previous Article in Journal
Psychometric Analysis of the Shortened Version of the Spiritual Well-Being Scale on the Slovak Population (SWBS-Sk)
Previous Article in Special Issue
Prediction of Cardiorespiratory Fitness in Czech Adults: Normative Values and Association with Cardiometabolic Health
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Adding Estimated Cardiorespiratory Fitness to the Framingham Risk Score and Mortality Risk in a Korean Population-Based Cohort Study

College of Sport Science, Sungkyunkwan University, Suwon 16419, Korea
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(1), 510; https://doi.org/10.3390/ijerph19010510
Submission received: 6 December 2021 / Revised: 30 December 2021 / Accepted: 1 January 2022 / Published: 3 January 2022

Abstract

:
Background: The added value of non-exercise-based estimation of cardiorespiratory fitness (eCRF) to cardiovascular disease (CVD) risk factors for mortality risk has not been examined in Korean populations. Methods: This population-based prospective cohort study examined the relationship of the 10-year Framingham risk score (FRS) for CVD risk and eCRF with all-cause and CVD mortality in a representative sample of Korean adults aged 30 years and older. Data regarding a total of 38,350 participants (16,505 men/21,845 women) were obtained from the 2007–2015 Korea National Health and Nutrition Examination Survey (KNHANES). All-cause and CVD mortality were the main outcomes. The 10-year FRS point sum and eCRF level were the main exposures. Results: All-cause and CVD mortality was positively correlated with the 10-year FRS point summation and inversely correlated with eCRF level in this study population. The protective of high eCRF against all-cause and CVD mortality was more prominent in the middle and high FRS category than in the low FRS category. Notably, the FRS plus eCRF model has better predictor power for estimating mortality risk compared to the FRS only model. Conclusions: The current findings indicate that eCRF can be used as an alternative to objectively measured CRF for mortality risk prediction.

1. Introduction

Cardiovascular disease (CVD) is the leading cause of global death, especially in low- and middle-income countries [1]. Traditional risk factors, such as hypertension, diabetes, cigarette smoking, family history of premature CVD, chronic kidney disease, and obesity, are well-established predictors of morbidity and mortality in Western [2,3] and Asian populations [4,5]. Thus, identifying individuals at increased CVD risk as early as possible is critical so that an appropriate therapeutic strategy can be timely implemented.
The Framingham risk score (FRS) for estimating 10-year CVD risk was developed with the data of coronary risk factors obtained from the Framingham Heart Study [6]. The 10-year FRS is the most widely used and validated tool for estimating CVD risk score [7,8]. However, the 10-year FRS does not account for physical activity (PA) and cardiorespiratory fitness (CRF) although they are well-established predictors for morbidity and mortality [9,10].
CRF is defined as the oxygen-delivering capacity of the circulatory and respiratory systems into skeletal muscles during sustained physical activity. Substantial evidence supporting CRF as an independent determinant of CVD mortality exists in the literature [11,12]. For example, a 1-unit increase in metabolic equivalent of task (MET; approximating 3.5 mL O2/kg/min) at baseline was associated with an 18% decrease in 30-year CVD mortality in FRS-based low-risk adults [2]. Higher CRF was significantly associated with lower CVD mortality risk in a 10-year follow-up study [13]. The inverse relationship between CRF and CVD mortality has been investigated systematically and well-summarized in meta-analysis studies [14,15].
In South Korea, CVD is the leading cause of mortality second to malignancy. The predictive role of CVD risk factors for estimating mortality risk has been reported [15,16]. However, only a handful number of population-based studies has examined the potential of CRF as a predictor for all-cause and CVD mortality [17,18]. Furthermore, the addition of estimated CRF (eCRF) to the 10-year FRS for predicting mortality risk in Korean populations is not known.
Together, poor CRF along with the traditional risk factors is now considered as an important indicator of mortality risk in Korean adults. To the best of our knowledge, however, the added value of eCRF to the 10-year FRS for estimating mortality risk in South Korea has not been examined in previous studies. Therefore, in this population-based cohort study, the prognostic value of adding eCRF to the 10-year FRS was investigated by examining improvement in prediction of all-cause and CVD mortality risk.

2. Materials and Methods

2.1. Study Population and Database Information

The Korea National Health and Nutrition Examination Survey (KNHANES) is a nationwide surveillance system designed to assess health and nutritional status of Koreans in South Korea (http://knhanes.cdc.go.kr/ accessed on 5 May 2021). The detailed design and procedures of the KNHANES are described elsewhere [19]. In brief, the survey includes three parts: health examination, health interview, and nutrition survey. With consent obtained from participants (response rate of 92.1%), the KNHANES 2007–2015 data were linked to death certificates and medical records from 1 January 2007 to 31 December 2016. All participants provided informed consent and the survey was approved by the Ethics Committee of the Korea Centre for Disease Control and Prevention (2007-02CON-04-P, 2008-04EXP-01-C, 2009-01COM-03-2C, 2010-02CON-21-C, 2011-02CON-06-C, 2012-01EXP-01-2C, 2013-07CON-03-4C, and 2013-12EXP-03-5C).
The current study included KNHANES participants who responded to the survey from 2007 to 2015, and consented to mortality follow-up (n = 51,603). Among the participants, those who were aged <30 years (n = 6341) were excluded because no parameters were available to calculate the FRS. In addition, subjects with missing or no data for BMI (n = 188), resting HR (n = 343), physical activity (n = 2295), blood chemistry (n = 2278), blood pressure or smoking (n = 211) or covariates (i.e., income, education, marriage, drinking, menopause status, chronic diseases, and hospitalization) (n = 1045) were excluded. Lastly, subjects who died during the first 3 years of follow-up (n = 552) were excluded to minimize the influence of preexisting health conditions on mortality risk. Consequently, a total of 38,350 participants (16,505 men/21,845 women) were included in the final data analysis (Figure 1).

2.2. Determination of Anthropometrics and CVD Risk Factors

Height and weight were measured using an automatic height scale (SECA-225, SECA, Hamburg, Germany) and a weight scale (GL-6000-20, G tech, Uijeongbu, Korea), respectively, and body mass index (BMI) was calculated by dividing weight (kg) by height (m2). Resting heart rate (RHR) was obtained by counting pulse rate at the right radial for 15 s and multiplying it by 4. Physical activity was assessed using the Korean version of the international physical activity questionnaire short form [20].
For blood chemistry, 13 mL of venous blood was collected from the median cubital vein of the non-dominant arm after an 8 h fast. Blood concentrations of high-density lipoprotein cholesterol (HDLC), low-density lipoprotein cholesterol (LDLC), total cholesterol (TC), fasting blood glucose (FBG) were assessed using standard procedures [19]. Concentrations of LDLC was calculated using the Friedewald formula [21]; LDLC (mg/dL) = TC-HDLC-TG/5. Blood pressure was measured 3 times at the right brachial artery with a mercury sphygmomanometer (Baumanometer desk model 0320, Baum, NY, USA) in a comfortable sitting position. The average of 2–3 measurements was recorded. Use of antihypertensive therapy, current/past smoking, and presence of diabetes was assessed using a health questionnaire. Smoking was defined as currently smoking or having had smoked more than 100 cigarettes in the past. Diabetes was defined as fasting blood glucose of ≥126 mg/dL or physician-diagnosed diabetes.

2.3. Determination of the 10-Year FRS and eCRF

The 10-year FRS was calculated based on sex, age, TC, HDLC, SBP, use of antihypertensive therapy, smoking, and presence of diabetes, as described previously [22]. The FRS point summation was classified as low (<10%), moderate (10–20%), or high (>20%) risk.
eCRF was determined using the algorithm developed by Jurca et al. [23]; eCRF (METs) = (ex (women = 0, men = 1) × 2.77) − (age (years) × 0.10) − (BMI (kg/m2) × 0.17) − (RHR (beats/min) × 0.03) + (physical activity score × 1.00) + 18.07. Then, receiver operating characteristic (ROC) analysis was conducted to evaluate the gender-specific relationship between individual eCRF values and all-cause mortality. Area under the curve (AUC) value of eCRF for predicting all-cause mortality was 0.754 (95% CI = 0.737–0.770, p < 0.001) in males and 0.776 (95% CI = 0.757–0.796, p < 0.001) in females. The optimal cut-off value for all-cause mortality was 9.16 and 6.14 METs for men and women, respectively. The sensitivity and specificity of the cut-off values were 0.587 and 0.786, respectively, for men, and 0.668 and 0.776, respectively, for women (Supplemental Tables S1 and S2).

2.4. Determination of Mortality

Main outcomes were all-cause and CVD mortality, which were defined as death from all causes or CVD. Date of causes of death from 1 October 2007 to 31 December 2018 were identified by medical records filed at the Korean Statistical Information Service (KOSIS). Using the International Classification of Disease, 10th version (ICD-10), all-cause mortality (n = 1474) and deaths from cardiovascular diseases (I00-I99) (n = 325) were identified. Follow-up time was defined as the period from the baseline visit to the day of death for participants who died or to the last contact date for those who did not experience the outcome event (censored).

2.5. Determination of Covariates

The covariates used in the study were age, sex, household income, education (i.e., less than elementary, middle school, high school, college and higher), marital status (i.e., married or widowed/divorced or unmarried), residence area (i.e., urban vs. rural). The covariates were assessed at baseline using a questionnaire [19].

2.6. Statistics

Characteristics of study participants were compared using analysis of variance and χ2 tests and presented as mean ± standard deviation (SD) for continuous variables and number (percentages) for categorical variables. Death rate per 1000 person/years of follow-up was calculated for e-CRF level and the 10-year FRS point sum.
Cox proportional hazards model was used to estimate hazard ratio (HR) and 95% confidence intervals (CIs) for all-cause and CVD mortality according to 10-year FRS category (i.e., low or moderate or high risk) or eCRF level (i.e., unfit vs. fit) in two models. Model 1 was adjusted for age and sex. Model 2 was additionally adjusted for household income, education, marital status, and residence area. The Kaplan–Meier procedure with log-rank tests was used to estimate mortality functions according to number of baseline lifestyle risk factors. Survival time was measured as the time from the baseline survey to death or the censor point (31 December 2018). In addition, the significance of multiplicate interaction between eCRF and FRS levels for all-cause and CVD mortality was tested by adding cross-products term in the Cox proportional hazard models.
The ROC curves were plotted with the MedCalc statistical software (version 20.009, MedCalc Software Ltd., Ostend, Belgium) to determine if significant improvement in the predictive accuracy of mortality existed when adding eCRF to the FRS point sum. The χ2 test was used to determine significant difference in AUC between the FRS plus eCRF model, and the FRS only model for estimating mortality risk.

3. Results

As shown in Table 1, there were 5.26 deaths per 1000 person/years (PY) in total, 7.27 deaths per 1000 PY in men, and 3.75 deaths per 1000 PY in women. In general, males were older (p = 0.027), heavier (p < 0.001), had higher income (p < 0.001), higher education level (p < 0.001), tended to be married (p < 0.001) and religious (p < 0.001) compared with females. Males were more active (p < 0.001), had higher SBP (p < 0.001), higher smoking rate (p < 0.001), higher at-risk alcohol consumption (p < 0.001), and higher rates of diabetes (p < 0.001) with lower resting heart rate (p < 0.001), lower HDLC (p < 0.001), lower LDLC (p < 0.001), and lower TC (p < 0.001) than females.
Table 2 shows the descriptive statistics of study participants dichotomized based on eCRF. Fit individuals were younger (p < 0.001), had higher income (p < 0.001) and higher education level (p < 0.001), tended to be married (p < 0.001), reside in urban areas (p < 0.001), had lower BMI (p < 0.001), lower LDLC (p < 0.001), lower TC (p < 0.001), lower SBP (p < 0.001), lower smoking rate (p < 0.001), and lower rate of diabetes (p < 0.001) with higher HDLC (p < 0.001) than did unfit individuals.
Table 3 shows the mortality risk stratified based on eCRF and the FRS. Fit individuals had lower all-cause (HR (95% CI) = 0.74 (0.66–0.84), p < 0.001) and CVD mortality (HR (95% CI) = 0.77 (0.68–0.87, p < 0.001) compared with unfit individuals (HR = 1). The lower HR remained significant for all-cause (HR (95% CI) = 0.62 (0.47–0.80), p < 0.001) and CVD mortality (HR (95% CI) = 0.64 (0.49–0.83, p = 0.001) even after adjustments for all covariates. In addition, based on FRS category, a positive linear trend was observed for all-cause (HR (95% CI) = 1.14 (0.95–1.36) and HR (95% CI) = 1.31 (1.07–1.60), p = 0.025), and CVD mortality (HR (95% CI) = 1.31 (0.87–1.96) and HR (95% CI) = 2.16 (1.41–3.30), respectively, p < 0.001). Furthermore, the linear trend in FRS category remained significant for all-cause (HR (95% CI) = 1.08 (0.91–1.30), HR (95% CI) = 1.26 (1.03–1.54, respectively, p = 0.046), and CVD mortality (HR (95% CI) = 1.25 (0.83–1.87) and HR (95% CI) = 2.07 (1.36–3.16, respectively, p < 0.001) even after adjustments for all covariates. As illustrated in Figure 2, the Kaplan–Meier mortality functions showed that the survival rates of all-cause and CVD mortality decreased significantly by decremental eCRF (from fit to unfit) and incremental FRS (from low to high), respectively.
Table 4 represents HRs and 95% CI for all-cause and CVD mortality by eCRF and FRS. There were significant interaction effects between eCRF and FRS on all-cause and CVD mortality in all models (p < 0.001 and p < 0.001, respectively). With respect to all-cause mortality, the negative impact of elevated FRS was found to be significant in unfit category but not in fit category. Compared to the combination of fit eCRF and low FRS (HR = 1), the combination of unfit eCRF and moderate FRS and the combination of unfit eCRF and high FRS were associated with the higher risk of all-cause mortality (HR = 1.21, p < 0.001 and HR = 1.34, p < 0.001, respectively) in the fully adjusted model. With respect to CVD mortality, the negative impact of elevated FRS was observed in the combination of fit eCRF and high FRS (HR = 1.53, p < 0.001) as well as the combination of unfit eCRF and moderate FRS (HR = 1.55, p < 0.001) and the combination of unfit eCRF and high FRS (HR = 2.55, p < 0.001).
Figure 3 shows the ROC curves plotted for the FRS only model and the FRS plus eCRF model. The AUC for the FRS plus e-CRF model was significantly greater (p < 0.001) than that for the FRS only model, indicating that the FRS plus eCRF model compared with the FRS only model has higher predictive power for estimating all-cause and CVD mortality.

4. Discussion

In this population-based prospective cohort study, we found that all-cause and CVD mortality were positively correlated with the 10-year FRS and inversely correlated with eCRF in Korean adults. In addition, we found that the protective effect of high eCRF against all-cause and CVD mortality was more prominent in the moderate and high FRS category than in the low FRS category, suggesting that fitness promotion via regular exercise may protect the premature death risks from elevated FRS. Notably, the FRS plus eCRF model has better predictive power for all-cause and CVD mortality compared with the FRS only model, implying the added value of eCRF to the traditional risk factor model.
In agreement with the current findings of the study, CRF has been well-established as an independent predictor of CVD mortality in healthy adults [14] as well as in CVD patients [24]. Specifically, Imboden et al. [25] showed that a 1-unit increase in MET was associated with 20% and 38% decreases in 30-year mortality risk for males and females, respectively. Cao et al. [12] showed that CRF was associated inversely with all-cause and cancer mortality in a US Baby Boomers and Generation Xers-based cohort study. The association between higher CRF and lower mortality also is observed in in a 10-year prospective cohort study involving 59,941 Koreans 30–84 years [17], and in a retrospective cohort study of 18,775 Korean men [18].
Although CRF is a well-known predictor of all-cause and CVD mortality risk, it is not routinely checked during clinical visits because specialized equipment, trained personnel, and sufficient time are needed. However, in recent studies, CRF was estimated with an acceptable accuracy using easily obtained health indicators [23], and it has been used as a parameter for predicting morbidity and mortality risk. In a population-based cohort study involving 12,834 participants aged 20 to 86 years, Zhang et al. [26] showed that higher eCRF was significantly associated with lower all-cause and CVD mortality risk in males and females. Wang et al. [27] also showed that a 1-MET increase in eCRF was associated with 30% and 27% lower risk of all-cancer mortality in males and females, respectively. In a prospective cohort study involving 29,850 men from the Aerobics Center Longitudinal Study, Gander et al. [28] showed that higher eCRF was associated with lower coronary heart disease (CHD) and lower CHD mortality. In our previous study with Korean older adults, we also found that higher eCRF was significantly associated with lower all-cause mortality risk [29]. However, we are not sure if eCRF can replace direct measurement of CRF as a predictor for mortality risk, which should be further investigated in a future study involving both predictors.
The prognostic value of adding eCRF to the FRS only model observed in the current study is of particular interest and supports the findings from previous studies. For example, Gupta et al. [24] examined the effect of adding CRF to traditional risk factors on risk classification in 66,371 healthy adults and found that the addition of CRF to the traditional risk factors improved classification of short- and long-term risk for CVD mortality. In a cross-sectional study involving 6962 patients, Myers et al. [30] showed that addition of physical activity and CRF to the traditional risk factor model improved risk classification accuracy for mortality by 22.8% and 43.5%, respectively. The prognostic role of adding other health indicators to the CVD risk factors-based model for mortality risk has been reported in previous studies [31,32]. Likewise, the role of adding other health indicators to the CVD risk factors-based model for predicting mortality risk has also been reported in previous studies [31,32]. For example, the algorithm for eCRF used in the current study includes age, sex, PA, RHR, and BMI as parameters. Among them, PA, RHR, and BMI are additional predictors that are not included in the FRS model. Therefore, the added value of eCRF to the FRS only model may imply the potential of the three parameters. In support of this notion, elevated RHR is a significant predictor of all-cause mortality in ambulatory patients with heart failure [33] and middle-aged populations [34,35]. In addition, mortality risk is significantly and independently associated with overweight/obesity [36,37], and physical inactivity [9,10] in different populations. Together, the findings from the current and previous studies suggest that adding eCRF to the traditional risk factor-based models may positively contribute to the accuracy for predicting mortality risk.
Several explanations are possible for the protective effect of high CRF against increased all-cause and CVD mortality risk associated with the traditional risk factors. CVD risk factors either individually and/or additively contribute to increased mortality risk via (i) decreased insulin sensitivity and increased insulin resistance, (ii) unfavorable blood lipids and lipoproteins, (iii) obesity/overweight, (iv) inflammatory responses, (v) elevated blood pressure, (vi) increased sympathetic and decreased parasympathetic tone, and (vii) metabolic disorders [38,39,40]. Conversely, obtaining and/or maintaining high CRF through regular physical activity provides morbidity and mortality benefits via improved insulin sensitivity and decreased insulin resistance [41], favorable changes in blood lipids and lipoproteins [42], anti-inflammation [43], decreased blood pressure [44], increased parasympathetic/decreased sympathetic tone [45], and decreased clustering of metabolic risk factors [46].
This study has limitations. First, the cross-sectional nature of the study limits any cause-and-effect explanation for the relationships between exposures and mortality. Second, parameters used to obtain eCRF would change during follow-up. However, only baseline eCRF was used in this study, which could result in an underestimation of the study’s associations. Third, the possibility cannot be excluded that other biomarkers or covariates not included in the current study may mediate or modulate the relationships between exposures and mortality [47]. This study also has strengths. First, this study included a large and representative sample of Korean adults for generalization of study outcomes. Second, the mortality data were gathered from a reliable register. Third, eCRF classification (unfit vs. fit) was based on the ROC analysis of the current data, which would provide a more reliable and reproducible association between the exposure and mortality.

5. Conclusions

In summary, we examined the relationship of the 10-year FRS point summation and eCRF levels with all-cause and CVD mortality in a representative sample of Korean adults and found that all-cause and CVD mortality risk was positively correlated with the FRS but inversely correlated with eCRF. The protective effect of high eCRF against all-cause and CVD mortality was noticeable in moderate and high FRS categories. In addition, we found that the FRS plus eCRF model has better predictor power for mortality risk than the FRS only model. Together, the current findings indicate that CRF promotion via physical activity should be encouraged for subjects at increased CVD risk, and incorporating eCRF into the traditional risk factors could be an alternative to objectively measuring CRF for mortality risk prediction.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph19010510/s1, Table S1: The receiver operating curve analysis of estimated cardiorespiratory fitness for all-cause mortality; Table S2: Cut-off values of estimated cardiorespiratory fitness for all-cause mortality.

Author Contributions

Study design, I.L. and H.K.; study conduct: I.L., J.K. and H.K.; data collection, I.L., J.K. and H.K.; data analysis, I.L., J.K. and H.K.; data interpretation, I.L., J.K. and H.K.; manuscript drafting, I.L. and H.K. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Research Foundation Grant funded by the Korean Government (NRF-2019R1I1A1A01043771).

Institutional Review Board Statement

The Institutional Review Board of the Korea Centre for Disease Control and Prevention in accordance with the Declaration of Helsinki reviewed and approved the KNHANES (2007-02CON-04-P, 2008-04EXP-01-C, 2009-01COM-03-2C, 2010-02CON-21-C, 2011-02CON-06-C, 2012-01EXP-01-2C, 2013-07CON-03-4C, and 2013-12EXP-03-5C).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author on request.

Acknowledgments

Data in this study were from the Korea National Health and Nutrition Examination Survey, Centers for Disease Control and Prevention, Ministry for Health and Welfare and Cause of Death Statistics, Statistics Korea.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. World Health Organization. Fact Sheets; Cardiovascular Diseases (CVDs). 2021. Available online: https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds) (accessed on 11 June 2021).
  2. Barlow, C.E.; Defina, L.F.; Radford, N.B.; Berry, J.D.; Cooper, K.H.; Haskell, W.L.; Jones, L.W.; Lakoski, S.G. Cardiorespiratory fitness and long-term survival in “low-risk” adults. J. Am. Heart Assoc. 2021, 1, e001354. [Google Scholar] [CrossRef] [Green Version]
  3. Rawshani, A.; Rawshani, A.; Rawshani, A.; Franzén, S.; Sattar, N.; Eliasson, B.; Svensson, A.M.; Zethelius, B.; Miftaraj, M.; McGuire, D.K.; et al. Risk factors, mortality, and cardiovascular outcomes in patients with type 2 diabetes. N. Engl. J. Med. 2018, 379, 633–644. [Google Scholar] [CrossRef]
  4. Kim, S.M.; Kim, S.M.; Lee, G.; Choi, S.; Kim, K.; Jeong, S.M.; Son, J.S.; Yun, J.M.; Kim, S.G.; Hwang, S.S.; et al. Association of early-onset diabetes, prediabetes and early glycaemic recovery with the risk of all-cause and cardiovascular mortality. Diabetologia 2020, 63, 2305–2314. [Google Scholar] [CrossRef]
  5. Li, T.C.; Li, C.I.; Liu, C.S.; Lin, W.Y.; Lin, C.H.; Yang, S.Y.; Lin, C.C. Derivation and validation of 10-year all-cause and cardiovascular disease mortality prediction model for middle-aged and elderly community-dwelling adults in Taiwan. PLoS ONE 2020, 15, e0239063. [Google Scholar] [CrossRef] [PubMed]
  6. Wilson, P.W.; D’Agostino, R.B.; Levy, D.; Belanger, A.M.; Silbershatz, H.; Kannel, W.B. Prediction of coronary heart disease using risk factor categories. Circulation 1998, 97, 1837–1847. [Google Scholar] [CrossRef] [Green Version]
  7. Bitton, A.; Gaziano, T. The Framingham heart study’s impact on global risk assessment. Prog. Cardiovasc. Dis. 2020, 53, 68–78. [Google Scholar] [CrossRef] [Green Version]
  8. Collins, G.; Altman, D. An independent and external validation of QRISK2 cardiovascular disease risk score: A prospective open cohort study. BMJ 2010, 340, c2442. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Äijö, M.; Kauppinen, M.; Kujala, U.M.; Parkatti, T. Physical activity, fitness, and all-cause mortality: An 18-year follow-up among old people. J. Sport Health Sci. 2016, 5, 437–442. [Google Scholar] [CrossRef]
  10. Min, C.; Yoo, D.M.; Wee, J.H.; Lee, H.J.; Byun, S.H.; Choi, H.G. Mortality and cause of death in physical activity and insufficient physical activity participants: A longitudinal follow-up study using a national health screening cohort. BMC Public Health 2020, 20, 1469. [Google Scholar] [CrossRef]
  11. Imboden, M.T.; Harber, M.P.; Whaley, M.H.; Finch, W.H.; Bishop, D.L.; Kaminsky, L.A. Cardiorespiratory fitness and mortality in healthy men and women. J. Am. Coll. Cardiol. 2018, 72, 2283–2292. [Google Scholar] [CrossRef]
  12. Cao, C.; Yang, L.; Cade, W.T.; Racette, S.B.; Park, Y.; Cao, Y.; Friedenreich, C.M.; Hamer, M.; Stamatakis, E.; Smith, L. Cardiorespiratory fitness is associated with early death among healthy young and middle-aged baby boomers and generation Xers. Am. J. Med. 2020, 133, 961–968.e3. [Google Scholar] [CrossRef]
  13. Mitchell, J.A.; Bornstein, D.B.; Sui, X.; Hooker, S.P.; Church, T.S.; Lee, C.D.; Lee, D.C.; Blair, S.N. The impact of combined health factors on cardiovascular disease mortality. Am. Heart J. 2010, 160, 102–108. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Ross, R.; Blair, S.N.; Arena, R.; Church, T.S.; Després, J.P.; Franklin, B.A.; Haskell, W.L.; Kaminsky, L.A.; Levine, B.D.; Lavie, C.J.; et al. Importance of assessing cardiorespiratory fitness in clinical practice: A case for fitness as a clinical vital sign: A scientific statement from the American Heart Association. Circulation 2016, 134, e653–e699. [Google Scholar] [CrossRef]
  15. Schmid, D.; Leitzmann, M.F. Cardiorespiratory fitness as predictor of cancer mortality: A systematic review and meta-analysis. Ann. Oncol. 2015, 26, 272–278. [Google Scholar] [CrossRef]
  16. Kim, J.Y.; Ko, Y.J.; Rhee, C.W.; Park, B.J.; Kim, D.H.; Bae, J.M.; Shin, M.H.; Lee, M.S.; Li, Z.M.; Ahn, Y.O. Cardiovascular health metrics and all-cause and cardiovascular disease mortality among middle-aged men in Korea: The Seoul male cohort study. J. Prev. Med. Public Health 2013, 46, 319–328. [Google Scholar] [CrossRef]
  17. Yun, J.E.; Won, S.; Kimm, H.; Jee, S.H. Effects of a combined lifestyle score on 10-year mortality in Korean men and women: A prospective cohort study. BMC Public Health 2012, 12, 673. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Park, M.S.; Chung, S.Y.; Chang, Y.; Kim, K. Physical activity and physical fitness as predictors of all-cause mortality in Korean men. J. Korean Med. Sci. 2009, 24, 13–19. [Google Scholar] [CrossRef]
  19. Kweon, S.; Kim, Y.; Jang, M.J.; Kim, Y.; Kim, K.; Choi, S.; Chun, C.; Khang, Y.H.; Oh, K. Data resource profile: The Korea National Health and Nutrition Examination Survey (KNHANES). Int. J. Epidemiol. 2014, 43, 69–77. [Google Scholar] [CrossRef] [Green Version]
  20. Oh, J.Y.; Yang, Y.J.; Kim, B.S.; Kang, J.H. Validity and reliability of Korean version of international physical activity questionnaire (IPAQ) short form. J. Korean Acad. Fam. Med. 2007, 28, 532–541. [Google Scholar]
  21. Friedewald, W.T.; Levy, R.I.; Fredrickson, D.S. Estimation of low-density lipoprotein cholesterol in plasma without use of preparative ultracentrifuge. Clin. Chem. 1972, 18, 499–502. [Google Scholar] [CrossRef]
  22. D’Agostino, R.B.S.; Vasan, R.S.; Pencina, M.J.; Wolf, P.A.; Cobain, M.; Massaro, J.M.; Kannel, W.B. General cardiovascular risk profile for use in primary care: The Framingham Heart Study. Circulation 2008, 117, 743–753. [Google Scholar] [CrossRef] [Green Version]
  23. Jurca, R.; Jackson, A.S.; LaMonte, M.J.; Morrow, J.R., Jr.; Blair, S.N.; Wareham, N.J.; Haskell, W.L.; van Mechelen, W.; Church, T.S.; Jakicic, J.M.; et al. Assessing cardiorespiratory fitness without performing exercise testing. Am. J. Prev. Med. 2005, 29, 185–193. [Google Scholar] [CrossRef]
  24. Gupta, S.; Rohatgi, A.; Ayers, C.R.; Willis, B.L.; Haskell, W.L.; Khera, A.; Drazner, M.H.; de Lemos, J.A.; Berry, J.D. Cardiorespiratory fitness and classification of risk of cardiovascular disease mortality. Circulation 2011, 123, 1377–1383. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Imboden, M.T.; Harber, M.P.; Whaley, M.H.; Finch, W.H.; Bishop, D.A.; Fleenor, B.S.; Kaminsky, L.A. The influence of change in cardiorespiratory fitness with short-term exercise training on mortality risk from the Ball State Adults Fitness Longitudinal Lifestyle Study (BALL ST). Mayo Clin. Proc. 2019, 94, 1406–1414. [Google Scholar] [CrossRef]
  26. Zhang, Y.; Zhang, J.; Zhou, J.; Ernstsen, L.; Lavie, C.J.; Hooker, S.P.; Sui, X. Nonexercise estimated cardiorespiratory fitness and mortality due to all causes and cardiovascular disease: The NHANES III study. Mayo Clin. Proc. Innov. Qual. Outcomes 2017, 1, 16–25. [Google Scholar] [CrossRef] [PubMed]
  27. Wang, Y.; Chen, S.; Zhang, J.; Zhang, Y.; Ernstsen, L.; Lavie, C.J.; Hooker, S.P.; Chen, Y.; Sui, X. Nonexercise estimated cardiorespiratory fitness and all-cancer mortality: The NHANES III Study. Mayo Clin. Proc. 2018, 93, 848–856. [Google Scholar] [CrossRef]
  28. Gander, J.C.; Sui, X.; Hébert, J.R.; Lavie, C.J.; Hazlett, L.J.; Cai, B.; Blair, S.N. Addition of estimated cardiorespiratory fitness to the clinical assessment of 10-year coronary heart disease risk in asymptomatic men. Prev. Med. Rep. 2017, 7, 30–37. [Google Scholar] [CrossRef] [PubMed]
  29. Song, M.; Lee, I.; Kang, H. Cardiorespiratory Fitness without Exercise Testing Can Predict All-Cause Mortality Risk in a Representative Sample of Korean Older Adults. Int. J. Environ. Res. Public Health 2019, 16, 1633. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Myers, J.; Nead, K.T.; Chang, P.; Abella, J.; Kokkinos, P.; Leeper, N.J. Improved reclassification of mortality risk by assessment of physical activity in patients referred for exercise testing. Am. J. Med. 2015, 128, 396–402. [Google Scholar] [CrossRef] [PubMed]
  31. Nishimoto, M.; Tagawa, M.; Matsui, M.; Eriguchi, M.; Samejima, K.I.; Iseki, K.; Iseki, C.; Asahi, K.; Yamagata, K.; Konta, T.; et al. A prediction model with lifestyle in addition to previously known risk factors improves its predictive ability for cardiovascular death. Sci. Rep. 2019, 9, 12953. [Google Scholar] [CrossRef]
  32. Lee, W.J.; Peng, L.N.; Chiou, S.T.; Chen, L.K. Physical health indicators improve prediction of cardiovascular and all-cause mortality among middle-aged and older people: A national population-based study. Sci. Rep. 2017, 7, 40427. [Google Scholar] [CrossRef]
  33. Lau, K.; Malik, A.; Foroutan, F.; Buchan, T.A.; Daza, J.F.; Sekercioglu, N.; Orchanian-Cheff, A.; Alba, A.C. Resting heart rate as an important predictor of mortality and morbidity in ambulatory patients with heart failure: A systematic review and meta-analysis. J. Card. Fail. 2021, 27, 349–363. [Google Scholar] [CrossRef]
  34. Seccareccia, F.; Pannozzo, F.; Dima, F.; Minoprio, A.; Menditto, A.; Lo Noce, C.; Giampaoli, S.; Malattie Cardiovascolari Aterosclerotiche Istituto Superiore di Sanita Project. Heart rate as a predictor of mortality: The MATISS project. Am. J. Public Health 2001, 91, 1258–1263. [Google Scholar] [CrossRef]
  35. Jensen, M.T.; Suadicani, P.; Hein, H.O.; Gyntelberg, F. Elevated resting heart rate, physical fitness and all-cause mortality: A 16-year follow-up in the Copenhagen Male Study. Heart 2013, 99, 882–887. [Google Scholar] [CrossRef]
  36. Global BMI Mortality Collaboration; Di Angelantonio, E.; Bhupathiraju, S.N.; Wormser, D.; Gao, P.; Kaptoge, S.; Berrington de Gonzalez, A.; Cairns, B.J.; Huxley, R.; Jackson, C.L.; et al. Body-mass index and all-cause mortality: Individual-participant-data meta-analysis of 239 prospective studies in four countries. Lancet 2016, 388, 776–786. [Google Scholar] [CrossRef] [Green Version]
  37. Yang, Y.; Dugué, P.A.; Lynch, B.M.; Hodge, A.M.; Karahalios, A.; MacInnis, R.J.; Milne, R.L.; Giles, G.G.; English, D.R. Trajectories of body mass index in adulthood and all-cause and cause-specific mortality in the Melbourne collaborative cohort study. BMJ Open 2019, 9, e030078. [Google Scholar] [CrossRef] [Green Version]
  38. Adeva-Andany, M.M.; Martínez-Rodríguez, J.; González-Lucán, M.; Fernández-Fernández, C.; Castro-Quintela, E. Insulin resistance is a cardiovascular risk factor in humans. Diabetes Metab. Syndr. 2019, 13, 1449–1455. [Google Scholar] [CrossRef]
  39. Agostinis-Sobrinho, C.A.; Ruiz, J.R.; Moreira, C.; Abreu, S.; Luís, L.; Oliveira-Santos, J.; Mota, J.; Santos, R. Cardiorespiratory fitness and inflammatory profile on cardiometabolic risk in adolescents from the LabMed Physical Activity Study. Eur. J. Appl. Physiol. 2017, 117, 2271–2279. [Google Scholar] [CrossRef]
  40. Oliveira, C.; Silveira, E.A.; Rosa, L.; Santos, A.; Rodrigues, A.P.; Mendonça, C.; Silva, L.; Gentil, P.; Rebelo, A.C. Risk factors associated with cardiac autonomic modulation in obese individuals. J. Obes. 2020, 7185249. [Google Scholar] [CrossRef]
  41. Haufe, S.; Engeli, S.; Budziarek, P.; Utz, W.; Schulz-Menger, J.; Hermsdorf, M.; Wiesner, S.; Otto, C.; Haas, V.; de Greiff, A.; et al. Cardiorespiratory fitness and insulin sensitivity in overweight or obese subjects may be linked through intrahepatic lipid content. Diabetes 2001, 59, 1640–1647. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Breneman, C.B.; Polinski, K.; Sarzynski, M.A.; Lavie, C.J.; Kokkinos, P.F.; Ahmed, A.; Sui, X. The impact of cardiorespiratory fitness levels on the risk of developing atherogenic dyslipidemia. Am. J. Med. 2016, 129, 1060–1066. [Google Scholar] [CrossRef] [Green Version]
  43. González-Gil, E.M.; Santaliestra-Pasías, A.M.; Buck, C.; Gracia-Marco, L.; Lauriam, F.; Pala, V.; Molnar, D.; Veidebaum, T.; Iacoviello, L.; Tornaritis, M.; et al. Improving cardiorespiratory fitness protects against inflammation in children: The IDEFICS study. Pediatr. Res. 2021. [Google Scholar] [CrossRef]
  44. Agostinis-Sobrinho, C.; Ruiz, J.R.; Moreira, C.; Abreu, S.; Lopes, L.; Oliveira-Santos, J.; Mota, J.; Santos, R. Cardiorespiratory fitness and blood pressure: A longitudinal analysis. J. Pediatr. 2018, 192, 130–135. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Röhling, M.; Strom, A.; Bönhof, G.J.; Roden, M.; Ziegler, D. Cardiorespiratory Fitness and Cardiac Autonomic Function in Diabetes. Curr. Diab. Rep. 2017, 17, 125. [Google Scholar] [CrossRef]
  46. Wedell-Neergaard, A.S.; Krogh-Madsen, R.; Petersen, G.L.; Hansen, Å.M.; Pedersen, B.K.; Lund, R.; Bruunsgaard, H. Cardiorespiratory fitness and the metabolic syndrome: Roles of inflammation and abdominal obesity. PLoS ONE 2018, 13, e0194991. [Google Scholar] [CrossRef]
  47. Shigdel, R.; Dalen, H.; Sui, X.; Lavie, C.J.; Wisløff, U.; Ernstsen, L. Cardiorespiratory fitness and the risk of first acute myocardial infarction: The HUNT study. J. Am. Heart Assoc. 2019, 8, e010293. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. A flow chart of selection of study participants.
Figure 1. A flow chart of selection of study participants.
Ijerph 19 00510 g001
Figure 2. The Kaplan–Meier survival curves for all-cause and cardiovascular disease (CVD) mortality by estimated cardiorespiratory fitness (eCRF) and 10-yr Framingham risk score (FRS): (a) eCRF and all-cause mortality; (b) eCRF and CVD mortality; (c) FRS and all-cause mortality, and (d) FRS and CVD mortality.
Figure 2. The Kaplan–Meier survival curves for all-cause and cardiovascular disease (CVD) mortality by estimated cardiorespiratory fitness (eCRF) and 10-yr Framingham risk score (FRS): (a) eCRF and all-cause mortality; (b) eCRF and CVD mortality; (c) FRS and all-cause mortality, and (d) FRS and CVD mortality.
Ijerph 19 00510 g002
Figure 3. Receiver operating characteristic curve (AUC) comparing the predictive ability of the Framingham risk score (FRS) (Model A) compared to the FRS plus estimated cardiorespiratory fitness (eCRF) (Model B).
Figure 3. Receiver operating characteristic curve (AUC) comparing the predictive ability of the Framingham risk score (FRS) (Model A) compared to the FRS plus estimated cardiorespiratory fitness (eCRF) (Model B).
Ijerph 19 00510 g003
Table 1. Characteristics of study participants.
Table 1. Characteristics of study participants.
Total
(n = 38,350)
Men
(n = 16,505)
Women
(n = 21,845)
p-Value
Age (years)52.5 ± 13.752.7 ± 13.652.4 ± 13.80.027
BMI (kg/m2)23.9 ± 3.324.2 ± 3.123.7 ± 3.4<0.001
Income (10,000/won)329.2 ± 271.2338.2 ± 270.4322.3 ± 271.7<0.001
Educational background, n (%) <0.001
Elementary or less10,460 (27.3)3155 (19.1)7305 (33.4)
Middle4670 (12.2)2132 (12.9)2538 (11.6)
High12,237 (31.9)5407 (32.8)6830 (31.3)
College or higher10,983 (28.6)5811 (35.2)5172 (23.7)
Marital status, n (%) <0.001
Married 31,121 (81.1)14,405 (87.3)16,716 (76.5)
Widowed/divorced5383 (14.0)971 (5.9)4412 (20.2)
Unmarried1846 (4.9)1129 (6.8)717 (3.3)
Residence area, n (%) <0.001
Urban29,550 (77.1)12,573 (76.2)16,977 (77.7)
Rural8800 (22.9)3932 (23.8)4868 (22.3)
CVD risk factors
HDL-C (mg/dL)49.1 ± 11.746.1 ± 10.951.4 ± 11.8<0.001
LDL-C (mg/dL)114.1 ± 33.6110.4 ± 3.9117.0 ± 32.3<0.001
TC (mg/dL)190.9 ± 35.9189.0 ± 35.4192.4 ± 36.3<0.001
SBP (mmHg)119.8 ± 17.3122.2 ± 16.0118.1 ± 18.1<0.001
SBP treat, n (%)8209 (21.4)3486 (21.1)4723 (21.6)0.238
Smoking, n (%)15,004 (39.1)13,251 (80.3)1753 (8.0)<0.001
Diabetes, n (%)2885 (7.5)1530 (9.3)1352 (6.2)<0.001
Resting heart rate (beats/min)69.3 ± 9.768.5 ± 10.169.9 ± 9.3<0.001
Physically inactive, n (%)25,146 (65.6)9741 (59.0)15,405 (70.5)<0.001
Follow-up time (years)7.3 ± 2.47.3 ± 2.47.3 ± 2.40.241
Death, n 1474875599<0.001
Person/years (PY)280,315120,368159,946<0.001
Death rate per 1000 PY5.37.33.8<0.001
BMI: body mass index, HDL-C: high density lipoprotein cholesterol, LDL-C: low density lipoprotein cholesterol, TC: total cholesterol, SBP: systolic blood pressure.
Table 2. Demographics of participants stratified by estimated cardiorespiratory fitness (eCRF).
Table 2. Demographics of participants stratified by estimated cardiorespiratory fitness (eCRF).
eCRFp-Value
Fit
(n = 29,339)
Unfit
n = 9011)
eCRF (METs)10.1 ± 2.16.4 ± 1.7<0.001
Women, n (%)16,692 (56.9)5153 (57.2)0.624
Age (years)47.7 ± 11.468.1 ± 7.6<0.001
BMI (kg/m2)23.6 ± 3.125.0 ± 3.4<0.001
Income (10,000/won)367.6 ± 271.2203.9 ± 230.4<0.001
Education, n (%) <0.001
Elementary or less4861 (16.6)5599 (62.1)
Middle3394 (11.6)1276 (14.2)
High10,771 (36.7)1466 (16.3)
College or higher10,313 (35.1)670 (7.4)
Marital status, n (%) <0.001
Married 24,841 (84.7)6280 (69.7)
Widowed/divorced2718 (9.3)2665 (29.6)
Unmarried1780 (6.0)66 (0.7)
Residence area, n (%) <0.001
Urban23,331 (79.5)6219 (69.0)
Rural6008 (20.5)2792 (31.0)
CVD risk factors
HDL-C (mg/dL)49.9 ± 11.846.6 ± 11.1<0.001
LDL-C (mg/dL)113.8 ± 33.0115.3 ± 35.6<0.001
TC (mg/dL)190.6 ± 35.2192.0 ± 38.40.001
SBP (mmHg)117.1 ± 16.4128.9 ± 17.2<0.001
SBP treat, n (%)3940 (13.4)4269 (47.4)<0.001
Smoking, n (%)11,477 (39.1)3527 (39.1)0.970
Diabetes, n (%)1611 (5.5)1274 (14.1)<0.001
10-year FRS (%)8.3 ± 8.719.4 ± 9.4<0.001
eCRF: estimated cardiorespiratory fitness, MET: metabolic equivalent task, BMI: body mass index, CVD: cardiovascular disease, FRS: Framingham risk score, HDL-C: high density lipoprotein cholesterol, LDL-C: low density lipoprotein cholesterol, TC: total cholesterol, SBP: systolic blood pressure.
Table 3. All-cause and CVD mortality risks stratified by 10-year Framingham risk score (FRS) and estimated cardiorespiratory fitness (eCRF).
Table 3. All-cause and CVD mortality risks stratified by 10-year Framingham risk score (FRS) and estimated cardiorespiratory fitness (eCRF).
NNumber of DeathsDeath Rate aHR (95% CI) for All-Cause Mortality bHR (95% CI) for All-Cause Mortality cHR (95% CI) for CVD Mortality bHR (95% CI) for CVD Mortality c
eCRF
Fit29,3395622.61111
Unfit901191215.11.35 (1.20–1.52)1.31 (1.16–1.47)1.62 (1.24–2.12)1.57 (1.20–2.05)
p-value <0.001<0.001<0.0010.001
10-year FRS
Low22,6532301.41111
Moderate74433446.41.14
(0.95–1.36)
1.08
(0.91–1.30)
1.31
(0.87–1.96)
1.25
(0.83–1.87)
High825490015.51.31
(1.07–1.60)
1.26
(1.03–1.54)
2.16
(1.41–3.30)
2.07
(1.36–3.16)
p-value for trend0.0250.046<0.001<0.001
a Deaths per 1000 person/years of follow-up. b Adjusted for age and sex. c Adjusted for household income, marital status, and residence area. HR: hazard ratio; CI: confidence interval. Low: <10% of 10-year FRS; moderate: 10–20% of 10-year FRS; high: ≥20% of 10-year FRS.
Table 4. The combined associations of estimated cardiorespiratory fitness (eCRF) and 10-year Framingham risk score (FRS) with all-cause and cardiovascular disease (CVD) mortality.
Table 4. The combined associations of estimated cardiorespiratory fitness (eCRF) and 10-year Framingham risk score (FRS) with all-cause and cardiovascular disease (CVD) mortality.
HR (95% CI) for All-Cause MortalityHR (95% CI) for CVD Mortality
TotalNumber of DeathsDeath Rate aModel 1Model 2Model 1Model 2
Fit
   Low FRS20,6351691.11 (ref)1 (ref)1 (ref)1 (ref)
   Moderate FRS45861263.60.89 (0.67–1.14)0.84 (0.66–1.07)1.05 (0.58–1.90)0.98 (0.54–1.77)
   High FRS35562679.41.03 (0.80–1.31)0.98 (0.77–1.26)1.62 (0.92–2.86)1.53 (0.87–2.69)
Unfit
   Low FRS1788614.90.84 (0.61–1.15)0.79 (0.58–1.08)1.07 (0.54–2.09)0.98 (0.50–1.93)
   Moderate FRS251321811.91.32 (1.04–1.67)1.21 (0.95–1.54)1.70 (0.98–2.93)1.55 (0.90–2.67)
   High FRS379863321.41.44 (1.13–1.84)1.34 (1.05–1.71)2.73 (1.58–4.72)2.50 (1.45–4.31)
P for interaction <0.001<0.001<0.001<0.001
a Deaths per 1000 person/years of follow-up. Model 1 adjusted for age and sex. Model 2 adjusted for household income, marital status, and residence area. HR: hazard ratio; CI: confidence interval. Low: <10% of 10-year FRS; moderate: 10~20% of 10-year FRS; high: >20% of 10-year FRS.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lee, I.; Kim, J.; Kang, H. Adding Estimated Cardiorespiratory Fitness to the Framingham Risk Score and Mortality Risk in a Korean Population-Based Cohort Study. Int. J. Environ. Res. Public Health 2022, 19, 510. https://doi.org/10.3390/ijerph19010510

AMA Style

Lee I, Kim J, Kang H. Adding Estimated Cardiorespiratory Fitness to the Framingham Risk Score and Mortality Risk in a Korean Population-Based Cohort Study. International Journal of Environmental Research and Public Health. 2022; 19(1):510. https://doi.org/10.3390/ijerph19010510

Chicago/Turabian Style

Lee, Inhwan, Jeonghyeon Kim, and Hyunsik Kang. 2022. "Adding Estimated Cardiorespiratory Fitness to the Framingham Risk Score and Mortality Risk in a Korean Population-Based Cohort Study" International Journal of Environmental Research and Public Health 19, no. 1: 510. https://doi.org/10.3390/ijerph19010510

APA Style

Lee, I., Kim, J., & Kang, H. (2022). Adding Estimated Cardiorespiratory Fitness to the Framingham Risk Score and Mortality Risk in a Korean Population-Based Cohort Study. International Journal of Environmental Research and Public Health, 19(1), 510. https://doi.org/10.3390/ijerph19010510

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop