Next Article in Journal
Literature Cases Summarized Based on Their Polysomnographic Findings in Rett Syndrome
Previous Article in Journal
Sport Psychological Skill Factors and Scale Development for Taekwondo Athletes
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Effects of Resistance Training on C-Reactive Protein and Inflammatory Cytokines in Elderly Adults: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

Department of Nursing, College of Health Science, Kangwon National University, Samcheok 25949, Korea
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(6), 3434; https://doi.org/10.3390/ijerph19063434
Submission received: 26 January 2022 / Revised: 8 March 2022 / Accepted: 9 March 2022 / Published: 14 March 2022
(This article belongs to the Section Exercise and Health)

Abstract

:
Chronic low-grade inflammation that accompanies aging is associated with adverse health outcomes and may exacerbate the severity of infectious disease such as COVID-19. Resistance training (RT) has the potential to improve chronic low-grade inflammation, but the evidence remains inconclusive. This study evaluated the effects of RT on chronic low-grade inflammation in elderly adults. MEDLINE, EMBASE, Cochrane Library, CINAHL, RISS, NDSL, and KoreaMed were searched. We included studies that assessed the effect of RT on C-reactive protein (CRP), interleukin (IL)-6, IL-10, and tumor necrosis factor (TNF)-α in those aged ≥60 years. The effect size was estimated using fixed or random-effects models. Subgroup analysis was performed regarding age, health status, training method, number of exercises, intensity, weekly frequency, and duration. In the 18 randomized controlled trials (539 patients) included, RT was effective in alleviating CRP (effect size = −0.72, 95% confidence interval = −1.06 to −0.38, p < 0.001), IL-10 (−3.34, −6.16 to −0.53, p = 0.02), and TNF-α (−0.56, −1.08 to −0.03, p = 0.04) in elderly adults and tended to reduce IL-6 (−0.59, −1.18 to 0.00, p = 0.05). Subgroup analyses showed CRP reduction regardless of age, training method, number of exercises, intensity, weekly frequency, and duration. RT can be used to ameliorate chronic low-grade inflammation in elderly adults.

1. Introduction

The global coronavirus disease 2019 (COVID-19) pandemic caused by severe acute respiratory syndrome virus (SARS-CoV-2) is still ongoing. As of 24 November 2021, the cumulative number of confirmed cases worldwide was 257,046,038, and the death toll was 5,154,865 [1]. In particular, people over 60 years of age are considered a major high-risk group for COVID-19. Data from 16 countries, reporting a relatively high number of COVID-19 cases, showed that the death rate from COVID-19 among individuals aged ≥65 years was 62.1 times higher than among those aged <54 years [2]. In South Korea, approximately 21% of COVID-19 cases involve individuals aged >60 years, accounting for 92% of deaths [1].
A normal inflammatory response is characterized by a transient increase in inflammatory activity that develops when an infection is present and resolves once the infection has subsided [3]. However, certain social, psychological, environmental, and biological factors have been associated with inhibition of acute inflammation resolution and development of a low-grade, non-infectious chronic inflammation state [4]. Both short- and long-term changes in the inflammatory response can affect immunity and cause major changes in tissues and organs, as well as the normal cellular metabolism; this, in turn, may increase the risk of various communicable and noncommunicable diseases [5,6]. Chronic low-grade inflammation, also known as inflammaging, is associated with aging and is characterized by increases in serum inflammatory mediators such as C-reactive protein (CRP), interleukin (IL)-6, and tumor necrosis factor (TNF)-α [7].
Although the causes of chronic low-grade inflammation are not fully understood, results from previous studies have suggested that a combination of endogenous and non-endogenous social, environmental, and lifestyle risk factors may be involved [8]. Among endogenous causes are DNA damage, dysfunctional telomeres, and oxidative stress [9]. Non-endogenous contributors include chronic infection, lifestyle-induced obesity, unbalanced diet, and environmental toxicants [10,11,12,13]. Inflammation is linked to a decreased ability to mount effective immune responses in the elderly [14,15]. A decline in immune function not only increases the morbidity and mortality due to COVID-19, it also causes several degenerative diseases. Cancer, cardiovascular diseases, Alzheimer’s disease, Parkinson’s disease, osteoporosis, muscle loss, and depression are all related to inflammation [16,17,18].
Exercise and weight control have been suggested as methods for mitigating the negative effects of chronic inflammation. Obese people have continuously high levels of inflammation because fat cells secrete numerous types of inflammatory substances called adipokines, such as leptin, adiponectin, and resistin [19]. Proper exercise has a dual effect in reducing chronic inflammation via weight reduction and lowering adipokines in cells [20]. Although aerobic exercise is generally recommended, it is difficult for the elderly to perform weight-bearing exercises such as walking due to various problems including degenerative arthritis, cardiovascular disease, and diabetes [21]. In addition, the elderly have reduced muscle strength and range of motion, and the stimulation response period and nerve conduction speed are slow; thus, injuries and safety accidents easily occur during exercise [22]. Resistance training (RT), a strength training exercise involving progressive overload in which the muscles exert force against an external load, could be a safe and effective method of improving chronic low-grade inflammation in older individuals [23,24].
A previous study [25] reported that RT was associated with anti-inflammatory effects by decreasing serum levels of IL-6 and CRP, in addition to inducing changes in TNF-α gene expression in elderly women. However, another study [26] suggested that RT was not related to TNF-α, IL-6, IL-10, and CRP improvement. As a result, an integrated and clear conclusion on the effects of RT in the elderly is currently unavailable. The purpose of this review was to critically examine the effects of RT on chronic low-grade inflammation in elderly adults through a systematic review and meta-analysis of randomized controlled trials (RCTs). These findings will enable practitioners in prevention and treatment to present fundamental evidence for developing interventions to manage chronic low-grade inflammation.

2. Materials and Methods

2.1. Eligibility Criteria

The systematic review was conducted in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. Data selection was performed according to PICOTS-SD (Participants, Intervention, Comparisons, Outcomes, Timing of Outcome Measurement, Settings, Study Design), which is a guideline for conducting systematic reviews and meta-analyses. The participants (P) were elderly persons aged ≥60 years. The intervention (I) was RT. The comparison (C) was no intervention or other comparative intervention. The outcomes (O) were CRP, IL-6, IL-10, or TNF-α levels. The timing of outcome measurement (T) was post-intervention. The setting (S) was hospital, nursing home, or community, and the study design (D) was RCT. The exclusion criteria for the studies in this review were participant age <60 years; absence of RT or combined intervention (RT + other intervention); use of outcome variables other than CRP, IL-6, IL-10, or TNF-α, non-RCT study design; and unavailable mean and standard deviation values.

2.2. Literature Search

The searched databases included MEDLINE, EMBASE, Cochrane Library, CINAHL, RISS, NDSL, and KoreaMed. The search formula was combined by merging terms representing the elderly, RT, inflammatory biomarkers, and RCT. The search terms were as follows: (elder * OR old * OR aged OR aging OR senior) AND (resistance training OR strength training OR weight-lifting strengthening program OR weight-bearing strengthening program OR elastic OR machines OR equipment OR fitness) AND (biomarker * OR inflamm * OR marker * OR cytokine * OR C-reactive protein OR interleukin OR tumor necrosis factor) AND (randomized controlled trial OR controlled clinical trial OR randomized OR randomly). The search was conducted from the year of the first RT article to November 2021 and was limited to the English and Korean languages.

2.3. Data Extraction

First, the authors deleted duplicate articles and screened the titles and abstracts of the selected articles to eliminate irrelevant studies. Thereafter, the full texts of the remaining references were evaluated, and ineligible studies were excluded based on the criteria, and the reasons for exclusion were recorded. Two authors performed this process independently, and any disagreements were discussed until a compromise was achieved. The data of the finally selected studies were collected using the coding table. The coding table consisted of author, publication year, country of study, study subject, intervention method, control group, statistical value, methodological quality, etc.

2.4. Data Analysis

The effect size and homogeneity of RT was analyzed using RevMan 5.3 (The Cochrane Collaboration, Oxford, UK). An effect size of 0.20–0.50 indicated a ‘small’ effect, 0.50–0.80 indicated a ‘medium’ effect, and ≥0.8 indicated a ‘large’ effect [27]. The statistical significance of the effect size was determined using the overall effect test and 95% confidence interval (CI) with the significance level set at 0.05 (p < 0.05). Heterogeneity was explored using I2 statistics, with values of 25%, 50%, and 75% considered low, moderate, and high heterogeneity, respectively [28]. When calculating the pooled effect size, the effect size was estimated using the fixed-effects model if the studies were homogeneous and with the random-effects model if the studies were heterogeneous. Publication bias was inspected with funnel plots. Subgroup analysis was performed by participant age, health status, training type, number of exercises, intensity, weekly frequency, and duration to identify potential moderators. According to the American College of Sports Medicine, moderate RT loading is defined as 60% of 1 repetition maximum (RM) [29]. Thus, intensity was classified as vigorous if the maximum intensity was >60 of I RM, and moderate if it was ≤60 of I RM.

2.5. Methodological Quality

Methodological quality was assessed using the Cochrane assessment of risk of bias (ROB). The tool is structured to evaluate the risk of bias in six domains as follows: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective outcome reporting. The risk of bias was determined as ‘low’, ‘high’, and ‘unclear’ for each item. The authors independently evaluated the items, and inconsistent items were reviewed and reevaluated.

3. Results

3.1. Literature Search

A total of 1044 articles were identified through the database search. After removing duplicate studies, the titles and abstracts of 546 studies suggested that 74 studies were eligible for inclusion. Thereafter, full text reviews identified 18 studies that met the inclusion criteria (Figure 1) [26,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46].

3.2. Study Characteristics

The characteristics of the included studies are summarized in Table 1. The total number of participants was 539, with 268 in the intervention groups and 271 in the control groups. Sample sizes ranged from 14 to 48, with a median size of 30. The participants’ mean age ranged from 62.0 to 82.7 years, with an average of 70.2 years (76.7% women). Ten of the 18 interventions included healthy older individuals and 8 included individuals with specific diseases such as metabolic syndrome, type 2 diabetes, and cognitive impairment. Intervention durations ranged from 6 to 32 weeks, with a mean of 12 weeks. Of the 18 RT programs, 9 involved elastic band exercise, and the RT session frequency ranged from two to four sessions per week. The number of different muscle group exercises ranged from 6 to 14, and RT session duration ranged from 20 to 70 min (mode 60 min). Exercise intensity was moderate in 12 studies.

3.3. RT Effects

The results of the pooled meta-analysis for all outcomes are shown in Figure 2, and the results of the subgroup analysis are displayed in Table 2. Sixteen studies performed RT for CRP improvement, with a medium effect size of −0.72 (95% CI = −1.06 to −0.38, p < 0.001) and moderate heterogeneity (I2 = 69%). In the subgroup analysis, RCTs that included older individuals (>70 years), healthier individuals, machines or equipment, lower number of exercises (≤8), vigorous intensity, lower weekly frequency (≤2 times/week), and shorter duration (≤8 weeks) reported larger effect sizes for CRP. There was no publication bias on the Funnel plot.
Eleven studies examined IL-6, and a non-significant effect size was noted (−0.59, 95% CI = −1.18 to 0.00, p = 0.05, I2 = 84%) but tended towards reduction. In the subgroup analysis, IL-6 was significantly decreased in the studies that included healthier individuals, machines or equipment, lower number of exercises (≤8), and shorter duration (≤8 weeks). Publication bias was noted for IL-6 studies.
Three studies examined IL-10, with a large effect size of −3.34 (95% CI = −6.16 to −0.53, p = 0.002) and low heterogeneity (I2 = 45%). Unfortunately, subgroup analysis was not possible for IL-10 due to its heterogeneity between studies.
The effect on TNF-α was assessed in 11 studies, with a significant effect size of −0.56 (95% CI = −1.08 to −0.03, p < 0.001, I2 = 81%). RCTs with specific health conditions, machines or equipment, higher weekly frequency (>2 times/week), and longer duration (>8 weeks) showed significant decreases in the subgroup analysis. There was some publication bias.

3.4. Methodological Quality

In sequence generation, 11 studies were rated as unclear because randomization procedures were not sufficiently described. Four of the studies were classified as having low bias in relation to the allocation concealment domain. Only two studies had a low risk of bias regarding the blinding of participants and personnel. In the blinding of outcome assessment, all the studies were evaluated as having a low risk of bias because the RT results were calculated via blood analysis performed at an external clinical pathology center. Thirteen studies were determined as having low bias related to incomplete outcome domains (Figure 3).

4. Discussion

As the prolonged COVID-19 pandemic has had a significant negative impact on the health of the elderly, public health interventions are needed to mitigate the situation. Chronic low-grade inflammation is associated with several adverse health outcomes and may exacerbate the severity and mortality of COVID-19. Therefore, this study was conducted to objectively synthesize studies evaluating the effects of RT on CRP and inflammatory cytokines in the elderly.
A meta-analysis of 18 RCTs demonstrated that RT had a medium effect size for reducing CRP, IL-10, and TNF-α in the elderly, and tended to reduce IL-6. Similarly, Sardel i et al. [47] showed that RT had a medium effect size in lowering CRP and tended to lower IL-6, but did not induce changes in TNF-α based on a meta-analysis of 13 RCTs in individuals > 50 years of age. Furthermore, Rose et al. [48] conducted a meta-analysis of 59 studies involving people of various ages and found that while RT had a small effect size for lowering CRP, it had no effect on IL-6 or TNF. Combining the above studies, CRP is reduced by RT regardless of age in the elderly, but IL-6, IL-10, and TNF-α may be affected by moderators such as participant age, participant condition, and intervention type.
CRP is a major marker of systemic inflammation. When inflammation or tissue damage occurs in the body, IL-6 stimulates hepatocytes to promote CRP production, raising the CRP concentration in the blood [49]. In addition to being a simple inflammatory marker, CRP is also associated with a decrease in muscle strength that causes deterioration in physical function among the elderly. According to a study by Schaap et al. [50], an increase in CRP and IL-6 levels by two to three times was associated with a decrease in grip strength by more than 40% in a sample of elderly patients. Another study showed a similar association between CRP and thigh muscle strength, with higher levels being associated with decreased thigh muscle strength [51]. In a rat study by Goodman et al., the authors found that administration of IL-6 and TNF-α was associated with muscle tissue destruction [52]. In a subgroup analysis of the current study, CRP levels showed statistically significant results, regardless of the training method, in those aged ≤70 and >70 years. Similarly, a previous meta-analysis investigating the effect of exercise on CRP levels found that exercising reduced CRP levels regardless of age or sex [53]. These results show that elderly individuals with a relatively high risk of injury from exercise can manage chronic inflammation using available RT. In addition, the effect size of CRP differed depending on exercise intensity, with the vigorous group having a larger effect size than the moderate group. Rose et al., who investigated the chronic inflammatory response according to the intensity of aerobic and resistance exercise, reported that higher-intensity training may be more efficacious than lower-intensity training for middle-aged adults, which supports the results of our study [54].
The pooled effect size for IL-6 was not significant for RT, but in the subgroup analysis, the effect size was significantly decreased in the studies that included healthier individuals, machines or equipment; lower number of exercises (≤8); and shorter duration (≤8 weeks). The pooled effect size for TNF-α was significantly decreased by RT. Particularly, it was significantly decreased in studies that consisted of healthier individuals, machines or equipment,; higher weekly frequency (>2 times/week); and longer duration (>8 weeks) in the subgroup analysis. Pérez Chaparro et al. [55] found that both RT alone and RT combined with aerobic exercise decreased IL-6 levels in HIV patients. Similarly, Hayashino et al. [56] discovered that aerobic and resistance exercises were more helpful for IL-6 in patients with type 2 diabetes when performed over a longer duration and in a larger number of sessions. Furthermore, Khalafi et al. [57] reported that exercise training significantly reduced IL-6 and TNF-α in postmenopausal women in a meta-analysis of 22 studies. Generally, TNF-α is produced in adipose tissues. It has been shown to aggravate inflammation and promote IL-6 secretion, which in turn, leads to metabolic disease [58,59]. Considering that TNF-α and IL-6 are closely related to an increase in adipose tissue [60] and a loss in muscle strength [61], the reduction of TNF-α and IL-6 may be because of a decrease in fat and an increase in muscle mass due to RT. In particular, IL-6 is secreted from both adipocytes and muscle cells. IL-6 secreted from adipocytes causes inflammatory action, whereas IL-6 secreted from muscle has anti-inflammatory action [62]. It is thought that the immune function of the elderly improves by exercising the muscles. Previous studies also emphasized the importance of increasing muscle mass in the anti-inflammatory effect of RT [26].
IL-10, while a pleiotropic cytokine, is an immune cytokine with anti-inflammatory potential. Petersen and Pedersen [63] reported that the process of muscle contraction during exercise may increase the transcription of IL-6 mRNA, resulting in activation of anti-inflammatory cytokines such as IL-10. Surprisingly, RT was found to lower IL-10 levels in this study, both in the elderly group (>70 years) and in the group with shorter duration (≤8 weeks), while previous studies showed no change or an increase in IL-10 after exercise. Gonzalo-Encabo et al. [64] showed that IL-10 increased after RT in adults with overweight and obesity, based on a systematic review of 27 studies. Yousefabadi et al. [65] in a meta-analysis of 20 RCTs also demonstrated that combined aerobic and resistance exercise increased IL-10 levels in patients with metabolic syndrome.
Patients with pathological conditions, such as obesity, diabetes, and coronary artery disease often have high basal levels of pro-inflammatory cytokines [66]. As results, a reduction in IL-10 could be linked to a lower release. In addition, Cerqueira, et al. [67] reported that an elevation in pro-inflammatory cytokines was more evident after a prolonged bout of intense exercise. However, the intensity of RT was lower than that of the general population in this study in order to prevent inadvertent damage in elderly individuals. This could have resulted in a decrease in IL-10 levels. More research is needed to better understand the impact of RT on IL-10 in elderly adults without clinical comorbidities.
A strength of this study is that it is the first study to reveal the positive effects of RT on CRP and inflammatory cytokines in elderly adults aged ≥60 years through a systematic review and meta-analysis of RCTs. The limitations of this study in deriving such results are as follows. First, blinding of participants and personnel is an important aspect in evaluating the methodological quality of selected studies, but this was insufficiently performed in most studies. Second, a high level of heterogeneity was observed in the integrated meta-analysis, and a subgroup analysis was performed for further evaluation, but some subgroups still showed high heterogeneity. Therefore, the results of the meta-analysis should be interpreted with caution. Third, although a comprehensive method was applied for the literature search, publication bias was found. It is necessary to estimate the change in the pooled effect size after adjusting for publication bias.

5. Conclusions

The pooled results of the present study indicated that RT had significant effects on reducing CRP, IL-10, and TNF-α in the elderly, and tended to reduce IL-6. Particularly, RT reduced CRP regardless of the age of the elderly, training method, number of exercises, intensity, weekly frequency, and duration. However, IL-6 and TNF-α were affected by various moderators. Based on the results of this study, it is recommended that optimal RT for the elderly aged ≥60 years consists of vigorous intensity, lower number of exercises (≤8), lower weekly frequency (≤2 times/week), and shorter duration (≤8) using machines or equipment. RT may be safer to administer and could be an effective option to improve chronic low-grade inflammation in elderly adults.

Author Contributions

Y.-R.Y. contributed to the conception and design of the study; Y.-R.Y. and S.-D.K. searched the literatures and organized the database; Y.-R.Y. and S.-D.K. evaluated the quality appraisal of trials included in this review; Y.-R.Y. analyzed the effect size and homogeneity; Y.-R.Y. wrote the first draft of the manuscript; Y.-R.Y. and S.-D.K. reviewed and revised the drafts of manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Korea Disease Control and Prevention Agency. Current Status of Overseas Outbreaks of Coronavirus Infectious Diseases. Available online: http://ncov.mohw.go.kr/bdBoardList_Real.do?brdId=1&brdGubun=14&ncvContSeq=&contSeq=&board_id=&gubun= (accessed on 24 November 2021).
  2. Levin, A.T.; Hanage, W.P.; Owusu-Boaitey, N.; Cochran, K.B.; Walsh, S.P.; Meyerowitz-Katz, G. Assessing the age specificity of infection fatality rates for COVID-19: Systematic review, meta-analysis, and public policy implications. Eur. J. Epidemiol. 2020, 35, 1123–1138. [Google Scholar] [CrossRef] [PubMed]
  3. Fullerton, J.N.; Gilroy, D.W. Resolution of inflammation: A new therapeutic frontier. Nat. Rev. Drug Discov. 2016, 15, 551–567. [Google Scholar] [CrossRef] [PubMed]
  4. Straub, R.H. The brain and immune system prompt energy shortage in chronic inflammation and ageing. Nat. Rev. Rheumatol. 2017, 13, 743–751. [Google Scholar] [CrossRef] [PubMed]
  5. Furman, D.; Chang, J.; Lartigue, L.; Bolen, C.R.; Haddad, F.; Gaudilliere, B.; Ganio, E.A.; Fragiadakis, G.K.; Spitzer, M.H.; Douchet, I.; et al. Expression of specific inflammasome gene modules stratifies older individuals into two extreme clinical and immunological states. Nat. Med. 2017, 23, 174–184. [Google Scholar] [CrossRef]
  6. Kotas, M.E.; Medzhitov, R. Homeostasis, inflammation, and disease susceptibility. Cell 2015, 160, 816–827. [Google Scholar] [CrossRef] [Green Version]
  7. Pietrobon, A.J.; Teixeira, F.M.E.; Sato, M.N. Immunosenescence and Inflammaging: Risk factors of severe COVID-19 in older people. Front. Immunol. 2020, 11, 579220. [Google Scholar] [CrossRef]
  8. Furman, D.; Campisi, J.; Verdin, E.; Carrera-Bastos, P.; Targ, S.; Franceschi, C.; Ferrucci, L.; Gilroy, D.W.; Fasano, A.; Miller, G.W.; et al. Chronic inflammation in the etiology of disease across the life span. Nat. Med. 2019, 25, 1822–1832. [Google Scholar] [CrossRef]
  9. Calcinotto, A.; Kohli, J.; Zagato, E.; Pellegrini, L.; Demaria, M.; Alimonti, A. Cellular senescence: Aging, cancer, and injury. Physiol. Rev. 2019, 99, 1047–1078. [Google Scholar] [CrossRef]
  10. Byrd, D.A.; Judd, S.E.; Flanders, W.D.; Hartman, T.J.; Fedirko, V.; Bostick, R.M. Development and validation of novel dietary and lifestyle inflammation scores. J. Nutr. 2019, 149, 2206–2218. [Google Scholar] [CrossRef]
  11. De Rosa, V.; La Cava, A.; Matarese, G. Metabolic pressure and the breach of immunological self-tolerance. Nat. Immunol. 2017, 18, 1190–1196. [Google Scholar] [CrossRef]
  12. Franceschi, C.; Garagnani, P.; Parini, P.; Giuliani, C.; Santoro, A. Inflammaging: A new immune-metabolic viewpoint for age-related diseases. Nat. Rev. Endocrinol. 2018, 14, 576–590. [Google Scholar] [CrossRef]
  13. Margină, D.; Ungurianu, A.; Purdel, C.; Tsoukalas, D.; Sarandi, E.; Thanasoula, M.; Tekos, F.; Mesnage, R.; Kouretas, D.; Tsatsakis, A. chronic inflammation in the context of everyday life: Dietary changes as mitigating factors. Int. J. Environ. Res. Public Health 2020, 17, 4135. [Google Scholar] [CrossRef]
  14. Franceschi, C.; Salvioli, S.; Garagnani, P.; de Eguileor, M.; Monti, D.; Capri, M. Immunobiography and the heterogeneity of immune responses in the elderly: A focus on inflammaging and trained immunity. Front. Immunol. 2017, 8, 982. [Google Scholar] [CrossRef]
  15. Fuentes, E.; Fuentes, M.; Alarcón, M.; Palomo, I. Immune system dysfunction in the elderly. An. Acad. Bras. Cienc. 2017, 89, 285–299. [Google Scholar] [CrossRef] [Green Version]
  16. Fulop, T.; Witkowski, J.M.; Olivieri, F.; Larbi, A. The integration of inflammaging in age-related diseases. Semin Immunol. 2018, 40, 17–35. [Google Scholar] [CrossRef]
  17. Li, T.; Huang, Y.; Cai, W.; Chen, X.; Men, X.; Lu, T.; Wu, A.; Lu, Z. Age-related cerebral small vessel disease and inflammaging. Cell Death Dis. 2020, 11, 932. [Google Scholar] [CrossRef]
  18. Mészáros, Á.; Molnár, K.; Nógrádi, B.; Hernádi, Z.; Nyúl-Tóth, Á.; Wilhelm, I.; Krizbai, I.A. Neurovascular Inflammaging in Health and Disease. Cells 2020, 9, E1614. [Google Scholar] [CrossRef]
  19. Peluso, I.; Palmery, M. The relationship between body weight and inflammation: Lesson from anti-TNF-α antibody therapy. Hum. Immunol. 2016, 77, 47–53. [Google Scholar] [CrossRef]
  20. Krüger, K.; Mooren, F.C.; Pilat, C. The immunomodulatory effects of physical activity. Curr. Pharm. Des. 2016, 22, 3730–3748. [Google Scholar] [CrossRef]
  21. Liberman, K.; Forti, L.N.; Beyer, I.; Bautmans, I. The effects of exercise on muscle strength, body composition, physical functioning and the inflammatory profile of older adults: A systematic review. Curr. Opin. Clin. Nutr. Metab. Care 2017, 20, 30–53. [Google Scholar] [CrossRef]
  22. de Labra, C.; Guimaraes-Pinheiro, C.; Maseda, A.; Lorenzo, T.; Millán-Calenti, J.C. Effects of physical exercise interventions in frail older adults: A systematic review of randomized controlled trials. BMC Geriatr. 2015, 15, 154. [Google Scholar] [CrossRef] [Green Version]
  23. Lopez, P.; Pinto, R.S.; Radaelli, R.; Rech, A.; Grazioli, R.; Izquierdo, M.; Cadore, E.L. Benefits of resistance training in physically frail elderly: A systematic review. Aging Clin. Exp. Res. 2018, 30, 889–899. [Google Scholar] [CrossRef]
  24. Kraemer, W.J.; Ratamess, N.A. Fundamentals of resistance training: Progression and exercise prescription. Med. Sci. Sports Exerc. 2004, 36, 674–688. [Google Scholar] [CrossRef]
  25. Macêdo Santiago, L.Â.; Neto, L.G.L.; Borges Pereira, G.; Leite, R.D.; Mostarda, C.T.; de Oliveira Brito Monzani, J.; Sousa, W.R.; Rodrigues Pinheiro, A.J.M.; Navarro, F. Effects of resistance training on immunoinflammatory response, TNF-alpha gene expression, and body composition in elderly women. J. Aging Res. 2018, 2018, 1467025. [Google Scholar] [CrossRef] [Green Version]
  26. Furtado, G.E.; Chupel, M.U.; Minuzzi, L.G.; Rama, L.; Colado, J.C.; Hogervorst, E.; Ferreira, J.P.; Teixeira, A.M. The mediating effect of different exercise programs on the immune profile of frail older women with cognitive impairment. Curr. Pharm. Des. 2020, 26, 906–915. [Google Scholar]
  27. Cohen, J. Statistical Power Analysis for the Behavioral Science, 2nd ed.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1998. [Google Scholar]
  28. Higgins, J.P.; Thompson, S.G.; Deeks, J.J.; Altman, D.G. Measuring inconsistency in meta-analyses. BMJ 2003, 327, 557–560. [Google Scholar] [CrossRef] [Green Version]
  29. American College of Sports Medicine. American College of Sports Medicine position stand. Progression models in resistance training for healthy adults. Med. Sci. Sports Exerc. 2009, 41, 687–708. [Google Scholar] [CrossRef]
  30. Chupel, M.U.; Direito, F.; Furtado, G.E.; Minuzzi, L.G.; Pedrosa, F.M.; Colado, J.C.; Ferreira, J.P.; Filaire, E.; Teixeira, A.M. Strength training decreases inflammation and increases cognition and physical fitness in older women with cognitive impairment. Front. Physiol. 2017, 8, 377. [Google Scholar] [CrossRef] [Green Version]
  31. Cunha, P.M.; Ribeiro, A.S.; Nunes, J.P.; Tomeleri, C.M.; Nascimento, M.A.; Moraes, G.K.; Sugihara, P., Jr.; Barbosa, D.S.; Venturini, D.; Cyrino, E.S. Resistance training performed with single-set is sufficient to reduce cardiovascular risk factors in untrained older women: The randomized clinical trial. Active aging longitudinal study. Arch. Gerontol. Geriatr. 2019, 81, 171–175. [Google Scholar] [CrossRef]
  32. Hsieh, P.L.; Tseng, C.H.; Tseng, Y.J.; Yang, W.S. Resistance training improves muscle function and cardiometabolic risks but not quality of life in older people with type 2 diabetes mellitus: A randomized controlled trial. J. Geriatr. Phys. Ther. 2018, 41, 65–76. [Google Scholar] [CrossRef]
  33. Kim, C.H.; Lee, J.W.; Park, H.Y. The effect of yoga exercise and band exercise erformance in elderly women on the inflammatory makers and ageing hormone. KSW 2014, 9, 209–220. [Google Scholar]
  34. Kim, K.T.; Cho, J.H. The effect of elastic band and aerobic exercise on fitness, blood lipids, and vascular inflammatory markers in elderly women. A. J. Kinesiol. 2013, 15, 129–138. [Google Scholar]
  35. Kim, N.R.; Park, H.Y. Change of cellular immunity substance by exercise types in elderly women. KJGD 2010, 18, 163–170. [Google Scholar]
  36. Kim, Y.Y.; Lim, S.; Choi, S.M.; Lee, M.G. Effects of 12 weeks of aerobic and resistance training on abdominal fat, physical fitness, adipokines, and inflammatory markers in female elderly patients with type 2 diabetes. KJSS 2012, 23, 489–501. [Google Scholar]
  37. Lee, J.W. The Effect of aerobic exercise and type of band exercise on inflammatory markers of elderly women patients with metabolic syndrome. KJSS 2011, 20, 1041–1053. [Google Scholar]
  38. Martins, R.A.; Neves, A.P.; Coelho-Silva, M.J.; Veríssimo, M.T.; Teixeira, A.M. The effect of aerobic versus strength-based training on high-sensitivity C-reactive protein in older adults. Eur. J. Appl. Physiol. 2010, 110, 161–169. [Google Scholar] [CrossRef]
  39. Rall, L.C.; Roubenoff, R.; Cannon, J.G.; Abad, L.W.; Dinarello, C.A.; Meydani, S.N. Effects of progressive resistance training on immune response in aging and chronic inflammation. Med. Sci. Sports Exerc. 1996, 28, 1356–1365. [Google Scholar] [CrossRef]
  40. Redwine, L.S.; Pung, M.A.; Wilson, K.; Bangen, K.J.; Delano-Wood, L.; Hurwitz, B. An exploratory randomized sub-study of light-to-moderate intensity exercise on cognitive function, depression symptoms and inflammation in older adults with heart failure. J. Psychosom. Res. 2020, 128, 109883. [Google Scholar] [CrossRef]
  41. Rodriguez-Miguelez, P.; Fernandez-Gonzalo, R.; Almar, M.; Mejías, Y.; Rivas, A.; de Paz, J.A.; Cuevas, M.J.; González-Gallego, J. Role of Toll-like receptor 2 and 4 signaling pathways on the inflammatory response to resistance training in elderly subjects. Age 2014, 36, 9734. [Google Scholar] [CrossRef] [Green Version]
  42. So, W.Y.; Song, M.; Park, Y.H.; Cho, B.L.; Lim, J.Y.; Kim, S.H.; Song, W. Body composition, fitness level, anabolic hormones, and inflammatory cytokines in the elderly: A randomized controlled trial. Aging Clin. Exp. Res. 2013, 25, 167–174. [Google Scholar] [CrossRef]
  43. Theodorou, A.A.; Panayiotou, G.; Volaklis, K.A.; Douda, H.T.; Paschalis, V.; Nikolaidis, M.G.; Smilios, I.; Toubekis, A.; Kyprianou, D.; Papadopoulos, I.; et al. Aerobic, resistance and combined training and detraining on body composition, muscle strength, lipid profile and inflammation in coronary artery disease patients. Res. Sports Med. 2016, 24, 171–184. [Google Scholar] [CrossRef]
  44. Tomeleri, C.M.; Ribeiro, A.S.; Souza, M.F.; Schiavoni, D.; Schoenfeld, B.J.; Venturini, D.; Barbosa, D.S.; Landucci, K.; Sardinha, L.B.; Cyrino, E.S. Resistance training improves inflammatory level, lipid and glycemic profiles in obese older women: A randomized controlled trial. Exp. Gerontol. 2016, 84, 80–87. [Google Scholar] [CrossRef]
  45. Tomeleri, C.M.; Souza, M.F.; Burini, R.C.; Cavaglieri, C.R.; Ribeiro, A.S.; Antunes, M.; Nunes, J.P.; Venturini, D.; Barbosa, D.S.; Sardinha, L.B.; et al. Resistance training reduces metabolic syndrome and inflammatory markers in older women: A randomized controlled trial. J. Diabetes 2018, 10, 328–337. [Google Scholar] [CrossRef]
  46. Wanderley, F.A.; Moreira, A.; Sokhatska, O.; Palmares, C.; Moreira, P.; Sandercock, G.; Oliveira, J.; Carvalho, J. Differential responses of adiposity, inflammation and autonomic function to aerobic versus resistance training in older adults. Exp. Gerontol. 2013, 48, 326–333. [Google Scholar] [CrossRef]
  47. Sardeli, A.V.; Tomeleri, C.M.; Cyrino, E.S.; Fernhall, B.; Cavaglieri, C.R.; Chacon-Mikahil, M.P.T. Effect of resistance training on inflammatory markers of older adults: A meta-analysis. Exp. Gerontol. 2018, 111, 188–196. [Google Scholar] [CrossRef]
  48. Rose, G.L.; Mielke, G.I.; Durr, M.; Schaumberg, M.A. Effect of resistance training on chronic inflammation: A systematic review and meta-analysis. Transl. Sports Med. 2021, 4, 900–913. [Google Scholar] [CrossRef]
  49. Wu, Y.; Potempa, L.A.; El Kebir, D.; Filep, J.G. C-reactive protein and inflammation: Conformational changes affect function. Biol. Chem. 2015, 396, 1181–1197. [Google Scholar] [CrossRef]
  50. Schaap, L.A.; Pluijm, S.M.; Deeg, D.J.; Visser, M. Inflammatory markers and loss of muscle mass (sarcopenia) and strength. Am. J. Med. 2006, 119, 526. [Google Scholar] [CrossRef]
  51. Schaap, L.A.; Pluijm, S.M.; Deeg, D.J.; Harris, T.B.; Kritchevsky, S.B.; Newman, A.B.; Colbert, L.H.; Pahor, M.; Rubin, S.M.; Tylavsky, F.A.; et al. Higher inflammatory marker levels in older persons: Associations with 5-year change in muscle mass and muscle strength. J. Gerontol. A Biol. Sci. Med. Sci. 2009, 64, 1183–1189. [Google Scholar] [CrossRef] [Green Version]
  52. Goodman, M.N. Interleukin-6 induces skeletal muscle protein breakdown in rats. Proc. Soc. Exp. Biol. Med. 1994, 205, 182–185. [Google Scholar] [CrossRef]
  53. Fedewa, M.V.; Hathaway, E.D.; Ward-Ritacco, C.L. Effect of exercise training on C reactive protein: A systematic review and meta-analysis of randomised and non-randomised controlled trials. Br. J. Sports Med. 2017, 51, 670–676. [Google Scholar] [CrossRef]
  54. Rose, G.L.; Skinner, T.L.; Mielke, G.I.; Schaumberg, M.A. The effect of exercise intensity on chronic inflammation: A systematic review and meta-analysis. J. Sci. Med. Sport 2021, 24, 345–351. [Google Scholar] [CrossRef]
  55. Pérez Chaparro, C.G.A.; Zech, P.; Schuch, F.; Wolfarth, B.; Rapp, M.; Heiβel, A. Effects of aerobic and resistance exercise alone or combined on strength and hormone outcomes for people living with HIV. A meta-analysis. PLoS ONE 2018, 13, e0203384. [Google Scholar] [CrossRef]
  56. Hayashino, Y.; Jackson, J.L.; Hirata, T.; Fukumori, N.; Nakamura, F.; Fukuhara, S.; Tsujii, S.; Ishii, H. Effects of exercise on C-reactive protein, inflammatory cytokine and adipokine in patients with type 2 diabetes: A meta-analysis of randomized controlled trials. Metabolism 2014, 63, 431–440. [Google Scholar] [CrossRef]
  57. Khalafi, M.; Malandish, A.; Rosenkranz, S.K. The impact of exercise training on inflammatory markers in postmenopausal women: A systemic review and meta-analysis. Exp. Gerontol. 2021, 150, 111398. [Google Scholar] [CrossRef]
  58. Idriss, H.T.; Naismith, J.H. TNF alpha and the TNF receptor superfamily: Structure-function relationship(s). Microsc. Res. Tech. 2000, 50, 184–195. [Google Scholar] [CrossRef]
  59. Tzanavari, T.; Giannogonas, P.; Karalis, K.P. TNF-alpha and obesity. Curr. Dir. Autoimmun. 2010, 11, 145–156. [Google Scholar] [PubMed]
  60. Giannopoulou, I.; Fernhall, B.; Carhart, R.; Weinstock, R.S.; Baynard, T.; Figueroa, A.; Kanaley, J.A. Effects of diet and/or exercise on the adipocytokine and inflammatory cytokine levels of postmenopausal women with type 2 diabetes. Metabolism 2005, 54, 866–875. [Google Scholar] [CrossRef]
  61. Ferrucci, L.; Penninx, B.W.; Volpato, S.; Harris, T.B.; Bandeen-Roche, K.; Balfour, J.; Leveille, S.G.; Fried, L.P.; Md, J.M. Change in muscle strength explains accelerated decline of physical function in older women with high interleukin-6 serum levels. J. Am. Geriatr. Soc. 2002, 50, 1947–1954. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  62. Degens, H. Age-related skeletal muscle dysfunction: Causes and mechanisms. J. Musculoskelet. Neuronal Interact. 2007, 50, 246–252. [Google Scholar]
  63. Petersen, A.M.; Pedersen, B.K. The anti-inflammatory effect of exercise. J. Appl. Physiol. 2005, 98, 1154–1162. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  64. Gonzalo-Encabo, P.; Maldonado, G.; Valadés, D.; Ferragut, C.; Pérez-López, A. The role of exercise training on low-grade systemic inflammation in adults with overweight and obesity: A systematic review. Int. J. Environ. Res. Public Health 2021, 18, 13258. [Google Scholar] [CrossRef] [PubMed]
  65. Alizaei Yousefabadi, H.; Niyazi, A.; Alaee, S.; Fathi, M.; Mohammad Rahimi, G.R. Anti-inflammatory effects of exercise on metabolic syndrome patients: A systematic review and meta-analysis. Biol. Res. Nurs. 2021, 23, 280–292. [Google Scholar] [CrossRef]
  66. Tam, C.S.; Garnett, S.P.; Cowell, C.T.; Heilbronn, L.K.; Lee, J.W.; Wong, M.; Baur, L.A. IL-6, IL-8 and IL-10 levels in healthy weight and overweight children. Horm. Res. Paediatr. 2010, 73, 128–134. [Google Scholar] [CrossRef]
  67. Cerqueira, É.; Marinho, D.A.; Neiva, H.P.; Lourenço, O. Inflammatory effects of high and moderate intensity exercise-a systematic review. Front. Physiol. 2019, 10, 1550. [Google Scholar] [CrossRef]
Figure 1. Flow diagram of the study selection process.
Figure 1. Flow diagram of the study selection process.
Ijerph 19 03434 g001
Figure 2. Forest plot of effect size by resistance training (RT) on (a) CRP, (b) IL-6, (c) IL-10, and (d) TNF-α.
Figure 2. Forest plot of effect size by resistance training (RT) on (a) CRP, (b) IL-6, (c) IL-10, and (d) TNF-α.
Ijerph 19 03434 g002
Figure 3. Methodological quality.
Figure 3. Methodological quality.
Ijerph 19 03434 g003
Table 1. Characteristics of included studies.
Table 1. Characteristics of included studies.
First Author and YearParticipants:
Health Condition,
Sample Size n (EG, CG),
Mean Age, % of Females
Interventions:
Method, Number of Exercises (Ex), Intensity, Number of Sets and Repetitions, Minute, Weekly Frequency, Duration
ComparisonsOutcome Measures
Chupel
2017
Cognitive impairment,
33(16/17),
82.7, 100.0
Elastic band, 8–10 ex, vigorous,
1–2 × 10–12 Rep, 30–50 min,
2–3 times/week, 28 weeks
Usual careCRP (↓), IL-10 (↔), TNF-α (↔)
Cunha
2019
Healthy
48(25/23),
70.3, 100.0
Machines, 8 ex, moderate,
1 × 10–15 Rep, 20 min,
3 times/week, 12 weeks
InactiveCRP (↔)
Furtado
2020
Cognitive impairment
40(21/19),
80.5, 100.0
Elastic band, 8–10 ex, moderate,
2–3 × 10–15 Rep, 45 min,
2–3 times/week, 28 weeks
Usual careCRP (↓), IL-6 (↔),
IL-10 (↓), TNF-α (↔)
Hsieh
2018
Type 2 diabetes
30(15/15),
71.2, 63.3
Machines, 8 ex, vigorous,
3 × 8–12 Rep, NR,
3 times/week, 12 weeks
Usual careCRP (↔)
Kim
2010
Healthy
16(8/8),
67.1, 100.0
Elastic band, 14 ex, moderate,
2 × 10–15 Rep, 60 min,
4 times/week, 12 weeks
InactiveIL-6 (↔), TNF-α (↔)
Kim
2012
Type 2 diabetes
21(12/9),
68.8, 100.0
Machines, 8 ex, moderate,
2–3 × 10–15 Rep, 40–60 min,
3 times/week, 12 weeks
Usual careCRP (↔), IL-6 (↔),
TNF-α (↔)
Kim
2013
Healthy
20(10/10),
66.9, 100.0
Elastic band, 8 ex, vigorous,
3 × 10–12 Rep, 70 min,
3 times/week, 12 weeks
InactiveCRP (↔), IL-6 (↔),
TNF-α (↔)
Kim
2014
Healthy
14(7/7),
71.5, 100.0
Elastic band, 9 ex, moderate,
3 × 12 Rep, 60 min,
3 times/week, 12 weeks
InactiveCRP (↓)
Lee
2011
Metabolic syndrome
26(13/13),
68.8, 100.0
Elastic band, 12 ex, moderate,
3 × 10–15 Rep, 50 min,
3 times/week, 12 weeks
Usual careCRP (↓)
Martins
2010
Healthy
27(14/13),
73.2, 59.3
Callisthenic and elastic band,
8 ex, moderate, 6 × 8–12 Rep,
45 min, 3 times/week, 16 weeks
InactiveCRP (↔)
Rall
1996
Healthy
14(8/6),
69.7, 31.4
Machines, 5 ex, vigorous,
3 × 8 Rep, 45 min,
2 times/week, 12 weeks
InactiveIL-6 (↓), TNF-α (↔)
Redwine 2020Heart Failure
45(22/23),
66.0, 13.5
Elastic band, NR, vigorous,
NR, 60 min, 2 times/week,
16 weeks
InactiveCRP (↔), IL-6 (↑),
TNF-α (↓)
Rodriguez-Miguelez 2014Healthy
26(16/10),
69.1, 73.1
Machines, 3 ex, vigorous,
3 × 8–12 Rep, NR,
2 times/week, 8 weeks
InactiveCRP (↓), IL-6 (↓)
So
2013
Healthy
40(18/22),
69.8, 67.5
Elastic band, 14 ex, moderate,
2–3 × 15–25 Rep, 60 min,
3 times/week, 12 weeks
InactiveCRP (↔), IL-6 (↔),
TNF-α (↓)
Theodorou
2016
Coronary artery disease
26(11/15),
62.0, 0.0
Machines, 8 ex, moderate,
2 × 12–15 Rep, NR,
3 times/week, 32 weeks
Usual careCRP (↔)
Tomeleri
2016
Obese
38(19/19),
68.2, 100.0
Machines, 8 ex, moderate,
3 × 10–15 Rep, 45–50 min,
3 times/week, 12 weeks
InactiveCRP (↓), IL-6 (↔),
TNF-α (↔)
Tomeleri
2018
Healthy
45(22/23),
70.4, 100.0
Machines, 8 ex, moderate,
3 × 10–15 Rep, NR,
3 times/week, 12 weeks
InactiveCRP (↓), IL-6 (↓),
TNF-α (↔)
Wanderley
2013
Healthy
30(11/19),
67.6, 73.3
Machines, 9 ex, moderate,
2 × 10–15 Rep, 50 min,
3 times/week, 32 weeks
InactiveCRP (↓), IL-6 (↔),
IL-10 (↔), TNF-α (↔)
EG, experimental group; CG, control group; NR, not reported; CRP, C- reactive protein; IL-6, interleukin-6; IL-10, interleukin-10; TNF-α, tumor necrosis factors –α; ↓, decreased; ↑, increased; ↔, no change.
Table 2. Subgroup analysis of the effects of RT of elderly adults.
Table 2. Subgroup analysis of the effects of RT of elderly adults.
SubgroupCRPIL-6TNF-α
Effect Size95% CInI2Effect Size95% CInI2Effect Size95% CInI2
Age
  ≤70 years−0.69−1.19, −0.20973−0.70−1.43, 0.03985−0.73−1.46, −0.00884
  >70 years−0.83−1.24, −0.42754−0.31−1.49, 0.86286−0.15−0.65, 0.34346
Health condition
  Healthy −0.10−0.12, −0.07887−0.57−0.65, −0.49762−0.47−0.78, −0.16636
  Specific condition−0.04−0.08, 0.008840.16−0.18, 0.51464−0.73−1.90, 0.45592
Training method
  Elastic band−0.64−1.07, −0.218610.17−0.29, 0.63550−0.89−1.87, 0.09689
  Machines−0.82−1.37, −0.27877−1.28−2.21, −0.36685−0.33−0.63, −0.0450
Number of exercises
  ≤8−0.12−0.15, −0.08988−0.59−0.67, −0.51627−0.04−0.17, 0.0954
  >8−0.06−0.09, −0.0440−0.20−0.86, 0.46353−0.52−0.93, −0.1230
Intensity
  Moderate−0.63−0.93, −0.331145−0.23−0.60, 0.14746−0.23−0.52, 0.06715
  Vigorous−1.11−2.02, −0.20584−1.73−3.84, 0.39494−1.21−2.73, 0.31491
Weekly frequency
  ≤2−0.26−0.32, −0.2020−0.55−0.63, −0.47393−1.93−5.43, 1.57296
  >2−0.05−0.08, −0.031269−0.37−0.61, −0.12731−0.38−0.65, −0.1070
Duration (week)
  ≤8−0.26−0.32, −0.20279−0.59−0.67, −0.50362−0.24−0.87, 0.401-
  >8−0.05−0.08, −0.031478−0.14−0.36, 0.09868−0.59−1.18, −0.011082
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kim, S.-D.; Yeun, Y.-R. Effects of Resistance Training on C-Reactive Protein and Inflammatory Cytokines in Elderly Adults: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Int. J. Environ. Res. Public Health 2022, 19, 3434. https://doi.org/10.3390/ijerph19063434

AMA Style

Kim S-D, Yeun Y-R. Effects of Resistance Training on C-Reactive Protein and Inflammatory Cytokines in Elderly Adults: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. International Journal of Environmental Research and Public Health. 2022; 19(6):3434. https://doi.org/10.3390/ijerph19063434

Chicago/Turabian Style

Kim, Sang-Dol, and Young-Ran Yeun. 2022. "Effects of Resistance Training on C-Reactive Protein and Inflammatory Cytokines in Elderly Adults: A Systematic Review and Meta-Analysis of Randomized Controlled Trials" International Journal of Environmental Research and Public Health 19, no. 6: 3434. https://doi.org/10.3390/ijerph19063434

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop