Next Article in Journal
Artificial Intelligence Supports Decision Making during Open-Chest Surgery of Rare Congenital Heart Defects
Next Article in Special Issue
Relationship between Muscle Mass, Bone Density and Vascular Calcifications in Elderly People with SARS-CoV-2 Pneumonia
Previous Article in Journal
Targeting the Gut Microbiome in Prader-Willi Syndrome
Previous Article in Special Issue
Sarcopenia, Obesity, and Sarcopenic Obesity: Relationship with Skeletal Muscle Phenotypes and Single Nucleotide Polymorphisms
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Sarcopenia’s Prognostic Impact on Patients Treated with Immune Checkpoint Inhibitors: A Systematic Review and Meta-Analysis

1
Department of Surgery, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul 06273, Korea
2
Medical Library, Yonsei University College of Medicine, Seoul 03722, Korea
3
Yonsei University College of Medicine, Seoul 03722, Korea
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2021, 10(22), 5329; https://doi.org/10.3390/jcm10225329
Submission received: 22 September 2021 / Revised: 10 November 2021 / Accepted: 12 November 2021 / Published: 16 November 2021
(This article belongs to the Collection Sarcopenia: Skeletal Muscle Health and Ageing)

Abstract

:
Background: Although sarcopenia has been reported to predict survival in cancer patients, its impact on patients who received immune checkpoint inhibitors (ICIs) has not been thoroughly investigated. This systematic review aimed to assess the long-term oncologic impact of sarcopenia on patients who received ICIs. Methods: A systematic review of studies indexed in the PubMed, Embase, and Cochrane databases, up to April 1, 2021, was conducted. Studies that reported hazard ratios (HRs) for overall survival (OS) and progression-free survival (PFS) based on sarcopenia in patients treated with ICIs were included. The inverse variance method was used with a random-effects model for data analysis. Results: A total of 1284 patients from 14 studies were included. Among the patients who received ICIs, patients with sarcopenia had a significant increase in overall mortality compared to patients without sarcopenia in univariate analyses (HR = 1.66, 95% CI = 1.20–2.29, p = 0.002) and in adjusted HRs (HR = 1.55, 95% CI = 1.15–2.10, p = 0.004). The same results were obtained for PFS by both univariate analysis (HR = 1.75, 95% CI = 1.37–2.23, p < 0.001) and adjusted HRs (HR = 1.63, 95% CI 1.28–2.09, p < 0.001). Conclusions: Sarcopenia appears to be an effective biomarker for predicting long-term oncologic outcomes in patients receiving ICI therapy and hence plays an important role when making treatment decisions. However, the fundamental role of this association with survival should be further investigated in large cohorts and clinical trials.

1. Introduction

Sarcopenia and changes in muscle mass during a certain treatment period have been evaluated as important prognosticators in cancer patients [1,2,3,4,5]. The clinical importance of myosteatosis, which indicates the mean Hounsfield Unit of skeletal muscle measured using computed tomography (CT), has also been studied, and its association with oncologic outcomes has also been determined in patients with various types of cancers [6,7].
Immunotherapy with immune checkpoint inhibitors (ICIs) has become one of the major breakthroughs in advanced cancers [8]. The clinical benefits and side effects of ICIs vary, and they include hyperprogression among patients with cancer [9,10,11]. Therefore, investigating the biomarkers relevant to ICI response is important for predicting patients’ clinical outcomes. Recently, the clinical significance of sarcopenia in patients who have undergone ICI therapy has been reported. Cortellini et al. reported that a low skeletal muscle index (SMI) was associated with poor oncologic outcomes in advanced cancer patients treated with ICIs [12]. Shimizu et al. reported that the psoas muscle index (PMI) might be a significant prognostic factor for progression-free survival (PFS) and overall survival (OS) following ICI therapy for metastatic urothelial carcinoma [13]. In contrast, Minami et al. observed no significant correlation between sarcopenia and clinical outcomes in patients treated with ICIs [14]. To incorporate these findings and investigate the role of sarcopenia as a possible prognostic factor in patients who underwent ICIs, a review of the existing evidence seemed appropriate. Although a recent meta-analysis reported that sarcopenia could be used as a viable option for predicting prognosis in non-small cell lung cancer (NSCLC) patients who received ICIs [15], the impact of sarcopenia has not been thoroughly investigated in patients with other types of cancer.
Therefore, we performed a systematic review and meta-analysis to evaluate the long-term effects of sarcopenia on survival in cancer patients treated with ICIs.

2. Materials and Methods

2.1. Search Strategy

This systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [16]. We performed computerized bibliographic searches from inception to April 2021. The detailed search strategies and associated lists following PubMed/Medline, Embase, and Cochrane library databases were extracted by one of our authors (N.W.K.) and are included in the Supplementary Tables S1 and S2. Manual searching of reference lists from the included articles was also performed to identify additional potential results.

2.2. Study Selection and Data Extraction

The search identified 1706 articles. Two authors (D.L. and J.K.) screened all the titles and abstracts of the identified articles. After screening, full-text reviews were independently performed by the authors to determine the eligibility of the studies. Discrepancies were resolved through discussion. Studies were selected based on several inclusion criteria. We defined the study population as various types of cancer patients who underwent ICI therapy. The primary outcomes were measured as hazard ratios (HRs) of OS and/or PFS with sarcopenia as one of the variables. In general, the previously established definition of sarcopenia is what was most commonly used; however, some studies employed newly defined criteria for sarcopenia, which were also included in our study. Conference abstracts, editorials, comments, and publications published in languages other than English were not eligible for inclusion.
Two authors (D.L. and J.K.) extracted data details on authors, year, publication country, cancer type, stage, time point of CT examination, ICI type, measurement of sarcopenia, cut-off value of sarcopenia, number of included patients, and number and proportion of sarcopenia patients. Any discrepancies in study selection and data extraction between the two authors were resolved by discussion.

2.3. Study Quality Assessments

The risk of bias and quality of individual studies were assessed using the Newcastle-Ottawa Scale (NOS) [17]. The NOS uses three categories with each item: (1) selection (representativeness of the exposed cohort, selection of the non-exposed cohort, ascertainment of exposure, and demonstration that the outcome of interest was not present at the start of the study); (2) comparability (comparability of cohorts based on the design or analysis); and (3) outcome (assessment of outcome, follow-up long enough for outcomes to occur, and adequacy of follow-up of cohorts). A study could be awarded a maximum of one point for each numbered item within the selection and outcome categories. A maximum of two points could be given for comparability. The NOS scores were up to nine points. A study that scored 7 or higher points was considered to be of high quality, 5–6 points moderate quality, and below 4 points low quality.
In addition, we used the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system to assess the certainty of the evidence for each outcome of our meta-analysis [18]. The GRADE system consists of study limitations (risk of bias), inconsistency, indirectness, imprecision, and publication. For our observational study, the GRADE system also considers a large effect, plausible confounding, and the dose-response gradient. Based on these categories, two authors classified the certainty of the results as high, moderate, low, or very low.

2.4. Statistical Analysis

The effects of sarcopenia on outcomes were measured using the HRs and CIs of OS and PFS. Meta-analysis was performed using the two different outcomes, such as HRs in univariate analysis and adjusted HRs, of each individual studies. Pooled HRs with 95% CIs indicated the effects of sarcopenia on outcomes. A Cochran Q statistic p-value < 0.05 or an I2 statistic > 50% were evaluated to represent statistically significant heterogeneity between the studies. A random-effects model was used to pool the HRs. Inverse-variance weighting was used to pool estimates from the included studies. While measuring the heterogeneity between the included studies, we excluded one study because it used the HR of sarcopenia as a continuous variable [19]. Subgroup analyses were conducted according to the type of cancer, receipt of different types of ICIs, or different definitions of sarcopenia. Funnel plots and Egger linear regression analysis were not possible because fewer than ten studies were included.
The meta-analysis was performed using Review Manager software (RevMan, Version 5.41, for Windows, Oxford, UK; the Cochrane Collaboration, 2014). All p-values were two-sided, and except for the test of discrepancy, the p-value < 0.05 was considered statistically significant.

3. Results

3.1. Search Results and Study Population Characteristics

The results of the PRISMA flow diagram are shown in Figure 1. Our initial database searches and manual searches identified 1710 articles. After the duplicates were removed, 1555 articles remained. We then examined the titles and abstracts, and 1529 articles were excluded. We then performed full-text reviews based on the inclusion criteria. We excluded articles that defined sarcopenia as differences in SMI between pre-treatment and post-treatment [20,21,22] or those that did not show HRs of OS and/or PFS [23,24,25,26,27,28,29,30,31]. Finally, 14 studies with a total of 1284 patients were included in this study [12,13,14,19,32,33,34,35,36,37,38,39,40,41].
The details of the included studies are presented in Table 1. All the included studies had a retrospective design. No prospective studies were available. All the included studies were published after 2019. The majority of the cancer types in the included studies were NSCLC (7), followed by hepatocellular carcinoma (2), melanoma (2), urothelial carcinoma (2), renal cell carcinoma, and gastric cancer. In terms of the types of ICIs, 13 studies used anti-PD-1/PD-L1 ICIs, and only one study used anti-CTLA-4. The cut-off values and sarcopenia reference varied across studies. SMI was used to define sarcopenia in 10 studies, whereas PMI was used to measure sarcopenia in 4 studies. Result types of the studies included in the meta-analysis are shown in Supplementary Table S3.

3.2. Primary Outcome

The meta-analysis of the included studies showed the pooled HR of 1.66 for OS in the studies that used univariate analysis (95% CI = 1.20–2.29, p = 0.002). The heterogeneity among these studies was significant (Cochran’s Q statistic, p < 0.0001, I2 = 80%). For the studies that used adjusted HRs, the meta-analysis of the included studies showed the pooled HR of 1.55 for OS (95% CI = 1.15–2.10, p = 0.004). The heterogeneity among these studies was not significant (Cochran’s Q statistic, p = 0.07, I2 = 44%) (Figure 2). In addition, we performed a meta-analysis of PFS. For the studies that used univariate analysis, the pooled HR was 1.75 for PFS (95% CI = 1.37–2.23, p < 0.00001). The heterogeneity among these studies was not significant (Cochran’s Q statistic, p = 0.07, I2 = 45%). For the studies that used adjusted HRs, the pooled HR was 1.63 for PFS (95% CI = 1.28–2.09, p < 0.0001). The heterogeneity among these studies was also not significant (Cochran’s Q statistic, p = 0.12, I2 = 36%) (Figure 3).

3.3. Subgroup Analyses

To investigate the cause of the heterogeneity in the OS in the studies using HRs extracted from univariate analysis, we excluded the study by Magri et al., which expressed the HR of sarcopenia uniquely as a continuous variable. After excluding that study, the heterogeneity decreased in the studies using HRs extracted from the univariate analysis of OS (HR = 1.74, 95% CI = 1.39–2.19, p < 0.00001, Cochran Q statistic, p = 0.27, I2 = 21%) and the studies using adjusted HRs of OS (HR = 1.64, 95% CI = 1.21–2.21, p = 0.001, Cochran Q statistic, p = 0.10, I2 = 41%) (Supplementary Figure S1).
We performed subgroup analysis according to cancer type. Because of the study number, only OS of the studies with adjusted HRs were eligible (Supplementary Figure S2). The test for subgroup differences suggested that there was no statistically significant subgroup effect (p = 0.42, I2 = 0%), implying that cancer type does not modify the effect of ICIs on OS in different cancer types. However, this analysis may not be able to detect subgroup differences due to the discrepancies in the study numbers and participant numbers. In the case of the studies derived from univariate analysis of PFS, the test for subgroup differences suggested that there was a statistically significant subgroup effect (p = 0.05, I2 = 81.0%) (Supplementary Figure S3).
We performed subgroup analysis by type of ICI using the HRs of PFS. The test for sub-group differences suggested that there was no statistically significant subgroup effect (p = 0.22, I2 = 34.4%) (Supplementary Figure S4).
In this meta-analysis, either SMI or PMI was used to define sarcopenia. Comparing SMI and PMI, a significant subgroup effect was observed in the studies using univariate analysis of OS (p = 0.06, I2 = 71.0%); however, there was no statistically significant subgroup effect (p = 0.80, I2 = 0%) OS of the studies using adjusted HRs (p = 0.80, I2 = 0%) (Supplementary Figure S5). With respect to PFS, there was a statistically significant subgroup effect (p = 0.01, I2 = 83.2%) in the studies using univariate analysis of PFS, whereas no statistically significant subgroup effect was observed in the PFS of the studies using adjusted HRs (p = 0.69, I2 = 0%) (Supplementary Figure S6).

3.4. Quality Assessment

We used the NOS to assess the risk of bias and the quality of the individual studies. Of the 14 studies, five were assessed as high quality and nine as moderate quality. Detailed assessments are presented in Table 2. We then used the GRADE system to assess the certainty of the evidence in our meta-analysis. The results of the GRADE certainty assessment for the meta-analysis of each outcome are presented in Table 3. We are very uncertain about the prognostic effect of sarcopenia in cancer patients treated with ICIs.

4. Discussion

This systematic review and meta-analysis evaluated the prognostic effects of sarcopenia in patients with ICIs. The meta-analysis of the included studies showed poor prognosis of patients with sarcopenia with respect to OS (HR, 1.55; 95% CI = 1.15–2.10, p = 0.004 for OS using studies with adjusted HRs) and PFS (HR, 1.75; 95% CI = 1.37–2.23, p < 0.001 for PFS using studies with adjusted HRs).
Since ICIs were introduced as an alternative treatment option in various advanced and refractory cancer patients [42], identifying the biomarkers relevant to ICI outcomes has been actively investigated. Higher tumor mutational burden, microsatellite instability, and PD-L1 immunohistochemical staining have been proven to be strong predictive markers for better responses [43]. However, these biomarkers are difficult to use because additional laborious work or obtaining adequate tissue is required. Recently, a growing body of evidence has reported that patient host factors, such as body composition, are associated with the clinical efficacy of ICIs. Sarcopenia, which was initially defined as age-associated loss of muscle mass in elderly persons [44], has been incorporated into the oncology field, and the prognostic impact of sarcopenia or myosteatosis in cancer patients treated with surgery and/or palliative or adjuvant chemotherapy has been well studied [1,6,7]. Several retrospective studies have explored the impact of sarcopenia on ICIs, and most of them were included in this meta-analysis. Most patients diagnosed with a certain type of cancer underwent abdominopelvic CT to determine the extent of the disease at the initial stage. The lack of additional demand to assess sarcopenia is a very important advantage in terms of clinical use.
The impact of sarcopenia on ICIs can be explained in several ways. Chronic inflammation in cancer, a major contributor to the sarcopenia cascade [45], causes immune dysfunction, such as T cell exhaustion, which is characterized by a loss of effector function, prolonged and high expression of multiple inhibitory receptors, and specific transcriptional pathways [46]. It is mediated by changes in the functions of cytokines and results in a reduced response of ICIs. In addition, skeletal muscle tissue synthesizes cytokines and other proteins. They are collectively called myokines. Myokines, such as IL-6, IL-15, TNF-α, and TGF-β, exert autocrine, endocrine, and paracrine effects on many tissues. With altered activities of these myokines in the setting of sarcopenia, the immune system leans towards exhibiting pro-inflammatory effects and muscle catabolism, as well as inducing immune senescence [47,48]. Additionally, the role of gut microbiome in developing sarcopenia and modulating ICIs’ responses was recently introduced. The gut microbiome is extensively involved in the immune system by anatomic features and the need to modulate the numerous variant species in the gastrointestinal tract. In patients with altered gut microbiome, the gut dysbiosis may result in promoting a pro-inflammatory pathway related to sarcopenia. However, the pathophysiology of the role of gut microbiome to regulate the response to ICIs was not fully investigated yet [49,50].
Whether sarcopenia could be used as a predictive marker for immune-related adverse events (irAEs) remains unclear. A recent meta-analysis by Wang et al. reported that sarcopenia was not associated with an increased rate of irAEs (relative risk = 0.99, 95% CI = 0.21–4.67) in patients with NSCLC [15]. Another systematic review reported that sarcopenia was correlated with adverse events; however, no association with increased irAEs was noted [51]. Therefore, it is unclear whether poor oncologic outcomes for sarcopenic patients are directly derived from ICI-induced toxicities or reduced adherence to ICI treatments. Further research needs to be done to reveal the fundamental mechanism of this correlation.
Our study had some limitations. The included studies were mainly retrospective in design, with a relatively small number of included patients. The inevitable selection bias of the original studies might be one of the reasons for the heterogeneity. Additionally, we could not adjust covariates equally due to the different variables used in each included study, which are shown in Supplementary Table S4. This situation caused heterogeneity, which resulted in increased values of Q-statistic’s p-value and I-square. Thus, we estimated a pooled HR considering these variabilities via the random effect model. We excluded several studies that did not provide adequate survival outcomes, such as HRs and 95% CI. Notably, there were severe methodological variations in defining sarcopenia and its cut-off values across the studies in this meta-analysis. Most of the included studies used the SMI cut-off value suggested by Martin et al. However, PMI was alternatively used because of its potential ease of measurement. Studies comparing the skeletal muscle area (SMA) and psoas muscle area (PMA) showed discordant results. Rutten et al. reported a weak correlation (Pearson correlation of 0.52) between the SMA and PMA in ovarian cancer patients [52], whereas Jones et al. reported a better correlation (Spearman correlation of 0.8) in patients with colorectal cancer [53]. The reason for this discrepancy might be explained by different statistical methods, sex differences, and the relatively small number of patients. When we tried to measure the effect size, these different definitions caused heterogeneity and other unexpected biases. Based on the subgroup analysis according to the different methods used to define sarcopenia, some subgroup differences were observed between SMI and PMI with respect to the studies from the univariate analysis of OS and PFS. Nevertheless, there was no difference in the overall trends favoring a better prognosis in patients without sarcopenia. Although the lack of a consensus on the definition of sarcopenia may hamper the wide application of our observations in clinical practice, our results show the possibility that PMI can be used to predict sarcopenia in patients receiving ICIs. Additionally, we could not exclude publication bias where studies of opposite results were not published. We could not use a funnel plot to distinguish real asymmetry due to the small sample sizes. Finally, the measured time gap between CT examination and treatment initiation showed a huge diversity, ranging from 1 month to 6 months across the included studies. Considering malnourishment and advanced tumor stage, the time gap might be one of the critical determinants of sarcopenia. The different time gaps between studies may hinder the standardization of this parameter. For future studies dealing with this issue, these limitations should be considered.

5. Conclusions

Our study investigated the effect of sarcopenia on clinical outcomes across all types of cancer in patients treated with ICIs. The results suggested that sarcopenia could negatively affect OS and PFS in patients with cancer who were treated with ICIs in both univariate and with adjusted HRs. Sarcopenia could be used as a marker for predicting ICI clinical outcomes and needs to be assessed before treatment with ICIs.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jcm10225329/s1. Supplementary Figure S1. Forest plots of hazard ratios of overall survival according to the sarcopenia, excluding Magri et al.’s study. Supplementary Figure S2. Forest plots of hazard ratios for overall survival with adjusted HRs by cancer type. Supplementary Figure S3. Forest plots of hazard ratios for progression-free survival in the univariate analysis by cancer type. Supplementary Figure S4. Forest plots of hazard ratios for progression-free survival in the univariate analysis according to the type of immune checkpoint inhibitors. Supplementary Figure S5. Forest plots of hazard ratios for overall survival by skeletal muscle index and psoas muscle index. Supplementary Figure S6. Forest plots of hazard ratios for progression-free survival by skeletal muscle index and psoas muscle index. Supplementary Table S1. Search strategy. Supplementary Table S2. Search query and number of items found. Supplementary Table S3. Result types of the studies included in the meta-analysis. Supplementary Table S4. Covariates of the studies using adjusted HRs.

Author Contributions

Conceptualization, D.L. and J.K.; methodology, N.W.K. and J.Y.K.; formal analysis, D.L., N.W.K. and J.K.; investigation, J.H.L., J.W.J. and S.L.; data curation, J.H.N. and H.L.; writing—original draft preparation, D.L.; writing—review and editing, D.L. and J.K.; supervision, J.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Shachar, S.S.; Williams, G.R.; Muss, H.B.; Nishijima, T.F. Prognostic value of sarcopenia in adults with solid tumours: A meta-analysis and systematic review. Eur. J. Cancer 2016, 57, 58–67. [Google Scholar] [CrossRef]
  2. Trejo-Avila, M.; Bozada-Gutiérrez, K.; Valenzuela-Salazar, C.; Herrera-Esquivel, J.; Moreno-Portillo, M. Sarcopenia predicts worse postoperative outcomes and decreased survival rates in patients with colorectal cancer: A systematic review and meta-analysis. Int. J. Colorectal Dis. 2021, 36, 1077–1096. [Google Scholar] [CrossRef] [PubMed]
  3. Guo, Z.; Gu, C.; Gan, S.; Li, Y.; Xiang, S.; Gong, L.; Chan, L.C.; Wang, S. Sarcopenia as a predictor of postoperative outcomes after urologic oncology surgery: A systematic review and meta-analysis. Urol. Oncol. 2020, 38, 560–573. [Google Scholar] [CrossRef]
  4. van Rijn-Dekker, M.I.; van den Bosch, L.; van den Hoek, J.G.M.; Bijl, H.P.; van Aken, E.S.M.; van der Hoorn, A.; Oosting, S.F.; Halmos, G.B.; Witjes, M.J.H.; van der Laan, H.P.; et al. Impact of sarcopenia on survival and late toxicity in head and neck cancer patients treated with radiotherapy. Radiother. Oncol. 2020, 147, 103–110. [Google Scholar] [CrossRef] [PubMed]
  5. Olson, B.; Edwards, J.; Stone, L.; Jiang, A.; Zhu, X.; Holland, J.; Li, R.; Andersen, P.; Krasnow, S.; Marks, D.L.; et al. Association of Sarcopenia with Oncologic Outcomes of Primary Surgery or Definitive Radiotherapy Among Patients with Localized Oropharyngeal Squamous Cell Carcinoma. JAMA Otolaryngol. Head Neck Surg. 2020, 146, 714–722. [Google Scholar] [CrossRef]
  6. Lee, C.M.; Kang, J. Prognostic impact of myosteatosis in patients with colorectal cancer: A systematic review and meta-analysis. J. Cachexia Sarcopenia Muscle 2020, 11, 1270–1282. [Google Scholar] [CrossRef] [PubMed]
  7. Aleixo, G.F.P.; Shachar, S.S.; Nyrop, K.A.; Muss, H.B.; Malpica, L.; Williams, G.R. Myosteatosis and prognosis in cancer: Systematic review and meta-analysis. Crit. Rev. Oncol. Hematol. 2020, 145, 102839. [Google Scholar] [CrossRef]
  8. Darvin, P.; Toor, S.M.; Sasidharan Nair, V.; Elkord, E. Immune checkpoint inhibitors: Recent progress and potential biomarkers. Exp. Mol. Med. 2018, 50, 1–11. [Google Scholar] [CrossRef] [Green Version]
  9. Cristescu, R.; Mogg, R.; Ayers, M.; Albright, A.; Murphy, E.; Yearley, J.; Sher, X.; Liu, X.Q.; Lu, H.; Nebozhyn, M.; et al. Pan-tumor genomic biomarkers for PD-1 checkpoint blockade-based immunotherapy. Science 2018, 362, 6411. [Google Scholar] [CrossRef] [Green Version]
  10. Kim, J.Y.; Lee, K.H.; Kang, J.; Borcoman, E.; Saada-Bouzid, E.; Kronbichler, A.; Hong, S.H.; de Rezende, L.F.M.; Ogino, S.; Keum, N.; et al. Hyperprogressive Disease during Anti-PD-1 (PDCD1) / PD-L1 (CD274) Therapy: A Systematic Review and Meta-Analysis. Cancers 2019, 11, 1699. [Google Scholar] [CrossRef] [PubMed]
  11. Martins, F.; Sofiya, L.; Sykiotis, G.P.; Lamine, F.; Maillard, M.; Fraga, M.; Shabafrouz, K.; Ribi, C.; Cairoli, A.; Guex-Crosier, Y.; et al. Adverse effects of immune-checkpoint inhibitors: Epidemiology, management and surveillance. Nat. Rev. Clin. Oncol. 2019, 16, 563–580. [Google Scholar] [CrossRef] [PubMed]
  12. Cortellini, A.; Bozzetti, F.; Palumbo, P.; Brocco, D.; Di Marino, P.; Tinari, N.; De Tursi, M.; Agostinelli, V.; Patruno, L.; Valdesi, C.; et al. Weighing the role of skeletal muscle mass and muscle density in cancer patients receiving PD-1/PD-L1 checkpoint inhibitors: A multicenter real-life study. Sci. Rep. 2020, 10, 1456. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Shimizu, T.; Miyake, M.; Hori, S.; Ichikawa, K.; Omori, C.; Iemura, Y.; Owari, T.; Itami, Y.; Nakai, Y.; Anai, S.; et al. Clinical Impact of Sarcopenia and Inflammatory/Nutritional Markers in Patients with Unresectable Metastatic Urothelial Carcinoma Treated with Pembrolizumab. Diagnostics 2020, 10, 310. [Google Scholar] [CrossRef] [PubMed]
  14. Minami, S.; Ihara, S.; Tanaka, T.; Komuta, K. Sarcopenia and Visceral Adiposity Did Not Affect Efficacy of Immune-Checkpoint Inhibitor Monotherapy for Pretreated Patients with Advanced Non-Small Cell Lung Cancer. World J. Oncol. 2020, 11, 9–22. [Google Scholar] [CrossRef] [PubMed]
  15. Wang, J.; Cao, L.; Xu, S. Sarcopenia affects clinical efficacy of immune checkpoint inhibitors in non-small cell lung cancer patients: A systematic review and meta-analysis. Int. Immunopharmacol. 2020, 88, 106907. [Google Scholar] [CrossRef]
  16. Moher, D.; Shamseer, L.; Clarke, M.; Ghersi, D.; Liberati, A.; Petticrew, M.; Shekelle, P.; Stewart, L.A.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 2015, 4, 1. [Google Scholar] [CrossRef] [Green Version]
  17. Wells, G.A.; Shea, B.; O’Connell, D.; Peterson, J.; Welch, V.; Losos, M.; Tugwell, P. The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses. Available online: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (accessed on 11 November 2021).
  18. Guyatt, G.H.; Oxman, A.D.; Vist, G.E.; Kunz, R.; Falck-Ytter, Y.; Alonso-Coello, P.; Schünemann, H.J.; GRADE Working Group. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336, 924–926. [Google Scholar] [CrossRef] [Green Version]
  19. Magri, V.; Gottfried, T.; Di Segni, M.; Urban, D.; Peled, M.; Daher, S.; Stoff, R.; Bar, J.; Onn, A. Correlation of body composition by computerized tomography and metabolic parameters with survival of nivolumab-treated lung cancer patients. Cancer Manag. Res. 2019, 11, 8201–8207. [Google Scholar] [CrossRef] [Green Version]
  20. Loosen, S.H.; van den Bosch, V.; Gorgulho, J.; Schulze-Hagen, M.; Kandler, J.; Jördens, M.S.; Tacke, F.; Loberg, C.; Antoch, G.; Brümmendorf, T.; et al. Progressive Sarcopenia Correlates with Poor Response and Outcome to Immune Checkpoint Inhibitor Therapy. J. Clin. Med. 2021, 10, 1361. [Google Scholar] [CrossRef] [PubMed]
  21. Nishioka, N.; Uchino, J.; Hirai, S.; Katayama, Y.; Yoshimura, A.; Okura, N.; Tanimura, K.; Harita, S.; Imabayashi, T.; Chihara, Y.; et al. Association of Sarcopenia with and Efficacy of Anti-PD-1/PD-L1 Therapy in Non-Small-Cell Lung Cancer. J. Clin. Med. 2019, 8, 450. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Petrova, M.P.; Donev, I.S.; Radanova, M.A.; Eneva, M.I.; Dimitrova, E.G.; Valchev, G.N.; Minchev, V.T.; Taushanova, M.S.; Boneva, M.V.; Karanikolova, T.S.; et al. Sarcopenia and high NLR are associated with the development of hyperprogressive disease after second-line pembrolizumab in patients with non-small-cell lung cancer. Clin. Exp. Immunol. 2020, 202, 353–362. [Google Scholar] [CrossRef]
  23. Nishioka, N.; Naito, T.; Notsu, A.; Mori, K.; Kodama, H.; Miyawaki, T.; Mamesaya, N.; Kobayashi, H.; Omori, S.; Wakuda, K.; et al. Unfavorable impact of decreased muscle quality on the efficacy of immunotherapy for advanced non-small cell lung cancer. Cancer Med. 2021, 10, 247–256. [Google Scholar] [CrossRef]
  24. Bilen, M.A.; Martini, D.J.; Liu, Y.; Shabto, J.M.; Brown, J.T.; Williams, M.; Khan, A.I.; Speak, A.; Lewis, C.; Collins, H.; et al. Combined Effect of Sarcopenia and Systemic Inflammation on Survival in Patients with Advanced Stage Cancer Treated with Immunotherapy. Oncologist 2020, 25, e528–e535. [Google Scholar] [CrossRef] [Green Version]
  25. Qayyum, A.; Bhosale, P.; Aslam, R.; Avritscher, R.; Ma, J.; Pagel, M.D.; Sun, J.; Mohamed, Y.; Rashid, A.; Beretta, L.; et al. Effect of sarcopenia on systemic targeted therapy response in patients with advanced hepatocellular carcinoma. Abdom. Radiol. 2021, 46, 1008–1015. [Google Scholar] [CrossRef]
  26. Cortellini, A.; Verna, L.; Porzio, G.; Bozzetti, F.; Palumbo, P.; Masciocchi, C.; Cannita, K.; Parisi, A.; Brocco, D.; Tinari, N.; et al. Predictive value of skeletal muscle mass for immunotherapy with nivolumab in non-small cell lung cancer patients: A “hypothesis-generator” preliminary report. Thorac. Cancer 2019, 10, 347–351. [Google Scholar] [CrossRef]
  27. Daly, L.E.; Power, D.G.; O’Reilly, Á.; Donnellan, P.; Cushen, S.J.; O’Sullivan, K.; Twomey, M.; Woodlock, D.P.; Redmond, H.P.; Ryan, A.M. The impact of body composition parameters on ipilimumab toxicity and survival in patients with metastatic melanoma. Br. J. Cancer 2017, 116, 310–317. [Google Scholar] [CrossRef]
  28. Heidelberger, V.; Goldwasser, F.; Kramkimel, N.; Jouinot, A.; Huillard, O.; Boudou-Rouquette, P.; Chanal, J.; Arrondeau, J.; Franck, N.; Alexandre, J. Sarcopenic overweight is associated with early acute limiting toxicity of anti-PD1 checkpoint inhibitors in melanoma patients. Investig. New Drugs 2017, 35, 436–441. [Google Scholar] [CrossRef] [PubMed]
  29. Kano, M.; Hihara, J.; Tokumoto, N.; Kohashi, T.; Hara, T.; Shimbara, K.; Takahashi, S. Association between skeletal muscle loss and the response to nivolumab immunotherapy in advanced gastric cancer patients. Int. J. Clin. Oncol. 2021, 26, 523–531. [Google Scholar] [CrossRef] [PubMed]
  30. Hu, J.B.; Ravichandran, S.; Rushing, C.; Beasley, G.M.; Hanks, B.A.; Jung, S.H.; Salama, A.; Ho, L.; Mosca, P.J. Higher BMI, But Not Sarcopenia, Is Associated With Pembrolizumab-related Toxicity in Patients With Advanced Melanoma. Anticancer Res. 2020, 40, 5245–5254. [Google Scholar] [CrossRef]
  31. Hirsch, L.; Bellesoeur, A.; Boudou-Rouquette, P.; Arrondeau, J.; Thomas-Schoemann, A.; Kirchgesner, J.; Gervais, C.; Jouinot, A.; Chapron, J.; Giraud, F.; et al. The impact of body composition parameters on severe toxicity of nivolumab. Eur. J. Cancer 2020, 124, 170–177. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Roch, B.; Coffy, A.; Jean-Baptiste, S.; Palaysi, E.; Daures, J.P.; Pujol, J.L.; Bommart, S. Cachexia-sarcopenia as a determinant of disease control rate and survival in non-small lung cancer patients receiving immune-checkpoint inhibitors. Lung Cancer 2020, 143, 19–26. [Google Scholar] [CrossRef]
  33. Shiroyama, T.; Nagatomo, I.; Koyama, S.; Hirata, H.; Nishida, S.; Miyake, K.; Fukushima, K.; Shirai, Y.; Mitsui, Y.; Takata, S.; et al. Impact of sarcopenia in patients with advanced non-small cell lung cancer treated with PD-1 inhibitors: A preliminary retrospective study. Sci. Rep. 2019, 9, 2447. [Google Scholar] [CrossRef] [Green Version]
  34. Takada, K.; Yoneshima, Y.; Tanaka, K.; Okamoto, I.; Shimokawa, M.; Wakasu, S.; Takamori, S.; Toyokawa, G.; Oba, T.; Osoegawa, A.; et al. Clinical impact of skeletal muscle area in patients with non-small cell lung cancer treated with anti-PD-1 inhibitors. J. Cancer Res. Clin. Oncol. 2020, 146, 1217–1225. [Google Scholar] [CrossRef]
  35. Tsukagoshi, M.; Yokobori, T.; Yajima, T.; Maeno, T.; Shimizu, K.; Mogi, A.; Araki, K.; Harimoto, N.; Shirabe, K.; Kaira, K. Skeletal muscle mass predicts the outcome of nivolumab treatment for non-small cell lung cancer. Medicine 2020, 99, e19059. [Google Scholar] [CrossRef]
  36. Akce, M.; Liu, Y.; Zakka, K.; Martini, D.J.; Draper, A.; Alese, O.B.; Shaib, W.L.; Wu, C.; Wedd, J.P.; Sellers, M.T.; et al. Impact of Sarcopenia, BMI, and Inflammatory Biomarkers on Survival in Advanced Hepatocellular Carcinoma Treated with Anti-PD-1 Antibody. Am. J. Clin. Oncol. 2021, 44, 74–81. [Google Scholar] [CrossRef]
  37. Kim, N.; Yu, J.I.; Park, H.C.; Yoo, G.S.; Choi, C.; Hong, J.Y.; Lim, H.Y.; Lee, J.; Choi, M.S.; Lee, J.E.; et al. Incorporating sarcopenia and inflammation with radiation therapy in patients with hepatocellular carcinoma treated with nivolumab. Cancer Immunol. Immunother. 2020, 70, 1593–1603. [Google Scholar] [CrossRef]
  38. Chu, M.P.; Li, Y.; Ghosh, S.; Sass, S.; Smylie, M.; Walker, J.; Sawyer, M.B. Body composition is prognostic and predictive of ipilimumab activity in metastatic melanoma. J. Cachexia Sarcopenia Muscle 2020, 11, 748–755. [Google Scholar] [CrossRef]
  39. Young, A.C.; Quach, H.T.; Song, H.; Davis, E.J.; Moslehi, J.J.; Ye, F.; Williams, G.R.; Johnson, D.B. Impact of body composition on outcomes from anti-PD1 +/- anti-CTLA-4 treatment in melanoma. J. Immunother. Cancer 2020, 8, e000821. [Google Scholar] [CrossRef] [PubMed]
  40. Fukushima, H.; Fukuda, S.; Moriyama, S.; Uehara, S.; Yasuda, Y.; Tanaka, H.; Yoshida, S.; Yokoyama, M.; Matsuoka, Y.; Fujii, Y. Impact of sarcopenia on the efficacy of pembrolizumab in patients with advanced urothelial carcinoma: A preliminary report. Anticancer Drugs 2020, 31, 866–871. [Google Scholar] [CrossRef] [PubMed]
  41. Kim, Y.Y.; Lee, J.; Jeong, W.K.; Kim, S.T.; Kim, J.H.; Hong, J.Y.; Kang, W.K.; Kim, K.M.; Sohn, I.; Choi, D. Prognostic significance of sarcopenia in microsatellite-stable gastric cancer patients treated with programmed death-1 inhibitors. Gastric Cancer 2021, 24, 457–466. [Google Scholar] [CrossRef] [PubMed]
  42. Snyder, A.; Makarov, V.; Merghoub, T.; Yuan, J.; Zaretsky, J.M.; Desrichard, A.; Walsh, L.A.; Postow, M.A.; Wong, P.; Ho, T.S.; et al. Genetic basis for clinical response to CTLA-4 blockade in melanoma. N. Engl. J. Med. 2014, 371, 2189–2199. [Google Scholar] [CrossRef] [Green Version]
  43. Kim, J.Y.; Kronbichler, A.; Eisenhut, M.; Hong, S.H.; van der Vliet, H.J.; Kang, J.; Shin, J.I.; Gamerith, G. Tumor Mutational Burden and Efficacy of Immune Checkpoint Inhibitors: A Systematic Review and Meta-Analysis. Cancers 2019, 11, 1798. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Baumgartner, R.N.; Koehler, K.M.; Gallagher, D.; Romero, L.; Heymsfield, S.B.; Ross, R.R.; Garry, P.J.; Lindeman, R.D. Epidemiology of sarcopenia among the elderly in New Mexico. Am. J. Epidemiol. 1998, 147, 755–763. [Google Scholar] [CrossRef] [PubMed]
  45. Chhetri, J.K.; de Souto Barreto, P.; Fougère, B.; Rolland, Y.; Vellas, B.; Cesari, M. Chronic inflammation and sarcopenia: A regenerative cell therapy perspective. Exp. Gerontol. 2018, 103, 115–123. [Google Scholar] [CrossRef] [PubMed]
  46. Wherry, E.J. T cell exhaustion. Nat. Immunol. 2011, 12, 492–499. [Google Scholar] [CrossRef]
  47. Nelke, C.; Dziewas, R.; Minnerup, J.; Meuth, S.G.; Ruck, T. Skeletal muscle as potential central link between sarcopenia and immune senescence. EBioMedicine 2019, 49, 381–388. [Google Scholar] [CrossRef] [Green Version]
  48. Schnyder, S.; Handschin, C. Skeletal muscle as an endocrine organ: PGC-1α, myokines and exercise. Bone 2015, 80, 115–125. [Google Scholar] [CrossRef] [Green Version]
  49. Nardone, O.M.; de Sire, R.; Petito, V.; Testa, A.; Villani, G.; Scaldaferri, F.; Castiglione, F. Inflammatory Bowel Diseases and Sarcopenia: The Role of Inflammation and Gut Microbiota in the Development of Muscle Failure. Front. Immunol. 2021, 12, 2783. [Google Scholar] [CrossRef]
  50. Roviello, G.; Iannone, L.F.; Bersanelli, M.; Mini, E.; Catalano, M. The gut microbiome and efficacy of cancer immunotherapy. Pharmacol. Ther. 2021, 107973, Epub ahead of printing. [Google Scholar] [CrossRef]
  51. Guzman-Prado, Y.; Ben Shimol, J.; Samson, O. Sarcopenia and the risk of adverse events in patients treated with immune checkpoint inhibitors: A systematic review. Cancer Immunol. Immunother. 2021, 70, 2771–2780. [Google Scholar] [CrossRef]
  52. Rutten, I.J.G.; Ubachs, J.; Kruitwagen, R.; Beets-Tan, R.G.H.; Olde Damink, S.W.M.; Van Gorp, T. Psoas muscle area is not representative of total skeletal muscle area in the assessment of sarcopenia in ovarian cancer. J. Cachexia Sarcopenia Muscle 2017, 8, 630–638. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Jones, K.I.; Doleman, B.; Scott, S.; Lund, J.N.; Williams, J.P. Simple psoas cross-sectional area measurement is a quick and easy method to assess sarcopenia and predicts major surgical complications. Colorectal Dis. 2015, 17, O20–O26. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Flow diagram of the study selection process. Abbreviation: HR, hazard ratios.
Figure 1. Flow diagram of the study selection process. Abbreviation: HR, hazard ratios.
Jcm 10 05329 g001
Figure 2. Forest plots of hazard ratios of sarcopenia on overall survival in cancer patients treated with immune checkpoint inhibitors. (A) Overall survival in the univariate analysis. (B) Overall survival with adjusted HRs.
Figure 2. Forest plots of hazard ratios of sarcopenia on overall survival in cancer patients treated with immune checkpoint inhibitors. (A) Overall survival in the univariate analysis. (B) Overall survival with adjusted HRs.
Jcm 10 05329 g002
Figure 3. Forest plots of hazard ratios of sarcopenia on progression-free survival in cancer patients treated with immune checkpoint inhibitors. (A) Progression-free survival in the univariate analysis. (B) Progression-free survival with adjusted HRs.
Figure 3. Forest plots of hazard ratios of sarcopenia on progression-free survival in cancer patients treated with immune checkpoint inhibitors. (A) Progression-free survival in the univariate analysis. (B) Progression-free survival with adjusted HRs.
Jcm 10 05329 g003
Table 1. Characteristics of the studies included in the meta-analysis.
Table 1. Characteristics of the studies included in the meta-analysis.
Author, YearCountryCancerStage Time Point of CT Exam aICI TypeMeasurement of SarcopeniaCut-Off Value of Sarcopenia bNo. of PatientsMedian Age of PatientsNo. of Sarcopenia(%)
Minami 2020 [14]JapanNSCLCAdvanced90 daysNivolumab, Pembrolizumab, AtezolizumabPMIMale:6.36, Female:3.92747053(71)
Magri 2019 [19]ItalyNSCLCStage IV10 weeksNivolumabSMINA4665NA
Roch 2020 [32]FranceNSCLCMetastaticNANivolumab, PembrolizumabSMIMale: 52.4, Female: 38.51426492(66)
Shiroyama 2019 [33]JapanNSCLCAdvanced90 daysNivolumab, PembrolizumabPMIMale:6.36, Female:3.92427122(52)
Takada 2020 [34]JapanNSCLCStage III, IV/RecurredNANivolumab, PembrolizumabSMIMale: 25.63, Female: 21.731036751(49)
Tsukagoshi 2020 [35]JapanNSCLCStage III, IV30 daysNivolumabPMIMale:6.36, Female:3.92306713(43)
Akce 2020 [36]USAHCCAdvanced2 monthsAnti PD-1, Anti-PD-1 + others(not specified)SMIMale: 43, Female: 39576628(49)
Kim N 2020 [37]KoreaHCCAdvancedNANivolumabSMIMale: 42, Female: 381026123(23)
Chu 2020 [38]CanadaMelanomaMetastatic/ advanced30 daysIpilimumabSMIMale: 43(52 c), Female: 419756NA
Young 2020 [39]USAMelanomaMetastatic/ advanced6 monthsNivolumab, Pembrolizumab, Atezolizumab, Ipilimunab + nivolumabSMIMale: 43(52 c), Female: 4128763154(54)
Shimizu 2020 [13]JapanUrothelial carcinomaMetastatic/ advancedNAPembrolizumabPMIMale:6.36, Female:3.92277315(56)
Fukushima 2020 [40]JapanUrothelial carcinomaAdvanced30 daysPembrolizumabSMIMale: 43(52 c), Female: 41287419(68)
Kim Y 2020 [41]KoreaGastric cancerMetastatic3 monthsNivolumab, PembrolizumabSMIMale: 49, Female: 311495779(53)
Cortellini 2020 [12]ItalyNSCLC, Melanoma, RCC, othersAdvanced90 daysPembrolizumab, Nivolumab, Atezolizumab, and othersSMIMale: 48.4(50.2 c), Female: 36.9(59.6 c)1006650(50)
Abbreviations: ICI, immune-checkpoint inhibitor; NSCLC, non-small cell lung cancer; HCC, hepatocellular carcinoma; NA, not available; SMI, skeletal muscle index; PMI, psoas muscle index; a: time within initiation of ICI therapy; b: cm2 /m2; c: patient with body mass index >25kg/m2.
Table 2. The Newcastle-Ottawa Scale (NOS) of the studies included in the meta-analysis.
Table 2. The Newcastle-Ottawa Scale (NOS) of the studies included in the meta-analysis.
SelectionComparabilityOutcomeTotal
Author,
Year
Representativeness of the Exposed CohortSelection of the Non-Exposed CohortAscertainment of ExposureDemonstration That Outcome of Interest Was Not Present at Start of StudyComparability of Cohorts on the Basis of the Design or Analysis aAssessment of Outcome Was Follow-Up Long Enough for Outcomes to Occur bAdequacy of Follow Up of Cohorts c
Young 2020 [39]11110 1117
Minami 2020 [14]111101117
Shimizu 2020 [13]111101117
Shiroyama 2019 [33]111101117
Kim Y 2020 [41]111101117
Cortellini 2020 [12]11110110 6
Roch 2020 [32]11110110 6
Fukushima 2020 [40]11110110 6
Takada 2020 [34]11110110 6
Akce 2020 [36]11110110 6
Chu 2020 [38]111101016
Kim N 2020 [37]11110110 6
Magri 2019 [19]11110110 6
Tsukagoshi 2020 [35]111101005
a: None of the included studies adequately controlled confounders. b: If individual studies specified follow-up duration, we assessed long enough for outcomes to occur. c: If individual studies lost more than 30% of the included patients or did not specify those lost, we assessed as inadequate.
Table 3. Results of GRADE certainty assessment for evidence of impact of sarcopenia in patients treated with immune checkpoint inhibitors.
Table 3. Results of GRADE certainty assessment for evidence of impact of sarcopenia in patients treated with immune checkpoint inhibitors.
Certainty AssessmentEffectCertainty
No of StudiesStudy DesignRisk of BiasInconsistencyIndirectnessImprecisionOther ConsiderationsRelative (95% CI)
Overall survival in univariate analysis
8 observational studies serious anot serious bnot serious not serious none HR 1.66
(1.20 to 2.29)
⨁◯◯◯
VERY LOW
Overall survival with adjusted HRs
9 observational studies serious anot serious serious cnot serious none HR 1.55
(1.15 to 2.10)
⨁◯◯◯
VERY LOW
Progression free survival in univariate analysis
9 observational studies serious anot serious not serious not serious none HR 1.75
(1.37 to 2.23)
⨁◯◯◯
VERY LOW
Progression free survival with adjusted HRs
10 observational studies serious anot serious serious cnot serious none HR 1.63
(1.28 to 2.09)
⨁◯◯◯
VERY LOW
Explanations: CI: confidence interval; HR: hazard ratio; a. Failure to adequately control confounding; b. Not downgraded for inconsistency despite substantial heterogeneity given its likely not clinical meaningful; c. Indirectness on the outcome in concept of adjusted HRs.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lee, D.; Kim, N.W.; Kim, J.Y.; Lee, J.H.; Noh, J.H.; Lee, H.; Jeong, J.W.; Lee, S.; Kang, J. Sarcopenia’s Prognostic Impact on Patients Treated with Immune Checkpoint Inhibitors: A Systematic Review and Meta-Analysis. J. Clin. Med. 2021, 10, 5329. https://doi.org/10.3390/jcm10225329

AMA Style

Lee D, Kim NW, Kim JY, Lee JH, Noh JH, Lee H, Jeong JW, Lee S, Kang J. Sarcopenia’s Prognostic Impact on Patients Treated with Immune Checkpoint Inhibitors: A Systematic Review and Meta-Analysis. Journal of Clinical Medicine. 2021; 10(22):5329. https://doi.org/10.3390/jcm10225329

Chicago/Turabian Style

Lee, Donggun, Na Won Kim, Jong Yeob Kim, Joo Hyung Lee, Ji Hyun Noh, Haejun Lee, Jin Woon Jeong, Seungeun Lee, and Jeonghyun Kang. 2021. "Sarcopenia’s Prognostic Impact on Patients Treated with Immune Checkpoint Inhibitors: A Systematic Review and Meta-Analysis" Journal of Clinical Medicine 10, no. 22: 5329. https://doi.org/10.3390/jcm10225329

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop