Efficacy of Cancer Immunotherapy: An Umbrella Review of Meta-Analyses of Randomized Controlled Trials

We conducted a systematic review for evidence of the clinical efficacy of cancer immunotherapies. We searched PubMed from inception to 14 February 2018 for meta-analyses of randomized controlled trials (RCTs) of cancer immunotherapies. Re-analyses were performed to estimate the summary effect size under random-effects, the 95% confidence interval (CI), heterogeneity, and the 95% prediction interval, and we determined the strength of the evidence. We examined publication bias and excess significance bias. 63 articles corresponding to 247 meta-analyses were eligible. Nine meta-analyses were classified to have convincing evidence, and 75 were classified as suggestive evidence. The clinical benefit of immunotherapy was supported by convincing evidence in the following settings: anti-PD-1/PD-L1 monoclonal antibody (mAb) therapy for treating advanced melanoma and non-small cell lung cancer (NSCLC), the combination of rituximab and chemotherapy for treating chronic lymphocytic leukemia and B-cell non-Hodgkin’s lymphoma, adoptive cell immunotherapy for NSCLC, and the combination of interferon α and chemotherapy for metastatic melanoma. A further meta-analysis of 16 RCTs showed that anti-PD-1/PD-L1 mAb therapy had a benefit in patients with solid tumors (overall survival; hazard ratio = 0.73, 95% CI: 0.68–0.79; p < 0.001), supported by convincing evidence. In the future, rigorous approaches are needed when interpreting meta-analyses to gain better insight into the true efficacy of cancer immunotherapy.

. Flow chart of literature search.      Figure S8. Level of evidence algorithm *When a meta-analysis had no evidence of publication bias or ES but had high in-between study heterogeneity (I 2 > 50), we rechecked the results of its component studies to find out whether high heterogeneity was due to the differences in the direction of effects or due to the differences in the size of the associations. When the number of statistically significant component studies was the same or greater than the number of studies which were not significant or significant in the opposite direction, the comparison was classified as suggestive evidence, or convincing evidence if further criteria were met.

101(41% )
Analyses of RCTs plus non- RCTs   6  35  288  --34,326  ---------------------Abbreviations: N., number; IQR, interquartile range; NA, not available; mAB, monoclonal antibody; ACI, adoptive cell immunotherapy; DC/CIK, dendritic cells with cytokine-induced killer cells; CIK, cytokine-induced killer cells; IFN-α, interferon alpha; IL-2, interleukin-2; DC, dendritic cells; IT, immunotherapy; RCT, randomized controlled trials. * 4 results from one article showed weak evidence in favor of the control therapy. † Value is rounded to the nearest unit.     Abbreviations: RCT, randomized controlled trial; N., number; SE, standard effect; CI, confidence interval; M, model; F, fixed effect; R, random effect; NA, not available; C, concordance with largest study; Y, concordant with largest study; N, not concordant with largest study; OS, overall survival; PFS progression-free survival; RR, risk ratio; HR, hazard ratio; OR, odds ratio; IFN-α, interferon alpha; 5FU, fluorouracil; IFN, interferon; p, placebo; CTx, chemotherapy; CT, conventional therapy; obs, observation; VAX, cancer vaccine; IL-2, interleukin; IFN-γ, interferon gamma; AT, adjuvant therapy; HCC, hepatocellular carcinoma; NSCLC, nonsmall cell lung cancer; RCC, renal cell carcinoma. * Value reported in original article of the meta-analysis. † Number of individual studies of effect size with statistical significance in the reverse direction/not statistically significant/statistically significant. ‡ Concordance of fixed and random effects summary outcome with outcome of largest individual study. § Value obtained from re-analysis of original meta-analysis. ES is estimated by assuming power of each study could be replaced by power of the study with most cases and controls. All p-Values are two-sided.

No association
Abbreviations: RCT, randomized controlled trial; N., number; SE, standard effect; CI, confidence interval; M, model; F, fixed effect; R, random effect; NA, not available; C, concordance with largest study; Y, concordant with largest study; N, not concordant with largest study; OS, overall survival; PFS progression-free survival; RR, risk ratio; HR, hazard ratio; OR, odds ratio; DC, dendritic cell based vaccine; P, placebo; VAX, cancer vaccine; CTx, chemotherapy; obs, observation; P, placebo; AT, adjuvant therapy; CT, conventional therapy; NSCLC, non-small cell lung cancer; RCC, renal cell carcinoma. * Value reported in original article of the meta-analysis. † Number of individual studies of effect size with statistical significance in the reverse direction/not statistically significant/statistically significant. ‡ Concordance of fixed and random effects summary outcome with outcome of largest individual study. § Value obtained from re-analysis of original meta-analysis. Re-analysis was performed after excluding non-RCTs from original meta-analysis. All p-Values are two-sided. Abbreviations: RCT, randomized controlled trial; N., number; SE, standard effect; CI, confidence interval; M, model; F, fixed effect; R, random effect; NA, not available; C, concordance with largest study; Y, concordant with largest study; N, not concordant with largest study; OS, overall survival; PFS progression-free survival; RR, risk ratio; HR, hazard ratio; OR, odds ratio; AT, adjuvant therapy; mAB, monoclonal antibodies; VAX, cancer vaccine; IL-2, interleukin-2; CIK, cytokineinduced killer cells; DC/CIK, dendritic cells with cytokine-induced killer cells; GM-CSF, Granulocyte-macrophage colony-stimulating factor; DC, dendritic cell; p, placebo; IFN, interferon; HCC, hepatocellular carcinoma; NSCLC, non-small cell lung cancer. * Value reported in original article of the meta-analysis. † Number of individual studies of effect size with statistically significant in reverse direction/not statistically significant/statistically significant. ‡ Concordance of fixed and random effects summary outcome with outcome of largest individual study. § Value obtained from re-analysis of original meta-analysis. Re-analysis was performed after excluding non-RCTs from original meta-analysis. All p-Values are two-sided.  Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale. 14 Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched. 14 Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated. 14 Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis). 14, figure S1 Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators. 14-15 Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made. 14-15

15
Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified.

Study selection 17
Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.

2-3
Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.