Next Article in Journal
Implications for Immunotherapy of Breast Cancer by Understanding the Microenvironment of a Solid Tumor
Next Article in Special Issue
Prognostic Value of KRAS Mutations in Colorectal Cancer Patients
Previous Article in Journal
Postoperative Delirium in Patients with Oral Cancer: Is Intraoperative Fluid Administration a Neglected Risk Factor?
Previous Article in Special Issue
Role of Radiology in the Preoperative Detection of Arterial Calcification and Celiac Trunk Stenosis and Its Association with Anastomotic Leakage Post Esophagectomy, an Up-to-Date Review of the Literature
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 14
Issue 13
10.3390/cancers14133177
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Robot-Assisted Minimally Invasive Esophagectomy versus Open Esophagectomy for Esophageal Cancer: A Systematic Review and Meta-Analysis

by
Stepan M. Esagian
1,
Ioannis A. Ziogas
1,
Konstantinos Skarentzos
1,
Ioannis Katsaros
1,2,
Georgios Tsoulfas
3,
Daniela Molena
4,
Michalis V. Karamouzis
5,
Ioannis Rouvelas
6,7,
Magnus Nilsson
6,7 and
Dimitrios Schizas
2,*
1
Surgery Working Group, Society of Junior Doctors, 151-23 Athens, Greece
2
First Department of Surgery, National and Kapodistrian University of Athens, Laikon General Hospital, 115-27 Athens, Greece
3
First Department of Surgery, Aristotle University of Thessaloniki, 541-24 Thessaloniki, Greece
4
Thoracic Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA
5
Molecular Oncology Unit, Department of Biological Chemistry, National and Kapodistrian University of Athens, 115-27 Athens, Greece
6
Division of Surgery, Department of Clinical Science, Intervention and Technology (CLINTEC), Karolinska Institutet, 141-86 Stockholm, Sweden
7
Department of Upper Abdominal Diseases, Karolinska University Hospital, 171-77 Stockholm, Sweden
*
Author to whom correspondence should be addressed.
Cancers 2022, 14(13), 3177; https://doi.org/10.3390/cancers14133177
Submission received: 17 February 2022 / Revised: 12 June 2022 / Accepted: 23 June 2022 / Published: 29 June 2022
(This article belongs to the Special Issue Updates on the Treatment of Gastroesophageal Cancer)
Browse Figures
Review Reports Versions Notes

Abstract

:

Simple Summary

Robot-assisted minimally invasive esophagectomy (RAMIE) constitutes a newly developed surgical technique for the treatment of resectable esophageal cancer, aiming to further improve the high morbidity and mortality associated with open esophagectomy. We performed a systematic review of the literature and compared the outcomes of RAMIE and open esophagectomy. RAMIE is a safe and feasible procedure, resulting in decreased cardiopulmonary morbidity, wound infections, blood loss, and hospital stays compared to open esophagectomy.

Abstract

Robot-assisted minimally invasive esophagectomy (RAMIE) was introduced as a further development of the conventional minimally invasive esophagectomy, aiming to further improve the high morbidity and mortality associated with open esophagectomy. We aimed to compare the outcomes between RAMIE and open esophagectomy, which remains a popular approach for resectable esophageal cancer. Ten studies meeting our inclusion criteria were identified, including five retrospective cohort, four prospective cohort, and one randomized controlled trial. RAMIE was associated with significantly lower rates of overall pulmonary complications (odds ratio (OR): 0.38, 95% confidence interval (CI): [0.26, 0.56]), pneumonia (OR: 0.39, 95% CI: [0.26, 0.57]), atrial fibrillation (OR: 0.53, 95% CI: [0.29, 0.98]), and wound infections (OR: 0.20, 95% CI: [0.07, 0.57]) and resulted in less blood loss (weighted mean difference (WMD): −187.08 mL, 95% CI: [−283.81, −90.35]) and shorter hospital stays (WMD: −9.22 days, 95% CI: [−14.39, −4.06]) but longer operative times (WMD: 69.45 min, 95% CI: [34.39, 104.42]). No other statistically significant difference was observed regarding surgical and short-term oncological outcomes. Similar findings were observed when comparing totally robotic procedures only to OE. RAMIE is a safe and feasible procedure, resulting in decreased cardiopulmonary morbidity, wound infections, blood loss, and shorter hospital stays compared to open esophagectomy.

1. Introduction

Esophagectomy is the mainstay of treatment for resectable esophageal cancer [1]. The technical optimization of the procedure over the years has led to improved short- and long-term outcomes [2]. Despite this progress, esophagectomy is still a highly invasive and technically complex operation with considerable morbidity and mortality [3]. This aspect remains problematic, as postoperative complications are strongly associated with worse long-term outcomes and a higher cost of care [4,5]. Minimally invasive esophagectomy (MIE) has emerged as an alternative to the conventional open esophagectomy (OE) and has become more broadly adopted after early case series demonstrated lower complication and mortality rates compared to OE [6,7]. The TIME (traditional invasive vs. minimally invasive esophagectomy) trial and several meta-analyses validated these preliminary findings and convincingly showed that MIE, at least in the context of selected high-volume centers, is superior to OE regarding short- and long-term outcomes [8,9,10]. However, evidence from population-based cohort studies is less clear and in some cases even suggests worse short-term outcomes after MIE [11].
Minimally invasive esophagectomy is often used as an umbrella term for many different approaches, including the conventional thoracoscopic/laparoscopic MIE, hybrid MIE, and robot-assisted minimally invasive esophagectomy (RAMIE) [12]. More specifically, RAMIE was introduced as a solution to the inherent limitations and technical difficulties of conventional MIE, such as the most commonly used two-dimensional operative field vision, the limited surgical instrument mobility, and the well-documented long learning curve [13,14]. Since its implementation into clinical practice, large case series have consistently shown that RAMIE is safe and feasible with favorable short- and long-term outcomes [15,16,17,18], while a recent meta-analysis suggested that RAMIE is equivalent to conventional MIE [19].
Although RAMIE and conventional MIE have both demonstrated promising outcomes, OE is still a commonly utilized approach in many centers for the management of resectable esophageal cancer [20]. The results of the ROBOT randomized controlled trial showed that RAMIE was superior to OE for numerous outcomes of interest [21]. However, no systematic comparison between RAMIE and OE has been conducted to date. Thus, the aim of the current study is to perform a systematic review and meta-analysis to compare the short-term surgical and oncological outcomes between RAMIE and OE in patients with esophageal cancer, as the proposed benefits of RAMIE may soon lead to a shift in the standard of care for patients with resectable disease.

2. Materials and Methods

2.1. Study Design and Inclusion/Exclusion Criteria

This systematic review and meta-analysis was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines and in line with the protocol developed and agreed by all authors (Supplemental Table S1) [22]. Our systematic review was registered at Open science (https://osf.io/prereg/ (accessed on 3 May 2022)) with a registration number of 10.17605/OSF.IO/XEPHB. The study selection criteria were defined by applying the PICO (Population/Participants, Intervention, Comparison, and Outcome) framework:
  • Participants: Patients of any race, age, or sex undergoing esophagectomy for esophageal cancer.
  • Interventions: RAMIE and OE. Studies were included independent of the surgical approach (i.e., transthoracic (Ivor-Lewis or McKeown), or transhiatal). Procedures were identified as RAMIE if the robotic approach was utilized for the thoracic phase of the operation, irrespective of the abdominal phase approach. The procedures were further classified as a) totally robotic (TRAMIE) if the robotic approach was utilized for both the thoracic and the abdominal part of the operation and b) hybrid if the robotic approach was utilized for the thoracic part of the operation or both types of the aforementioned approaches were included in a single sample.
  • Comparison: Studies were deemed eligible only if RAMIE was directly compared to OE.
  • Outcomes: The primary surgical outcome measures were the rates of overall complications, overall pulmonary complications, anastomotic leakage, 30-day mortality, and 90-day mortality. The primary oncological outcomes were the number of total lymph nodes resected, the margin-negative resection (R0) rate, the overall survival, (OS) and the disease-free survival (DFS). The secondary outcome measures were total operative time, estimated blood loss (EBL), hospital length of stay (LOS), and length of intensive care unit (ICU) stay as well the rates of pneumonia, acute respiratory distress syndrome (ARDS), postoperative hemorrhage, chylothorax, recurrent laryngeal nerve (RLN) palsy, atrial fibrillation, and wound infection.
Original clinical studies, including both randomized trials and non-randomized prospective/retrospective comparative studies, published in English, reporting on both RAMIE and OE for the outcomes of interest, were deemed eligible for inclusion. The exclusion criteria for the present systematic review were: (i) non-English language articles, (ii) irrelevant articles, (iii) non-comparative studies (<2 study arms), (iv) studies not directly comparing RAMIE and OE for the outcomes of interest, (v) animal and in-vitro studies, (vi) case reports, (vii) narrative or systematic reviews and meta-analyses, (viii) editorials, letters to the editor, perspectives, comments, and errata that did not provide any primary patient data, and (ix) published abstracts with no published full text. No search filters were applied.
All eligible studies were assessed for overlap based on the author list, center and country of origin, and dates of patient enrollment. Between studies with overlapping populations, we included those with the largest number of patients or reporting granular data on the outcomes of interest. On one occasion, analyses on additional outcomes were presented in two eligible articles; hence, data were extracted from both, but their population was not summed in the overall cohort numbers, as they constituted additional analyses on the same subjects [23,24]. Data from national registry studies were qualitatively assessed separately and were excluded from the final data synthesis to avoid overlap with single-center studies from the same country.

2.2. Literature Search Strategy

Eligible studies were identified by searching through the MEDLINE (via PubMed), Scopus, and Cochrane Library databases (end-of-search date: 2 May 2020) by two independent reviewers. The following search algorithm was used: ((robot-assisted OR robotic OR RAMIE OR minimally invasive) OR (transthoracic or open)) AND (oesophagectomy or esophagectomy). Any disagreements on article inclusion were resolved by a third reviewer when necessary. The reference lists of the included studies were also thoroughly searched to identify eligible, missed studies based on the “snowball” methodology [25].

2.3. Data Tabulation and Extraction

Data tabulation and extraction from the eligible records were performed using a standardized, pre-piloted form. Two independent reviewers extracted the data, and any disagreements were identified and resolved after the involvement of a third reviewer when necessary. Whenever a study provided data for multiple esophagectomy approaches, including those that were not of interest to this study (i.e., conventional MIE), we only extracted data pertaining to OE and RAMIE. The following data were collected: (i) study characteristics (first author, year of publication, study design, study center, study period for each intervention, and number of patients for each study group), (ii) patient characteristics (age in years, sex, body mass index (BMI) in kg/m2, prior comorbidities according to the Charlson–Deyo score or the American Society of Anesthesiologists (ASA) classification, clinical stage and tumor location, histology, degree of differentiation, and size in cm), (iii) primary surgical outcomes (overall complication rate, overall pulmonary complication rate, anastomotic leakage rate, 30-day mortality rate, and 90-day mortality rate), (iv) primary oncological outcomes (total number of lymph nodes resected and R0 resection rate), and (v) secondary outcomes (operative time in minutes, EBL in mL, ICU length of stay in days, LOS in days, pneumonia rate, ARDS rate, postoperative hemorrhage rate, chylothorax rate, RLN palsy rate, atrial fibrillation rate, and wound infection rate). Overall complication and overall pulmonary complication rates were defined as the number of patients who had at least one complication or one pulmonary complication, respectively. The umbrella term “overall pulmonary complications” included pneumonia, pneumothorax, pulmonary embolism, ARDS, and pleural effusion.

2.4. Quality of Evidence Assessment

An assessment of study quality for non-randomized studies was performed with the Newcastle–Ottawa scale (NOS) [26]; a score of ≥6 denoted high study quality. In the item assessing whether follow-up was long enough for outcomes to occur, the cut-off was a priori set at 90 days after esophagectomy, while regarding the item about follow-up adequacy, a rate of 90% was also adopted a priori.
For randomized controlled trials (RCTs), quality of evidence assessment was conducted with the Cochrane Collaboration’s tool, which assesses for selection, performance, detection, attrition, and reporting biases [27].

2.5. Statistical Analysis

2.5.1. Data Pooling

Categorical variables were summarized as frequencies and percentages, while continuous variables were summarized as means and standard deviations (SDs). When continuous data were presented as medians and ranges or interquartile ranges, we estimated the respective means and SDs by applying the methods described by Hozo et al. and Wan et al. [28,29]. We estimated all relative rates based on the available data for the variables of interest, and all data handling was performed according to the principles of the Cochrane Handbook [30].

2.5.2. Meta-Analysis

A meta-analysis was carried out to compare RAMIE and OE for all primary surgical, primary oncological, and secondary outcomes. Based on the extracted data, odds ratios (ORs) and 95% confidence intervals (CIs) were calculated by means of 2 × 2 tables for each categorical outcome; OR > 1 indicated that the outcome was more frequently present in the RAMIE group. A continuity correction of 0.5 in studies with zero cell frequencies was adopted [31,32]. Weighted mean differences (WMD) and 95% CIs were estimated for each continuous outcome; WMD > 0 corresponded to larger values in the RAMIE group. Between-study heterogeneity was assessed through the Cochran Q statistic and by estimating I2. High heterogeneity was confirmed with a significance level of p < 0.05 and I2 ≥ 50%. Due to the significant between-study clinical heterogeneity, we used the random-effects model (DerSimonian–Laird) to calculate the pooled effect estimates for all outcomes [33]. An assessment of publication bias was conducted by means of funnel plots for each outcome of interest. Statistical significance was set at 0.05, and all p-values were two-tailed. Subgroup analyses were performed according to the characteristics of the RAMIE technique in order to compare the TRAMIE and hybrid approaches, as previously defined. Statistical analyses were performed using STATA IC 16.0 (StataCorp LLC, College Station, TX, USA).

3. Results

3.1. Study Selection and Characteristics

Through our systematic search, 3479 unique articles were retrieved, of which 111 underwent full-text evaluation for eligibility. Ultimately, 10 studies reporting on 1977 patients undergoing esophagectomy for esophageal cancer (674 RAMIE and 1303 open) fulfilled the inclusion criteria and were included in our quantitative data synthesis (Figure 1) [21,23,34,35,36,37,38,39,40,41]. Three analyses of the National Cancer Database (NCDB) (USA) were identified [42,43,44]; among these, we selected the one reporting on the largest number of patients and the pre-specified outcomes of interest to serve as a measure of comparison for the results of our systematic review [42]. Six of the included studies (60%) were published in 2019 or 2020. Detailed study and patient characteristics of the included studies are presented in Table 1 and Table 2, respectively.

3.2. Study Quality and Publication Bias Assessment

Of the 10 included studies in the final data synthesis, 9 were non-randomized studies (5 retrospective and 4 prospective), and 1 was a randomized controlled trial. The nine non-randomized studies were assessed using the NOS, with a mean score of 6.9 ± 1.45 (Supplemental Table S2).
One randomized controlled trial was included in the final analysis and was assessed separately using the Cochrane Collaboration’s tool. A low risk of bias was detected for selection, attrition, and reporting bias. Performance and detection bias could not be assessed, as blinding of the medical personnel performing the operations and assessing the outcomes is not possible in trials comparing surgical procedures (Supplemental Table S3).
The funnel plots assessing for publication bias are presented in Supplemental Figure S1 for the primary surgical and oncological outcomes and in Supplemental Figure S2 for the secondary outcomes. No publication was evident through visual assessment of the funnel plots.

3.3. Primary Surgical Outcomes

The results of the meta-analyses are summarized in Table 3.

3.3.1. Overall Complication Rate

The overall complication rate was reported in five studies [24,34,36,37,41]. No statistically significant difference was found between the RAMIE group (27.88%; n = 109/391) and the OE group (33.93%; n = 303/893) (OR: 0.66, 95% CI: 0.42–1.05; p = 0.08). The statistical heterogeneity was high (I2 = 50.17%). No statistically significant difference was detected between TRAMIE and OE (OR: 0.81, 95% CI: 0.49–1.34) (Figure 2A).

3.3.2. Overall Pulmonary Complication Rate

The overall pulmonary complication rate was reported in six studies [21,35,37,38,40,41] and was significantly lower in the RAMIE group (14.29%; n = 49/343) compared to the OE group (25.32%; n = 174/687) (OR: 0.38, 95% CI: 0.26–0.56; p < 0.001). The statistical heterogeneity was low (I2 = 0.00%). TRAMIE was also associated with a significantly lower overall pulmonary complication rate (OR: 0.38; 95% CI: 0.24–0.60) (Figure 2B).

3.3.3. Anastomotic Leakage Rate

The anastomotic leakage rate was reported in 10 studies [21,23,34,35,36,37,38,39,40,41]. No statistically significant difference was found between the RAMIE group (6.82%; n = 46/674) and the OE group (6.06%; n = 79/1303) (OR: 0.93, 95% CI: 0.60–1.44; p = 0.76). The statistical heterogeneity was low (I2 = 0.00%). No statistically significant difference was detected between TRAMIE and OE (OR: 0.91, 95% CI: 0.45–1.44) (Figure 2C).

3.3.4. Thirty-Day Mortality Rate

The thirty-day mortality rate was reported in three studies [21,23,35]. No statistically significant difference was found between the RAMIE group (0.81%; n = 2/248) and the OE group (1.49%; n = 6/402) (OR: 0.75, 95% CI: 0.15–3.78; p = 0.73). The statistical heterogeneity was low (I2 = 0.00%) (Figure 2D).
In comparison, the NCDB study by Weksler et al. reported a 30-day mortality rate of 5.6% for the RAMIE group and 2.7% for the OE group (p = 0.060) [42].

3.3.5. Ninety-Day Mortality Rate

The ninety-day mortality rate was reported in six studies [21,24,34,35,37,41]. No statistically significant difference was found between the RAMIE group (2.59%; n = 10/421) and the OE group (2.239%; n = 18/808) (OR: 0.80, 95% CI: 0.31–2.05; p = 0.64). The statistical heterogeneity was low (I2 = 0.00%). No statistically significant difference was detected between TRAMIE and OE (OR: 0.57, 95% CI: 0.17–1.86) (Figure 2E).
In comparison, the NCDB study by Weksler et al. reported a 90-day mortality rate of 8.1% for the RAMIE group and 6.4% for the OE group (p = 0.399) [42].

3.4. Primary Oncological Outcomes

3.4.1. Total Lymph Nodes Resected

The number of total lymph nodes resected was reported in eight studies [21,23,34,35,37,38,40,41]. No statistically significant difference was found between the RAMIE group (28.45 ± 14.71 lymph nodes resected) and the OE group (21.44 ± 15.53 lymph nodes resected) (WMD: 3.26 lymph nodes resected, 95% CI: −0.89–7.41; p = 0.12). The statistical heterogeneity was high (I2 = 91.41%). No statistically significant difference was detected between TRAMIE and OE (WMD: 5.01 lymph nodes resected, 95% CI: −0.93–10.96) (Figure 3A).
In comparison, the NCDB study by Weksler et al. reported that the mean number of resected lymph nodes was 16.25 ± 2.17 in the RAMIE group and 13.25 ± 2.17 in the OE group (p = 0.087) [42].

3.4.2. R0 Resection Rate

The R0 resection rate was reported in six studies [21,23,34,35,37,39]. No statistically significant difference was found between the RAMIE group (98.02%; n = 495/505) and the OE group (95.52%; n = 1109/1161) (OR: 1.36, 95% CI 0.56–3.32; p = 0.85). The statistical heterogeneity was low (I2 = 18.07%). No statistically significant difference was detected between TRAMIE and OE (OR: 1.20, 95% CI: 0.34–4.28) (Figure 3B).
In comparison, the NCDB study by Weksler et al. reported a R0 resection rate of 95.1% (n = 541/569) for the RAMIE group and 93.1% (n = 530/569) for the OE group (p = 0.165) [42].

3.4.3. Overall Survival

The OS was reported by two studies [21,23]. Yun et al. reported a 1-year OS of 95.1% and 85.6% and a 3-year OS of 81.7% and 73.7% for the RAMIE and OE groups, respectively [23]. Van der Sluis et al. had a follow-up of 40 months, during which the median OS was not reached for either group [21].
In comparison, the NCDB study by Weksler et al. reported a median OS of 48 months (95% CI: 34–55 months) for the RAMIE group and 44 months (95% CI: 38–53 months) for the OE group (p = 0.53) [42].

3.4.4. Disease-Free Survival

The DFS was reported by two studies [21,23]. Yun et al. reported a 1-year DFS of 54.4% and 53.2% and a 3-year OS of 49.2% and 45.6% for the RAMIE and OE groups, respectively [23]. Van der Sluis reported a median DFS of 26 months for the RAMIE group and 28 months for the OE group [21].

3.5. Secondary Outcomes

3.5.1. Operative Time

The operative time was reported in eight studies [21,23,34,35,36,37,38,39] and was significantly longer in the RAMIE group (360.39 ± 115.44 min) compared to the OE group (306.21 ± 94.58 min) (WMD: 69.45 min, 95% CI: 34.39–104.42; p < 0.001). The statistical heterogeneity was high (I2 = 96.58%). TRAMIE was also associated with significantly longer operative times (WMD: 69.50 min, 95% CI: 13.74–125.25) (Figure 4A).

3.5.2. Estimated Blood Loss

Estimated blood loss was reported in eight studies [21,23,34,35,36,37,38,39] and was significantly lower in the RAMIE group (209.59 ± 169.02 mL) compared to the OE group (374.38 ± 415.34 mL) (WMD: −187.08 mL, 95% CI: −283.81–(−90.35); p < 0.001). The statistical heterogeneity was high (I2 = 95.50%). TRAMIE was also associated with significantly lower estimated blood loss (WMD: −296.94 mL, 95% CI: −503.03–(−90.85)) (Figure 4B).

3.5.3. ICU Length of Stay

The intensive care unit length of stay was reported in three studies [21,23,36]. No statistically significant difference was found between the RAMIE group (1.37 ± 0.56 days) and the OE group (1.59 ± 1.55 days) (WMD: −0.13 days, 95% CI: −0.28–0.02); p = 0.09). The statistical heterogeneity was low (I2 = 27.67%) (Figure 4C).

3.5.4. Length of Hospital Stay

The length of hospital stay was reported in nine studies [21,23,34,35,36,37,38,39,41] and was significantly shorter in the RAMIE group (17.10 ± 9.39 days) compared to the OE group (30.68 ± 23.88 days) (WMD: −9.22 days, 95% CI: −14.39–(−4.06); p < 0.001). The statistical heterogeneity was high (I2 = 96.13%). TRAMIE was also associated with significantly a shorter length of hospital stay (WMD: −20.35, 95% CI: −34.75–(−5.95) (Figure 4D).

3.5.5. Pneumonia Rate

The rate of pneumonia was reported in six studies [21,23,34,36,37,39] and was significantly lower in the RAMIE group (7.94%; n = 42/529) compared to the OE group (14.90%; n = 181/1146, 15.79%) (OR: 0.39, 95% CI: 0.26–0.57; p < 0.001). The statistical heterogeneity was low (I2 = 0.00%). The rate of pneumonia was also significantly lower in the TRAMIE group compared to the OE group (OR: 0.43, 95% CI: 0.23–0.80) (Figure 5A).

3.5.6. ARDS Rate

The rate of ARDS was reported in three studies [21,23,41]. No statistically significant difference was found between the RAMIE group (0.46%; n= 1/217) and the OE group (1.34%; n = 4/298, 1.34%) (OR: 0.27, 95% CI: 0.04–1.73; p = 0.17). The statistical heterogeneity was low (I2 = 0.00%) (Figure 5B).

3.5.7. Atrial Fibrillation Rate

The rate of atrial fibrillation was reported in five studies [21,23,34,35,36] and was significantly lower in the RAMIE group (6.79%, n = 29/427) compared to the OE group (8.46%, n = 54/638) (OR: 0.53, 95% CI: 0.29–0.98; p = 0.04). The statistical heterogeneity was low (I2 = 14.60%). No statistically significant difference was detected between TRAMIE and OE (OR: 1.08, 95% CI: 0.47–2.48) (Figure 5C).

3.5.8. Postoperative Hemorrhage Rate

The rate of postoperative hemorrhage was reported in four studies [21,23,34,35]. No statistically significant difference was found between the RAMIE group (1.18%; n= 4/339) and the OE group (2.51%, n = 12/479) (OR: 0.59, 95% CI: 0.18, 1.92]; p = 0.38). The statistical heterogeneity was low (I2 = 0.00%). No statistically significant difference was detected between TRAMIE and OE (OR: 0.83, 95% CI: 0.11–6.53) (Figure 6A).

3.5.9. Chylothorax Rate

The rate of chylothorax was reported in seven studies [21,23,34,35,37,39,41]. No statistically significant difference was found between the RAMIE group (5.39%; n = 29/538) and the OE group (3.01%; n = 33/1095) (OR: 1.31, 95% CI: 0.75–2.29; p = 0.35). The statistical heterogeneity was low (I2 = 0.00%). No statistically significant difference was detected between TRAMIE and OE (OR: 0.69, 95% CI: 0.21–2.31) (Figure 6B).

3.5.10. Recurrent Laryngeal Nerve Palsy Rate

The rate of RLN palsy was reported in seven studies [21,23,34,35,36,38,39]. No statistically significant difference was found between the RAMIE group (13.99%; n = 67/479) and the OE group (10.41%; n = 84/807) (OR: 1.31, 95% CI 0.90–1.90; p = 0.16). The statistical heterogeneity was low (I2 = 0.00%). No statistically significant difference was detected between TRAMIE and OE (OR: 1.46, 95% CI: 0.59–3.64) (Figure 6C).

3.5.11. Wound Infection Rate

The rate of wound infection was reported in four studies [21,34,37,38] and was significantly lower in the RAMIE group (1.25%; n = 4/319) compared to the OE group (5.95%; n = 38/637) (OR: 0.20, 95% CI: 0.07–0.57; p < 0.001). The statistical heterogeneity was low (I2 = 0.00%). TRAMIE was also associated with a significantly lower wound infection rate (OR: 0.21, 95% CI 0.04–0.96) (Figure 6D).

4. Discussion

Open esophagectomy, a highly invasive procedure associated with significant morbidity and mortality [1,3], is still a commonly utilized technique for the management of resectable esophageal cancer in many centers internationally [20]. Robot-assisted minimally invasive esophagectomy was introduced as a potential solution to these problems [13]. This systematic review and meta-analysis of 2046 patients, to our knowledge the first one directly comparing open to robotic MIE, robustly shows an advantage for robotic MIE with regard to several short-term outcomes.
The overall complication rate following esophagectomy is directly linked to the mortality rate of the procedure, as most postoperative deaths occur in patients with multiple complications [45]. No statistically significant difference was observed in the overall complication rate and 30-day and 90-day mortality rates, suggesting that RAMIE might be an overall safe procedure. The limited number of studies reporting on these outcomes prevent us from drawing any definitive conclusions in spite of the lower absolute rates seen with RAMIE. The length of hospital stay was much shorter with RAMIE. As with MIE, the less invasive nature of the procedure minimizes surgical trauma and seemingly allows for faster recovery [8]. However, postoperative complications also prolong hospitalization [46] and thus may be partially responsible for the longer LOS of the OE group, with a WMD of 9.22 days (95% CI: [−14.39, −4.06]). An interesting observation is that in our study the average LOS in both groups was prolonged compared to what has previously been reported in the literature regarding both open and robot-assisted approaches [15,47]. Again, this may be explained by the fact that some patients in both groups experienced severe complications that may have skewed the LOS towards higher average values. TRAMIE, in particular, was associated with about a 20-day shorter length of hospital stay, contrary to hybrid operations which did not statistically differ from the open ones, and, thus, was the main contributary to the observed effect. Future studies specifically focusing on the LOS in patients without any postoperative complications may provide more insight into the true effect of the technique itself on this outcome. Even though only a minority of studies focused on the quality of life and postoperative recovery, data from Van der Sluis et al. and Sugawara et al. suggest RAMIE may have an advantage over OE in regard to these outcomes that extends beyond the initial hospitalization period [21,40].
Pulmonary complications constitute a major cause of morbidity and account for a large proportion of postoperative mortality following esophagectomy [45,48]. Among all pulmonary complications, pneumonia is the most common and has been identified as an independent prognostic factor of mortality and worse long-term outcomes [3,49,50]. Soon after the introduction of conventional MIE, the decreased incidence of pulmonary complications stood out as its main advantage over the open approach [6,7] In the TIME randomized controlled trial, the MIE group had an almost 70% relative risk reduction of pneumonias the first 2 weeks after the procedure [8]. The lower incidence of pulmonary complications with MIE has been attributed to many different factors. Compared to the lateral decubitus position used during OE, the prone positioning often employed during MIE decreases the compression of the lung parenchyma and may bypass the limitation of single lung ventilation by avoiding total lung collapse [8,51]. A significant proportion of the included studies in this meta-analysis used the prone [38,41] or semi-prone [21,23,34] positioning during the thoracic part of the robot-assisted procedure. In addition, the minimally invasive nature of the procedure decreases pulmonary tissue trauma and postoperative pain, leading to decreased inflammation, more effective breathing mechanics, and decreased basal atelectasis [52,53]. Van der Sluis et al. found that postoperative pain was significantly lower with RAMIE compared to the open procedure [21]. As with the conventional MIE, this can be attributed to the avoidance of thoracotomy [8]. Robot-assisted surgery also has the advantage of three-dimensional operative field visualization and offers seven degrees of freedom using articulated instruments [15]. This may enable improved preservation of the pulmonary branches of the vagus nerve, which regulates many important pulmonary functions and reflexes and, if compromised, may set the stage for postoperative pulmonary complications to occur [34].
Anastomotic leakage is common but is also one of the most feared complications following esophagectomy. It is associated with significant morbidity, as it tends to co-occur with other complications, prolonged hospitalization, and high mortality [3,54]. In terms of technical considerations, leakage is more common with cervical anastomoses and the McKeown approach compared to intrathoracic anastomoses and the Ivor-Lewis approach [54,55]. In our meta-analysis, the rate of anastomotic leakage was almost identical between RAMIE and OE. As with conventional MIE, RAMIE does not seem to offer any benefit for this outcome per se but appears to be as safe as OE nonetheless [56]. Another complication that is often associated with postoperative morbidity following esophagectomy is atrial fibrillation [57,58]. We found that atrial fibrillation was observed at significantly higher rates in the OE. The pathophysiology of atrial fibrillation after esophagectomy is complex and involves multiple factors. In this instance, the decreased rate in RAMIE might be explained by a combination of decreased intravascular depletion through less EBL and decreased oxidative stress because of fewer infectious complications and better lung ventilation in the prone position [57]. Interestingly TRAMIE was not associated with lower rates of atrial fibrillation, while hybrid procedures were found to lead to significantly lower rates, but the limited number of included studies could lead to this result.
Recurrent laryngeal nerve palsy and chylothorax rates were slightly higher in the RAMIE group. The relatively high RLN palsy rates observed after RAMIE in the past were mainly attributed to the extensive en bloc lymphadenectomy of the superior mediastinum [59]. The findings of Gong et al. support this association, as higher numbers of resected superior mediastinal lymph nodes were also combined with a higher incidence of vocal cord paralysis in the RAMIE group [34]. Van der Sluis et al. also attributed the relatively high incidence of chylothorax to the same phenomenon [21]. Regardless of that, the differences in the rates of RLN palsy and chylothorax between the two groups were found to be non-statistically significant.
RAMIE displayed a clear advantage over OE in terms of EBL. In line with MIE, the minimally invasive nature of RAMIE reduces tissue trauma and thus decreases the risk of blood vessel injury. The prone positioning of the patient during MIE and RAMIE allows for better visualization of the operative field compared to the lateral decubitus position of the open approach. In the latter case, blood tends to pool within the operative field and consequently obstructs the intraoperative view [60]. As a result, the prone position allows for higher surgical precision in both dissection and hemostasis, hence minimizing blood loss [61]. RAMIE has the additional advantage of three-dimensional visualization of the operating field as well as greater mobility and precision of the surgical instrumentation. This may explain why RAMIE results in lower EBL compared to both OE and MIE [19,38,62]. This decrease in intraoperative blood loss may also lead to reduced transfusion requirements, which are generally associated with inferior outcomes [63]. Nevertheless, the main drawback of RAMIE was the significantly longer operative time compared to the open esophagectomy group [59,64]. The docking and undocking of the robotic equipment is a lengthy process that prolongs the total operative time [64]. RAMIE has a learning curve of around 70 cases, but as experience accumulates, intraoperative parameters, including operative time and EBL are expected to decrease [65].
Open esophagectomy and RAMIE were comparable in terms of oncological outcomes and resulted in similar numbers of resected lymph nodes and R0 resection rates. Both these outcomes are strongly linked to long-term survival following esophagectomy [66,67,68]. Compared to the respective groups from the national database analysis by Weksler et al., the pooled numbers of resected lymph nodes in the current meta-analysis in both the RAMIE and OE groups were higher, while the R0 resection rates were similar [42]. Van der Sluis et al. reported comparable OS and DFS rates up to 5 years between the two groups [21]. Weksler et al. also came to the same conclusion about overall survival up to 4 years [42]. In comparison, Yun et al. found that RAMIE had an advantage in 1-year (95.1% in RAMIE vs. 85.6% in OE) and 3-year OS (81.7% in RAMIE vs 73.7% in OE) [23]. Although these preliminary data suggest that RAMIE demonstrates equivalent or even superior long-term survival compared to OE, additional data are required to deduce more meaningful conclusions.
Nonetheless, certain limitations should be considered when interpreting the results of the present study. First, we included patients undergoing esophagectomy with either the transthoracic or the transhiatal approach. Although both approaches lead to similar outcomes, the transhiatal approach offers a limited capacity for an extended lymphadenectomy [47]. Similar concerns about the heterogeneity of our sample arise due to varying definitions and assessment protocols for complications, as well as technical variations, such as the role of the robot in creating the anastomosis and performing the lymphadenectomy, which warrant a careful interpretation of our results. However, performing a subgroup analysis for any of these parameters was not feasible, as the available data were limited and in some occasions the approaches differed between the RAMIE and OE groups within the same study [37,39,40,41]. Second, a very small proportion of patients with high-grade dysplasia/esophageal cancer in situ originating from two studies [35,37] had to be included in the patient sample, as selective data extraction for esophageal cancer patients only was not feasible; we decided to include these two studies as the significant increase in the study sample and the accuracy of the effect estimates was deemed to be more important than the slight increase in disease heterogeneity. In addition, the RAMIE group included a slightly higher proportion of stage I patients and a lower proportion of stage III patients compared to the OE group. Third, none of the included studies evaluated the health-related economic cost difference between the two procedures. Although the robotic approaches in surgery are typically associated with higher costs compared to their open counterparts [64], future studies focusing on the cost effectiveness of the different approaches are warranted. Fourth, RAMIE is a relatively new procedure compared to OE, and surgical expertise varied among the included studies; however, the effects of the learning curve on the outcomes could not be sufficiently evaluated. It is therefore expected that as surgeons acquire more expertise in RAMIE, the outcomes will continue to improve [65,69]. Finally, the limited study sample for certain outcomes of interest deserves careful interpretation of the non-statistically significant differences and also precluded us from conducting meta-analyses on the outcomes of OS and DFS.

5. Conclusions

In conclusion, the results of this systematic review and meta-analysis suggest that RAMIE is a feasible and safe procedure in carefully selected patients. Compared to OE, it may lead to less EBL, shorter LOS, and reduced rates of pulmonary morbidity, atrial fibrillation, and wound infection. More studies in the form of randomized controlled trials and well-designed large population-based studies are required to evaluate the potential advantage of RAMIE in terms of overall complications and long-term oncological results as well as the impact of the different surgical approaches (transthoracic and transhiatal) on outcomes.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/cancers14133177/s1, Supplemental Table S1: PRISMA 2009 Checklist, Supplemental Table S2: Quality assessment of included non-randomized studies utilizing the Newcastle–Ottawa scale, Supplemental Table S3: Quality assessment of included randomized controlled trials utilizing the Cochrane Collaboration’s tool, Supplemental Figure S1: Funnel plots to assess publication bias (primary surgical and primary oncological outcomes), Supplemental Figure S2: Funnel plots to assess publication bias (secondary outcomes).

Author Contributions

Conceptualization, S.M.E., I.A.Z., I.K. and D.S.; methodology, S.M.E., I.A.Z., I.K., G.T., D.M., M.V.K., I.R., M.N. and D.S.; software, S.M.E., I.A.Z. and I.K.; validation, S.M.E., I.A.Z., K.S., I.K., G.T., D.M., M.V.K., I.R., M.N. and D.S.; formal analysis, I.A.Z.; investigation, S.M.E.; resources, S.M.E., I.A.Z. and K.S.; data curation, S.M.E. and K.S.; writing—original draft preparation, S.M.E.; writing—review and editing, I.A.Z., K.S., I.K., G.T., D.M., M.V.K., I.R., M.N. and D.S.; visualization, S.M.E., I.A.Z. and I.K.; supervision, D.S.; project administration, D.S.; funding acquisition, N/A. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ajani, J.A.; Barthel, J.S.; Bentrem, D.J.; Amico, T.A.D.; Das, P.; Denlinger, C.S.; Fuchs, C.S.; Gerdes, H.; Glasgow, R.E.; Hayman, J.A.; et al. Esophageal and Esophagogastric Junction Cancers Clinical Practice Guidelines in Oncology. J. Natl. Compr. Cancer Netw. 2011, 9, 830–887. [Google Scholar] [CrossRef] [Green Version]
  2. Ando, N.; Ozawa, S.; Kitagawa, Y.; Shinozawa, Y.; Kitajima, M. Improvement in the Results of Surgical Treatment of Advanced Squamous Esophageal Carcinoma During 15 Consecutive Years. Ann. Surg. 2000, 232, 225–232. [Google Scholar] [CrossRef] [PubMed]
  3. Takeuchi, H.; Miyata, H.; Gotoh, M.; Kitagawa, Y.; Baba, H.; Kimura, W.; Tomita, N.; Nakagoe, T.; Shimada, M.; Sugihara, K.; et al. A Risk Model for Esophagectomy Using Data of 5354 Patients Included in a Japanese Nationwide Web-Based Database. Ann. Surg. 2014, 260, 259–266. [Google Scholar] [CrossRef] [PubMed]
  4. Lerut, T.; Moons, J.; Coosemans, W.; Van Raemdonck, D.; De Leyn, P.; Decaluwé, H.; Decker, G.; Nafteux, P. Postoperative Complications after Transthoracic Esophagectomy for Cancer of the Esophagus and Gastroesophageal Junction Are Correlated with Early Cancer Recurrence: Role of Systematic Grading of Complications Using the Modified Clavien Classification. Ann. Surg. 2009, 250, 798–806. [Google Scholar] [CrossRef] [PubMed]
  5. Jiang, R.; Liu, Y.; Ward, K.C.; Force, S.D.; Pickens, A.; Sancheti, M.S.; Javidfar, J.; Fernandez, F.G.; Khullar, O.V. Excess Cost and Predictive Factors of Esophagectomy Complications in the SEER-Medicare Database. Ann. Thorac. Surg. 2018, 106, 1484–1491. [Google Scholar] [CrossRef] [Green Version]
  6. Luketich, J.D.; Alvelo-Rivera, M.; Buenaventura, P.O.; Christie, N.A.; McCaughan, J.S.; Litle, V.R.; Schauer, P.R.; Close, J.M.; Fernando, H.C.; Zinner, M.J. Minimally Invasive Esophagectomy: Outcomes in 222 Patients. Ann. Surg. 2003, 238, 486–495. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Palanivelu, C.; Prakash, A.; Senthilkumar, R.; Senthilnathan, P.; Parthasarathi, R.; Rajan, P.S.; Venkatachlam, S. Minimally Invasive Esophagectomy: Thoracoscopic Mobilization of the Esophagus and Mediastinal Lymphadenectomy in Prone Position—Experience of 130 Patients. J. Am. Coll. Surg. 2006, 203, 7–16. [Google Scholar] [CrossRef] [PubMed]
  8. Biere, S.S.A.Y.; van Berge Henegouwen, M.I.; Maas, K.W.; Bonavina, L.; Rosman, C.; Garcia, J.R.; Gisbertz, S.S.; Klinkenbijl, J.H.G.; Hollmann, M.W.; de Lange, E.S.; et al. Minimally invasive versus open oesophagectomy for patients with oesophageal cancer: A multicentre, open-label, randomised controlled trial. Lancet 2012, 379, 1887–1892. [Google Scholar] [CrossRef]
  9. Van Der Sluis, P.C.; Schizas, D.; Liakakos, T.; van Hillegersberg, R. Minimally Invasive Esophagectomy. Dig. Surg. 2020, 37, 93–100. [Google Scholar] [CrossRef]
  10. Gottlieb-Vedi, E.; Kauppila, J.H.; Malietzis, G.; Nilsson, M.; Markar, S.R.; Lagergren, J. Long-term Survival in Esophageal Cancer After Minimally Invasive Compared to Open Esophagectomy: A Systematic Review and Meta-Analysis. Ann. Surg. 2019, 270, 1005–1017. [Google Scholar] [CrossRef]
  11. Markar, S.R.; Ni, M.; Gisbertz, S.S.; van der Werf, L.; Straatman, J.; van der Peet, D.; Cuesta, M.A.; Hanna, G.B.; van Berge Henegouwen, M.I. Implementation of Minimally Invasive Esophagectomy From a Randomized Controlled Trial Setting to National Practice. J. Clin. Oncol. 2020, 38, 2130. [Google Scholar] [CrossRef]
  12. Decker, G.; Coosemans, W.; De Leyn, P.; Decaluwé, H.; Nafteux, P.; Van Raemdonck, D.; Lerut, T. Minimally Invasive Esophagectomy for Cancer. Eur. J. Cardio-Thorac. Surg. 2009, 35, 13–21. [Google Scholar] [CrossRef]
  13. Van Hillegersberg, R.; Boone, J.; Draaisma, W.A.; Broeders, I.A.M.J.; Giezeman, M.J.M.M.; Rinkes, I.H.M.B. First Experience with Robot-Assisted Thoracoscopic Esophagolymphadenectomy for Esophageal Cancer. Surg. Endosc. Other Interv. Tech. 2006, 20, 1435–1439. [Google Scholar] [CrossRef]
  14. van Workum, F.; Stenstra, M.H.B.C.; Berkelmans, G.H.K.; Slaman, A.E.; van Berge Henegouwen, M.I.; Gisbertz, S.S.; van den Wildenberg, F.J.H.; Polat, F.; Irino, T.; Nilsson, M.; et al. Learning Curve and Associated Morbidity of Minimally Invasive Esophagectomy: A Retrospective Multicenter Study. Ann. Surg. 2019, 269, 88–94. [Google Scholar] [CrossRef]
  15. van der Sluis, P.C.; Tagkalos, E.; Hadzijusufovic, E.; Babic, B.; Uzun, E.; van Hillegersberg, R.; Lang, H.; Grimminger, P.P. Robot-Assisted Minimally Invasive Esophagectomy with Intrathoracic Anastomosis (Ivor Lewis): Promising Results in 100 Consecutive Patients (the European Experience). J. Gastrointest. Surg. 2020, 25, 1–8. [Google Scholar] [CrossRef] [Green Version]
  16. Park, S.Y.; Kim, D.J.; Yu, W.S.; Jung, H.S. Robot-assisted thoracoscopic esophagectomy with extensive mediastinal lymphadenectomy: Experience with 114 consecutive patients with intrathoracic esophageal cancer. Dis. Esophagus 2016, 29, 326–332. [Google Scholar] [CrossRef]
  17. van der Sluis, P.C.; Ruurda, J.P.; Verhage, R.J.J.; van der Horst, S.; Haverkamp, L.; Siersema, P.D.; Borel Rinkes, I.H.M.; ten Kate, F.J.W.; van Hillegersberg, R. Oncologic Long-Term Results of Robot-Assisted Minimally Invasive Thoraco-Laparoscopic Esophagectomy with Two-Field Lymphadenectomy for Esophageal Cancer. Ann. Surg. Oncol. 2015, 22, 1350–1356. [Google Scholar] [CrossRef] [Green Version]
  18. Zhang, X.; Su, Y.; Yang, Y.; Sun, Y.; Ye, B.; Guo, X.; Mao, T.; Hua, R.; Li, Z. Robot assisted esophagectomy for esophageal squamous cell carcinoma. J. Thorac. Dis. 2018, 10, 3767–3775. [Google Scholar] [CrossRef]
  19. Jin, D.; Yao, L.; Yu, J.; Liu, R.; Guo, T.; Yang, K.; Gou, Y. Robotic-assisted minimally invasive esophagectomy versus the conventional minimally invasive one: A meta-analysis and systematic review. Int. J. Med. Robot. Comput. Assist. Surg. 2019, 15, e1988. [Google Scholar] [CrossRef]
  20. Haverkamp, L.; Seesing, M.F.J.; Ruurda, J.P.; Boone, J.; van Hillegersberg, R. Worldwide trends in surgical techniques in the treatment of esophageal and gastroesophageal junction cancer. Dis. Esophagus 2017, 30, 1–7. [Google Scholar] [CrossRef] [Green Version]
  21. Van der Sluis, P.C.; van der Horst, S.; May, A.M.; Schippers, C.; Brosens, L.A.A.; Joore, H.C.A.; Kroese, C.C.; Mohammad, N.H.; Mook, S.; Vleggaar, F.P.; et al. Robot-assisted Minimally Invasive Thoracolaparoscopic Esophagectomy Versus Open Transthoracic Esophagectomy for Resectable Esophageal Cancer: A Randomized Controlled Trial. Ann. Surg. 2019, 269, 621–630. [Google Scholar] [CrossRef]
  22. Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gotzsche, P.C.; Ioannidis, J.P.A.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Healthcare Interventions: Explanation and Elaboration. BMJ 2009, 339, b2700. [Google Scholar] [CrossRef] [Green Version]
  23. Yun, J.K.; Chong, B.K.; Kim, H.J.; Lee, I.-S.; Gong, C.-S.; Kim, B.S.; Lee, G.D.; Choi, S.; Kim, H.R.; Kim, D.K.; et al. Comparative Outcomes of Robot-Assisted Minimally Invasive versus Open Esophagectomy in Patients with Esophageal Squamous Cell Carcinoma: A Propensity Score-Weighted Analysis. Dis. Esophagus 2020, 33, doz071. [Google Scholar] [CrossRef]
  24. Yun, J.K.; Lee, I.-S.; Gong, C.-S.; Kim, B.S.; Kim, H.R.; Kim, D.K.; Park, S.-I.; Kim, Y.-H. Clinical utility of robot-assisted transthoracic esophagectomy in advanced esophageal cancer after neoadjuvant chemoradiation therapy. J. Thorac. Dis. 2019, 11, 2913–2923. [Google Scholar] [CrossRef]
  25. Wohlin, C. Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering. In Proceedings of the the 18th International Conference on Evaluation and Assessment in Software Engineering, ACM, New York, NY, USA, 13 May 2014; pp. 1–10. [Google Scholar]
  26. Stang, A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur. J. Epidemiol. 2010, 25, 603–605. [Google Scholar] [CrossRef] [Green Version]
  27. Higgins, J.P.T.; Altman, D.G.; Gøtzsche, P.C.; Jüni, P.; Moher, D.; Oxman, A.D.; Savović, J.; Schulz, K.F.; Weeks, L.; Sterne, J.A.C.; et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011, 343, d5928. [Google Scholar] [CrossRef] [Green Version]
  28. Hozo, S.P.; Djulbegovic, B.; Hozo, I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med. Res. Methodol. 2005, 5, 13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Wan, X.; Wang, W.; Liu, J.; Tong, T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med. Res. Methodol. 2014, 14, 135. [Google Scholar] [CrossRef] [Green Version]
  30. Higgins, J.; Green, S. (Eds.) Cochrane Handbook for Systematic Reviews of Interventions; Version 5.1.0 (Updated March 2011); The Cochrane Collaboration: London, UK, 2011. [Google Scholar]
  31. Haldane, J.B.S. The estimation and significance of the logarithm of a ratio of frequencies. Ann. Hum. Genet. 1956, 20, 309–311. [Google Scholar] [CrossRef]
  32. Anscombe, F.J. On Estimating Binomial Response Relations. Biometrika 1956, 43, 461. [Google Scholar] [CrossRef]
  33. DerSimonian, R.; Laird, N. Meta-analysis in clinical trials. Control Clin. Trials 1986, 7, 177–188. [Google Scholar] [CrossRef]
  34. Gong, L.; Jiang, H.; Yue, J.; Duan, X.; Tang, P.; Ren, P.; Zhao, X.; Liu, X.; Zhang, X.; Yu, Z. Comparison of the short-term outcomes of robot-assisted minimally invasive, video-assisted minimally invasive, and open esophagectomy. J. Thorac. Dis. 2020, 12, 916–924. [Google Scholar] [CrossRef] [PubMed]
  35. Sarkaria, I.S.; Rizk, N.P.; Goldman, D.A.; Sima, C.; Tan, K.S.; Bains, M.S.; Adusumilli, P.S.; Molena, D.; Bott, M.; Atkinson, T.; et al. Early Quality of Life Outcomes After Robotic-Assisted Minimally Invasive and Open Esophagectomy. Ann. Thorac. Surg. 2019, 108, 920–928. [Google Scholar] [CrossRef] [PubMed]
  36. Jeong, D.M.; Kim, J.A.; Ahn, H.J.; Yang, M.; Heo, B.Y.; Lee, S.H. Decreased Incidence of Postoperative Delirium in Robot-assisted Thoracoscopic Esophagectomy Compared with Open Transthoracic Esophagectomy. Surg. Laparosc. Endosc. Percutaneous Tech. 2016, 26, 516–522. [Google Scholar] [CrossRef]
  37. Meredith, K.L.; Maramara, T.; Blinn, P.; Lee, D.; Huston, J.; Shridhar, R. Comparative Perioperative Outcomes by Esophagectomy Surgical Technique. J. Gastrointest. Surg. 2019, 24, 1261–1268. [Google Scholar] [CrossRef]
  38. Osaka, Y.; Tachibana, S.; Ota, Y.; Suda, T.; Makuuti, Y.; Watanabe, T.; Iwasaki, K.; Katsumata, K.; Tsuchida, A. Usefulness of robot-assisted thoracoscopic esophagectomy. Gen. Thorac. Cardiovasc. Surg. 2018, 66, 225–231. [Google Scholar] [CrossRef]
  39. Mori, K.; Yamagata, Y.; Aikou, S.; Nishida, M.; Kiyokawa, T.; Yagi, K.; Yamashita, H.; Nomura, S.; Seto, Y. Short-term outcomes of robotic radical esophagectomy for esophageal cancer by a nontransthoracic approach compared with conventional transthoracic surgery. Dis. Esophagus 2016, 29, 429–434. [Google Scholar] [CrossRef] [Green Version]
  40. Sugawara, K.; Yoshimura, S.; Yagi, K.; Nishida, M.; Aikou, S.; Yamagata, Y.; Mori, K.; Yamashita, H.; Seto, Y. Long-term health-related quality of life following robot-assisted radical transmediastinal esophagectomy. Surg. Endosc. 2019, 34, 1602–1611. [Google Scholar] [CrossRef]
  41. del Val, I.D.; Loureiro, C.; McCulloch, P. The IDEAL prospective development study format for reporting surgical innovations. An illustrative case study of robotic oesophagectomy. Int. J. Surg. 2015, 19, 104–111. [Google Scholar] [CrossRef]
  42. Weksler, B.; Sullivan, J.L. Survival After Esophagectomy: A Propensity-Matched Study of Different Surgical Approaches. Ann. Thorac. Surg. 2017, 104, 1138–1146. [Google Scholar] [CrossRef] [Green Version]
  43. Espinoza-Mercado, F.; Imai, T.A.; Borgella, J.D.; Sarkissian, A.; Serna-Gallegos, D.; Alban, R.F.; Soukiasian, H.J. Does the Approach Matter? Comparing Survival in Robotic, Minimally Invasive, and Open Esophagectomies. Ann. Thorac. Surg. 2019, 107, 378–385. [Google Scholar] [CrossRef]
  44. Yerokun, B.A.; Sun, Z.; Jeffrey Yang, C.F.; Gulack, B.C.; Speicher, P.J.; Adam, M.A.; D’Amico, T.A.; Onaitis, M.W.; Harpole, D.H.; Berry, M.F.; et al. Minimally Invasive Versus Open Esophagectomy for Esophageal Cancer: A Population-Based Analysis. Ann. Thorac. Surg. 2016, 102, 416–423. [Google Scholar] [CrossRef] [Green Version]
  45. Schieman, C.; Wigle, D.A.; Deschamps, C.; NIcholsii, F.C.; Cassivi, S.D.; Shen, K.R.; Allen, M.S. Patterns of operative mortality following esophagectomy. Dis. Esophagus 2012, 25, 645–651. [Google Scholar] [CrossRef]
  46. Morita, M.; Nakanoko, T.; Fujinaka, Y.; Kubo, N.; Yamashita, N.; Yoshinaga, K.; Saeki, H.; Emi, Y.; Kakeji, Y.; Shirabe, K.; et al. In-Hospital Mortality After a Surgical Resection for Esophageal Cancer: Analyses of the Associated Factors and Historical Changes. Ann. Surg. Oncol. 2011, 18, 1757–1765. [Google Scholar] [CrossRef]
  47. Mertens, A.C.; Kalff, M.C.; Eshuis, W.J.; Van Gulik, T.M.; Henegouwen, M.I.V.B.; Gisbertz, S.S. Transthoracic Versus Transhiatal Esophagectomy for Esophageal Cancer: A Nationwide Propensity Score-Matched Cohort Analysis. Ann. Surg. Oncol. 2020, 28, 175–183. [Google Scholar] [CrossRef]
  48. Bailey, S.H.; Bull, D.A.; Harpole, D.H.; Rentz, J.J.; Neumayer, L.A.; Pappas, T.N.; Daley, J.; Henderson, W.G.; Krasnicka, B.; Khuri, S.F. Outcomes after esophagectomy: A ten-year prospective cohort. Ann. Thorac. Surg. 2003, 75, 217–222. [Google Scholar] [CrossRef]
  49. Atkins, B.Z.; Shah, A.S.; Hutcheson, K.A.; Mangum, J.H.; Pappas, T.N.; Harpole, D.H.; D’Amico, T.A. Reducing Hospital Morbidity and Mortality Following Esophagectomy. Ann. Thorac. Surg. 2004, 78, 1170–1176. [Google Scholar] [CrossRef]
  50. Kataoka, K.; Takeuchi, H.; Mizusawa, J.; Igaki, H.; Ozawa, S.; Abe, T.; Nakamura, K.; Kato, K.; Ando, N.; Kitagawa, Y. Prognostic Impact of Postoperative Morbidity After Esophagectomy for Esophageal Cancer. Ann. Surg. 2017, 265, 1152–1157. [Google Scholar] [CrossRef]
  51. Murthy, R.A.; Clarke, N.S.; Kernstine, K.H. Minimally Invasive and Robotic Esophagectomy: A Review. Innov. Technol. Tech. Cardiothorac. Vasc. Surg. 2018, 13, 391–403. [Google Scholar] [CrossRef]
  52. Lv, L.; Hu, W.; Ren, Y.; Wei, X. Minimally invasive esophagectomy versus open esophagectomy for esophageal cancer: A meta-analysis. OncoTargets Ther. 2016, 9, 6751–6762. [Google Scholar] [CrossRef] [Green Version]
  53. Mariette, C.; Markar, S.R.; Dabakuyo-Yonli, T.S.; Meunier, B.; Pezet, D.; Collet, D.; D’Journo, X.B.; Brigand, C.; Perniceni, T.; Carrère, N.; et al. Hybrid Minimally Invasive Esophagectomy for Esophageal Cancer. N. Engl. J. Med. 2019, 380, 152–162. [Google Scholar] [CrossRef]
  54. Kassis, E.S.; Kosinski, A.S.; Ross, P.; Koppes, K.E.; Donahue, J.M.; Daniel, V.C. Predictors of Anastomotic Leak After Esophagectomy: An Analysis of The Society of Thoracic Surgeons General Thoracic Database. Ann. Thorac. Surg. 2013, 96, 1919–1926. [Google Scholar] [CrossRef] [PubMed]
  55. Deng, J.; Su, Q.; Ren, Z.; Wen, J.; Xue, Z.; Zhang, L.; Chu, X. Comparison of short-term outcomes between minimally invasive McKeown and Ivor Lewis esophagectomy for esophageal or junctional cancer: A systematic review and meta-analysis. OncoTargets Ther. 2018, 11, 6057–6069. [Google Scholar] [CrossRef] [Green Version]
  56. Zhou, C.; Ma, G.; Li, X.; Li, J.; Yan, Y.; Liu, P.; He, J.; Ren, Y. Is minimally invasive esophagectomy effective for preventing anastomotic leakages after esophagectomy for cancer? A systematic review and meta-analysis. World J. Surg. Oncol. 2015, 13, 269. [Google Scholar] [CrossRef] [Green Version]
  57. Seesing, M.F.J.; Borggreve, A.S.; Ruurda, J.P.; van Hillegersberg, R. New-onset atrial fibrillation after esophagectomy for cancer. J. Thorac. Dis. 2019, 11, S831–S834. [Google Scholar] [CrossRef]
  58. Schizas, D.; Kosmopoulos, M.; Giannopoulos, S.; Kokkinidis, D.G.; Karampetsou, N.; Papanastasiou, C.; Rouvelas, I.; Liakakos, T. Meta-analysis of risk factors and complications associated with atrial fibrillation after oesophagectomy. Br. J. Surg. 2019, 106, 534–547. [Google Scholar] [CrossRef]
  59. Boone, J.; Schipper, M.E.I.; Moojen, W.A.; Borel Rinkes, I.H.M.; Cromheecke, G.J.E.; van Hillegersberg, R. Robot-assisted thoracoscopic oesophagectomy for cancer. Br. J. Surg. 2009, 96, 878–886. [Google Scholar] [CrossRef]
  60. Fabian, T.; Martin, J.; Katigbak, M.; McKelvey, A.A.; Federico, J.A. Thoracoscopic esophageal mobilization during minimally invasive esophagectomy: A head-to-head comparison of prone versus decubitus positions. Surg. Endosc. 2008, 22, 2485–2491. [Google Scholar] [CrossRef] [PubMed]
  61. Noshiro, H.; Iwasaki, H.; Kobayashi, K.; Uchiyama, A.; Miyasaka, Y.; Masatsugu, T.; Koike, K.; Miyazaki, K. Lymphadenectomy along the left recurrent laryngeal nerve by a minimally invasive esophagectomy in the prone position for thoracic esophageal cancer. Surg. Endosc. 2010, 24, 2965–2973. [Google Scholar] [CrossRef]
  62. Ruurda, J.P.; van Vroonhoven, T.J.M.V.; Broeders, I.A.M.J. Robot-assisted surgical systems: A new era in laparoscopic surgery. Ann. R. Coll. Surg. Engl. 2002, 84, 223–226. [Google Scholar] [CrossRef] [Green Version]
  63. Boshier, P.R.; Ziff, C.; Adam, M.E.; Fehervari, M.; Markar, S.R.; Hanna, G.B. Effect of perioperative blood transfusion on the long-term survival of patients undergoing esophagectomy for esophageal cancer: A systematic review and meta-analysis. Dis. Esophagus 2017, 31, dox134. [Google Scholar] [CrossRef] [PubMed]
  64. Clark, J.; Sodergren, M.H.; Purkayastha, S.; Mayer, E.K.; James, D.; Athanasiou, T.; Yang, G.-Z.; Darzi, A. The role of robotic assisted laparoscopy for oesophagogastric oncological resection; an appraisal of the literature. Dis. Esophagus 2010, 24, 240–250. [Google Scholar] [CrossRef] [PubMed]
  65. Van Der Sluis, P.C.; Ruurda, J.P.; Van Der Horst, S.; Goense, L.; Van Hillegersberg, R. Learning Curve for Robot-Assisted Minimally Invasive Thoracoscopic Esophagectomy: Results From 312 Cases. Ann. Thorac. Surg. 2018, 106, 264–271. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  66. Visser, E.; Markar, S.R.; Ruurda, J.P.; Hanna, G.B.; van Hillegersberg, R. Prognostic Value of Lymph Node Yield on Overall Survival in Esophageal Cancer Patients: A Systematic Review and Meta-Analysis. Ann. Surg. 2019, 269, 261–268. [Google Scholar] [CrossRef]
  67. Samson, P.; Puri, V.; Broderick, S.; Patterson, G.A.; Meyers, B.; Crabtree, T. Extent of Lymphadenectomy Is Associated with Improved Overall Survival After Esophagectomy with or without Induction Therapy. Ann. Thorac. Surg. 2016, 103, 406–415. [Google Scholar] [CrossRef] [Green Version]
  68. Markar, S.R.; Gronnier, C.; Duhamel, A.; Pasquer, A.; Théreaux, J.; du Rieu, M.C.; Lefevre, J.H.; Turner, K.; Luc, G.; Mariette, C. Significance of Microscopically Incomplete Resection Margin After Esophagectomy for Esophageal Cancer. Ann. Surg. 2016, 263, 712–718. [Google Scholar] [CrossRef]
  69. Park, S.; Hyun, K.; Lee, H.J.; Park, I.K.; Kim, Y.T.; Kang, C.H. A study of the learning curve for robotic oesophagectomy for oesophageal cancer. Eur. J. Cardio-Thoracic Surg. 2017, 53, 862–870. [Google Scholar] [CrossRef]
Cancers 14 03177 g001 550
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of the study selection process.
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of the study selection process.
Cancers 14 03177 g001
Cancers 14 03177 g002 550
Figure 2. Forest plots of overall complication rate (A), overall pulmonary complication rate (B), anastomotic leakage rate (C), 30-day mortality rate (D), and 90-day mortality rate (E) [21,23,24,25,34,35,36,37,38,39,40,41].
Figure 2. Forest plots of overall complication rate (A), overall pulmonary complication rate (B), anastomotic leakage rate (C), 30-day mortality rate (D), and 90-day mortality rate (E) [21,23,24,25,34,35,36,37,38,39,40,41].
Cancers 14 03177 g002
Cancers 14 03177 g003 550
Figure 3. Forest plots of total lymph nodes resected (A) and margin-negative resection (R0) rate (B) [21,23,34,35,37,38,39,40,41].
Figure 3. Forest plots of total lymph nodes resected (A) and margin-negative resection (R0) rate (B) [21,23,34,35,37,38,39,40,41].
Cancers 14 03177 g003
Cancers 14 03177 g004 550
Figure 4. Forest plots of operative time (A), estimated blood loss (B), intensive care unit (ICU) length of stay (C), and hospital length of stay (D) [21,23,34,35,37,38,39,40,41].
Figure 4. Forest plots of operative time (A), estimated blood loss (B), intensive care unit (ICU) length of stay (C), and hospital length of stay (D) [21,23,34,35,37,38,39,40,41].
Cancers 14 03177 g004
Cancers 14 03177 g005 550
Figure 5. Forest plots of pneumonia rate (A), acute respiratory distress syndrome rate (B), and atrial fibrillation rate (C) [21,23,34,35,36,37,39,41].
Figure 5. Forest plots of pneumonia rate (A), acute respiratory distress syndrome rate (B), and atrial fibrillation rate (C) [21,23,34,35,36,37,39,41].
Cancers 14 03177 g005
Cancers 14 03177 g006 550
Figure 6. Forest plots of postoperative hemorrhage rate (A), chylothorax rate (B), recurrent laryngeal nerve palsy rate (C), and wound infection rate (D) [21,23,34,35,36,37,38,39,41].
Figure 6. Forest plots of postoperative hemorrhage rate (A), chylothorax rate (B), recurrent laryngeal nerve palsy rate (C), and wound infection rate (D) [21,23,34,35,36,37,38,39,41].
Cancers 14 03177 g006
Table
Table 1. Characteristics of the included studies.
Table 1. Characteristics of the included studies.
Single-Center Studies
AuthorYearCountryStudy PeriodType of StudyRAMIEOERAMIE Technique
Gong et al.2020ChinaJan 2016 to Dec 2018Retrospective cohort7791Robotic thoracic + abdominal phase;
cervical anastomosis
Sugawara et al.2020JapanApr 2015 to Jan 2017Prospective cohort1819Robotic transhiatal combined with a video-assisted cervical approach;
cervical anastomosis
Sarkaria et al.2019USAMar 2012 to Aug 2014Prospective cohort64106Robotic thoracic + abdominal phase;
cervical or intrathoracic anastomosis
Yun et al.2019South KoreaJan 2012 to Dec 2016Retrospective cohort130241Robotic thoracic phase;
robotic abdominal or laparoscopic phase; cervical or intrathoracic anastomosis
Meredith et al. 2019USA1999 to 2016Retrospective cohort144475Robotic thoracic + abdominal phase; intrathoracic anastomosis
van der Sluis et al.2019The NetherlandsJan 2012 to Aug 2016Randomized controlled trial5455Robotic thoracic phase;
laparoscopic abdominal phase; cervical anastomosis
Osaka et al.2018JapanJun 2010 to Dec 2013 (RAMIE), 2006–2010 (OE)Retrospective cohort3030Robotic thoracic phase
Jeong et al.2016South KoreaDec 2012 to Apr 2015Retrospective cohort88159Robotic thoracic phase; open abdominal phase; cervical anastomosis
Mori et al.2016JapanNov 2012 to Jul 2014 (RAMIE), May 2008 to Jul 2014 (OE)Prospective cohort22139Robotic transhiatal combined with a video-assisted cervical approach;
cervical anastomosis
Diez del Val et al.2015SpainDec 2009 to N/AProspective cohort332Robotic thoracic or transhiatal phase; intrathoracic or cervical anastomosis (respectively)
Total 1999 to 2019 6741303
National Registry Studies
AuthorYearCountryStudy PeriodType of StudyRAMIEOpen
Weksler et al.2017USA2010 to 2013Retrospective cohort569569N/A
N/A: not available; RAMIE: Robot-assisted minimally invasive esophagectomy; OE: Open esophagectomy.
Table
Table 2. Demographic, preoperative, and tumor characteristics.
Table 2. Demographic, preoperative, and tumor characteristics.
RAMIE (n = 674)OE (n = 1303)
Demographic characteristics
Age (years)63.5 ± 8.562.7 ± 9.27
Male/Female545 (85.0)/96 (15.0)1133 (87.1)/168 (12.9))
BMI (kg/m2)25.8 ± 6.825.7 ± 5.4
Preoperative characteristics
ASA physical status
1–2195 (55.7)427 (60.7)
3–4155 (44.3)276 (39.3)
Charlson–Deyo score
018 (16.5)17 (17.7)
131 (28.4)33 (34.4)
242 (38.5)35 (36.5)
314 (12.8)11 (11.5)
44 (3.7)0 (0.0)
Tumor characteristics
Tumor location
Proximal 1/357 (13.9)94 (14.1)
Middle 1/3136 (33.3)246 (36.9)
Distal 1/3 + GEJ216 (52.8)327 (49.0)
Tumor histology
Squamous cell carcinoma271 (72.7)472 (74.4)
Adenocarcinoma100 (26.8)154 (24.3)
Other2 (0.5)8 (1.3)
Tumor differentiation
G1 (well-differentiated)13 (14.3)11 (14.3)
G2 (moderately differentiated)54 (59.3)51 (66.2)
G3 (poorly differentiated)24 (26.3)15 (19.5)
Clinical stage (according to TNM)
Stage 01 (0.2)2 (0.2)
Stage I243 (39.9)318(30.3)
Stage II181 (29.7)333 (31.7)
Stage III178 (29.2)375 (35.7)
Stage IV6 (1.0)23 (2.2)
Tumor size (cm)5.0 ± 2.14.4 ± 1.8
Neoadjuvant treatment292 (51.8)596 (59.3)
Values are given as means ± SD or n (%). ASA = American Society of Anesthesiologists; GEJ = gastroesophageal junction; RAMIE: Robot-assisted minimally invasive esophagectomy; OE: Open esophagectomy.
Table
Table 3. Summary of meta-analyses for all outcomes.
Table 3. Summary of meta-analyses for all outcomes.
OutcomesSum RAMIESum OpenOR/WMD95% CIp ValueI2 (%)Result
Primary surgical
Overall complications *109/391303/8930.66[0.42, 1.05]0.0850.17NS
Overall pulmonary complications *49/343174/6870.38[0.26, 0.56]<0.0010.00Favors RAMIE
Anastomotic leakage46/67479/13030.93[0.60, 1.44]0.760.00NS
30-day mortality2/2486/4020.75[0.15, 3.78]0.730.00NS
90-day mortality10/42118/8080.80[0.31, 2.05]0.640.00NS
Primary oncological
Lymph nodes resected28.45 ± 14.7121.44 ± 15.533.26[−0.89, 7.41]0.1291.41NS
R0 resection495/6491047/10921.34[0.56, 3.17]0.3313.82NS
Secondary
Operative time †360.39 ± 115.44306.21 ± 94.5869.45[34.39, 104.42]< 0.00196.58Favors OE
EBL ‡209.59 ± 169.02374.38 ± 415.34−187.08[−283.81, −90.35]< 0.00195.50Favors RAMIE
ICU length of stay §1.37 ± 0.561.59 ± 1.55−0.13[−0.28, 0.02]0.0927.67NS
LOS §17.10 ± 9.3930.68 ± 23.88−9.22[−14.39, −4.06]<0.00196.13Favors RAMIE
Pneumonia42/529181/11460.39[0.26, 0.57]<0.0010.00Favors RAMIE
ARDS1/2174/2980.27[0.04, 1.73]0.170.00NS
Atrial fibrillation29/42754/6380.53[0.29, 0.98]0.0414.60Favors RAMIE
Postoperative hemorrhage4/33912/4790.59[0.18, 1.92]0.380.00NS
Chylothorax29/53833/10951.31[0.75, 2.29]0.350.00NS
RLN palsy67/47984/8071.31[0.90, 1.90]0.160.00NS
Wound infection4/31938/6370.20[0.07, 0.57]<0.0010.00Favors RAMIE
* expressed as patients affected/total patients, † in minutes; ‡ in mL; § in days; RAMIE: robot-assisted minimally invasive esophagectomy; OE: open esophagectomy; OR: odds ratio; WMD: weighted mean difference; CI: confidence interval; EBL: estimated blood loss; ICU: intensive care unit; LOS: length of hospital stay; ARDS: acute respiratory distress syndrome; RLN: recurrent laryngeal nerve.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Esagian, S.M.; Ziogas, I.A.; Skarentzos, K.; Katsaros, I.; Tsoulfas, G.; Molena, D.; Karamouzis, M.V.; Rouvelas, I.; Nilsson, M.; Schizas, D. Robot-Assisted Minimally Invasive Esophagectomy versus Open Esophagectomy for Esophageal Cancer: A Systematic Review and Meta-Analysis. Cancers 2022, 14, 3177. https://doi.org/10.3390/cancers14133177

AMA Style

Esagian SM, Ziogas IA, Skarentzos K, Katsaros I, Tsoulfas G, Molena D, Karamouzis MV, Rouvelas I, Nilsson M, Schizas D. Robot-Assisted Minimally Invasive Esophagectomy versus Open Esophagectomy for Esophageal Cancer: A Systematic Review and Meta-Analysis. Cancers. 2022; 14(13):3177. https://doi.org/10.3390/cancers14133177

Chicago/Turabian Style

Esagian, Stepan M., Ioannis A. Ziogas, Konstantinos Skarentzos, Ioannis Katsaros, Georgios Tsoulfas, Daniela Molena, Michalis V. Karamouzis, Ioannis Rouvelas, Magnus Nilsson, and Dimitrios Schizas. 2022. "Robot-Assisted Minimally Invasive Esophagectomy versus Open Esophagectomy for Esophageal Cancer: A Systematic Review and Meta-Analysis" Cancers 14, no. 13: 3177. https://doi.org/10.3390/cancers14133177

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop