Next Article in Journal
Adenotonsillectomy in Children with Obstructive Sleep Apnea Syndrome: Clinical and Functional Outcomes
Next Article in Special Issue
Endoscopic Vacuum-Assisted Closure (E-VAC) in Septic Shock from Perforated Duodenal Ulcers with Abscess Formations
Previous Article in Journal
Role of Macrophage Colony-Stimulating Factor for Staphylococcal Infection in the Oral Cavity
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 12
Issue 18
10.3390/jcm12185823
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Reducing the Risks of Esophagectomies: A Retrospective Comparison of Hybrid versus Full-Robotic-Assisted Minimally Invasive Esophagectomy (RAMIE) Approaches

by
Jens Peter Hoelzen
1,*,†,
Brooke E. Frankauer
1,†,
Carsten Szardenings
2,
Dhruvajyoti Roy
3,
Lukas Pollmann
1,
Lukas Fortmann
1,
Jennifer Merten
1,
Emile Rijcken
1,
Mazen A. Juratli
1,‡ and
Andreas Pascher
1,‡
1
Department of General, Visceral and Transplant Surgery, University Hospital Muenster, 48149 Muenster, Germany
2
Institute of Biostatistics and Clinical Research, University of Muenster, 48149 Muenster, Germany
3
Department of Surgical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
‡
This research was conducted with the joint efforts of two senior authors, Mazen A. Juratli and Andreas Pascher, who contributed equally to the study.
J. Clin. Med. 2023, 12(18), 5823; https://doi.org/10.3390/jcm12185823
Submission received: 29 July 2023 / Revised: 30 August 2023 / Accepted: 5 September 2023 / Published: 7 September 2023
(This article belongs to the Special Issue Minimally Invasive Surgery: Current Challenges and New Perspectives)
Browse Figures
Versions Notes

Abstract

:
This retrospective analysis aimed to assess and compare the short-term perioperative outcomes and morbidity of hybrid and full-Robotic-Assisted Minimally Invasive Esophagectomy (RAMIE) surgical techniques. A total of 168 robotic-assisted Ivor Lewis esophagectomy procedures performed at Muenster University Hospital were included in the study, with 63 cases in the hybrid group and 105 cases in the full-robotic group. Demographic factors, comorbidities, and tumor stages showed no significant differences between the two groups. However, the full-RAMIE technique demonstrated superiority in terms of overall operative time, postoperative pain levels, and patient morphine consumption. Additionally, the full-RAMIE group exhibited better perioperative outcomes, with significantly shorter ICU stays and fewer occurrences of pneumonias and severe complications. While there was a trend favoring the full-RAMIE technique in terms of severe postoperative complications and anastomotic insufficiencies, further research is required to establish it as the gold standard surgical technique for Ivor Lewis esophagectomy.

1. Introduction

Esophageal cancer is the eighth most common type of cancer worldwide and the sixth leading cause of cancer deaths [1]. Despite the expansion of preventive measures and a heightened medical and societal awareness of risk factors (i.e., nutrition, alcohol, smoking, GERD), the incidence of esophageal cancer is “expected to increase by roughly 35% from 2018 to 2030” [2], thus making it one of the fastest growing malignancies. With an approximate 40% five-year survival rate, esophageal cancer also remains one of the deadliest cancers perioperatively. However, advances in surgical–oncological care have the potential to lead to higher survival rates and better long-term treatment options [3]. To this end, it is not just the pathology itself that remains challenging: esophagectomy as a surgical procedure suffers from an array of varying routine approaches (open, minimally invasive, hybrid or full robotic) and lack of a clear gold standard technique. Ultimately, this plurality leads to a more complex assessment of perioperative outcomes, and as such, imminently suboptimal disease management.
While it is clear that surgical resection with neoadjuvant therapy continues to be the optimal treatment as it provides potential curative outcomes, the extent of resection and ideal surgical approach remain contentious, and consequently varies greatly between institutions [4,5,6]. The growing adoption of robotic-assisted minimally invasive esophagectomy (RAMIE) presents one pillar of a safe surgical resection that yields significant short-term benefits over other hybrid minimally invasive esophagectomy (MIE) and open surgery (OE) techniques, such as reduced hospital stays, faster functional recovery, lower postoperative pain, and fewer overall severe complications [7,8,9,10,11].
Historically, esophagectomy procedures have widely been recognized as one of the most complex major oncologic operations [12]. MIE approaches have been shown to reduce surgical trauma, pain, and postoperative complications, thus affecting both short-term and long-term quality of life [13,14,15]. Reduced peri- and postoperative complications have allowed RAMIE to emerge not only as a desired treatment option for patients but also as an optimal solution to the technical limitations of conventional MIE, namely the use of rigid, straight instruments that have a large distance between trocar and the target anatomy [9,10,16]. Furthermore, RAMIE is associated with an increased rate of textbook outcomes that could prevent postoperative complications and improve long-term survival [17,18].
These factors support the clear trend in the literature towards increasingly minimally invasive procedures, which favors RAMIE to MIE. Although RAMIE has additional benefits such as better visualization, ability to perform more complex surgical maneuvers, and more precise lymph node dissections in smaller spaces [19,20,21,22,23], the evidence for robotic use in the abdominal phase is minimal [24,25]. With continuous advancement in the technological capabilities of robotic software between the Si and Xi da Vinci Robot systems (Da Vinci Robotic System Intuitive Surgical Inc., Sunnyvale, CA, USA), machine learning, and surgeon experience, the RAMIE method has an ever-growing list of potential benefits to patients, making robotic dissection in tight spaces overall superior [9,23,26]. It is this notable innovation in robotic technology that now allows for a full-robotic approach. It is also what motivated this retrospective study.
Initially, our center performed a hybrid-RAMIE technique with conventional abdominal laparoscopy and robot-assisted thoracic phase using the Si da Vinci robot system. With the implementation of the Xi da Vinci robot system, the minimally invasive technique went a step further. The ability to use Endowrist instruments and the smaller robotic arms made multiquadrant surgery in the abdominal phase possible [24]. The trajectory of RAMIE approaches at our center highlights an inherent issue: although gradual advancements in technology allow for what we can safely, if cautiously, call a potential refinement of techniques, the procedure’s nuanced spectrum and limited (yet growing) availability of new robotic systems means that a surgical gold standard for RAMIE techniques has yet to be established [5]. It is from these considerations that this study aims to compare the two most commonly performed RAMIE techniques, and in doing so, hopes to make a valuable contribution not only to the field of upper gastrointestinal (robotic) surgery, but also to the perioperative care, treatment, and outcomes of esophagectomy patients. As a practical and patient-centered way forward, we hope that our findings lead to the establishment of a gold standard technique, and yet more effectively, the widespread adoption of the full-RAMIE procedure.

2. Materials and Methods

2.1. Study Design

This study is a university single-center, investigator-initiated, retrospective comparison efficacy trial of robot-assisted hybrid-RAMIE and full-RAMIE surgical techniques in patients with resectable intrathoracic esophageal cancer. The Muenster University Hospital is a high-volume esophagectomy center with experience in OE, MIE, and RAMIE surgical techniques and therefore optimally positioned to report on short-term postoperative findings. Consequently, the main objective was to determine the efficacy of two RAMIE surgical approaches regarding short-term perioperative parameters including operating time, blood loss, incidence of severe postoperative complications (Clavien-Dindo Classification > 3b), length of hospital stay, length of Intensive Care Unit (ICU) stay, postoperative incidence of pneumonia, and anastomosis insufficiency, as defined by the Esophageal Complications Consensus Group (ECCG) [27]. Secondary endpoints include median opioid consumption in Morphine milligram Equivalent (MME), and median postoperative pain NRS (numeric rating scale).

2.2. Patients

Inclusion criteria were as follows: all adult patients (≥18 years) with histologically proven, surgically resectable (at the time of diagnosis) intrathoracic or abdominal esophageal carcinoma treated surgically at the Muenster University Hospital between January 2019 and December 2022. Selection between groups was established retrospectively and based on the RAMIE technique performed at the time of surgery. The hybrid-RAMIE procedures using the da Vinci Si were performed from January 2019 until December 2020, whereas the full-RAMIE procedures using the da Vinci Xi were performed from January 2021 through December 2022. All patients with cervical esophageal carcinoma, patients with carcinoma of the gastro-esophageal junction with gastric cardia infiltration (Siewert Type-III), and all patients whose surgery was intraoperatively terminated upon diagnosis of inoperability, were excluded from this study. All patients were treated with a neoadjuvant therapy, either radiochemotherapy or chemotherapy, following either the CROSS- (Carboplatin/Paclitaxel) or FLOT- schemes (Fluorouracil/Leucovorin/Oxaliplatin/Docetaxel) [4,28,29,30]. Optimal treatment was decided individually on recommendation from an interdisciplinary tumor conference board and in accordance with the guidelines set forth by the German Cancer Society (DKG). All data were gathered retrospectively from electronic patient charts and medical records. Prior to surgery, all patients received standard diagnostic care which included an esophagogastrodudodenoscopy, abdominal ultrasound, and pulmonary function test. All patients in this study were treated by the same, highly experienced surgeon, who has been performing upper gastrointestinal tract surgeries at our high-volume center since 2008. To ensure adequate perioperative pain management, each patient received an epidural catheter that was placed prior to surgery. Lastly, each patient was given intravenous antibiotics (3 g Cefuroxime and 500 mg Metronidazole) as a preoperative prophylaxis thirty minutes prior to the start of surgery.

2.3. Surgical Technique

The robotic-assisted Ivor Lewis minimally invasive esophagectomy (RAMIE) consists of two phases: abdominal and thoracic, with gastrolysis, en-bloc esophagectomy with a D2 lymphadenectomy and gastric conduit formation via a 45 linear gastric stapler [8,20,24,26,31]. The anastomosis perfusion is evaluated using ICG fluorescence imaging. The thoracic phase is robotic-assisted and identical in both approaches. It is only in the abdominal phase that the two groups and procedures differ. Additionally, all patients received preemptive endoluminal vacuum therapy (EndoVAC) placement under endoscopic control. Finally, it is worth pointing out our nomenclature for the surgical techniques, as what has previously been referred to in other publications as “RAMIE”, we define here as “hybrid-RAMIE”, whereas the abdominal phase is performed via conventional laparoscopy and only the thoracic phase assisted by the da Vinci Si System. In the full-RAMIE group, both thoracic and abdominal phases are robotically performed using the da Vinci Xi system.

2.4. Postoperative Care

Postoperative patients were first transferred to the ICU and then, once hemodynamically and respiratorily stable, moved to the general surgical ward. Physical therapy and rehabilitation were the same in both groups and included early mobilization. Patient oral intake and nutrition began with liquids and increased gradually to soft food within three postoperative days. Patients received multimodal pain therapy including epidural catheter, non-steroid anti-inflammatory drugs, Metamizol and opioid analgesics, as needed for breakthrough pain [32]. Quality of the epidural anesthesia was controlled daily by experienced acute pain specialists. On average, epidural catheters were removed on the fifth postoperative day. Only two patients did not have an epidural catheter, both of which belonged to the full-RAMIE group. Evaluation of subjective pain levels occurred routinely multiple times per day using the NRS-11 for pain. On the fifth postoperative day, a routine esophagogastroduodenoscopy (EGD) was performed to ensure anastomotic sufficiency. If irregularities were found, EndoVAC therapy was continued. Otherwise, the EndoVAC sponge was removed and a gastrin esophagogram was performed. Chest tube and wound drains were removed as soon as the drainage output was at or below 200 mL per day. Throughout the study, there were no significant changes in postoperative care or pain management.

2.5. Endpoints

To compare the extent of robotic assistance in RAMIE techniques, we assessed perioperative and short-term postoperative parameters retrospectively using patient charts and data. Perioperative primary endpoints included baseline demographics such as age, sex, body mass index, Charlson-Comorbidity Index, ASA score (American Society of Anesthesiologists), tumor entities, preoperative tumor stage, and preoperative lymph node stage. Additional primary endpoints included surgery duration, blood loss, duration of postoperative ICU-stay, total hospital stay, anastomotic insufficiency, pneumonia, re-operation rates, postoperative resection status, 30-day morbidity, and percentage rates of severe postoperative complications (Clavien-Dindo > 3b). Secondary endpoints included mean subjective pain NRS levels during rest and stress on the first, third and fifth postoperative days (POD), cumulative morphine milligram equivalents (MME), as well as MME normalized by patient weight and duration of stay.

2.6. Statistical Analysis

Data were analyzed using SPSS Version 29.0 (IBM Corp. Released 2022. IBM SPSS Statistics for Windows, Version 29.0, IBM Corp., Armonk, NY, USA) and R Version 4.20 (R Foundation for Statistical Computing, Wirtschaftsuniversität Wien, Vienna, Austria). Levels of categorical and ordinal variables are displayed as percentages, except for NRS-pain values and postoperative complications, which are summarized by medians and interquartile ranges. Continuous variables and durations in days are summarized by medians and interquartile ranges. One patient, who died of sepsis on the fourth postoperative day, was excluded from the analyses of NRS-pain POD 5 and time until bowel movement. Otherwise, there were no missing data. Significance testing was conducted in R employing Fisher’s exact test for categorical variables and Wilcoxon–Mann–Whitney test for all other variables with all p-values two sided, and a local significance level of 0.05; unadjusted p-values are reported.

3. Results

Our cohort of 168 patients was homogeneous in terms of baseline characteristics. As seen in Table 1, demographic factors, comorbidities (ASA score and Charlson Comorbidity Index), pre-therapeutic tumor stages, and neoadjuvant therapies are comparable between groups.
The total operating time was significantly shorter in the full-RAMIE group (median 410, range 367–470) compared with the hybrid-RAMIE group (median 494, range 409–593, p < 0.001). While the median blood loss between groups was similar, the upper quartile was substantially lower in the full-RAMIE group (125 mL vs. 300 mL Q3 upper quartile, Table 2). There were no intraoperative conversions and oncological results were similar between groups with a R0 resection rate of about 97%. Median postoperative opioid consumption (Figure 1) was significantly lower in the full-RAMIE group (median 2.28, range 1.12–4.83 MME/kg bw vs. hybrid median 3.66, range 2.54–7.83, p < 0.001) (Table 3). Postoperative defecation was similar between groups, and there is no evidence for opioid-adverse events. In addition, mean pain NRS scores were reduced on the first, and third postoperative days, and comparable on the fifth (Figure 2 and Figure 3).
There was a tendency for fewer severe complications (Clavien-Dindo > 3b) to be registered postoperatively in the full-RAMIE group (hybrid 14.3% vs. full 8.5%; p = 0.304). Similarly, regarding anastomotic insufficiencies, there was a trend in favor of the full-RAMIE (hybrid 22.2% vs. full 14.2%; p = 0.21). More importantly, we observed significantly fewer pneumonias in the full-RAMIE group (hybrid 20.6% vs. full 7.6%; p = 0.017). While total postoperative stays were the same for both patient groups, intensive care stays were significantly shorter in the full-RAMIE group (median 2, range 1–3 vs. median 3, range 2–6; p < 0.001). One patient in the full-RAMIE group died within 30 days of surgery due to sepsis (Table 3).

4. Discussion

Our results add to the growing body of evidence that robotic-assisted procedures provide better outcomes. The findings highlight the clinical superiority of the full-RAMIE approach both in terms of better surgical outcome and optimized patient care. This is most visible when comparing the risks of postoperative complications such as pneumonia and anastomotic insufficiencies, which were higher or significantly higher among patients who had undergone the hybrid-RAMIE. Conversely, the full-RAMIE group reported less pain, and fewer complications, and experienced shorter ICU stays. We attribute all of these outcomes to the differences in surgical techniques, namely the full-robotic abdominal approach.
Esophagectomy as a procedure is associated with relatively high levels of experienced pain, which may be worsened by excessive manipulation of the abdominal wall. Specifically, we believe that the remote center in the full-RAMIE group leads to reduced trocar movements when compared to the hybrid-RAMIE, and therefore to less manipulation of the abdominal wall during surgery. To this end, between the abdominal and thoracic phases, the abdominal pain appears to have a larger relevance on overall disease-specific pain experienced by the patient—a crucial aspect that would benefit from further evaluation in a randomized control trial.
Using the robotic system in both abdominal and thoracic phases significantly reduced the median operating time and upper quartile blood loss. We acknowledge that intraoperative parameters such as blood loss and operating time are influenced by surgeon and operating team proficiency, and therefore represent a learning curve that shows consistent improvement overtime. Although there is a significant learning curve associated with RAMIE, structured proctoring learning curves tend to be reached faster and have been shown to plateau after around 22–25 cases [24,28,33,34]. While we acknowledge that some results such as surgery time may be a result of systematic standardization and team experience, the quality of the surgery outcomes—especially pain levels and complications—are independent of any potential learning curves and as such can be imminently linked to the benefits of the full-robotic technique.
Anastomotic insufficiencies remain the most common postoperative complications [12], and while our results are no exception to this, the full-RAMIE group had relatively fewer incidences than the hybrid group. Although the benefit of a generalized, prophylactic EndoVAC therapy has not yet been established, its use has been standard procedure at our clinic as a way to proactively prevent and detect anastomotic insufficiencies [35]. This could also have had an impact on the overall severity and explain our significantly lower rate when compared to other studies. In both surgical techniques, the thoracic phases were robotic-assisted. Although more studies are needed to determine significance, we attribute the relative reduction to decreased surgical trauma experienced by patients. Minimally invasive surgery has been shown to cause less surgical trauma and immunologic stress, and it is in light of the latter that we observe our low postoperative complication rate [36]. Surgical trauma in both phases, but here specifically in the abdominal phase, directly influences recovery and healing after surgery.
This reduction in surgical trauma is also reflected in NRS pain scores, as is the need for supplemental opioid analgesics. In both groups, subjective pain levels were relatively low at rest and during activity, which shows effective multimodal pain therapy in both groups. Moreover, there is no evidence for opioid-related adverse outcomes in our patient collective. With multimodal analgesics, pain is expected to be comparable between groups; however, the full-RAMIE group required fewer opioid analgesics than the hybrid-RAMIE group. The full-RAMIE group not only had significantly less pain but as a result but also consumed significantly less morphine-equivalent doses. The significant difference in opioid consumption MME/kg bw/day shows that full-RAMIE patients had less breakthrough pain. This could be due to differences in abdominal insufflation and intraoperative intra-abdominal pressures and manipulation of the abdominal wall during surgery. In laparoscopic abdominal procedures an intraabdominal pressure between 12–15 mmHg is required for adequate visualization. In comparison, robotic assistance allows for intra-abdominal pressure ranging from 8–10 mmHg. Additionally, the robotic trocars hold up and stabilize the abdominal wall with the remote center mechanism, which substantially reduces abdominal wall manipulation and irritation.
With less pressure on the diaphragm and less abdominal wall irritation, the patients in the full-RAMIE group experienced less pain. Reduced pain could also help explain why the full-RAMIE group had a significantly lower incidence of pneumonia, as intense pain results in ineffective breathing, therefore potentially influencing the risk of lung infection [37]. Not only did we observe fewer postoperative complications and less pain, but the full-RAMIE group also subsequently had a faster recovery and spent less time in the ICU when compared to the hybrid-RAMIE group. To this end, we postulate that less manipulation of the abdominal wall could account for less surgical trauma, postoperative pain, complications, and recovery time. Therefore, the full-RAMIE minimizes patients’ perioperative morbidity and risk.
This study derives its strength from a robust retrospective data collection, which means biases, such as confounders and treatment bias are limited. All data used in the study are routine parameters that were collected for every patient and are independent of the treatment method or individual patient care. Additionally, each patient group received the most technically advanced robotic procedure available at the time of treatment, making a placebo effect between the groups nonexistent. Conversely, selection bias could affect results given the Muenster University Hospital’s national reputation as a high-volume esophagectomy center, which means that some patients could have elected to have their surgery performed here. In addition, information bias could have existed due to missing data and the inherent limitations of routine perioperative patient care management documentation. Our hospital has a quality-management team who continuously monitors medical record consistency and accuracy. Lastly, we acknowledge a time period bias as the procedures were not conducted simultaneously but rather consequently.
Other study limitations include the single-institution study design. While minimally invasive techniques are well established, there is currently only one randomized control trial evaluating RAMIE surgery, and no randomly controlled trials evaluating the differences between various RAMIE techniques [10]. Furthermore, the surgical techniques used (hybrid or full) and terminology applied differ significantly among healthcare providers and across clinical settings. Although structured training has been outlined, a universal standard does not yet exist [34]. This limits reproducibility and transferability to other institutions. Due to the popularity of the Ivor Lewis RAMIE, demand is increasing, and more centers have started offering the procedure, even without proctor-led training. Due to a lack of standardization of training, multiple surgery techniques have evolved, which makes comparison between centers difficult, and further highlights the need for establishing an ideal, universally accepted “gold standard” surgical technique. Under the status quo, patient perioperative risk depends on technique and individual surgical experience.
Further randomized control trial studies comparing full-RAMIE to full-MIE laparoscopic approaches (such as the ROBOT-2 trial) are needed [38]. Additionally, a random control trial comparing hybrid- and full-RAMIE techniques should be performed. While our study does not consider in-depth cost analysis for both RAMIE procedures, the hybrid-RAMIE has higher costs associated with it. By using both robotic and laparoscopic technologies in the hybrid-RAMIE, an additional laparoscopic surgical set, camera, tower, and surgical dissection/stapling systems are needed. These requirements influence not only operating room ergonomics but also hospital economics. Likewise, a quicker recovery, less analgesics, and fewer complications seen in the full-RAMIE have a major impact on hospital economics.

5. Conclusions

Our results show that the full-RAMIE is superior to the hybrid-RAMIE for short-term postoperative outcomes and morbidity. While the end anastomosis techniques are identical, we postulate that less manipulation of the abdominal wall in the full-RAMIE group results in less pain and surgical trauma, and therefore a lower incidence of complications. With the additional robotic-assisted abdominal phase in the full-RAMIE group, we see significantly fewer pneumonias, a faster ICU recovery and reportedly less pain. Minimally invasive surgery allows patients to recover more quickly due to less surgical trauma. This represents a potential impact minimally invasive procedures could have on future patient outcomes, if applied on an even greater scale. To this end, RAMIE is also a more patient-centered approach with the potential of improving patients’ immediate outcomes. The enhanced technical aspects in the full-RAMIE procedure allow for a more streamlined, accurate and standardized surgery, which affects disease management as much as it does hospital economics. The accumulative surgeon experience seen in the full-RAMIE also results in shorter operating times. Coupled with meticulous lymph node resection capabilities, the full-RAMIE approach represents a noticeable and permeable surgical–oncological advancement for future patients with esophageal cancer. Ultimately, our findings underscore the relevance of continued research into, and advancement of, full robotic-assisted procedures in the quest for improving surgical outcomes, reducing pain, and establishing a gold standard surgical technique.

Author Contributions

Conceptualization, J.P.H. and A.P.; Methodology, J.P.H., B.E.F. and M.A.J.; Validation, B.E.F. and C.S.; Formal analysis, J.P.H., B.E.F. and C.S.; Investigation, B.E.F., L.P., L.F., M.A.J., E.R. and J.M.; Resources, J.P.H. and A.P.; Data curation, B.E.F. and L.F.; Writing—original draft preparation, J.P.H. and B.E.F.; Writing—review and editing, B.E.F., L.P., D.R. and M.A.J.; Visualization, B.E.F.; Supervision, J.P.H. and A.P.; Project administration, J.P.H. and B.E.F.; Funding acquisition, J.P.H. and A.P. All authors contributed substantially to the interpretation of data, participation in drafting the article and revising it critically for important intellectual content, and final approval of the manuscript before submission. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy and integrity of any part of the work are appropriately investigated and resolved. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of the University of Muenster (Muenster, Germany) and Medical Association of Westphalia-Lippe (11 March 2022/2022-123-f-S).

Informed Consent Statement

Informed consent was obtained from all individual participants included in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available.

Conflicts of Interest

Jens P. Hoelzen is a proctor for Intuitive Surgical Inc. (Sunnyvale, CA, USA). Other authors have no conflicts of interest to declare.

References

  1. Bray, F.; Ferlay, J.; Soerjomataram, I.; Siegel, R.L.; Torre, L.A.; Jemal, A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 2018, 68, 394–424. [Google Scholar] [CrossRef]
  2. Uhlenhopp, D.J.; Then, E.O.; Sunkara, T.; Gaduputi, V. Epidemiology of esophageal cancer: Update in global trends, etiology and risk factors. Clin. J. Gastroenterol. 2020, 13, 1010–1021. [Google Scholar] [CrossRef]
  3. van der Sluis, P.C.; Ruurda, J.P.; Verhage, R.J.; van der Horst, S.; Haverkamp, L.; Siersema, P.D.; Borel Rinkes, I.H.; Ten Kate, F.J.; van Hillegersberg, R. Oncologic long-term results of robot-assisted minimally invasive thoraco-laparoscopic esophagectomy with two-field lymphadenectomy for esophageal cancer. Ann. Surg. Oncol. 2015, 22, 1350–1356. [Google Scholar] [CrossRef]
  4. Shapiro, J.; van Lanschot, J.J.B.; Hulshof, M.C.C.M.; van Hagen, P.; van Berge Henegouwen, M.I.; Wijnhoven, B.P.L.; van Laarhoven, H.W.M.; Nieuwenhuijzen, G.A.P.; Hospers, G.A.P.; Bonenkamp, J.J.; et al. Neoadjuvant chemoradiotherapy plus surgery versus surgery alone for oesophageal or junctional cancer (CROSS): Long-term results of a randomised controlled trial. Lancet Oncol. 2015, 16, 1090–1098. [Google Scholar] [CrossRef] [PubMed]
  5. Kingma, B.F.; Grimminger, P.P.; van der Sluis, P.C.; van Det, M.J.; Kouwenhoven, E.A.; Chao, Y.-K.; Tsai, C.-Y.; Fuchs, H.F.; Bruns, C.J.; Sarkaria, I.S.; et al. Worldwide Techniques and Outcomes in Robot-assisted Minimally Invasive Esophagectomy (RAMIE). Ann. Surg. 2020, 276, e386–e392. [Google Scholar] [CrossRef] [PubMed]
  6. Mariette, C.; Piessen, G.; Briez, N.; Gronnier, C.; Triboulet, J.P. Oesophagogastric junction adenocarcinoma: Which therapeutic approach? Lancet Oncol. 2011, 12, 296–305. [Google Scholar] [CrossRef]
  7. Babic, B.; Müller, D.T.; Jung, J.O.; Schiffmann, L.M.; Grisar, P.; Schmidt, T.; Chon, S.H.; Schröder, W.; Bruns, C.J.; Fuchs, H.F. Robot-assisted minimally invasive esophagectomy (RAMIE) vs. hybrid minimally invasive esophagectomy: Propensity score matched short-term outcome analysis of a European high-volume center. Surg. Endosc. 2022, 36, 7747–7755. [Google Scholar] [CrossRef] [PubMed]
  8. Hoelzen, J.P.; Sander, K.J.; Sesia, M.; Roy, D.; Rijcken, E.; Schnabel, A.; Struecker, B.; Juratli, M.A.; Pascher, A. Robotic-Assisted Esophagectomy Leads to Significant Reduction in Postoperative Acute Pain: A Retrospective Clinical Trial. Ann. Surg. Oncol. 2022, 29, 7498–7509. [Google Scholar] [CrossRef]
  9. Kingma, B.F.; de Maat, M.F.G.; van der Horst, S.; van der Sluis, P.C.; Ruurda, J.P.; van Hillegersberg, R. Robot-assisted minimally invasive esophagectomy (RAMIE) improves perioperative outcomes: A review. J. Thorac. Dis. 2019, 11 (Suppl. 5), 735–742. [Google Scholar] [CrossRef]
  10. van der Sluis, P.C.; van der Horst, S.; May, A.M.; Schippers, C.; Brosens, L.A.A.; Joore, H.C.A.; Kroese, C.C.; Mohammad, N.H.; Mook, S.; Vleggaar, F.P.; et al. Robot-assisted minimally invasive thoracolaparoscopic esophagectomy versus open transthoracic esophagectomy for resectable esophageal cancer. Ann. Surg. 2019, 269, 621–630. [Google Scholar] [CrossRef]
  11. Dindo, D.; Demartines, N.; Clavien, P.A. Classification of Surgical Complications: A New Proposal With Evaluation in a Cohort of 6336 Patients and Results of a Survey. Ann. Surg. 2004, 240, 205–213. [Google Scholar] [CrossRef]
  12. Low, D.E.; Kuppusamy, M.K.; Alderson, D.; Cecconello, I.; Chang, A.C.; Darling, G.; Davies, A.; D’journo, X.B.; Gisbertz, S.S.; Griffin, S.M.; et al. Benchmarking complications associated with esophagectomy. Ann. Surg. 2019, 269, 291–298. [Google Scholar] [CrossRef]
  13. Mariette, C.; Markar, S.R.; Dabakuyo-Yonli, T.S.; Meunier, B.; Pezet, D.; Collet, D.; D’journo, X.B.; Brigand, C.; Perniceni, T.; Carrère, N.; et al. Hybrid Minimally Invasive Esophagectomy for Esophageal Cancer. N. Engl. J. Med. 2019, 380, 152–162. [Google Scholar] [CrossRef]
  14. Biere, S.S.; Van Berge Henegouwen, M.I.; Maas, K.W.; Bonavina, L.; Rosman, C.; Garcia, J.R.; Gisbertz, S.S.; Klinkenbijl, J.H.G.; Hollmann, M.W.; de Lange, E.S.; et al. Minimally invasive versus open oesophagectomy for patients with oesophageal cancer: A multicentre, open-label, randomised controlled trial. Lancet 2012, 379, 1887–1892. [Google Scholar] [CrossRef]
  15. Straatman, J.; van der Wielen, N.; Cuesta, M.A.; Daams, F.; Garcia, J.R.; Bonavina, L.; Rosman, C.; van Berge Henegouwen, M.I.; Gisbertz, S.S.; van der Peet, D.L. Minimally invasive versus open esophageal resection: Three-year follow-up of the previously reported randomized con-trolled trial: The TIME trial. Ann. Surg. 2017, 266, 232–236. [Google Scholar] [CrossRef]
  16. van Hillegersberg, R.; Boone, J.; Draaisma, W.A.; Broeders, I.A.; Giezeman, M.J.; Rinkes, I.B. First experience with robot-assisted thoracoscopic esophagolymphadenectomy for esophageal cancer. Surg. Endosc. 2006, 20, 1435–1439. [Google Scholar] [CrossRef] [PubMed]
  17. Booka, E.; Takeuchi, H.; Suda, K.; Fukuda, K.; Nakamura, R.; Wada, N.; Kawakubo, H.; Kitagawa, Y. Meta-analysis of the impact of postoperative complications on survival after oesophagectomy for cancer. BJS Open 2018, 2, 276–284. [Google Scholar] [CrossRef] [PubMed]
  18. Kamarajah, S.K.; Griffiths, E.A.; Phillips, A.W.; Ruurda, J.; van Hillegersberg, R.; Hofstetter, W.L.; Markar, S.R. Robotic Techniques in Esophagogastric Cancer Surgery: An Assessment of Short- and Long-Term Clinical Outcomes. Ann. Surg. Oncol. 2022, 29, 2812–2825. [Google Scholar] [CrossRef] [PubMed]
  19. Tagkalos, E.; Goense, L.; Hoppe-Lotichius, M.; Ruurda, J.P.; Babic, B.; Hadzijusufovic, E.; Kneist, W.; van der Sluis, P.C.; Lang, H.; van Hillegersberg, R.; et al. Robot-assisted minimally invasive esophagectomy (RAMIE) compared to conventional minimally invasive esophagectomy (MIE) for esophageal cancer: A propensity-matched analysis. Dis. Esophagus 2020, 33, doz060. [Google Scholar] [CrossRef]
  20. Berlth, F.; Mann, C.; Uzun, E.; Tagkalos, E.; Hadzijusufovic, E.; Hillegersberg, R.; Li, H.; Egberts, J.H.; Lang, H.; Grimminger, P.P. Technical details of the abdominal part during full robotic-assisted minimally invasive esophagectomy. Dis. Esophagus 2020, 33 (Suppl. 2), 84. [Google Scholar] [CrossRef]
  21. Yang, Y.M.; Li, B.M.; Yi, J.M.; Hua, R.M.; Chen, H.M.; Tan, L.M.; Li, H.M.; He, Y.M.; Guo, X.M.; Sun, Y.M.; et al. Robot-assisted Versus Conventional Minimally Invasive Esophagectomy for Resectable Esophageal Squamous Cell Carcinoma: Early Results of a Multicenter Randomized Controlled Trial: The RAMIE Trial. Ann. Surg. 2022, 275, 646–653. [Google Scholar] [CrossRef]
  22. Angeramo, C.A.; Bras Harriott, C.; Casas, M.A.; Schlottmann, F. Minimally invasive Ivor Lewis esophagectomy: Robot-assisted versus laparoscopic–thoracoscopic technique. Systematic review and meta-analysis. Surgery 2021, 170, 1692–1701. [Google Scholar] [CrossRef]
  23. Park, S.; Hwang, Y.; Lee, H.J.; Park, I.K.; Kim, Y.T.; Kang, C.H. Comparison of robot-assisted esophagectomy and thoracoscopic esophagectomy in esophageal squamous cell carcinoma. J. Thorac. Dis. 2016, 8, 2853–2861. [Google Scholar] [CrossRef]
  24. de Groot, E.M.; Goense, L.; Ruurda, J.P.; van Hillegersberg, R. State of the art in esophagectomy: Robotic assistance in the abdominal phase. Updates Surg. 2021, 73, 823–830. [Google Scholar] [CrossRef] [PubMed]
  25. Grimminger, P.P.; Staubitz, J.I.; Perez, D.; Ghadban, T.; Reeh, M.; Scognamiglio, P.; Izbicki, J.R.; Biebl, M.; Fuchs, H.; Bruns, C.J.; et al. Multicenter Experience in Robot-Assisted Minimally Invasive Esophagectomy—A Comparison of Hybrid and Totally Robot-Assisted Techniques. J. Gastrointest. Surg. 2021, 25, 2463–2469. [Google Scholar] [CrossRef]
  26. Grimminger, P.P.; Hadzijusufovic, E.; Lang, H. Robotic-assisted Ivor Lewis esophagectomy (RAMIE) with a standardized intrathoracic circular end-to-side stapled anastomosis and a team of two (surgeon and assistant only). J. Thorac. Cardiovasc. Surg. 2017, 66, 404–406. [Google Scholar] [CrossRef]
  27. Low, D.E.; Alderson, D.; Cecconello, I.; Chang, A.C.; Darling, G.E.; D’Journo, X.B.; Griffin, S.M.; Hölscher, A.H.; Hofstetter, W.L.; Jobe, B.A.; et al. International Consensus on Standardization of Data Collection for Complications Associated With Esophagectomy: Esophagectomy Complications Consensus Group (ECCG). Ann. Surg. 2015, 262, 286–294. [Google Scholar] [CrossRef]
  28. Grimminger, P.P.; Tagkalos, E.; Hadzijusufovic, E.; Corvinus, F.; Babic, B.; Lang, H. Change from Hybrid to Fully Minimally Invasive and Robotic Esophagectomy is Possible without Compromises. J. Thorac. Cardiovasc. Surg. 2019, 67, 589–596. [Google Scholar] [CrossRef]
  29. Al-Batran, S.-E.; Hofheinz, R.D.; Pauligk, C.; Kopp, H.-G.; Haag, G.M.; Luley, K.B.; Meiler, J.; Homann, N.; Lorenzen, S.; Schmalenberg, H.; et al. Histopathological regression after neoadjuvant docetaxel, oxaliplatin, fluorouracil, and leucovorin versus epirubicin, cisplatin, and fluorouracil or capecitabine in patients with resectable gastric or gastro-oesophageal junction adenocarcinoma (FLOT4-AIO): Results from the phase 2 part of a multicentre, open-label, randomised phase 2/3 trial. Lancet Oncol. 2016, 17, 1697–1708. [Google Scholar] [CrossRef]
  30. van Hagen, P.; Hulshof, M.C.; van Lanschot, J.J.; Steyerberg, E.W.; van Berge Henegouwen, M.I.; Wijnhoven, B.P.; Richel, D.J.; Nieuwenhuijzen, G.A.; Hospers, G.A.; Bonenkamp, J.J.; et al. CROSS Group. Preoperative chemoradiotherapy for esophageal or junctional cancer. N. Engl. J. Med. 2012, 366, 2074–2084. [Google Scholar] [CrossRef]
  31. Grimminger, P.P.; Hadzijusufovic, E.; Ruurda, J.P.; Lang, H.; van Hillegersberg, R. The da Vinci Xi Robotic Four-Arm Approach for Robotic-Assisted Minimally Invasive Esophagectomy. J. Thorac. Cardiovasc. Surg. 2018, 66, 407–409. [Google Scholar] [CrossRef]
  32. Low, D.E.; Allum, W.; De Manzoni, G.; Ferri, L.; Immanuel, A.; Kuppusamy, M.; Law, S.; Lindblad, M.; Maynard, N.; Neal, J.; et al. Guidelines for Perioperative Care in Esophagectomy: Enhanced Recovery after Surgery (ERAS®) Society Recommendations. World J. Surg. 2019, 43, 299–330. [Google Scholar] [CrossRef] [PubMed]
  33. Kingma, B.F.; Hadzijusufovic, E.; Van der Sluis, P.C.; Bano, E.; Lang, H.; Ruurda, J.P.; van Hillegersberg, R.; Grimminger, P.P. A structured training pathway to implement robot-assisted minimally invasive esophagectomy: The learning curve results from a high-volume center. Dis. Esophagus 2020, 33 (Suppl. 2), 47. [Google Scholar] [CrossRef] [PubMed]
  34. Fuchs, H.F.; Collins, J.W.; Babic, B.; DuCoin, C.; Meireles, O.R.; Grimminger, P.P.; Read, M.; Abbas, A.; Sallum, R.; Müller-Stich, B.P.; et al. Robotic-assisted minimally invasive esophagectomy (RAMIE) for esophageal cancer training curriculum-a worldwide Delphi consensus study. Dis. Esophagus 2022, 35, 55. [Google Scholar] [CrossRef]
  35. Chon, S.H.; Brunner, S.; Müller, D.T.; Lorenz, F.; Stier, R.; Streller, L.; Eckhoff, J.; Straatman, J.; Babic, B.; Schiffmann, L.M.; et al. Time to endoscopic vacuum therapy-lessons learned after > 150 robotic-assisted minimally invasive esophagectomies (RAMIE) at a German high-volume center. Surg. Endosc. 2023, 37, 741–748. [Google Scholar] [CrossRef] [PubMed]
  36. Buunen, M.; Gholghesaei, M.; Veldkamp, R.; Meijer, D.W.; Bonjer, H.J.; Bouvy, N.D. Stress response to laparoscopic surgery: A review. Surg. Endosc. 2004, 18, 1022–1028. [Google Scholar] [CrossRef] [PubMed]
  37. Visser, E.; Marsman, M.; van Rossum, P.S.N.; Cheong, E.; Al-Naimi, K.; van Klei, W.A.; Ruurda, J.P.; van Hillegersberg, R. Postoperative pain management after esophagectomy: A systematic review and meta-analysis. Dis. Esophagus 2017, 30, 1–11. [Google Scholar] [CrossRef]
  38. Tagkalos, E.; van der Sluis, P.C.; Berlth, F.; Poplawski, A.; Hadzijusufovic, E.; Lang, H.; van Berge Henegouwen, M.I.; Gisbertz, S.S.; Müller-Stich, B.P.; Ruurda, J.P.; et al. Robot-assisted minimally invasive thoraco-laparoscopic esophagectomy versus minimally invasive esophagectomy for resectable esophageal adenocarcinoma, a randomized controlled trial (ROBOT-2 trial). BMC Cancer 2021, 21, 1060. [Google Scholar] [CrossRef]
Jcm 12 05823 g001
Figure 1. Box-plot chart presenting the Morphine Equivalent Dose in hybrid-RAMIE and full-RAMIE: (a) MME, (b) MME/kg bw, (c) MME/kg/day.
Figure 1. Box-plot chart presenting the Morphine Equivalent Dose in hybrid-RAMIE and full-RAMIE: (a) MME, (b) MME/kg bw, (c) MME/kg/day.
Jcm 12 05823 g001
Jcm 12 05823 g002
Figure 2. Line chart displaying mean NRS-pain scores at rest on the first, third, and fifth postoperative days.
Figure 2. Line chart displaying mean NRS-pain scores at rest on the first, third, and fifth postoperative days.
Jcm 12 05823 g002
Jcm 12 05823 g003
Figure 3. Line chart displaying mean NRS pains scores at stress on the first, third, and fifth postoperative days.
Figure 3. Line chart displaying mean NRS pains scores at stress on the first, third, and fifth postoperative days.
Jcm 12 05823 g003
Table 1. Baseline characteristics (n = 168).
Table 1. Baseline characteristics (n = 168).
Full-RAMIE (n = 105)
n (%)		Hybrid-RAMIE (n = 63)
n (%)		p-Value
Age, years p = 0.05
<6546 (43.8)39 (61.9)
65–7541 (39.1)14 (22.2)
>7518 (17.1)10 (15.9)
Sex p = 1
F17 (16)10 (15.9)
M88 (84)53 (84.1)
Ethnicity
White105 (100)63 (100)
BMI, kg/m2 p = 0.113
<208 (7.6)2 (3.2)
20–3077 (73.3)41 (65.1)
>3020 (19)20 (31.7)
ASA score p = 0.314
13 (2.9)2 (3.2)
250 (47.6)35 (55.6)
352 (49.5)26 (41.3)
40 (0)0 (0)
Type of carcinoma p = 0.513
Adenocarcinoma87 (82.9)55 (87.3)
Squamous cell carcinoma18 (17.1)8 (12.7)
Locations of tumor p = 0.555
Upper third1 (1.0)0 (0)
Middle third12 (11.4)4 (6.3)
Gastroesophageal junction92 (87.6)59 (93.7)
Neoadjuvant therapy p = 0.269
Chemotherapy31 (29.5)24 (38.1)
Chemoradiotherapy65 (61.9)31 (49.2)
None9 (8.6)8 (12.7)
Charlson Comorbidity Index p = 0.07
17 (6.7)0 (0)
28 (7.6)1 (1.6)
321 (20.0)11(17.5)
424 (22.9)13 (20.6)
514 (13.3)19 (30.2)
613 (12.4)10 (15.9)
78 (7.6)4 (6.3)
85 (4.8)3 (4.8)
94 (3.8)2 (3.2)
100 (0)0 (0)
111 (1.5)0 (0)
pretherapeutic T-status p = 0.645
T110 (9.5)7 (11.1)
T220 (19.0)13 (20.6)
T373 (69.5)42 (66.7)
T42 (1.9)1 (1.6)
pretherapeutic
N-status 		p < 0.01
N057 (54)16 (25)
N+48 (46)47 (74)
BMI: body mass index; ASA: American Society of Anesthesiologists.
Table 2. Intraoperative and pathological statistics (n = 168).
Table 2. Intraoperative and pathological statistics (n = 168).
Full-RAMIE (n = 105)Hybrid-RAMIE (n = 63)p-Values
Operating time,
min *		410 (367, 470)494 (409, 593)p < 0.001
Blood loss, mL *0 (0, 125)0 (0, 300)p = 0.5
Conversions
[n (%)]		0 (0)0 (0)
Conversion thorax0 (0)NA
Conversion abdomen0 (0)0 (0)
Pathological
T-status [n (%)]		 p = 0.8
T026 (24.8)14 (22.2)
T112 (11.4)12 (19.1)
T220 (19)6 (9.5)
T347 (44.8)31 (49.2)
Pathological
N-status [n (%)]		 p = 0.848
N057 (54.3)35 (55.6)
N123 (21.9)9 (14.3)
N216 (15.2)13 (20.6)
N39 (8.6)6 (9.5)
Radicality
of surgery [n (%)]		 p = 1.00
R0102 (97.1)61 (96.8)
R13 (2.9)2 (3.2)
* median (interquartile range).
Table 3. Postoperative statistics (n = 168).
Table 3. Postoperative statistics (n = 168).
Full-RAMIE
(n = 105)		Hybrid-RAMIE
(n = 63)		p-Values
Morphine-equivalent-dose total, mg *194 (105, 360)345 (215, 750.5)p < 0.001
Morphine-equivalent-dose total/kg bw, mg/kg *2.28 (1.12, 4.83)3.66 (2.54, 7.83)p < 0.001
Morphine-equivalent-dose total/kg bw/days hospital stay, mg/kg/days *0.13 (0.07, 0.26)0.24 (0.14, 0.36)p < 0.001
NRS pain POD 1 *
rest0 (0, 1)2 (1, 3)p < 0.001
stress2 (1, 3)3 (2, 4)p < 0.001
NRS pain POD 3 *
rest0 (0, 1)1 (0, 2)p = 0.006
stress1 (0, 2)2 (1, 3)p = 0.058
NRS pain POD 5 *
rest0 (0, 1)0 (0, 1)p = 0.14
stress1 (0, 2)1 (0, 3)p = 0.152
Complications, MCDC *0 (0, 3)2 (0, 3)p = 0.004
Severe complication, MCDC ≤ 3b [n (%)]9 (8.5)9 (14.3)p = 0.304
Pneumonia [n (%)]8 (7.6)13 (20.6)p = 0.017
Anastomotic leakage a
[n (%)]		15 (14.2)14 (22.2)p = 0.21
Type I (conservative)0 (0)0 (0)
Type II
(nonsurgical intervention)		13 (12.3)10 (15.9)
Type III (surgical intervention)2 (1.9)4 (6.3)
Reoperations [n (%)]5 (4.7)7 (11.1)p = 0.134
Hospital stay, days *18 (14, 25)19 (14, 27)p = 0.479
ICU stay, days *2 (1, 3)3 (2, 6)p < 0.001
Epidural anesthesia, days *5 (5, 6)5 (5, 6)p = 0.007
Time until bowel movement, days *5 (4, 7)4 (3, 6)p < 0.001
30-Day mortality [n (%)]1 (0.9)0 (0)p = 1.00
* median (interquartile range). bw, body weight; POD, Postoperative Day; MCDC, modified Clavien Dindo classification; ICU, intensive care unit. a Complications graded according to ECCG [12].
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Hoelzen, J.P.; Frankauer, B.E.; Szardenings, C.; Roy, D.; Pollmann, L.; Fortmann, L.; Merten, J.; Rijcken, E.; Juratli, M.A.; Pascher, A. Reducing the Risks of Esophagectomies: A Retrospective Comparison of Hybrid versus Full-Robotic-Assisted Minimally Invasive Esophagectomy (RAMIE) Approaches. J. Clin. Med. 2023, 12, 5823. https://doi.org/10.3390/jcm12185823

AMA Style

Hoelzen JP, Frankauer BE, Szardenings C, Roy D, Pollmann L, Fortmann L, Merten J, Rijcken E, Juratli MA, Pascher A. Reducing the Risks of Esophagectomies: A Retrospective Comparison of Hybrid versus Full-Robotic-Assisted Minimally Invasive Esophagectomy (RAMIE) Approaches. Journal of Clinical Medicine. 2023; 12(18):5823. https://doi.org/10.3390/jcm12185823

Chicago/Turabian Style

Hoelzen, Jens Peter, Brooke E. Frankauer, Carsten Szardenings, Dhruvajyoti Roy, Lukas Pollmann, Lukas Fortmann, Jennifer Merten, Emile Rijcken, Mazen A. Juratli, and Andreas Pascher. 2023. "Reducing the Risks of Esophagectomies: A Retrospective Comparison of Hybrid versus Full-Robotic-Assisted Minimally Invasive Esophagectomy (RAMIE) Approaches" Journal of Clinical Medicine 12, no. 18: 5823. https://doi.org/10.3390/jcm12185823

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop