Next Article in Journal
Transcatheter Aortic Valve Replacement for Failed Surgical or Transcatheter Bioprosthetic Valves: A Comprehensive Review
Previous Article in Journal
Vitamin D Deficiency Is Frequent in Patients with Rapidly Destructive Osteoarthritis—Data from a Single-Center Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 13
Issue 5
10.3390/jcm13051298
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Efficacy and Safety of Thermal Ablation after Endoscopic Mucosal Resection: A Systematic Review and Network Meta-Analysis

by
Hossein Haghbin
1,*,
Nuruddinkhodja Zakirkhodjaev
2,
Rawish Fatima
3,
Faisal Kamal
4 and
Muhammad Aziz
5
1
Division of Gastroenterology, Ascension Providence Hospital, Southfield, MI 48075, USA
2
Division of Occupational and Environmental Medicine, The University of Texas Health Science Center at Houston, Houston, TX 77021, USA
3
Division of Rheumatology, University of Toledo, Toledo, OH 43606, USA
4
Division of Gastroenterology, Thomas Jefferson University, Philadelphia, PA 19144, USA
5
Division of Gastroenterology and Hepatology, Bon Secours Mercy Health, Toledo, OH 43608, USA
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2024, 13(5), 1298; https://doi.org/10.3390/jcm13051298
Submission received: 7 January 2024 / Revised: 20 February 2024 / Accepted: 23 February 2024 / Published: 25 February 2024
(This article belongs to the Special Issue Colorectal Cancer: Clinical Practices and Challenges)
Browse Figures
Review Reports Versions Notes

Abstract

:
(1) Background: Large colonic polyps during colonoscopy can be managed by Endoscopic mucosal resection (EMR). To decrease the polyp recurrence rate, thermal ablation methods like argon plasma coagulation (APC) and snare tip soft coagulation (STSC) have been introduced. We performed this network meta-analysis to assess the efficacy and safety of these modalities. (2) Methods: We performed a comprehensive literature review, through 5 January 2024, of databases including Embase, PubMed, SciELO, KCI, Cochrane Central, and Web of Science. Using a random effects model, we conducted a frequentist approach network meta-analysis. The risk ratio (RR) with 95% confidence interval (CI) was calculated. Safety and efficacy endpoints including rates of recurrence, bleeding, perforation, and post polypectomy syndrome were compared. (3) Results: Our search yielded a total of 13 studies with 2686 patients. Compared to placebo, both APC (RR: 0.33 CI: 0.20–0.54, p < 0.01) and STSC (RR: 0.27, CI: 0.21–0.34, p < 0.01) showed decreased recurrence rates. On ranking, STSC showed the lowest recurrence rate, followed by APC and placebo. Regarding individual adverse events, there was no statistically significant difference between either of the thermal ablation methods and placebo. (4) Conclusions: We demonstrated the efficacy and safety of thermal ablation after EMR for decreasing recurrence of adenoma.

1. Introduction

Colonoscopy is the gold standard of colorectal cancer prevention through identification and removal of precancerous lesions [1] Large lateral spreading lesions (LSL) are considered high-risk lesions with higher chances of malignancy over time if left untreated [2]. Endoscopic mucosal resection (EMR) is a safe and effective technique to remove large LSLs, saving the patients from undergoing surgical procedures. Compared to surgery, EMR demonstrates a superior morbidity and mortality profile, in addition to being a better cost-effective procedure [3,4,5]. Despite its superior safety profile, EMR has some adverse events. EMR’s complications include perforation (0.5% of cases) and bleeding, whether during the procedure or in a delayed fashion, with rates of 15.8% and 5.9%, respectively [6,7,8].
Local recurrence of a lesion is a concern after performing EMR that requires follow-up surveillance colonoscopies [9]. This is attributed to multiple factors, most notably adherent macroscopic and microscopic disease not detectable by the endoscopist during the index procedure [10,11]. In support of this, thermal ablation of EMR edges through argon plasma coagulation (APC) or snare tip soft coagulation (STSC) has been shown to reduce the local recurrence rate in subsequent surveillance colonoscopies [12,13]. Thermal energy results in desiccation of cells, resulting in cell death in dysplastic tissue and, hence, less recurrence. STSC uses low-voltage pure sinusoidal current to achieve a more superficial and wider tissue ablation compared to cut current [14]. APC uses the flow of argon gas as a conductor to induce thermal effects in a non-contact manner [15]. For tissue ablation in EMR, APC and STSC generator settings range from 25–65 watts based on location of the lesion in the colon [16]. Through visualization of a 3 mm area of white coagulum after treatment, the endoscopists can identify areas that are adequately ablated and avoid applying energy to exposed muscle fibers to minimize the chance of perforation [16]. At the end of the procedure, endoclips are applied to areas of deep injury to minimize the chance of post-procedure bleeding or perforation [17]. While these methods may decrease recurrence rates, their potential drawbacks include increases in post polypectomy bleeding, post polypectomy syndrome (PPS), and perforation [12].
A recent multicenter three-arm randomized controlled trial (RCT) on 384 patients by Rex et al., comparing APC, STSC, and placebo, found superiority of thermal ablation compared to placebo [16]. Prior systematic reviews and meta-analyses have shown the efficacy and safety of thermal ablation modalities [18,19,20]. However, no updated network meta-analysis incorporating an RCT has been performed to draw comparison between these three modalities. Therefore, we decided to perform this updated network meta-analysis to compare the safety and efficacy of STSC, APC, and placebo.

2. Materials and Methods

2.1. Search Strategy

This study was conducted per the “Preferred Reporting Items for Systematic Reviews and Network Meta-Analyses (PRISMA-NMA) statement” [21]. We performed an electronic search of the following databases through 5 January 2024: Embase (https://www.embase.com), Medline (PubMed), Web of Science Core Collection (Web of Science, Clarviate), KCI- Korean Journal Index (Web of Science, Clarviate), SciELO, Global Index Medicus, and Cochrane Central Register of Controlled Trials (Cochrane Library, Wiley). The search strategy was defined by H.H. and M.A. This was refined by an experienced medical librarian. The core concepts of “Snare tip soft coagulation”, “Endoscopic Mucosal resection”, “Argon plasma coagulation”, “polyp recurrence”, and "colonoscopy” were used. We provide the search strategy for Embase as an example (Supplementary Table S1). The records were exported to EndNote 20 (Clarivate, Philadelphia, PA, USA), and duplicates were eliminated.

2.2. Selection Criteria/Included Interventions

All RCTs and prospective/retrospective studies adhering to the following criteria were included: (1) Patients: Adult patients undergoing EMR for colorectal lesions, (2) Outcomes: recurrence rate, adverse events including bleeding, post polypectomy syndrome, and perforation, (3) Comparison/Interventions/Control: STSC, APC, and/or placebo. We excluded reviews, case series, and case reports. We did not restrict the studies based on language or publication date.

2.3. Study Screening/Data Extraction

The authors performed the initial screening of studies by titles and abstracts. This screening and subsequent extraction of data were performed by two independent individuals (H.H. and M.A.). In case of any conflicts, the matters were resolved through mutual discussion. Subsequent screening was performed by full text of articles. Population and study characteristics in addition to outcome data were extracted and recorded in Microsoft Excel 2021 (Microsoft, Redmond, WA, USA).

2.4. Statistical Analysis

We performed direct and network meta-analyses for the outcomes. Direct meta-analysis was conducted for interventions, with head-to-head results reported in at least two studies. The DerSimonian–Laird approach was the primary tool due to presumed heterogeneity among studies. We used Open Meta Analyst v0.24.1 (CEBM, Brown University, Providence, RI, USA) as the software for direct meta-analysis. The frequentist approach was used for performing the network meta-analysis [22]. For proportional outcomes, the analysis generated the risk ratio (RR), 95% confidence interval (CI), p-value and forest plots with placebo as reference. ‘R’ package ‘Netmeta’ version 2.8-2 (Bell labs, Murray Hill, NY, USA) was used for analysis. A random effects model was chosen, given the presumed heterogeneity in studies, and the fixed effects model was used as a sensitivity tool. The intention-to-treat (ITT) protocol was our default assessment for outcomes. The I2 statistic was used for heterogeneity, with >50% indicating significant heterogeneity [23]. “Net splitting” was performed to detect differences between direct and indirect evidence for each network outcome (consistency). A p-value of < 0.05 was considered statistically significant. The treatment groups were then ranked using P-score, with higher values denoting decreased recurrence and/or adverse events [24].

2.5. Bias Assessment

The risk of bias for observational studies and RCTs was performed by risk of bias in non-randomized studies of interventions (ROBINS-I) and the Cochrane risk of bias tool, respectively [25,26]. Quantitative and qualitative publication bias analysis was performed using funnel plots and Egger’s and Thompson–Sharp regression analysis [27,28].

3. Results

Using the inclusion/exclusion criteria, a total of 13 studies with 2686 patients were included (Figure 1 and Table 1) [12,13,16,29,30,31,32,33,34,35,36,37,38,39]. Two of the studies were published as a single article [37]. Of these studies, six compared STSC to placebo, four compared APC to placebo, one APC to STSC, and one study compared all three modalities (Table 1). The studies were published between 2002 and 2023. Details and study characteristics are summarized in Table 1.

3.1. Direct Meta-Analysis

3.1.1. Snare Tip Soft Coagulation vs. Placebo

Nine studies compared STSC to placebo in terms of lesion recurrence rate. Regarding recurrence rate, STSC showed statistical superiority to placebo (RR: 0.252, CI: 0.199–0.320, p < 0.001, I2 = 0%) (Supplementary Figure S1). There was no difference in terms of PPS (two studies, RR: 1.031, CI: 0.108–9.828, p = 0.978, I2 = 0%), perforation (four studies, RR: 0.566, CI: 0.127–2.528, p = 0.456, I2 = 9.847%), delayed bleeding (five studies, RR: 0.821, CI: 0.473–1.427, p = 0.485, I2 = 0%), or intraprocedural bleeding (four studies, RR: 0.573, CI: 0.290–1.135, p = 0.110, I2 = 84.144%) (Supplementary Figure S1). Three studies reported total adverse events. Total adverse events in STSC were lower than in placebo (RR: 0.598, CI: 0.439–0.814, p < 0.001, I2 = 0%) (Supplementary Figure S1).

3.1.2. Argon Plasma Coagulation vs. Placebo

Five studies compared recurrence rates of APC to placebo. Recurrence rates were lower in APC compared to placebo (RR: 0.350, CI: 0.171–0.715, p = 0.004, I2 = 11.433%) (Supplementary Figure S2). There was no difference in terms of PPS (two studies, RR: 0.357, CI: 0.039–3.287, p = 0.363, I2 = 0%), delayed bleeding (three studies, RR: 0.454, CI: 0.191–1.079, p = 0.074, I2 = 0%), or intraprocedural bleeding (two studies, RR: 1.421, CI: 0.109–18.500, p = 0.789, I2 = 31.431%) (Supplementary Figure S2). Three studies reported total adverse events. Total adverse events of APC did not reach statistical significance compared to placebo (RR: 0.500, CI: 0.250–1.002, p = 0.051, I2 = 0%) (Supplementary Figure S2).

3.1.3. Snare Tip Soft Coagulation vs. Argon Plasma Coagulation

Only two studies compared STSC and APC. There were no statistically significant differences between STSC and APC in terms of recurrence rate (RR: 0.837, CI: 0.338–2.069, p = 0.699, I2 = 37.559%), PPS (RR: 0.760, CI: 0.215–2.690, p = 0.670, I2 = 0%), perforation (RR: 1.266, CI: 0.326–4.911, p = 0.733, I2 = 0%), or total adverse event rates (RR: 1.086, CI: 0.588–2.003, p = 0.793, I2 = 0%) (Supplementary Figure S3).

3.2. Network Meta-Analysis

3.2.1. Recurrence Rate

All studies reported recurrence rates. APC showed good efficacy in reducing the recurrence rate of lesions compared to placebo (RR: 0.33, CI: 0.20–0.54, p < 0.01) (Figure 2). Similarly, the use of STSC resulted in reduced recurrence of lesions compared to placebo (RR: 0.27, CI: 0.21–0.34, p < 0.01). There was no statistically significant difference between the two interventions (RR: 0.81, CI: 0.48–1.37, p = 0.44). On ranking, STSC showed the lowest recurrence rate, followed by APC (Figure 3). The heterogeneity for this analysis was 0%.

3.2.2. Total Adverse Events

Six studies reported the total adverse events. Both APC (RR: 0.53, CI: 0.31–0.91, p = 0.02) and STSC (RR: 0.58, CI: 0.43–0.79, p < 0.01) demonstrated lower overall adverse events compared to placebo (Figure 4). On ranking, APC showed the lowest total adverse events, followed by STSC (Figure 3). The heterogeneity for this model was 0%.

3.2.3. Post Polypectomy Syndrome

Four studies evaluated PPS. There was no difference between STSC and placebo in terms of PPS (RR: 0.67, CI: 0.10–4.33, p = 0.67) (Supplementary Figure S4). APC was also safe, with no difference compared to placebo (RR: 0.90, CI: 0.13–6.00, p = 0.91). On ranking, STSC was ranked the highest, followed by APC and placebo (Figure 3). The heterogeneity for this model was 0%.

3.2.4. Perforation

Five studies reported the perforation rates. Neither APC (RR: 0.71, CI: 0.11–4.71, p = 0.72) nor STSC (RR: 0.88, CI: 0.21–3.78, p = 0.86) showed higher rates of perforation in comparison to placebo (Supplementary Figure S5). There was no statistically significant difference between APC and STSC, either (RR: 1.24, CI: 0.33–4.70, p = 0.75). On ranking, APC was ranked the highest, followed by STSC and placebo (Figure 3) in terms of lower perforation rates. The heterogeneity for this model was 0%.

3.2.5. Delayed Bleeding

Seven studies reported the delayed bleeding rates. Neither APC (RR: 0.47, CI: 0.20–1.10, p = 0.08) nor STSC (RR: 0.82, CI 0.47–1.42, p = 0.47) showed higher rates of delayed bleeding in comparison to placebo (Supplementary Figure S6). On ranking, APC was superior to STSC in terms of delayed bleeding (Figure 3). Thermal ablation interventions were ranked higher than placebo regarding delayed bleeding. The heterogeneity for this model was 0%.

3.2.6. Intraprocedural Bleeding

Seven studies reported the intraprocedural bleeding rates. Neither APC (RR: 0.75, CI 0.17–3.22, p = 0.70) nor STSC (RR: 0.61, CI: 0.32–1.17, p = 0.13) showed statistically significant different rates of delayed bleeding compared to placebo (Supplementary Figure S7). On ranking, STSC was ranked the highest, followed by APC and placebo in intraprocedural bleeding (Figure 3). The heterogeneity for this model was 75.6%.

3.3. Bias Assessment

Using the ROBINS-I, ranging from low to high, all of the observational studies were noted to have low risk of bias (Supplementary Table S2). The Cochran risk of Bias tool showed high risk of bias in blinding of personnel in performance and outcome assessments (Supplementary Table S3). This was attributed to lack of blinding of endoscopists during the procedure and in follow-up surveillance. The consistency in primary outcomes of network meta-analysis was evaluated, and no significant difference between direct and indirect evidence was noted (Supplementary Table S4). No asymmetry was noted on visualization of funnel plot using recurrence rate as outcome (Supplementary Figure S8). Further, no significant publication bias was noted using Egger’s regression analysis (p = 0.6443) or Thompson–Sharp regression analysis (p = 0.7073) (Supplementary Figure S8). Publication bias for other outcomes was not applicable due to the low number of studies evaluating these outcomes.

4. Discussion

This systematic review and network meta-analysis showed the efficacy of thermal ablation in reduced recurrence of lesions after EMR, in addition to the safety of these interventions. Between APC and STSC, there was no statistical significance in terms of safety or efficacy.
Our study reaffirmed the efficacy of thermal ablation in reduced recurrence of lesions. Due to the absence of lymphatics in colonic mucosa, EMR is the preferred modality for most colonic LSLs. Although EMR’s cost-effectiveness and safety are better compared to surgery, EMR’s disadvantage is its higher recurrence rates [40,41]. The recurrence is attributed to macroscopic and microscopic residual tissue; therefore, auxiliary thermal ablation techniques like APC and STSC have been introduced to reduce recurrence rates [10]. In support of this, en bloc resection has significantly lower recurrence rates compared to piecemeal resection, as in the latter, inaccurate sequential snare resections may lead to residual tissue [13]. Emphasizing the need for methods to lower recurrence rates, a large Dutch study showed high recurrence rates of polyps larger than 30 mm when resected piecemeal [42]. US multi-society task force on colorectal cancer grades the use of thermal ablation as a conditional recommendation since prior the literature was only of moderate quality [9]. This study will provide further evidence for future guidelines in favor of recommending thermal ablation.
This network meta-analysis demonstrated the safety profile of thermal ablation after ablation of large non-pedunculated lesions. There was no difference in terms of PPS, perforation, intra-procedural bleeding, or delayed bleeding with thermal ablation compared to placebo. Since APC involves the superficial layers of the gastrointestinal lining, it is considered a safe modality. By the same token, only the snare tip is exposed in STSC. This should limit its ability to cause deep damage. Despite this safety profile, the use of APC with 40 watts of energy and inadvertent contact with tissue have been shown to affect deeper layers including muscularis mucosa, leading to complications like perforation and bleeding [43,44]. There is a need for larger studies and more data regarding the safety profile.
With similar safety and efficacy between APC and STSC, other factors like cost-effectiveness may become more important. Rex et al., in their multicenter RCT, demonstrated that STSC was more cost-effective than APC [16]. In STSC, we use the same snare that was already used for the EMR, whereas APC requires insertion of an APC catheter that costs approximately USD 175 to 275 and requires extra procedure time [16]. We could not assess cost-effectiveness, as the data were not available in other studies. We speculate that the requirement for the APC catheter and extra procedure time, combined with similar efficacy, would translate to inferiority of APC compared to STSC in terms of cost-effectiveness.
Auxiliary EMR techniques other than thermal ablation, such as underwater EMR, have been used to reduce polyp recurrence with an acceptable safety profile. Underwater EMR is a technique in which the lesion is immersed under water, which not only leads to elevation and magnification, but also eliminates the need for submucosal injection prior to resection [45]. In a recent meta-analysis, this method was shown to have less recurrence compared to conventional EMR, but was statistically inferior compared to the STSC group [46]. Of note, underwater EMR results may have been confounded by the use of practices like margin marking before starting the EMR in some of the studies [46]. Margin marking has been shown to lower the risk of recurrence, which is likely due to better delineation of edges and reducing the chance of marginal residual tissue [47]. This is a potential confounder for the efficacy of underwater EMR. In addition, the recent large multicenter trial by Rex et al. was not included in this recent meta-analysis, which could potentially affect the results [16].
Endoscopic submucosal dissection (ESD) is another endoscopic modality that is increasingly used to remove colonic neoplasms. ESD consists of the following steps: marking the margins of the lesion, circumferential incision with needle knife, serial submucosal injection, and hemostasis. ESD removes the neoplasm en bloc, leading to lower rates of recurrence [48]. Despite the aforementioned benefits, drawbacks of ESD include higher perforation rates with surgery requirement, post ESD electrocoagulation syndrome, procedure duration, cost, and limited availability of technical expertise [49,50]. Hence, there is a need for risk stratification before conducting ESD so that the benefits will outweigh the drawbacks. The 2019 American Gastroenterology Association (AGA) Institute clinical practice update encourages consideration of ESD in colonic lesions with features suggesting submucosal invasion like depressed component (Paris 0–IIc) and Kudo type V pit pattern, LSTs larger than two to three centimeters especially in the rectosigmoid area where ESD complications are lower, and last, residual or recurrent adenomas [51]. Therefore, many lesions will still require EMR, emphasizing the need for modalities to reduce recurrence rates. The results of our study are reassuring, as the use of ablation methods can reduce recurrence rates associated with EMR and can expand indications for EMR.
Our study had some limitations. First, due to limitations in the number of studies available, we used both RCTs and observational studies. This will lead to biases inherent to observational studies like confounding bias. However, as it is impossible to blind the endoscopist during the procedures and surveillance in RCTs, the bias gap between observational studies and RCTs may not be as pronounced. This is due to the nature of the intervention, in which endoscopists could not be blinded to the treatment group. This is common to many RCTs in endoscopy. Second, there was heterogeneity in terms of APC modalities, STSC coagulation modes, power settings, and generators of different companies. Despite this, the same ablation mechanism of shallow and wide thermal ablation of tissues with a lower voltage pure sinusoidal waveform holds true for all forms of STSC and superficial ablation for APC modalities [14,15] Another source of heterogeneity is that we pooled data of high volume (>100 patients, n = 8) and low volume (<35 patients, n = 3) studies that may mirror variable expertise and skills of the endoscopists. There is a need for future studies to evaluate the effect of endoscopic skills in utilization of these modalities, as personal competencies cannot be neglected. Despite heterogeneity being a limitation in meta-analyses, pooling of these diverse data can potentially provide us with a real-world estimate of safety and efficacy not skewed by expertise of a single endoscopist.

5. Conclusions

In conclusion, we demonstrated reduced recurrence of lesions and no increase in adverse events with the use of thermal ablation compared to placebo. There was no statistically significant difference between APC and STSC in terms of efficacy or safety. There is a need for more studies to compare these two modalities with other auxiliary techniques and evaluate their cost-effectiveness.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm13051298/s1, Figure S1: (A) Forest plot for recurrence rate in snare tip soft coagulation compared to placebo in studies. (B) Forest plot demonstrating post polypectomy syndrome in snare tip soft coagulation compared to placebo in studies. (C) Forest plot showing perforation rate in snare tip soft coagulation compared to placebo in studies. (D) Forest plot for delayed bleeding in snare tip soft coagulation compared to placebo in studies. (E) Forest plot demonstrating intraprocedural bleeding in snare tip soft coagulation compared to placebo in studies. (F) Forest plot illustrating total adverse events in snare tip soft coagulation compared to placebo in studies in this meta-analysis.; Figure S2: (A) Forest plot for recurrence rate in argon plasma coagulation compared to placebo in studies. (B) Forest plot demonstrating post polypectomy syndrome in studies comparing argon plasma coagulation to placebo. (C) Forest plot for delayed bleeding in studies comparing argon plasma coagulation to placebo. (D) Forest plot demonstrating intraprocedural bleeding in studies comparing argon plasma coagulation to placebo. (E) Forest plot illustrating total adverse events in studies comparing argon plasma coagulation to placebo in this meta-analysis.; Figure S3: (A) Forest plot for recurrence rate in snare tip soft coagulation compared to argon plasma coagulation in studies. (B) Forest plot demonstrating post polypectomy syndrome in snare tip soft coagulation compared to argon plasma coagulation in studies. (C) Forest plot showing perforation rate in snare tip soft coagulation compared to argon plasma coagulation in studies. (D) Forest plot illustrating total adverse events in snare tip soft coagulation compared to argon plasma coagulation in studies in this meta-analysis.; Figure S4: Post polypectomy syndrome events using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation); Figure S5: Perforation events using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation); Figure S6: Delayed bleeding events using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation); Figure S7: Intraprocedural bleeding events using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation); Figure S8: Publication bias analysis using funnel plots and Egger’s and Thompson–Sharp regression analysis. (APC: argon plasma coagulation, STSC: snare tip soft coagulation); Table S1: Embase search strategy; Table S2: Risk of bias in non-randomized studies of interventions (ROBINS-I) scale for illustration of risk of bias of studies; Table S3: Risk of bias table of randomized controlled trials using Cochrane risk of bias tool; Table S4: Sensitivity analysis of network meta-analysis using netsplit technique.

Author Contributions

Conceptualization, H.H. and M.A.; methodology, M.A.; software, H.H. and M.A.; validation, H.H., N.Z., R.F. and M.A.; formal analysis, H.H. and M.A.; investigation, H.H. and M.A.; resources, H.H. and M.A.; data curation, H.H., N.Z., R.F. and M.A.; writing—original draft preparation, H.H., N.Z., R.F., F.K. and M.A.; writing—review and editing, H.H., N.Z., R.F., F.K. and M.A.; supervision, M.A.; project administration, M.A.; funding acquisition, Not applicable. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data are contained within the article and Supplementary Materials.

Conflicts of Interest

M.A. is “Colorectal Cancer Clinical Practices and Challenges” special edition guest editor. Otherwise, authors declare no conflicts of interest.

References

  1. Shaukat, A.; Kahi, C.J.; Burke, C.A.; Rabeneck, L.; Sauer, B.G.; Rex, D.K. ACG Clinical Guidelines: Colorectal Cancer Screening 2021. Am. J. Gastroenterol. 2021, 116, 458–479. [Google Scholar] [CrossRef]
  2. Rotondano, G.; Bianco, M.A.; Buffoli, F.; Gizzi, G.; Tessari, F.; Cipolletta, L. The Cooperative Italian FLIN Study Group: Prevalence and clinico-pathological features of colorectal laterally spreading tumors. Endoscopy 2011, 43, 856–861. [Google Scholar] [CrossRef]
  3. Kandel, P.; Wallace, M.B. Colorectal endoscopic mucosal resection (EMR). Best Pract. Res. Clin. Gastroenterol. 2017, 31, 455–471. [Google Scholar] [CrossRef]
  4. Raju, G.S.; Lum, P.J.; Ross, W.A.; Thirumurthi, S.; Miller, E.; Lynch, P.M.; Lee, J.H.; Bhutani, M.S.; Shafi, M.A.; Weston, B.R.; et al. Outcome of EMR as an alternative to surgery in patients with complex colon polyps. Gastrointest. Endosc. 2016, 84, 315–325. [Google Scholar] [CrossRef]
  5. Law, R.; Das, A.; Gregory, D.; Komanduri, S.; Muthusamy, R.; Rastogi, A.; Vargo, J.; Wallace, M.B.; Raju, G.; Mounzer, R.; et al. Endoscopic resection is cost-effective compared with laparoscopic resection in the management of complex colon polyps: An economic analysis. Gastrointest. Endosc. 2015, 83, 1248–1257. [Google Scholar] [CrossRef]
  6. Sethi, A.; Song, L.M.W.K. Adverse Events Related to Colonic Endoscopic Mucosal Resection and Polypectomy. Gastrointest. Endosc. Clin. N. Am. 2015, 25, 55–69. [Google Scholar] [CrossRef] [PubMed]
  7. Ahmad, N.A.; Kochman, M.L.; Long, W.B.; Furth, E.E.; Ginsberg, G.G. Efficacy, safety, and clinical outcomes of endoscopic mucosal resection: A study of 101 cases. Gastrointest. Endosc. 2002, 55, 390–396. [Google Scholar] [CrossRef]
  8. Kantsevoy, S.V.; Adler, D.G.; Conway, J.D.; Diehl, D.L.; Farraye, F.A.; Kwon, R.; Mamula, P.; Rodriguez, S.; Shah, R.J.; Song, L.M.W.K.; et al. Endoscopic mucosal resection and endoscopic submucosal dissection. Gastrointest. Endosc. 2008, 68, 11–18. [Google Scholar] [CrossRef] [PubMed]
  9. Kaltenbach, T.; Anderson, J.C.; Burke, C.A.; Dominitz, J.A.; Gupta, S.; Lieberman, D.; Robertson, D.J.; Shaukat, A.; Syngal, S.; Rex, D.K. Endoscopic Removal of Colorectal Lesions—Recommendations by the US Multi-Society Task Force on Colorectal Cancer. Gastroenterology 2020, 158, 1095–1129. [Google Scholar] [CrossRef] [PubMed]
  10. Kumar, V.; Broadley, H.; Rex, D.K. Safety and efficacy of hot avulsion as an adjunct to EMR (with videos). Gastrointest. Endosc. 2019, 89, 999–1004. [Google Scholar] [CrossRef]
  11. Bahin, F.F.; Pellise, M.; Williams, S.J.; Bourke, M.J. Extended endoscopic mucosal resection does not reduce recurrence compared with standard endoscopic mucosal resection of large laterally spreading colorectal lesions. Gastrointest. Endosc. 2016, 84, 997–1006.e1. [Google Scholar] [CrossRef]
  12. Levenick, J.M.; Groff, A.J.; Manzo, C.; Lester, C.; Maranki, J.L. Hybrid APC Colon EMR, A Novel Approach to Reduce Local Recurrence. Tech. Innov. Gastrointest. Endosc. 2022, 24, 10–15. [Google Scholar] [CrossRef]
  13. Klein, A.; Tate, D.J.; Jayasekeran, V.; Hourigan, L.; Singh, R.; Brown, G.; Bahin, F.F.; Burgess, N.; Williams, S.J.; Lee, E.; et al. Thermal Ablation of Mucosal Defect Margins Reduces Adenoma Recurrence After Colonic Endoscopic Mucosal Resection. Gastroenterology 2019, 156, 604–613.e3. [Google Scholar] [CrossRef]
  14. Li, A.A.; Zhou, M.J.; Hwang, J.H. Understanding the Principles of Electrosurgery for Endoscopic Surgery and Third Space Endoscopy. Gastrointest. Endosc. Clin. N. Am. 2023, 33, 29–40. [Google Scholar] [CrossRef] [PubMed]
  15. Zenker, M. Argon plasma coagulation. GMS Krankenhhyg Interdiszip. 2008, 3, Doc15. [Google Scholar] [PubMed]
  16. Rex, D.K.; Haber, G.B.; Khashab, M.; Rastogi, A.; Hasan, M.K.; DiMaio, C.J.; Kumta, N.A.; Nagula, S.; Gordon, S.; Al-Kawas, F.; et al. Snare Tip Soft Coagulation vs. Argon Plasma Coagulation vs. No Margin Treatment after Large Nonpedunculated Colorectal Polyp Resection: A Randomized Trial. Clin. Gastroenterol. Hepatol. 2023, submitted; accepted; in press. [Google Scholar] [CrossRef] [PubMed]
  17. Shah, E.D.; Pohl, H.; Rex, D.K.; Wallace, M.B.; Crockett, S.D.; Morales, S.J.; Feagins, L.A.; Law, R. Valuing innovative endoscopic techniques: Prophylactic clip closure after endoscopic resection of large colon polyps. Gastrointest. Endosc. 2020, 91, 1353–1360. [Google Scholar] [CrossRef] [PubMed]
  18. Kemper, G.; Turan, A.S.; Schoon, E.J.; Schrauwen, R.W.M.; Epping, L.S.M.; Gerges, C.; Beyna, T.; Neuhaus, H.; Gündug, U.; Siersema, P.D.; et al. Endoscopic techniques to reduce recurrence rates after colorectal EMR: Systematic review and meta-analysis. Surg. Endosc. 2021, 35, 5422–5429. [Google Scholar] [CrossRef] [PubMed]
  19. Kandel, P.; Hussain, M.; Yadav, D.; Dhungana, S.K.; Brahmbhatt, B.; Raimondo, M.; Lukens, F.J.; Bachuwa, G.; Wallace, M.B. Post-EMR for colorectal polyps, thermal ablation of defects reduces adenoma recurrence: A meta-analysis. Endosc. Int. Open 2022, 10, E1399–E1405. [Google Scholar] [CrossRef]
  20. Chandan, S.; Facciorusso, A.; Ramai, D.; Deliwala, S.; Mohan, B.P.; Kassab, L.L.; Draganov, P.V.; Othman, M.O.; Kochhar, G.S. Snare tip soft coagulation (STSC) after endoscopic mucosal resection (EMR) of large (>20 mm) non pedunculated colorectal polyps: A systematic review and meta-analysis. Endosc. Int. Open 2022, 10, E74–E81. [Google Scholar] [CrossRef]
  21. Hutton, B.; Catalá-López, F.; Moher, D. The PRISMA statement extension for systematic reviews incorporating network meta-analysis: PRISMA-NMA. Med. Clínica (Engl. Ed.) 2016, 147, 262–266. [Google Scholar] [CrossRef]
  22. DerSimonian, R.; Laird, N. Meta-analysis in clinical trials. Control Clin. Trials. 1986, 7, 177–188. [Google Scholar] [CrossRef] [PubMed]
  23. Julian, P.T.H.; Simon, G.T.; Jonathan, J.D.; Douglas, G.A. Measuring inconsistency in meta-analyses. Br. Med. J. 2003, 327, 557. [Google Scholar]
  24. Rücker, G.; Schwarzer, G. Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med. Res. Methodol. 2015, 15, 58. [Google Scholar] [CrossRef] [PubMed]
  25. Sterne, J.A.; Hernán, M.A.; Reeves, B.C.; Savović, J.; Berkman, N.D.; Viswanathan, M.; Henry, D.; Altman, D.G.; Ansari, M.T.; Boutron, I.; et al. ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. Br. Med. J. 2016, 355, i4919. [Google Scholar] [CrossRef] [PubMed]
  26. Higgins, J.P.; Altman, D.G.; Gøtzsche, P.C.; Jüni, P.; Moher, D.; Oxman, A.D.; Savović, J.; Schulz, K.F.; Weeks, L.; Sterne, J.A. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Br. Med. J. 2011, 343, d5928. [Google Scholar] [CrossRef] [PubMed]
  27. Thompson, S.G.; Sharp, S.J. Explaining heterogeneity in meta-analysis: A comparison of methods. Stat. Med. 1999, 18, 2693–2708. [Google Scholar] [CrossRef]
  28. Matthias, E.; George Davey, S.; Martin, S.; Christoph, M. Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 1997, 315, 629. [Google Scholar]
  29. Abu Arisha, M.; Scapa, E.; Wishahi, E.; Korytny, A.; Gorelik, Y.; Mazzawi, F.; Khader, M.; Muaalem, R.; Bana, S.; Awadie, H.; et al. Impact of margin ablation after EMR of large nonpedunculated colonic polyps in routine clinical practice. Gastrointest. Endosc. 2023, 97, 559–567. [Google Scholar] [CrossRef]
  30. Albuquerque, W.; Arantes, V.N.; Coelho, L.G.; Dias, C.A.; Savassi-Rocha, P.R. Complementation by argon plasma coagulation after endoscopic piecemeal resection of large colorectal adenomas. Rev. Col. Bras. Cir. 2013, 40, 404–408. [Google Scholar] [CrossRef]
  31. Brooker, J.C.; Saunders, B.P.; Shah, S.G.; Thapar, C.J.; Suzuki, N.; Williams, C.B. Treatment with argon plasma coagulation reduces recurrence after piecemeal resection of large sessile colonic polyps: A randomized trial and recommendations. Gastrointest. Endosc. 2002, 55, 371–375. [Google Scholar] [CrossRef] [PubMed]
  32. Groff, A.; Manzo, C.E.; Lester, C.; Levenick, J.M. Mo1780 hybrid apc-assisted emr: A new technique to prevent polyp recurrence with 6 month follow-up data. Gastrointest. Endosc. 2020, 91, AB495–AB496. [Google Scholar] [CrossRef]
  33. Kandel, P.; Werlang, M.E.; Ahn, I.R.; Woodward, T.A.; Raimondo, M.; Bouras, E.P.; Wallace, M.B.; Gómez, V. Prophylactic Snare Tip Soft Coagulation and Its Impact on Adenoma Recurrence after Colonic Endoscopic Mucosal Resection. Dig. Dis. Sci. 2019, 64, 3300–3306. [Google Scholar] [CrossRef]
  34. Katsinelos, P.; Lazaraki, G.; Chatzimavroudis, G.; Anastasiadis, S.; Georgakis, N.; Xanthis, A.; Gatopoulou, A.; Anastasiadou, K.; Kountouras, J. A retrospective comparative study of argon plasma versus polypectome snare tip coagulation: Effect on recurrence rate after resection of large laterally spreading type lesions. Ann. Gastroenterol. 2019, 32, 178–184. [Google Scholar]
  35. Kudaravalli, P.; Suresh Kumar, V.C.; Mohan, B.; Kelita, S.; Adler, D.; Sapkota, B. Comparison of adenoma recurrence rate between standard endoscopic mucosal resection vs endoscopic mucosal resection with adjuvant thermal ablation including the novel hybrid apc: A systematic review and meta-analysis. Gastrointest. Endosc. 2023, 97, AB520–AB521. [Google Scholar] [CrossRef]
  36. Nader, S.M.; Lahr, R.E.; Rex, D.K. Impact of margin thermal treatment after EMR of giant (≥40 mm) colorectal lateral spreading lesions. Gastrointest. Endosc. 2023, 97, 544–548. [Google Scholar] [CrossRef]
  37. Perez, I.; Strand, D.S.; Copland, A.P. ID: 3518361 early recurrence of colonic polyps reduced with thermal ablation of endoscopic mucosal resection margins: Retrospective study. Gastrointest. Endosc. 2021, 93, AB105. [Google Scholar] [CrossRef]
  38. Senada, P.A.; Kandel, P.; Bourke, M.; Ball, C.; Crook, J.; Raimondo, M.; Song, L.W.K.; Fukami, N.; Pannala, R.; Gurudu, S.; et al. S0138 Soft Coagulation of the Resection Margin for the Prevention of Residual or Recurrent Adenoma After Endoscopic Mucosal Resection of Large Sessile Colonic Polyps: A Multi-Center, Randomized Controlled Trial. Am. J. Gastroenterol. 2020, 115, S68. [Google Scholar] [CrossRef]
  39. Wehbeh, A.; Vemulapalli, K.; Rex, D.K. Mo1729 snare tip soft coagulation reduces the recurrence rate after emr of large flat and sessile colonic adenomas. Gastrointest. Endosc. 2020, 91, AB470. [Google Scholar] [CrossRef]
  40. Jayanna, M.; Burgess, N.G.; Singh, R.; Hourigan, L.F.; Brown, G.J.; Zanati, S.A.; Moss, A.; Lim, J.; Sonson, R.; Williams, S.J.; et al. Cost Analysis of Endoscopic Mucosal Resection vs Surgery for Large Laterally Spreading Colorectal Lesions. Clin. Gastroenterol. Hepatol. 2015, 14, 271–278.e1-2. [Google Scholar] [CrossRef]
  41. Belderbos, T.D.G.; Leenders, M.; Moons, L.M.G.; Siersema, P.D. Local recurrence after endoscopic mucosal resection of nonpedunculated colorectal lesions: Systematic review and meta-analysis. Endoscopy 2014, 46, 388–402. [Google Scholar] [CrossRef]
  42. Meulen, L.W.; van der Zander, Q.E.; Bogie, R.M.; Keulen, E.T.; van Nunen, A.B.; Winkens, B.; Straathof, J.W.A.; Hoge, C.V.; de Ridder, R.; Moons, L.M.; et al. Evaluation of polypectomy quality indicators of large nonpedunculated colorectal polyps in a nonexpert, bowel cancer screening cohort. Gastrointest. Endosc. 2021, 94, 1085–1095.e2. [Google Scholar] [CrossRef]
  43. Wahab, P.J.; Mulder, C.J.J.; Hartog, G.D.; Thies, J.E. Argon plasma coagulation in flexible gastrointestinal endoscopy: Pilot experiences. Endoscopy 1997, 29, 176–181. [Google Scholar] [CrossRef]
  44. Canard, J.-M.; Védrenne, B.; Bors, G.; Claude, P.; Bader, R.; Sondag, D. Long term results of treatment of hemorrhagic radiation proctitis by argon plasma coagulation. Gastroenterol. Clin. Biol. 2003, 27, 455–459. [Google Scholar]
  45. Choi, A.Y.; Moosvi, Z.; Shah, S.; Roccato, M.K.; Wang, A.Y.; Hamerski, C.M.; Samarasena, J.B. Underwater versus conventional EMR for colorectal polyps: Systematic review and meta-analysis. Gastrointest. Endosc. 2021, 93, 378–389. [Google Scholar] [CrossRef]
  46. Radadiya, D.; Desai, M.; Patel, H.; Srinivasan, S.; Chandrasekar, V.T.; Hassan, C.; Repici, A.; Rex, D.; Sharma, P. Analyzing methods for reducing recurrence rates after endoscopic mucosal resection of large non-pedunculated colorectal polyps: An Indirect pairwise comparison. Gastrointest. Endosc. 2023, (submitted; accepted; in press). [Google Scholar]
  47. Yang, D.; Draganov, P.V.; King, W.; Liu, N.; Sarheed, A.; Bhat, A.; Jiang, P.; Ladna, M.; Ruiz, N.C.; Wilson, J.; et al. Margin marking before colorectal endoscopic mucosal resection and its impact on neoplasia recurrence (with video). Gastrointest. Endosc. 2022, 95, 956–965. [Google Scholar] [CrossRef] [PubMed]
  48. Keihanian, T.; O Othman, M. Colorectal Endoscopic Submucosal Dissection: An Update on Best Practice. Clin. Exp. Gastroenterol. 2021, 14, 317–330. [Google Scholar] [CrossRef] [PubMed]
  49. Fujiya, M.; Tanaka, K.; Dokoshi, T.; Tominaga, M.; Ueno, N.; Inaba, Y.; Ito, T.; Moriichi, K.; Kohgo, Y. Efficacy and adverse events of EMR and endoscopic submucosal dissection for the treatment of colon neoplasms: A meta-analysis of studies comparing EMR and endoscopic submucosal dissection. Gastrointest. Endosc. 2015, 81, 583–595. [Google Scholar] [CrossRef] [PubMed]
  50. Jung, D.; Youn, Y.H.; Jahng, J.; Kim, J.-H.; Park, H. Risk of electrocoagulation syndrome after endoscopic submucosal dissection in the colon and rectum. Endoscopy 2013, 45, 714–717. [Google Scholar] [CrossRef] [PubMed]
  51. Draganov, P.V.; Wang, A.Y.; Othman, M.O.; Fukami, N. AGA Institute Clinical Practice Update: Endoscopic Submucosal Dissection in the United States. Clin. Gastroenterol. Hepatol. 2019, 17, 16–25.e1. [Google Scholar] [CrossRef] [PubMed]
Jcm 13 01298 g001
Figure 1. PRISMA flow diagram.
Figure 1. PRISMA flow diagram.
Jcm 13 01298 g001
Jcm 13 01298 g002
Figure 2. Recurrence rate using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation).
Figure 2. Recurrence rate using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation).
Jcm 13 01298 g002
Jcm 13 01298 g003
Figure 3. Ranking using frequentist approach and graded using P-score 1–100. Note: Higher P-score represented lower rates of recurrence, post polypectomy syndrome, perforation, intraprocedural bleeding, delayed bleeding, and total adverse events (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation).
Figure 3. Ranking using frequentist approach and graded using P-score 1–100. Note: Higher P-score represented lower rates of recurrence, post polypectomy syndrome, perforation, intraprocedural bleeding, delayed bleeding, and total adverse events (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation).
Jcm 13 01298 g003
Jcm 13 01298 g004aJcm 13 01298 g004b
Figure 4. Total adverse events using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation).
Figure 4. Total adverse events using network meta-analysis: (A) Network diagram (the line represents a direct comparison in studies, and width of line represents number of studies), (B) Forest plot with Placebo as comparison group (APC: argon plasma coagulation, CI: confidence interval, RR: relative risk, STSC: snare tip soft coagulation).
Jcm 13 01298 g004aJcm 13 01298 g004b
Table 1. Baseline data of finalized studies in this systematic review and network meta-analysis.
Table 1. Baseline data of finalized studies in this systematic review and network meta-analysis.
Study, YearDesignInterventions ComparedMorphology/SizeTotal PatientsSTSC, nAPC, nPlacebo, n
Abu Arisha 2023 [29]Retrospective and prospectiveSTSC, PlaceboNon-pedunculated, >2 cm764422NR342
Albuqurque 2013 [30]RCTAPC, Placebosessile, >2 cm20NR1010
Brooker 2002 [31]RCTAPC, Placebosessile, >1.5 cm21NR1011
Groff 2020 [32]RetrospectiveAPC, PlaceboNR, >2 cm53NR2825
Kandel 2019 [33]RetrospectiveSTSC, PlaceboNR12060NR60
Katsinelos 2019 [34]RetrospectiveSTSC, APCLST1015150NR
Klein 2019 [13]RCTSTSC, PlaceboLST, >2 cm416210NR206
Levenick 2022 [12]RetrospectiveAPC, PlaceboNon-pedunculated, >2 cm59NR3029
Nader 2022 [36]RetrospectiveSTSC, PlaceboNon-pedunculated, >4 cm201133NR68
Perez 2021 (1) [37]RetrospectiveSTSC, Placebo>2 cm7643NR33
Perez 2021 (2) [37]RetrospectiveSTSC, Placebo<2 cm3518NR17
Rex 2023 [16]RCTSTSC, APC, PlaceboNon-pedunculated, >1.5 cm384126126132
Senada 2020 [38]RCTSTSC, PlaceboLST, >3 cm14873NR75
Wehbeh 2020 [39]RetrospectiveSTSC, PlaceboLST, >2 cm288148NR140
APC: Argon plasma coagulation, LST: lateral spreading tumor, NR: not reported, RCT: Randomized controlled trial, STSC: snare tip soft coagulation.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Haghbin, H.; Zakirkhodjaev, N.; Fatima, R.; Kamal, F.; Aziz, M. Efficacy and Safety of Thermal Ablation after Endoscopic Mucosal Resection: A Systematic Review and Network Meta-Analysis. J. Clin. Med. 2024, 13, 1298. https://doi.org/10.3390/jcm13051298

AMA Style

Haghbin H, Zakirkhodjaev N, Fatima R, Kamal F, Aziz M. Efficacy and Safety of Thermal Ablation after Endoscopic Mucosal Resection: A Systematic Review and Network Meta-Analysis. Journal of Clinical Medicine. 2024; 13(5):1298. https://doi.org/10.3390/jcm13051298

Chicago/Turabian Style

Haghbin, Hossein, Nuruddinkhodja Zakirkhodjaev, Rawish Fatima, Faisal Kamal, and Muhammad Aziz. 2024. "Efficacy and Safety of Thermal Ablation after Endoscopic Mucosal Resection: A Systematic Review and Network Meta-Analysis" Journal of Clinical Medicine 13, no. 5: 1298. https://doi.org/10.3390/jcm13051298

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop