Next Article in Journal
Progress of Multiparameter Magnetic Resonance Imaging in Bladder Cancer: A Comprehensive Literature Review
Previous Article in Journal
Stratifying Disease Severity in Pediatric COVID-19: A Correlative Study of Serum Biomarkers and Lung Ultrasound—A Retrospective Observational Dual-Center Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Radioactive Seed Localization for Nonpalpable Breast Lesions: Systematic Review and Meta-Analysis

by
Hortência H. J. Ferreira
,
Carla Daruich de Souza
*,
Lorena Pozzo
,
Martha S. Ribeiro
and
Maria Elisa C. M. Rostelato
Nuclear and Energy Research Institute (IPEN/CNEN—SP), University of São Paulo (USP), Av. Professor Lineu Prestes 2242, São Paulo 05508-000, SP, Brazil
*
Author to whom correspondence should be addressed.
Diagnostics 2024, 14(4), 441; https://doi.org/10.3390/diagnostics14040441
Submission received: 11 November 2023 / Revised: 22 December 2023 / Accepted: 30 December 2023 / Published: 17 February 2024
(This article belongs to the Special Issue Nuclear Medicine Imaging and Therapy in Breast Cancer)

Abstract

:
Background: This study is a systematic review with meta-analysis comparing radioactive seed localization (RSL) versus radio-guided occult lesion localization (ROLL) and wire-guided localization (WGL) for patients with impalpable breast cancer undergoing breast-conserving surgery and evaluating efficacy, safety, and logistical outcomes. The protocol is registered in PROSPERO with the number CRD42022299726. Methods: A search was conducted in the Embase, Lilacs, Pubmed, Scielo, Web of Science, and clinicaltrials.gov databases, in addition to a manual search in the reference list of relevant articles, for randomized clinical trials and cohort studies. Studies selected were submitted to their own data extraction forms and risk of bias analysis according to the ROB 2 and ROBINS 1 tools. A meta-analysis was performed, considering the random effect model, calculating the relative risk or the mean difference for dichotomous or continuous data, respectively. The quality of the evidence generated was analyzed by outcome according to the GRADE tool. Overall, 46 articles met the inclusion criteria and were included in this systematic review; of these, 4 studies compared RSL and ROLL with a population of 1550 women, and 43 compared RSL and WGL with a population of 19,820 women. Results: The results showed that RSL is a superior method to WGL in terms of surgical efficiency in the impalpable breast lesions’ intraoperative localization, and it is at least equivalent to ROLL. Regarding security, RSL obtained results equivalent to the already established technique, the WGL. In addition to presenting promising results, RSL has been proven to be superior to WGL and ROLL technologies.

1. Introduction

Breast cancer is classified as a malignant, genetic, and multifactorial disease. It usually starts in the lobular duct from cellular mutations that develop into DNA damage [1]. This is a complex and gradual process that is related to endocrine, nutritional, and environmental factors. According with the World Health Organization, 685,000 people worldwide died in 2020 as a result of breast cancer, which affected 2.3 million women. The most common cancer in the globe as of the end of 2020 was breast cancer, which had been diagnosed in 7.8 million women in the previous five years [2].
Recommendations for breast cancer detection include participation in a mammographic screening program, that is, a periodic mammography in women who do not present signs or symptoms of the disease [3], which facilitates an early diagnosis, when the cancer is in its early stages. As a result of such programs, currently, one-third of diagnosed lesions are non-palpable [3,4].
The main treatment for breast cancer is surgery, which can be associated with radiotherapy or systemic therapies. Currently, the first choice of treatment for patients with early breast cancer is breast-conserving surgery, a method that prioritizes tumor removal with a safety margin, keeping as much breast tissue as possible [5]. The therapeutic result of breast-conserving surgery does not differ from that of radical surgery [6]. Its benefits are associated with smaller excised tissue volumes, which contribute to the aesthetic needs of patients, better prognosis, and life quality preservation. As the breast is partially excised, without dissection of axillary lymph nodes, it results in conservation of the movement of the affected arm, reducing the lymphedema risk rate [6,7].
Conservative surgery of nonpalpable breast lesions requires an intraoperative localization method. The purpose of this method is to ensure accurate identification of the lesion site during surgery and guide the surgeon in the complete removal of the tumor, sparing tissues and structures not infiltrated by tumor cells [8].
Wire-guided localization (WGL) is currently the most used method [8]. Radio-guided localization techniques, radio-guided occult lesion localization (ROLL), and radioactive seed localization (RSL) are promising alternatives for use in the localization and excision of these lesions [9].
The WGL technique consists of introducing a wire through a guide needle into the lesion area. After stabilizing the wire position, the needle is withdrawn, and the wire projects out of the breast, which must be secured with adhesive tape to keep it in place until surgery. The marking can be guided by ultrasound or stereotaxic mammography. After the procedure, mammography is mandatory to confirm the wire’s location. In this method, the surgical procedure is performed on the same day as the wire implantation to avoid dislocations and infections [10].
The ROLL technique consists of the use of a radiotracer, where about 0.2 to 0.5 mL of 99m-technetium-labeled colloidal albumin (99mTc-MAA) is injected directly into the breast [9], under ultrasound or stereotactic mammography guidance. As technetium is a gamma-type radiation emitter [11], a gamma detector device is needed to identify the marker during surgery [12]. It is possible to perform ROLL in conjunction with sentinel lymph node biopsy (SLNB) using the same radioactive tracer. Because the half-life of technetium-99m is short (6 h), surgery should be performed within 24 h after the lesion is marked [13].
The RSL technique consists of implanting one or more iodine-125 radioactive seeds in the lesion area, under ultrasound or stereotaxic mammography guidance [13]. After the procedure, mammography is performed to confirm the seed location [14]. The use of a gamma detector device, such as a gamma probe, is necessary during surgery to identify the seed location [12,13,14].
The radioactive seed consists of a titanium capsule with millimeter dimensions, and its internal structure contains a radio-opaque marker radiolabeled with iodine-125, varying according to the manufacturer [15,16]. Some have the radioactive material distributed in resin spheres or ceramic matrix, others on a silver wire [17,18].
Iodine-125 has a half-life of approximately 60 days [11], emitting gamma radiation with an average energy of 29 KeV [9], and the activity per seed is between 2 and 13 MBq [15]. Thus, the iodine-125’s physical characteristics permit the surgery to be performed within 2 months, on average, after implantation [12]. In addition, seed insertion can be performed before the initiation of neoadjuvant chemotherapy, and patients can avoid a new procedure the day of the surgery [19,20]. The gamma probe can discriminate energy, so SLNB can be performed in the same surgical procedure because the iodine-125 seed emits ≈29 KeV photons and technetium-99m emits ≈140KeV photons [9,12,13].
The aim of this study was to develop a systematic review with meta-analysis to evaluate the technique of impalpable breast lesions’ intraoperative localization with RSL with regard to its efficacy, safety, and logistics compared to WGL and/or ROLL.

2. Materials and Methods

The systematic review protocol is registered in PROSPERO with the number CRD42022299726. In Supplementary File S1 the search strategy, eligibility criteria of studies for review, summary of the results, and observations are presented.

2.1. Research Methods to Identify and Select Studies

  • Databases: The searches were performed in the following databases: Embase, LILACS, PubMed, SciELO, Web of Science, and clinicaltrials.gov. In addition, a manual search was performed in reference lists. Studies published in languages other than English, Portuguese, and Spanish were excluded. The search was performed to include studies published during the researched period. The search was performed again before the final analysis to include possible recent eligible studies.
  • Search strategy: The identification scope of relevant studies was as wide as possible according to the inclusion criteria. Based on this, the search strategy was to write with controlled and free vocabulary, respecting the syntax of each database. Exclusion filters, including the publication year, were not used. The database search strategy can be found in the Supporting Information.
  • Outcomes: Positive surgical margins, intraoperative re-excision, reoperation, recurrence, failed identification of SLNB, specimen volume, complications, patient assessment of intervention, medical team assessment of intervention, localization time, operative time, and time between site and surgery.
  • Two-step triage: Studies identified through the search strategy were exported to Mendeley [21] for managing references and removing duplicates. The study selection was performed through a two-step triage process, performed independently by two of the authors (Ferreira, Rostelato). Any discrepancies regarding eligibility were resolved by discussion among all authors. In the first step, the triage was performed by reading the title and abstract, to select all studies that potentially met the inclusion criteria. In the second step, the full text of all studies selected in step one was analyzed to confirm eligibility or exclusion from the study. The research results, including the studies’ selection and justified exclusion, were organized in a selection flowchart, according to the guidelines presented in “Preferred Reporting Items for Systematic review and Meta-analysis Protocols” (PRISMA-P) [21].

2.2. Eligibility Criteria of Studies for Review

  • Population
    Inclusion: women, of any age over eighteen years old, diagnosed with a nonpalpable breast lesion, eligible for surgical intervention.
    Exclusion: age under 18, pregnancy, men.
  • Intervention
    Inclusion: RSL with iodine-125 for nonpalpable breast lesion conservative surgery.
    Exclusion: RSL for regions other than the breast.
  • Comparator
    Inclusion: WGL and ROLL for nonpalpable breast lesion conservative surgery.
    Exclusion: WGL and ROLL for regions other than the breast.
  • Study design
    Inclusion: randomized clinical trial and cohort study with more than 10 patients, comparing RSL against ROLL and/or WGL.
    Exclusion: reviews, letters, abstracts, comments, in vitro assays

2.3. Data Analysis

  • Data extraction: Data from all studies included were collected independently by two of the authors (Ferreira, Rostelato). Any discrepancies regarding the collected data were resolved by discussion among all authors. A standard form created by the authors was used in this process to collect the following data: study information (authors, title, year of publication, country, type of study); population characteristics (number of participants, age, clinical and pathological information about the lesion); methodology (assessed interventions, procedure description, assessed outcomes); results (description of results by outcome).
  • Risk of bias assessment: The risk of bias assessment was performed independently by two authors (Ferreira, Rostelato). Any discrepancies regarding the critical assessment were resolved by discussion among all authors. The evaluation was performed for all included studies, using its specific tool for each study design. The risk of bias assessment results was illustrated in graphs with the most important points that could add bias to the review, using the Robvis tool [22]. Risk of bias assessment tools are structured with fixed bias domains, focusing on different aspects of study design, conduct, and reporting. Within each domain, there is a series of questions that aim to obtain information about study characteristics that are relevant to the risk of bias.
    Randomized clinical trials were evaluated according to ROB 2, which is the Cochrane risk of bias tool for randomized trials [23]. The domains evaluated with this tool are random generation and allocation; blinding of participants, professionals, and outcome evaluators; incomplete outcomes; outcome measurement; and selective outcome reporting.
    Observational studies were evaluated according to ROBINS I [24], which is the Cochrane risk of bias tool for evaluating non-randomized studies of interventions. With this tool, the domains evaluated were confusion; study participants selection; intervention classification; deviations from the intended interventions; incomplete outcomes; outcome measurement; and selective outcome reporting.
  • Measures of treatment effectiveness: A combined estimate for effect size and 95% confidence interval (CI = ) was calculated by combining all studies to determine the likely beneficial therapeutic outcome between treatment approaches. Dichotomous data were measured using the relative risk (RR) for SLNB identification failure rate, complication rate, positive resection margin rate, intraoperative re-excision rate, reoperation rate, recurrence rate, and intervention evaluation by the patient. Mean difference (MD) was calculated to measure continuous data such as localization time, time between localization and surgery, operating time, and specimen volume. The relative risk (RR) or odds ratio is the ratio of the probability that an individual in the exposed group will develop the condition studied to the probability that an individual in the comparator group will develop the same condition.
  • Lost data: This was an available case analysis, where data were analyzed for known cases only. To solve the missing data problem, all data were obtained whenever possible by contacting the correspondent author. In cases where data were found to be randomly lost, only existing data were analyzed. All known missing data occurrences will be presented and explored in the Section 4.
  • Assessment of heterogeneity: A statistical heterogeneity test was conducted to detect any differences in effects between the studies. This was performed by reviewing the confidence intervals (CI) for each of the individual studies in the systematic review to assess overlap; and formally conducting a statistical test for heterogeneity using the I2 statistical test. An I2 greater than 50% indicates substantial heterogeneity, and above 75% considerable heterogeneity. The analyzed results were organized into forest plots using the Review Manager software, version 5.4, Cochrane Library, Hoboken, NJ, USA [25].
  • Assessment of publication bias: To analyze the issue of publication bias, risk assessment forms were used to determine the level and type of risk of bias. This information was then entered into the Review Manager 5 software, where the results were summarized in funnel plots, except for outcomes that had less than 10 studies grouped together, because the test’s power is too low to determine symmetry.
  • Data synthesis: The random effects model was chosen to group the data by outcome, using the Review Manager 5 software. Results were reported in forest plots presenting the effect estimate. For quantitative data that were not considered appropriate for grouping due to heterogeneity or missing data, the synthesis was performed qualitatively with a narrative description.
  • Evidence quality: The evidence quality was evaluated for each of the outcomes independently according to the orientations present in the GRADE tool, which is a system for grading evidence and strength of recommendations [26]. The level of evidence represents the confidence level in estimating the effects presented in support of a particular recommendation and can be classified into four levels: high, moderate, low, and very low. The factors responsible for classifying the evidence are related to methodological quality, inconsistency, imprecision, indirect evidence, and publication bias.

3. Results

The database search, performed until 30 November 2023, and the manual reference list search retrieved 4499 records without duplicates, of which 4218 were excluded after screening by title and abstract. Finally, we excluded a further 232 articles in the full-text screening. In total, 46 articles met the inclusion criteria and were included in this systematic review, comprising 10 randomized clinical trials and 36 observational studies. For the study selection flowchat, see Figure 1 [21].

3.1. RSL and ROLL

3.1.1. Comparative Analysis by Outcome

Four observational studies [19,27,28,29] compared RSL and ROLL. The data synthesis from the four studies comprised a population of 1550 women with nonpalpable breast cancer, with ages ranging from 28 to 91 years old, undergoing conservative surgery with RSL or ROLL. Patients with bracketing localization, when two or more markers are used per breast, and/or undergoing neoadjuvant chemotherapy were also included. The general description of the patients’ characteristics is presented in Table 1.
Outcomes of intraoperative re-excision, failed identification of SLNB, patient assessment of intervention, medical team assessment of intervention, localization time, and operative time were not reported by the studies included in this systematic review. Outcomes of specimen volume and complications were only reported by one study, and therefore data synthesis was not performed. The outcomes of positive surgical margins, reoperation, recurrence, and time between site and surgery are presented below.

3.1.2. Positive Surgical Margins

Positive surgical margin rates were reported by three studies [19,27,29], with a low heterogeneity (I2 = 39%). The positive surgical margins risk in conservative surgery with RSL was lower than with ROLL, but this value was not statistically relevant (RR = 0.83, 95% CI = 0.50–1.39; 763 patients; three studies; see Figure 2).

3.1.3. Reoperation

Reoperation rates were reported by four studies [19,27,28,29], with a low heterogeneity (I2 = 17%). The reoperation risk after conservative surgery with ROLL was lower than with RSL, but this value was not statistically relevant (RR = 1.14, 95% CI = 0.75–1.74; 1550 patients; four studies; see Figure 3).

3.1.4. Recurrence

Recurrence rates—local recurrence, regional recurrence, and distant metastasis—were reported by three studies [27,28,29], without heterogeneity (I2 = 0%). The recurrence risk after conservative surgery with RSL was lower than with ROLL (RR = 0.50, 95% CI = 0.29–0.87; 939 patients; three studies; see Figure 4).

3.1.5. Time between Localization and Surgery

The time interval between localization and surgery was reported by three studies [19,27,28], totaling 1301 patients (RSL = 625; ROLL = 676); however, due to missing data, it was not possible to perform the statistical synthesis. The time interval between localization and surgery was longer with RSL, 1 day–31 weeks, than with ROLL, 0–1 day.

3.1.6. Risk of Bias Assessment

The risk of bias in the included studies that evaluated RSL x ROLL was considered moderate to low. Figure 5 presents the general evaluation result of these studies, according to the ROBINS I tool, 2016 version, Cochrane Library, Hoboken, NJ, USA, as they are non-randomized studies.
The main limitations that may introduce bias in the review are related to the domains of result measurement and confounding. For the studies by Donker [27], Niinikoski [28], and Theunissen [29], the methods for evaluating the results were not comparable between the intervention groups for the outcome of recurrence, as the follow-up period was different between the studied groups.
The study by Van der Noorda [19] showed the potential for confounding the intervention effect at baseline because the clinical and pathological characteristics of patients are different between the two groups studied, which may influence treatment decisions, and there is no information on whether the authors used a method of analysis that controlled for important confounding domains. However, the surgical efficiency analyses were performed by subgroup so that the confounding domain did not interfere with these outcomes.

3.1.7. Evidence Quality Assessment

For the outcomes of surgical margins, reoperation, and recurrence, the quality of evidence was classified as high. There is no relevant presence of inconsistency (I2 = 39%, 17% and 0%) or imprecision (CI = 0.5–1.39; 0.75–1.74; 0.29–0.87) for surgical margins, reoperation, and recurrence, respectively. Indirect evidence was not used for any of the outcomes. The risk of bias in the individual studies is low for all outcomes except recurrence, which had a moderate risk of bias. In contrast, the effect magnitude presented on the recurrence outcome is high (RR ≤ 0.5). Publication bias was not assessed due to the small number of studies grouped for each outcome.
The assessment of evidence quality for the time between localization and surgery outcome is inconclusive due to the absence of statistical synthesis. It is not possible to evaluate the inconsistency, imprecision, or publication bias of the results. However, it is noteworthy that the risk of bias in the included studies concerning this outcome is low, and there is no significant presence of indirect evidence.

3.2. RSL and WGL

3.2.1. Comparative Analysis by Outcome

Overall, 43 studies compared RSL and WGL, including 7 randomized controlled trials [30,31,32,33,34,35,36,37], and 32 observational studies [29,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69]. The data synthesis from the 39 studies comprised a population of 19,820 women with nonpalpable breast lesions, with ages ranging from 20 to 92 years old, undergoing conservative surgery with RSL or WGL. Patients with bracketing localization and/or undergoing neoadjuvant chemotherapy were also included. The general description of the patients’ characteristics is presented in Table 2.

3.2.2. Positive Surgical Margins

Positive surgical margin rates were reported by 33 studies [29,30,31,33,34,35,38,39,40,41,42,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,60,64,65,67,68,69,70], with a low heterogeneity (I2 = 36%). The positive surgical margins risk in conservative surgery with RSL was lower than with WGL (RR = 0.78, 95% CI = 0.70–0.87; 15,610 patients; 33 studies; see Figure 6). The narrowest part of the funnel plot is where the most accurate studies are located. This indicates that there is a predominance of studies with higher prevalence, with low risk of publication bias.

3.2.3. Intraoperative Re-Excision

Intraoperative re-excision rates were reported by 15 studies [31,34,35,40,43,45,50,62,63,64,65,66,69,70], with a high heterogeneity (I2 = 95%). The intraoperative re-excision risk in conservative surgery with RSL was higher than with WGL, but this value was not statistically relevant (RR = 1.02, 95% CI = 0.75–1.38; 5623 patients; 15 studies; analysis Figure 7). In the funnel plot, there was a predominance of less accurate studies, symmetrically located in the widest region of the funnel, indicating possible publication bias.

3.2.4. Reoperation

Reoperation rates were reported by 27 studies [29,35,38,39,40,41,42,44,45,46,47,48,49,50,51,52,53,54,55,56,57,60,61,62,63,64,65,66,67,69,70], with a relevant heterogeneity (I2 = 61%). The reoperation risk after conservative surgery with RSL was lower than with WGL (RR = 0.71, 95% CI = 0.61–0.84; 13884 patients; 27 studies; see Figure 8). The narrowest part of the funnel plot, where the most accurate studies are located, has a predominance of studies with higher prevalence, signifying a low risk of publication bias.

3.2.5. Recurrence

Recurrence rates—local recurrence, regional recurrence, and distant metastasis—were reported by three studies [29,32,39], with no heterogeneity (I2 = 0%). The recurrence risk after conservative surgery with RSL was lower than with WGL (RR = 0.41, 95% CI = 0.19–0.86; 1525 patients; three studies; see Figure 9).

3.2.6. Sentinel Lymph Node Biopsy Failure Identification Rate

SLNB failure rates were reported by six studies [30,33,34,45,48,70], with no heterogeneity (I2 = 0%). The SLNB failure risk associated with conservative surgery with RSL and WGL was equivalent (RR = 1.00, 95% CI = 0.35–2.87; 1318 patients; six studies; see Figure 10).

3.2.7. Complications

The complications rates related to marker implantation were reported by 25 studies [30,33,34,35,41,42,43,44,45,46,47,48,49,50,51,52,55,57,60,61,64,67,68,69,70], with a relevant heterogeneity (I2 = 57%). The complications risk in conservative surgery with WGL was lower than with RSL, but this value was not statistically relevant (RR = 1.12, 95% CI = 0.66–1.78; 8672 patients; 25 studies; see Figure 11).
For RSL, complications involved: wrong initial implant of seeds (n = 45); marker placed more than 5 mm from the lesion (n = 17); seed displacement (n = 14); seed dislodgement during surgery (n = 10); failure to remove the seed in the first specimen (n = 10); seed loss (n = 1); seed transection (n = 1); and vasovagal reaction (n = 1).
For WGL, complications involved: wrong initial implant of the wire (n = 31); marker placed more than 5 mm from the lesion (n = 9); accidental marking (n = 1); wire displacement (n = 44); wire dislodgement during surgery (n = 6); failure to remove the wire in the first specimen (n = 7); wire breakage (n = 2); vasovagal reaction (n = 12).
In the funnel plot, there was a predominance of less precise studies, which are asymmetrically located in the widest region of the funnel, indicating possible publication bias.
Post-operative complication rates were reported by 13 studies [29,31,32,41,42,44,48,51,60,61,64,65,66], without heterogeneity (I2 = 0%). The post-operative complications risk after conservative surgery with WGL was lower than with RSL, but this value was not statistically relevant (RR = 1.17, 95% CI = 0.90–1.52; 5525 patients; 13 studies; see Figure 12). Post-operative complications include hematoma, infection, inflammation, seroma, wound rupture, lymphedema, and deep vein thrombosis. The funnel plot shows a predominance of studies with higher prevalence, which indicates a low risk of publication bias.

3.2.8. Time between Localization and Surgery

The time interval between localization and surgery was reported by 25 studies [30,33,34,39,40,41,42,43,44,45,47,48,49,50,52,53,55,56,60,61,64,66,67,69,70], totaling 10455 patients (RSL = 4895; WGL = 5560). However, due to missing data it was not possible to perform statistical synthesis. The time interval between localization and surgery was longer with RSL, 0–47 days, than with WGL, 0–1 day.

3.2.9. Risk of Bias Assessment

The risk of bias in the included randomized trials that evaluated RSL x WGL was considered moderate to low. Figure 13 presents the general result of the evaluation of these studies, according to the ROB 2 tool. Similarly, the risk of bias in the included non-randomized studies that evaluated RSL X WGL was considered moderate to low, except for one study that had a high risk of bias [42]. Figure 14 presents the general evaluation result of these studies, according to the ROBINS I tool.
The main limitations of randomized trials that may introduce bias in the review are related to the domains of incomplete outcomes and other risks of bias such as positive/negative memory cognitive. In Gray’s study [33], five participants were excluded due to missing data. In Bloomquist’s study [30], more participants in the WGL group completed the technology assessment compared to the RSL group. In Parvez’s study [37], there was a loss of participants in the follow-up period, in addition to a loss of data from the evaluation’s initial year.
The studies by Bloomquist [30], Chagpar [31], Fung [32], Gray [33], and Parvez [37] present a sampling bias risk due to the small sample size of the study or a specific group. The Bloomquist study [30] presented a risk of memory bias because the groups were not interviewed in the same period after the procedure for the intervention evaluation outcome. Ong’s study [36] presented a risk of positive/negative memory cognitive bias for the technology assessment outcome, as the potential benefits of RSL over WGL were detailed in the study’s consent information.
The main limitations of non-randomized studies that may introduce bias in the review are related to the domains of confounding, intervention classification, incomplete outcomes, result measurement, and selective outcomes report. The studies by Horwood [46], Jumaa [49], Parvez [55], and Silva [63] did not present information on the intervention classification.
The studies by Aljohani [39], Chiu [42], Diego [43], Horwood [46], Hout [47], Hughes [48], Jumaa [49], Murphy [53], Parvez [55], Pieri [56], Rao [57], Romanoff [59], Sanchez [60], Saphier [61], Srour [64], Srour [65], Stelle [66], and Zénzola [68] showed confusion about the intervention effect on the baseline as the patient’s clinical and pathological characteristics are different between the two study groups, being potential factors to influence treatment decisions, and the authors did not use a regressive analysis method that controlled for this domain.
Chiu’s study [42] did not present information on the intervention classification, nor on the methodology used for data collection and result evaluation. Result data were not available for all participants, and there is no information on the reason or proportion of missing data. Thus, there is insufficient information to assess whether the reported effect estimate was selected based on the results of multiple measurements or different subgroups.
In the studies by Gray [45] and Hughes [48], data for the technology assessment outcome were not available for all participants. The missing data are disproportionate between interventions, and the results were not robust to the missing data. In addition, the result assessment methods were not comparable between the intervention groups for this outcome, as the self-assessment questionnaire was not performed by all patients in the WGL group.
In Romanoff’s study [59], result data were not available for all participants. The missing data are disproportionate between interventions, and the outcomes were not robust to the missing data. In Theunissen’s study [29], the methods for evaluating the results were not comparable between the intervention groups for the recurrence outcome, as the follow-up period was different between the studied groups.
Because the high risk of bias and missing data between studies, the outcomes of specimen volume, operative time, localization time, and intervention evaluation by the patient were excluded from the review in the comparative analysis between RSL and WGL.

3.2.10. Evidence Quality Assessment

For the outcomes of surgical margins, reoperation, recurrence, marker-related complications, and post-operative complications, the evidence quality was classified as high. There is no relevant presence of inconsistency (I2 = 36%, 61%, 0%, 58%, 0%) or imprecision (CI = 0.7–0.87; 0.61–0.84; 0.19–0.86; 0.61–1.78; 0.90–1.52) for surgical margins, reoperation, recurrence, marker-related complications, and postoperative complications, respectively. Indirect evidence was not used for any of the outcomes. The risk of bias in the individual studies is low for all outcomes except recurrence, which had a moderate risk of bias. In contrast, the effect magnitude presented on the recurrence outcome is high (RR ≤ 0.5). The funnel plot of these outcomes was symmetrical, indicating a low risk of publication bias, except for the recurrence outcome, which was not evaluated due to the low number of summarized studies.
For the outcomes of intraoperative re-excision and SLNB identification failed, the evidence quality was moderate. No indirect evidence was used for these outcomes, and the risk of bias from individual studies was low. Specifically, the intraoperative re-excision outcome did not present a relevant imprecision (CI = 0.75–1.38) or publication bias in the funnel plot, however, it presented a relevant heterogeneity (I2 95%). The SLNB identification failure outcome did not present heterogeneity (I2 0%); however, it presented a relevant imprecision (CI = 0.35–2.87). Publication bias for this outcome was not assessed due to the low number of summarized studies.
The evidence quality assessment for the outcome of time between localization and surgery is uncertain because statistical synthesis was not performed. It is not feasible to assess the inconsistency, imprecision, or publication bias of the results. Overall, the risk of bias for this outcome is low, and there is no relevant presence of indirect evidence.

4. Discussion

4.1. Key Findings and Explanations

The outcomes of surgical margins, intraoperative re-excision, reoperation, recurrence, SLNB failure, and specimen volume are related to the intervention’s effectiveness. The outcome of complications addresses safety, and the time between localization and surgery is related to the service’s logistical organization.
The success of breast-conserving surgery depends on the tumor’s complete removal. Thus, intraoperative re-excision during the first surgery can be performed to excise possible positive margins identified in the specimen preparation. This is important because surgery completion with negative surgical margins decreases the risk of reoperation and disease recurrence. Conservative surgery together with SLNB is a common procedure in surgical practice. In this scenario, it is essential that one intervention does not interfere with the result of the other.
Regarding surgical efficacy, the comparison between RSL and ROLL results did not demonstrate statistically significant superiority of one technology over the other for the reported outcomes of positive surgical margins, reoperation, and recurrence. This suggests that RSL is at least equivalent in surgical efficacy to ROLL technology. Considering that there is strong confidence that the true effect is close to the estimated one, given the high evidence quality, it is unlikely that further work will modify the confidence in the effect estimate.
Regarding safety, there is no evidence to support one technique over another with respect to this outcome. Regarding the organization of services, RSL proved to be superior to ROLL, with a longer time interval between localization and surgery.
In the comparative analysis between RSL and WGL, regarding surgical efficacy, RSL, despite having higher intraoperative re-excision rates, obtained lower rates of positive surgical margins, reoperation, recurrence. Regarding the identification failure rate in the BLS, the two technologies demonstrated equivalent efficiency. These results suggest that RSL is superior to WGL in terms of surgical efficiency. Considering that there is strong confidence that the true effect is close to the estimated one, given the high evidence quality, it is unlikely that further work will modify the confidence in the effect estimate.
Regarding safety, WGL obtained superior, but not statistically significant, results compared to RSL in the outcomes of complications. This suggests that RSL is at least equivalent in safety to WGL technology and showed a longer time interval between localization and surgery.
The longer interval time between the localization with RSL and the surgery results in significant efficiency gains for the radiology team, due to the ability to perform all the procedures of the week in a single list, and for the surgical team, as the patient undergoing RSL can be the first of the day with no delays associated with waiting for the implant to be performed on the same day. Furthermore, in this scenario, the radioactive seed can be implanted before the initiation of neoadjuvant chemotherapy, which represents safety for patients who achieve a complete pathological response, because the seed will continue to mark the tumor site for further surgery even after tumor regression. In addition, patients who receive RSL before neoadjuvant chemotherapy avoid another invasive intraoperative localization device placement procedure before surgery.

4.2. Strengths and Limitations

The main limitations of this review are related to safety outcomes, regarding the comparison between RSL and ROLL, as no primary studies were found in the literature that would serve as a basis for this analysis. Regarding the comparison between RSL and WGL, the main limitations of the review are related to the outcomes of specimen volume, operative time, localization time, and intervention evaluation by the patient, which showed low evidence quality and were excluded from the review. In addition, the outcome of the intervention evaluation by the medical team was not evaluated, as no primary studies were found in the literature that reported this result.
These limitations demonstrate gaps in the literature that may serve as clinical questions for future research. This gap presents the need for new primary studies that address the comparison between RSL and ROLL with regard to safety and organization of services. Furthermore, even in the surgical efficiency outcomes, the sample available in the literature is small, requiring more randomized clinical trials and well-defined cohort studies to address these outcomes, so that the statistical analyses are more robust and significant. The same occurs when comparing RSL and WGL in terms of safety and organization of services. The available sample of primary studies available in the literature is small and heterogeneous.

4.3. Comparison with Similar Researches

The results presented in this study on the technology’s effectiveness corroborate the systematic reviews performed by Chan [71] and Moreira [72]. In Chan’s study [71], the risk of positive surgical margins (RR = 0.67, 95% CI = 0.43–1.06; 366 participants) and reoperation (RR = 0.80, 95% CI = 0.48–1.32; 305 participants) with RSL was lower than with WGL, but this value was not statistically relevant, probably due to the small sample present in the study. In Moreira’s study [72], the positive surgical margins risk with RSL was lower than with WGL (RR = 0.84, 95% CI = 0.71–0.99; 8702 participants). RSL also showed an advantage over ROLL in this outcome, but it was not significant (RR = 1.22, 95% CI = 0.83–1.81; 609 participants). Regarding reoperation rates, RSL had lower rates than WGL (RR = 0.73, 95% CI = 0.58–0.92; 7634 participants) and showed a non-significant advantage over ROLL (RR = 1.07, 95% CI = 0.63–1.83; 609 participants). A non-significant advantage of RSL over WGL was demonstrated in terms of recurrence rates (RR = 0.55, 95% CI = 0.25–1.17; 1526 participants).
In both studies by Chan [71] and Moreira [72], RSL proved to be superior in surgical efficiency outcomes when compared to ROLL and WGL. However, this result was not statistically relevant, which can be explained due to the smaller sample of primary studies incorporated in these systematic reviews. The systematic review developed in this research is updated with a more robust sample, presenting statistically significant comparative data that prove the superiority of RSL compared to WGL. Regarding the RSL x ROLL comparison, our sample was not yet large enough to demonstrate a relevant statistical difference between the two technologies.

4.4. Implications and Actions Needed

This systematic review finds RSL to be superior to other techniques. An implementation protocol is needed to aid physicians and hospitals to adopt it in their clinical practice.
Initiating the implementation of the RSL involves several crucial steps. These include the formation of a procedural committee, providing education to staff members (covering areas such as radiology, surgery, pathology, radiological safety, breast imaging, and nuclear medicine), establishing a radiological safety committee, and gaining approval from a dedicated committee to investigate and adhere to federal and state-specific nuclear regulatory standards. Additionally, creating an inventory of materials and instrumentation associated with the technique, implementing a seed-tracking system, and developing emergency recovery procedures are indispensable components.
The RSL program necessitates the formulation of an institution-specific standard operating procedure protocol. This protocol is designed to outline professional practices related to radiation safety aspects, the requisition and storage of seeds, obtaining patient consent, conducting radiological procedures for placing breast localization devices, carrying out surgical procedures, handling post-surgical pathology, managing seed disposal, and releasing the patient [72].

4.5. Costs

The invoicing for the operating room, anesthesia, radiology technicians, and hospital software for material inventory management is the same for RSL and WGL, not including startup costs. When the institution already has a nuclear medicine sector, implementation also becomes more sustainable, as there will be no cost to hire technical staff. Additionally, most materials will already be available. The equipment includes two sodium iodide detectors for seed documentation in the radiology and pathology departments, a navigation system and a portable gamma probe for surgery, and imaging equipment to guide the procedure and for specimen radiography [72].
The cost of materials, as well as the cost per case, which in addition to the materials considers the fees of the professional team and possible complications of the procedure, is similar between the technologies. However, in the long term, considering costs and efficiency in the use of resources, there is a cost saving with RSL per case in both the public and private health services, based on the flexibility of scheduling between marker implantation and surgery, with fewer delay times in the operating room. The longer interval time between RSL localization and surgery results in significant efficiency gains for the radiology team, due to the ability to perform all the week’s procedures in a single list, and for the surgical team, as the patient undergoing RSL can be the first of the day without the delays associated with waiting for the implant to be performed on the same day [50,69].
Assuming a decrease in the reoperation rate with RSL, in the public health service there is a decrease in costs per case, while in the fee-for-service model switching to RSL results in a loss of revenue per case, because without reoperation the number of surgeries decreases.

5. Conclusions

This study presented a systematic review and meta-analysis comparing radioactive seed localization (RSL) versus radio-guided occult lesion localization (ROLL) and wire-guided localization (WGL) for patients with impalpable breast cancer undergoing breast-conserving surgery, evaluating the efficacy, safety, and logistical outcomes. Selected studies were submitted to their own data extraction forms and risk of bias analysis according to the ROB 2 and ROBINS 1 tools. The meta-analysis was performed, considering the random effect model, calculating the relative risk or the mean difference for dichotomous or continuous data, respectively. The quality of the evidence generated was analyzed by outcome according to the GRADE tool. Overall, 46 articles met the inclusion criteria and were included in this systematic review, including 4 studies comparing RSL and ROLL with a population of 1550 women and 43 comparing RSL and WGL with a population of 19,820 women. The results showed that RSL is a superior method to WGL in terms of surgical efficiency in the impalpable breast lesions’ intraoperative localization and is at least equivalent to ROLL. Regarding security, RSL obtained results equivalent to the already established technique, the WGL. In addition to presenting promising results about the organization of services, RSL was proven to be superior to WGL and ROLL technologies.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics14040441/s1. File S1: It includes the search strategy, eligibility criteria of studies for review, summary of the results, and observations. References [71,72,73,74,75,76,77,78,79,80,81,82] cited in Supplementary Materials.

Author Contributions

H.H.J.F.: Conceptualization, data curation, formal analysis, investigation, methodology, software, validation, visualization, writing—original draft; C.D.d.S.: Formal analysis, validation, visualization, writing—original draft; L.P.: conceptualization, data curation, formal analysis, validation, visualization; M.S.R.: Conceptualization, data curation, formal analysis, validation, visualization; M.E.C.M.R.: Conceptualization, data curation, formal analysis, investigation, validation, visualization. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No new data were created.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Wilson, C.; Kuusk, U.; Dingee, C.; McKevitt, E. Coordination of radiological and clinical care for breast cancer diagnosis in BC. Br. Columbia Med. J. 2018, 60, 33–39. [Google Scholar]
  2. World Heath Organization. Breast Cancer. Available online: https://www.who.int/news-room/fact-sheets/detail/breast-cancer#:~:text=In%202020%2C%20there%20were%202.3,the%20world’s%20most%20prevalent%20cancer2021 (accessed on 18 August 2022).
  3. Scully, O.J.; Bay, B.-H.; Yip, G.; Yu, Y. Breast Cancer Metastasis. Cancer Genom.—Proteom. 2012, 9, 311–320. [Google Scholar]
  4. Veronesi, U.; Cascinelli, N.; Mariani, L.; Greco, M.; Saccozzi, R.; Luini, A.; Aguilar, M.; Marubini, E. Twenty-Year Follow-up of a Randomized Study Comparing Breast-Conserving Surgery with Radical Mastectomy for Early Breast Cancer. N. Engl. J. Med. 2002, 347, 1227–1232. [Google Scholar] [CrossRef] [PubMed]
  5. Apantaku, L.M. Breast-conserving surgery for breast cancer. Am. Fam. Physician 2002, 66, 2271–2278. [Google Scholar]
  6. Xiang, J.; Huang, S.; Tuo, Y.; Wang, Y. Effect of breast-conserving surgery combined with sentinel lymph node biopsy and axillary preservation on the recurrence, metastasis, complications and cosmetic results of early breast cancer patients. Gland. Surg. 2020, 9, 1019–1025. [Google Scholar] [CrossRef]
  7. Weber, W.P.; Soysal, S.D.; Fulco, I.; Barandun, M.; Babst, D.; Kalbermatten, D.; Schaefer, D.J.; Oertli, D.; Kappos, E.A.; Haug, M. Standardization of oncoplastic breast conserving surgery. Eur. J. Surg. Oncol. 2017, 43, 1236–1243. [Google Scholar] [CrossRef]
  8. Skinner, K.A.; Silberman, H.; Sposto, R.; Silverstein, M.J. Palpable breast cancers are inherently different from nonpalpable breast cancers. Ann. Surg. Oncol. 2001, 8, 705–710. [Google Scholar] [CrossRef]
  9. Ahmed, M.; Douek, M. ROLL versus RSL: Toss of a coin? Breast Cancer Res. Treat. 2013, 140, 213–217. [Google Scholar] [CrossRef]
  10. Ernst, M.F.; Avenarius, J.K.; Schuur, K.H.; Roukema, J.A. Wire localization of non-palpable breast lesions: Out of date? Breast 2002, 11, 408–413. [Google Scholar] [CrossRef]
  11. National Nuclear Data Center. Brookhaven National Laboratory. NuDAT 2.7. Available online: https://www.nndc.bnl.gov/nudat2 (accessed on 10 June 2019).
  12. Penna, A.C.; Franca, C.A.S.; Oliveira MVFTd Fonseca LMBd Saldanha, P.d.A.B.; Costa, M.M.; Balen, J.; Rocha, A.C.P.; Pereira, F.P.A.; Andrade, V.C.; Calas, M.J.G. Uso de sementes de iodo-125 (ROLLIS) para localização intraoperatória de lesões impalpáveis da mama: Análise do implante de 338 sementes em 284 pacientes. Mastology 2017, 27, 117–123. [Google Scholar] [CrossRef]
  13. Bourke, A.G.; Taylor, D.B.; Westcott, E.; Hobbs, M.; Saunders, C. Iodine-125 seeds to guide removal of impalpable breast lesions: Radio-guided occult lesion localization—A pilot study. ANZ J. Surg. 2017, 87, E178–E182. [Google Scholar] [CrossRef] [PubMed]
  14. Taylor, D.B.; Bourke, A.G.; Westcott, E.; Burrage, J.; Latham, B.; Riley, P.; Ballal, H.; Kamyab, R.; Frost, F.; Dissanayake, D.; et al. Radioguided occult lesion localisation using iodine-125 seeds (‘ROLLIS’) for removal of impalpable breast lesions: First Australian experience. J. Med. Imaging Radiat. Oncol. 2015, 59, 411–420. [Google Scholar] [CrossRef] [PubMed]
  15. Reed, A.J.; Kim, J.H.; Burrage, J.W. Development and application of a simple method for calculating breast dose from radio-guided occult lesion localisation using iodine-125 seeds (ROLLIS). Phys. Med. Biol. 2019, 64, 075020. [Google Scholar] [CrossRef]
  16. Dua, S.M.; Gray, R.J.; Keshtgar, M. Strategies for localisation of impalpable breast lesions. Breast 2011, 20, 246–253. [Google Scholar] [CrossRef]
  17. Rostelato, M.E.C.M. Estudo e Desenvolvimento de uma nova Metodologia para Confecção de Sementes de Iodo-125 para Aplicação em Braquiterapia. IPEN-CNEN/SP; Instituto de Pesquisas Energéticas e Nucleares: São Paulo, Brazil, 2006. [Google Scholar]
  18. Cardoso, R.M.; de Souza, C.D.; Rostelato, M.E.C.M.; Araki, K. Highly efficient method for production of radioactive silver seed cores for brachytherapy. Appl. Radiat. Isot. 2017, 120, 76–81. [Google Scholar] [CrossRef]
  19. van der Noordaa, M.E.; Pengel, K.E.; Groen, E.; van Werkhoven, E.; Rutgers, E.J.; Loo, C.E.; Vogel, W.; Vrancken Peeters, M.J.T.F.D. The use of radioactive iodine-125 seed localization in patients with non-palpable breast cancer: A comparison with the radioguided occult lesion localization with 99m technetium. Eur. J. Surg. Oncol. 2015, 41, 553–558. [Google Scholar] [CrossRef]
  20. Elsevier. Mendeley [Computer Program]. Available online: https://www.mendeley.com/2021 (accessed on 10 January 2021).
  21. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  22. McGuinness, L.A.; Higgins, J.P.T. Risk-of-bias visualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments. Res. Syn. Meth. 2020, 12, 55–61. [Google Scholar] [CrossRef]
  23. Sterne, J.A.C.; Savović, J.; Page, M.J.; Elbers, R.G.; Blencowe, N.S.; Boutron, I.; Cates, C.J.; Cheng, H.-Y.; Corbett, M.S.; Eldridge, S.M.; et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ 2019, 366, l4898. [Google Scholar] [CrossRef]
  24. Sterne, J.A.; Hernán, M.A.; Reeves, B.C.; Savović, J.; Berkman, N.D.; Viswanathan, M.; Henry, D.; Altman, D.G.; Ansari, M.T.; Boutron, I. ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016, 355, i4919. [Google Scholar] [CrossRef] [PubMed]
  25. The Cochrane Library. Review Manager Version 5.4 [Computer Program]. Available online: https://training.cochrane.org/online-learning/core-software/revman2020 (accessed on 5 January 2021).
  26. Guyatt, G.H.; Oxman, A.D.; Vist, G.E.; Kunz, R.; Falck-Ytter, Y.; Alonso-Coello, P.; Schünemann, H.J. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336, 924–926. [Google Scholar] [CrossRef]
  27. Donker, M.; Drukker, C.A.; Valdés Olmos, R.A.; Rutgers, E.J.T.T.; Loo, C.E.; Sonke, G.S.; Wesseling, J.; Alderliesten, T.; Marie-Jeanne, T.F.D.; Peeters, V. Guiding breast-conserving surgery in patients after neoadjuvant systemic therapy for breast cancer: A comparison of radioactive seed localization with the ROLL technique. Ann. Surg. Oncol. 2013, 20, 2569–2575. [Google Scholar] [CrossRef]
  28. Niinikoski, L.; Hukkinen, K.; Leidenius, M.H.K.; Vaara, P.; Voynov, A.; Heikkilä, P.; Mattson, J.; Meretoja, T.J. Resection margins and local recurrences of impalpable breast cancer: Comparison between radioguided occult lesion localization (ROLL) and radioactive seed localization (RSL). Breast 2019, 47, 93–101. [Google Scholar] [CrossRef] [PubMed]
  29. Theunissen, C.I.J.M.; Rust, E.A.Z.; Edens, M.A.; Bandel, C.; Van’t Ooster-van den Berg, J.G.; Jager, P.L.; Noorda, E.M.; Francken, A.B. Radioactive seed localization is the preferred technique in nonpalpable breast cancer compared with wire-guided localization and radioguided occult lesion localization. Nucl. Med. Commun. 2017, 38, 396–401. [Google Scholar] [CrossRef] [PubMed]
  30. Bloomquist, E.V.; Ajkay, N.; Patil, S.; Collett, A.E.; Frazier, T.G.; Barrio, A.V. A randomized prospective comparison of patient-assessed satisfaction and clinical outcomes with radioactive seed localization versus wire localization. Breast J. 2016, 22, 151–157. [Google Scholar] [CrossRef] [PubMed]
  31. Chagpar, A.B.; Garcia-Cantu, C.; Howard-McNatt, M.M.; Gass, J.S.; Levine, E.A.; Chiba, A.; Lum, S.; Martinez, R.; Brown, E.; Dupont, E. Does Localization Technique Matter for Non-palpable Breast Cancers? Am. Surg. 2022, 88, 2871–2876. [Google Scholar] [CrossRef] [PubMed]
  32. Fung, F.; Cornacchi, S.D.; Reedijk, M.; Hodgson, N.; Goldsmith, C.H.; McCready, D.; Gohla, G.; Boylan, C.; Lovrics, P.J. Breast cancer recurrence following radioguided seed localization and standard wire localization of nonpalpable invasive and in situ breast cancers: 5-Year follow-up from a randomized controlled trial. Am. J. Surg. 2017, 213, 798–804. [Google Scholar] [CrossRef]
  33. Gray, R.J.; Salud, C.; Nguyen, K.; Dauway, E.; Friedland, J.; Berman, C.; Peltz, E.; Whitehead, G.; Cox, C.E. Randomized prospective evaluation of a novel technique for biopsy or lumpectomy of nonpalpable breast lesions: Radioactive seed versus wire localization. Ann. Surg. Oncol. 2001, 8, 711–715. [Google Scholar] [CrossRef]
  34. Langhans, L.; Tvedskov, T.F.; Klausen, T.L.; Jensen, M.-B.B.; Talman, M.-L.L.; Vejborg, I.; Benian, C.; Roslind, A.; Hermansen, J.; Oturai, P.S.; et al. Radioactive Seed Localization or Wire-guided Localization of Nonpalpable Invasive and In Situ Breast Cancer: A Randomized, Multicenter, Open-label Trial. Ann. Surg. 2017, 266, 29–35. [Google Scholar] [CrossRef]
  35. Lovrics, P.J.; Goldsmith, C.H.; Hodgson, N.; McCready, D.; Gohla, G.; Boylan, C.; Cornacchi, S.; Reedijk, M. A multicentered, randomized, controlled trial comparing radioguided seed localization to standard wire localization for nonpalpable, invasive and in situ breast carcinomas. Ann. Surg. Oncol. 2011, 18, 3407–3414. [Google Scholar] [CrossRef]
  36. Ong, J.S.L.; Teh, J.; Saunders, C.; Bourke, A.G.; Lizama, C.; Newton, J.; Phillips, M.; Taylor, D.B. Patient satisfaction with Radioguided Occult Lesion Localisation using iodine-125 seeds (‘ROLLIS’) versus conventional hookwire localisation. Eur. J. Surg. Oncol. 2017, 43, 2261–2269. [Google Scholar] [CrossRef]
  37. Parvez, E.; Cornacchi, S.D.; Hodgson, N.; Thoma, A.; Kong, I.; Foster, G.; Cheng, J.; Goldsmith, C.H.; Dao, D.; Lovrics, P.J. A cosmesis outcome substudy in a prospective, randomized trial comparing radioguided seed localization with standard wire localization for nonpalpable, invasive, and in situ breast carcinomas. Am. J. Surg. 2014, 208, 711–718. [Google Scholar] [CrossRef]
  38. Agahozo, M.C.; Berghuis, S.A.M.; van den Broek, E.; Koppert, L.B.; Obdeijn, I.-M.; van Deurzen, C.H.M. Radioactive Seed Versus Wire-Guided Localization for Ductal Carcinoma in Situ of the Breast: Comparable Resection Margins. Ann. Surg. Oncol. 2020, 27, 5296–5302. [Google Scholar] [CrossRef] [PubMed]
  39. Aljohani, B.; Jumaa, K.; Kornecki, A.; Brackstone, M. Clinical utility of radioactive seed localization in nonpalpable breast cancer: A retrospective single institutional cohort study. Int. J. Surg. 2018, 60, 149–152. [Google Scholar] [CrossRef] [PubMed]
  40. Angarita, F.A.; Acuna, S.A.; Down, N.; Leung, C.S.; Pirmoradi, F.; Osman, F. Comparison of Radioactive Seed Localized Excision and Wire Localized Excision of Breast Lesions: A Community Hospital’s Experience. Clin. Breast Cancer 2019, 19, e364–e369. [Google Scholar] [CrossRef]
  41. Mendoza Arnau, I.; Sánchez Sánchez, R.; Culiáñez Casas, M.; Rebollo Aguirre, Á.C.; González Jiménez, A.D.; Martínez Meca, S. Surgical clips vs. iodine-125 (125I) seeds for marking the location of nonpalpable malignant breast lesions: Preliminary results. Radiologia 2020, 62, 38–45. [Google Scholar] [CrossRef]
  42. Chiu, J.C.; Ajmal, S.; Zhu, X.; Griffith, E.; Encarnacion, T.; Barr, L. Radioactive seed localization of nonpalpable breast lesions in an academic comprehensive cancer program community hospital setting. Am. Surg. 2014, 80, 675–679. [Google Scholar] [CrossRef] [PubMed]
  43. Diego, E.J.; Soran, A.; McGuire, K.P.; Costellic, C.; Johnson, R.R.; Bonaventura, M.; Ahrendt, G.M.; McAuliffe, P.F. Localizing High-Risk Lesions for Excisional Breast Biopsy: A Comparison Between Radioactive Seed Localization and Wire Localization. Ann. Surg. Oncol. 2014, 21, 3268–3272. [Google Scholar] [CrossRef]
  44. Dryden, M.J.; Dogan, B.E.; Fox, P.; Wang, C.; Black, D.M.; Hunt, K.; Yang, W.T. Imaging factors that influence surgical margins after preoperative 125I radioactive seed localization of breast lesions: Comparison with wire localization. Am. J. Roentgenol. 2016, 206, 1112–1118. [Google Scholar] [CrossRef]
  45. Gray, R.J.; Pockaj, B.A.; Karstaedt, P.J.; Roarke, M.C. Radioactive seed localization of nonpalpable breast lesions is better than wire localization. Am. J. Surg. 2004, 188, 377–380. [Google Scholar] [CrossRef]
  46. Horwood, C.R.; Grignol, V.; Lahey, S.; Pemmaraju, V.; Schafer, A.; Terando, A.M.; Agnese, D.M. Radioactive seed vs wire localization for nonpalpable breast lesions: A single institution review. Breast J. 2019, 25, 282–285. [Google Scholar] [CrossRef]
  47. Hout, B.A.; Schenk, K.E.; van der Linden, A.N.; Roumen, R.M.H. Efficacy of localization of non-palpable, invasive breast cancer: Wire localization vs. Iodine-125 seed: A historical comparison. Breast 2016, 29, 8–13. [Google Scholar]
  48. Hughes, J.H.; Mason, M.C.; Gray, R.J.; McLaughlin, S.A.; Degnim, A.C.; Fulmer, J.T.; Pockaj, B.A.; Karstaedt, P.J.; Roarke, M.C. A multi-site validation trial of radioactive seed localization as an alternative to wire localization. Breast J. 2008, 14, 153–157. [Google Scholar] [CrossRef]
  49. Jumaa, K.; Johani, B.A.; Brackstone, M.; Kornecki, A. A Single-Institute Experience with Radioactive Seed Localization of Breast Lesions-A Retrospective Study. Can. Assoc. Radiol. J. J. L’association Can. Des Radiol. 2020, 71, 58–62. [Google Scholar] [CrossRef] [PubMed]
  50. Law, W.; Cao, X.; Wright, F.C.; Slodkowska, E.; Look Hong, N.; Curpen, B. Adequacy of invasive and in situ breast carcinoma margins in radioactive seed and wire-guided localization lumpectomies. Breast J. 2021, 27, 134–140. [Google Scholar] [CrossRef]
  51. Luiten, J.D.; Beek, M.A.; Voogd, A.C.; Gobardhan, P.D.; Luiten, E.J.T. Iodine seed versus wire-guided localization in breast-conserving surgery for non-palpable ductal carcinoma in situ. Br. J. Surg. 2015, 102, 1665–1669. [Google Scholar] [CrossRef]
  52. Milligan, R.; Pieri, A.; Critchley, A.; Peace, R.; Lennard, T.; O’Donoghue, J.M.; Howitt, R.; Nicholson, S.; Cain, H.; Petrides, G.; et al. Radioactive seed localization compared with wire-guided localization of non-palpable breast carcinoma in breast conservation surgery- the first experience in the United Kingdom. Br. J. Radiol. 2018, 91, 20170268. [Google Scholar] [CrossRef]
  53. Murphy, J.O.; Moo, T.-A.A.; King, T.A.; Van Zee, K.J.; Villegas, K.A.; Stempel, M.; Eaton, A.; St. Germain, J.M.; Morris, E.; Morrow, M. Radioactive seed localization compared to wire localization in breast-conserving surgery: Initial 6-month experience. Ann. Surg. Oncol. 2013, 20, 4121–4127. [Google Scholar] [CrossRef]
  54. Murphy, B.L.; Boughey, J.C.; Keeney, M.G.; Glasgow, A.E.; Racz, J.M.; Keeney, G.L.; Habermann, E.B. Factors Associated with Positive Margins in Women Undergoing Breast Conservation Surgery. Mayo Clin. Proc. 2018, 93, 429–435. [Google Scholar] [CrossRef] [PubMed]
  55. Parvez, E.; Cornacchi, S.D.; Fu, E.; Hodgson, N.; Farrokhyar, F.; Reid, S.; Lovrics, P.J. Adoption and outcomes of radioguided seed localization for non-palpable invasive and in-situ breast cancer at three academic tertiary care centers. Am. J. Surg. 2018, 216, 1160–1165. [Google Scholar] [CrossRef]
  56. Pieri, A.; Milligan, R.; Critchley, A.; O’Donoghue, J.M.; Sibal, N.; Peace, R.; Petrides, G.; Howitt, R.; Nicholson, S.; Cain, H. The introduction of radioactive seed localisation improves the oncological outcome of image guided breast conservation surgery. Breast 2017, 36, 49–53. [Google Scholar] [CrossRef]
  57. Rao, R.; Moldrem, A.; Sarode, V.; White, J.; Amen, M.; Rao, M.; Andrews, V.; Euhus, D.; Radford, L.; Ulissey, M. Experience with seed localization for nonpalpable breast lesions in a public health care system. Ann. Surg. Oncol. 2010, 17, 3241–3246. [Google Scholar] [CrossRef]
  58. Rarick, J.; Kimler, B.F.; Tawfik, O. Comparison of margin status and lesional size between radioactive seed localized vs conventional wire localized breast lumpectomy specimens. Ann. Diagn. Pathol. 2016, 21, 47–52. [Google Scholar] [CrossRef] [PubMed]
  59. Romanoff, A.; Schmidt, H.; McMurray, M.; Burnett, A.; Condren, A.; Port, E. Physician preference and patient satisfaction with radioactive seed versus wire localization. J. Surg. Res. 2017, 210, 177–180. [Google Scholar] [CrossRef] [PubMed]
  60. Sanchez Sanchez, R.; Gonzalez Jimenez, A.D.; Rebollo Aguirre, A.C.; Mendoza Arnau, I.; Menjon Beltran, S.; Vergara Alcaide, M.E.; Osorio Ceballos, J.L.; Llamas Elvira, J.M. I-125 radioactive seed localization for non-palpable lesions in breast cancer. Rev. Esp. Med. Nucl. E Imagen Mol. 2019, 38, 343–347. [Google Scholar]
  61. Saphier, N.; Kondraciuk, J.; Morris, E.; Bernard-Davila, B.; Mango, V. Preoperative Localization of Breast MRI Lesions: MRI-guided Marker Placement with Radioactive Seed Localization as an Alternative to MRI-guided Wire Localization. J. Breast Imaging 2020, 2, 250–258. [Google Scholar] [CrossRef] [PubMed]
  62. Sharek, D.; Zuley, M.L.; Zhang, J.Y.; Soran, A.; Ahrendt, G.M.; Ganott, M.A. Radioactive seed localization versus wire localization for lumpectomies: A comparison of outcomes. AJR Am. J. Roentgenol. 2015, 204, 872–877. [Google Scholar] [CrossRef] [PubMed]
  63. Da Silva, M.; Porembka, J.; Mokdad, A.A.; Seiler, S.; Huth, J.; Rivers, A.; Wooldridge, R.; Chu, A.; Brown, E.; Bao, J.; et al. Bracketed radioactive seed localization vs bracketed wire-localization in breast surgery. Breast J. 2018, 24, 161–166. [Google Scholar] [CrossRef]
  64. Srour, M.K.; Kim, S.; Amersi, F.; Giuliano, A.E.; Chung, A. Comparison of wire localization, radioactive seed, and Savi scout(®) radar for management of surgical breast disease. Breast J. 2020, 26, 406–413. [Google Scholar] [CrossRef]
  65. Srour, M.K.; Kim, S.; Amersi, F.; Giuliano, A.E.; Chung, A. Comparison of Multiple Wire, Radioactive Seed, and Savi Scout(®) Radar Localizations for Management of Surgical Breast Disease. Ann. Surg. Oncol. 2020, 28, 2212–2218. [Google Scholar] [CrossRef]
  66. Stelle, L.; Schoenheit, T.; Brubaker, A.; Tang, X.; Qu, P.; Cradock, K.; Higham, A. Radioactive Seed Localization Versus Wire Localization for Nonpalpable Breast Lesions: A Two-Year Initial Experience at a Large Community Hospital. Ann. Surg. Oncol. 2018, 25, 131–136. [Google Scholar] [CrossRef] [PubMed]
  67. Tran, V.T.; David, J.; Patocskai, E.; Zummo-Soucy, M.; Younan, R.; Lalonde, L.; Labelle, M.; El Khoury, M.; Robidoux, A.; Trop, I. Comparative Evaluation of Iodine-125 Radioactive Seed Localization and Wire Localization for Resection of Breast Lesions. Can. Assoc. Radiol. J. J. L’association Can. Des Radiol. 2017, 68, 447–455. [Google Scholar] [CrossRef] [PubMed]
  68. Zénzola, V.; Hidalgo, F.; Paredtes, R.; Betancourt, L.; Mattar, D.; Andrade, A. Estudio comparativo entre radiolocalización con aguja arpón: Y uso de semillas radioactivas de yodo 125 en lesiones subclínicas de mama TT—Comparative study between radiolocation with needle harpoon and use of radioactive iodine seeds in 125 subclinica. Rev. Venez. Oncol. 2006, 18, 136–143. [Google Scholar]
  69. Zhang, Y.; Seely, J.; Cordeiro, E.; Hefler, J.; Thavorn, K.; Mahajan, M.; Domina, S.; Aro, J.; Ibrahim, A.M.; Arnaout, A.; et al. Radioactive Seed Localization Versus Wire-Guided Localization for Nonpalpable Breast Cancer: A Cost and Operating Room Efficiency Analysis. Ann. Surg. Oncol. 2017, 24, 3567–3573. [Google Scholar] [CrossRef] [PubMed]
  70. Taylor, D.B.; Bourke, A.G.; Westcott, E.J.; Marinovich, M.L.; Chong, C.Y.L.; Liang, R.; Hughes, R.L.; Elder, E.; Saunders, C.M. Surgical outcomes after radioactive 125I seed versus hookwire localization of non-palpable breast cancer: A multicentre randomized clinical trial. Br. J. Surg. 2021, 108, 40–48. [Google Scholar] [CrossRef] [PubMed]
  71. Chan, B.K.Y.; Wiseberg-Firtell, J.A.; Jois, R.H.S.; Jensen, K.; Audisio, R.A. Localization techniques for guided surgical excision of non-palpable breast lesions. Cochrane Database Syst. Rev. 2015, 2015, CD009206. [Google Scholar]
  72. Moreira, I.C.; Ventura, S.R.; Ramos, I.; Fougo, J.L.; Rodrigues, P.P. Preoperative localisation techniques in breast conservative surgery: A systematic review and meta-analysis. Surg. Oncol. 2020, 35, 351–373. [Google Scholar] [CrossRef]
  73. Ahmed, M.; Douek, M. Radioactive seed localisation (RSL) in the treatment of non-palpable breast cancers: Systematic review and meta-analysis. Breast 2013, 22, 383–388. [Google Scholar] [CrossRef]
  74. Barentsz, M.W.; van den Bosch, M.A.A.J.; Veldhuis, W.B.; van Diest, P.J.; Pijnappel, R.M.; Witkamp, A.J.; Verkooijen, H.M. Radioactive seed localization for non-palpable breast cancer. Br. J. Surg. 2013, 100, 582–588. [Google Scholar] [CrossRef]
  75. Gray, R.J.; Pockaj, B.A.; Garvey, E.; Blair, S. Intraoperative Margin Management in Breast-Conserving Surgery: A Systematic Review of the Literature. Ann. Surg. Oncol. 2018, 25, 18–27. [Google Scholar] [CrossRef]
  76. Fusco, R.; Petrillo, A.; Catalano, O.; Sansone, M.; Granata, V.; Filice, S.; D’aiuto, M.; Pankhurst, Q.; Douek, M. Procedures for location of non-palpable breast lesions: A systematic review for the radiologist. Breast Cancer 2014, 21, 522–531. [Google Scholar] [CrossRef] [PubMed]
  77. Pouw, B.; Veen, L.J.d.W.v.d.; Stokkel, M.P.; Loo, C.E.; Peeters, M.-J.T.V.; Olmos, R.A.V. Heading toward radioactive seed localization in non-palpable breast cancer surgery? A meta-analysis. J. Surg. Oncol. 2015, 111, 185–191. [Google Scholar] [CrossRef] [PubMed]
  78. Wang, G.-L.; Tsikouras, P.; Zuo, H.-Q.; Huang, M.-Q.; Peng, L.; Bothou, A.; Zervoudis, S.; Teichmann, A.T. Radioactive seed localization and wire guided localization in breast cancer: A systematic review and meta-analysis. JBUON 2019, 24, 48–60. [Google Scholar] [PubMed]
  79. Xu, X.J.; Li, J.J.; Bin Ji, W. An updated Meta-Analysis of radioactive seed localization versus wire-guided localization in the treatment of nonpalpable breast lesions. Breast J. 2018, 24, 673–675. [Google Scholar] [CrossRef]
  80. Aetna. Breast Biopsy Procedures. In Medical Clinical Policy Bulletins; Aetna: Hartford, CT, USA, 2020. [Google Scholar]
  81. CADTH. Preoperative Seed Placement for Breast Cancer Surgery: Clinical Effectiveness, Cost-Efectiveness, and Guidelines; CADTH: Ottawa, ON, Canada, 2019. [Google Scholar]
  82. INESSS. Utilization de la bille radioactive pour la localisation preoperatoire de tumeurs mammaires non palpables. In Note Informative; INESSS: Montreal, QC, Canada, 2016. [Google Scholar]
Figure 1. Study selection flowchart. New version adapted from Page, M.J., 2021 [21].
Figure 1. Study selection flowchart. New version adapted from Page, M.J., 2021 [21].
Diagnostics 14 00441 g001
Figure 2. Positive surgical margin forest plot for RSL and ROLL. Figure generated by Review Manager software. References cited are [19,27,29].
Figure 2. Positive surgical margin forest plot for RSL and ROLL. Figure generated by Review Manager software. References cited are [19,27,29].
Diagnostics 14 00441 g002
Figure 3. Reoperation forest plot for RSL and ROLL. Figure generated by Review Manager software. References cited are [19,27,28,29].
Figure 3. Reoperation forest plot for RSL and ROLL. Figure generated by Review Manager software. References cited are [19,27,28,29].
Diagnostics 14 00441 g003
Figure 4. Recurrence forest plot for RSL and ROLL. Figure generated by Review Manager software. References cited are [27,28,29].
Figure 4. Recurrence forest plot for RSL and ROLL. Figure generated by Review Manager software. References cited are [27,28,29].
Diagnostics 14 00441 g004
Figure 5. Risk of bias assessment result of non-randomized clinical trials comparing RSL and ROLL. Figure generated by Robvis software (R package, 2019) developed by University of Bristol, UK, 2019. References cited are [19,27,28,29].
Figure 5. Risk of bias assessment result of non-randomized clinical trials comparing RSL and ROLL. Figure generated by Robvis software (R package, 2019) developed by University of Bristol, UK, 2019. References cited are [19,27,28,29].
Diagnostics 14 00441 g005
Figure 6. (a) Forest plot and (b) funnel plot of positive surgical margins with RSL and WGL. Figure generated by the Review Manager software. References cited are [29,30,31,33,34,35,38,39,40,41,42,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,60,64,65,67,68,69,70]. Caption: SE = standard error; RR = risk ratio.
Figure 6. (a) Forest plot and (b) funnel plot of positive surgical margins with RSL and WGL. Figure generated by the Review Manager software. References cited are [29,30,31,33,34,35,38,39,40,41,42,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,60,64,65,67,68,69,70]. Caption: SE = standard error; RR = risk ratio.
Diagnostics 14 00441 g006
Figure 7. (a) Forest plot and (b) funnel plot of intraoperative re-excision with RSL and WGL. Figure generated by Review Manager software. References cited are [31,34,35,40,43,45,50,62,63,64,65,66,69,70]. Caption: SE = standard error; RR = risk ratio.
Figure 7. (a) Forest plot and (b) funnel plot of intraoperative re-excision with RSL and WGL. Figure generated by Review Manager software. References cited are [31,34,35,40,43,45,50,62,63,64,65,66,69,70]. Caption: SE = standard error; RR = risk ratio.
Diagnostics 14 00441 g007
Figure 8. (a) Forest plot and (b) funnel plot of reoperation with RSL and WGL. Figure generated by Review Manager software. References cited are [29,35,38,39,40,41,42,44,45,46,47,48,49,50,51,52,53,54,55,56,57,60,61,62,63,64,65,66,67,69,70]. Caption: SE = standard error; RR = risk ratio.
Figure 8. (a) Forest plot and (b) funnel plot of reoperation with RSL and WGL. Figure generated by Review Manager software. References cited are [29,35,38,39,40,41,42,44,45,46,47,48,49,50,51,52,53,54,55,56,57,60,61,62,63,64,65,66,67,69,70]. Caption: SE = standard error; RR = risk ratio.
Diagnostics 14 00441 g008
Figure 9. Forest plot of recurrence RSL and WGL. Figure generated by Review Manager software. References cited are [29,32,39].
Figure 9. Forest plot of recurrence RSL and WGL. Figure generated by Review Manager software. References cited are [29,32,39].
Diagnostics 14 00441 g009
Figure 10. Forest plot of failure identification sentinel lymph node biopsy RSL and WGL. Figure generated by Review Manage software. References cited are [30,33,34,45,48,70].
Figure 10. Forest plot of failure identification sentinel lymph node biopsy RSL and WGL. Figure generated by Review Manage software. References cited are [30,33,34,45,48,70].
Diagnostics 14 00441 g010
Figure 11. (a) Forest plot and (b) funnel plot of marker complications with RSL and WGL. Figure generated by Review Manager software. References cited [30,33,34,35,41,42,43,44,45,46,47,48,49,50,51,52,55,57,60,61,64,67,68,69,70]. Caption: SE = standard error; RR = risk ratio.
Figure 11. (a) Forest plot and (b) funnel plot of marker complications with RSL and WGL. Figure generated by Review Manager software. References cited [30,33,34,35,41,42,43,44,45,46,47,48,49,50,51,52,55,57,60,61,64,67,68,69,70]. Caption: SE = standard error; RR = risk ratio.
Diagnostics 14 00441 g011
Figure 12. (a) Forest plot and (b) funnel plot of post-operative complications with RSL and WGL. Figure generated by Review Manager software. References cited [29,31,32,41,42,44,48,51,60,61,64,65,66]. Caption: SE = standard error; RR = risk ratio.
Figure 12. (a) Forest plot and (b) funnel plot of post-operative complications with RSL and WGL. Figure generated by Review Manager software. References cited [29,31,32,41,42,44,48,51,60,61,64,65,66]. Caption: SE = standard error; RR = risk ratio.
Diagnostics 14 00441 g012
Figure 13. Risk of bias assessment result of randomized clinical trials comparing RSL and WGL. Figure generated by Robvis software. References cited are [30,31,32,33,34,35,36,50,70].
Figure 13. Risk of bias assessment result of randomized clinical trials comparing RSL and WGL. Figure generated by Robvis software. References cited are [30,31,32,33,34,35,36,50,70].
Diagnostics 14 00441 g013
Figure 14. Risk of bias assessment result of non-randomized clinical trials comparing RSL and WGL. Figure generated by Robvis software. References cited are [29,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69].
Figure 14. Risk of bias assessment result of non-randomized clinical trials comparing RSL and WGL. Figure generated by Robvis software. References cited are [29,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69].
Diagnostics 14 00441 g014
Table 1. Patients’ characteristics in RSL and ROLL groups.
Table 1. Patients’ characteristics in RSL and ROLL groups.
DataRSLROLL
Age28–91
Number of participants724826
Carcinoma ductal in situ117106
Carcinoma invasive607720
Neoadjuvant chemotherapy7183
Bracketing localization20
Unknown bracketing localization69-
Table 2. Patient’s characteristics in RSL and WGL groups.
Table 2. Patient’s characteristics in RSL and WGL groups.
DataRSLWGL
Number of participants867011,150
Age 20–92 years old69867990
Unknown age16843160
Carcinoma ductal in situ29173593
Carcinoma invasive44324481
Other pathologies 113213076
Neoadjuvant chemotherapy16177
Unknown neoadjuvant chemotherapy418839
Bracketing localization479577
Unknown bracketing localization10592579
Excisional biopsy112177
Unknown excisional biopsy16072556
1 Includes other types of carcinomas, high-risk lesions, benign lesions, and cases of unknown pathology.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ferreira, H.H.J.; de Souza, C.D.; Pozzo, L.; Ribeiro, M.S.; Rostelato, M.E.C.M. Radioactive Seed Localization for Nonpalpable Breast Lesions: Systematic Review and Meta-Analysis. Diagnostics 2024, 14, 441. https://doi.org/10.3390/diagnostics14040441

AMA Style

Ferreira HHJ, de Souza CD, Pozzo L, Ribeiro MS, Rostelato MECM. Radioactive Seed Localization for Nonpalpable Breast Lesions: Systematic Review and Meta-Analysis. Diagnostics. 2024; 14(4):441. https://doi.org/10.3390/diagnostics14040441

Chicago/Turabian Style

Ferreira, Hortência H. J., Carla Daruich de Souza, Lorena Pozzo, Martha S. Ribeiro, and Maria Elisa C. M. Rostelato. 2024. "Radioactive Seed Localization for Nonpalpable Breast Lesions: Systematic Review and Meta-Analysis" Diagnostics 14, no. 4: 441. https://doi.org/10.3390/diagnostics14040441

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop