Next Article in Journal
Synergistic Effect of Polysaccharides from Chamomile Tea with Nitazoxanide Increases Treatment Efficacy against Giardia intestinalis
Previous Article in Journal
Long-Term Predictors of Hospitalized Reinfarction after an Incident Acute Myocardial Infarction
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Implementation of a Commercial Deep Learning-Based Auto Segmentation Software in Radiotherapy: Evaluation of Effectiveness and Impact on Workflow

Department of Medical Physics, ASL TO4, Ivrea Community Hospital, 10015 Ivrea, Italy
Department of Radiation Oncology, ASL TO4, Ivrea Community Hospital, 10015 Ivrea, Italy
Department of Translational Medicine (DIMET), ‘Maggiore della Carità’ University Hospital, University of Eastern Piedmont, 28100 Novara, Italy
Author to whom correspondence should be addressed.
Life 2022, 12(12), 2088;
Submission received: 24 October 2022 / Revised: 30 November 2022 / Accepted: 9 December 2022 / Published: 13 December 2022
(This article belongs to the Section Medical Research)


Proper delineation of both target volumes and organs at risk is a crucial step in the radiation therapy workflow. This process is normally carried out manually by medical doctors, hence demanding timewise. To improve efficiency, auto-contouring methods have been proposed. We assessed a specific commercial software to investigate its impact on the radiotherapy workflow on four specific disease sites: head and neck, prostate, breast, and rectum. For the present study, we used a commercial deep learning-based auto-segmentation software, namely Limbus Contour (LC), Version 1.5.0 (Limbus AI Inc., Regina, SK, Canada). The software uses deep convolutional neural network models based on a U-net architecture, specific for each structure. Manual and automatic segmentation were compared on disease-specific organs at risk. Contouring time, geometrical performance (volume variation, Dice Similarity Coefficient—DSC, and center of mass shift), and dosimetric impact (DVH differences) were evaluated. With respect to time savings, the maximum advantage was seen in the setting of head and neck cancer with a 65%-time reduction. The average DSC was 0.72. The best agreement was found for lungs. Good results were highlighted for bladder, heart, and femoral heads. The most relevant dosimetric difference was in the rectal cancer case, where the mean volume covered by the 45 Gy isodose was 10.4 cm3 for manual contouring and 289.4 cm3 for automatic segmentation. Automatic contouring was able to significantly reduce the time required in the procedure, simplifying the workflow, and reducing interobserver variability. Its implementation was able to improve the radiation therapy workflow in our department.

1. Introduction

Radiation therapy (RT) is an important treatment option in the management of cancer. It aims at delivering a high radiation dose to target cancer cells to ensure clinically required tumor control probability and concomitantly spare the nearby healthy tissues to prevent acute RT-related toxicity and late effects.
Accurate contouring of Clinical Target Volumes (CTV) and Organ at Risk (OAR) is important for treatment planning and delivery. Generally, the segmentation of tumor regions and normal tissues is manually performed by the clinical staff, based on the images acquired during planning computed tomography (CT). This approach is prone to a high degree of inter and intra observers’ variability, being time-consuming, and representing a bottleneck in the planning workflow [1].
To improve the efficiency of this process, auto-contouring methods have been proposed. One of the most popular approaches is atlas-based segmentation [2,3]. However, contouring algorithms based on deep-learning techniques are being increasingly used, showing better results than atlas-based approaches [4,5]. The purpose of our study was to investigate the clinical implementation in our institution of a specific deep learning-based auto contour commercial software to assess the impact on the radiotherapy workflow in four specific disease sites: head and neck, prostate, breast and rectum.

2. Materials and Methods

2.1. Deep Learning Auto-Segmentation

A commercial deep learning-based auto-segmentation software, Limbus Contour (LC), Version 1.5.0 (Limbus AI Inc., Regina, SK, Canada), which uses deep convolutional neural network models based on a U-net architecture specific for each structure, was recently introduced in our institution. The software relies on models trained, using public datasets [6,7,8,9,10], as well as datasets obtained through institutional data agreements [11,12,13,14,15]. The number of scans used in the training set for each model varies: each model is trained on hundreds or thousands of scans. Models were trained using TensorFlow; typical image augmentation and regularization techniques were applied. Each model is validated internally by Limbus AI by comparing the model output on a set of test scans to expert human contours on the same test scans. The models are also validated in published studies that investigate qualitative and quantitative accuracy and time savings [1,16,17,18].
LC obtains information related to the acquisition protocol by reading the DICOM metadata of the CT images. The corresponding auto-segmentation model is then automatically used to create auto-segmented contours that are exported alongside the CT images to the treatment planning software to be eventually edited and then validated by the clinicians.

2.2. Patients’ Selection

Four disease sites were selected for the present study, namely Head and Neck (H&N), prostate, rectum, and breast cancer. We focused on these four settings, considering their high frequency and important impact on the radiotherapy workflow. For each type of treatment, three patients treated in our center were selected.
For H&N, we chose oropharyngeal cancer to guarantee the standardization of OARs contouring. Patients eligible for the study received radical radiotherapy. The prescription dose was 70 Gy delivered in 35 fractions for the curative setting. The prostate setting consisted of patients who received exclusive radiotherapy on prostate gland and seminal vesicles. A moderate hypofractionated schedule was proposed: 70 Gy on prostate gland and 63 Gy on seminal vesicles in 28 fractions, delivered with a simultaneous integrated boost. For rectal cancer, patients offered pre-operative RT were considered. The prescription dose was 50 Gy on the gross tumor volume and positive nodes, and 45 Gy on the elective volumes, in 25 fractions. Finally, patients with left-sided breast cancer who underwent conservative surgery were selected. In this case, the prescription dose was 45 Gy for whole breast irradiation and 50 Gy on the tumor bed, given with a concomitant boost, in 20 fractions.

2.3. Technical Setup

Each patient underwent a planning CT scan in supine position; to prevent patient’s displacements during treatment, immobilization devices were used: thermoplastic mask for H&N, knee wedge and foot lock for prostate and rectum treatments, breast board for breast cases. Planning CT images were acquired with a Canon Aquilion LB V6.3 series scanner (Canon medical system corporation—Ōtawara, Japan) with 120 kVp tube load. The slice thickness was 3 mm for H&N cancer and 5 mm for other diseases. The in-plane pixel size was 1 mm × 1 mm for all acquisitions.

2.4. Contour Methods

Four different RO, each with expertise in the specific clinical setting, manually delineated the four-treatment districts following national and international consensus guidelines [11,12,13,14,15,19,20,21,22]. For contouring purposes, additional imaging (e.g., magnetic resonance imaging, positron emission tomography or diagnostic CT) were used, if necessary. The clinically approved treatment plan was subsequently delivered.
The same CT acquisitions were contoured by LC and the images, together with the RTstructure DICOM file, were then sent to the Treatment Planning System (TPS) Eclipse (Version 15.6, Varian Medical Systems—A Siemens Healthineers Company, Palo Alto, CA, USA). The LC structure set was later duplicated on the TPS. One structure set has been reviewed by the competent RO and, if necessary, the contours were modified; the second was not submitted to any change.
For H&N cancer, contoured OARs were fifteen (brainstem, brachial plexuses, spinal cord, inner ears, parotid glands, thyroid, mandible, oral cavity, larynx, lungs and esophagus). For prostate cases, five structures were considered (bladder, femoral heads, rectum and penile bulb). For rectal cancer, four OARs were accounted for (femoral heads, bladder and bowel—as abdominal cavity). Finally, for breast cancer contoured structures were four (contralateral breast, heart, and both lungs).

2.5. Contouring Time

We recorded the time spent performing the manual contour for each CT scan. Moreover, the time required for LC to generate OARs on a consumer grade system (3.1 GHz Intel Core i7, 8 GB memory) was also evaluated. Finally, the time spent by the ROs to review and, if necessary, edit the contours performed by LC was measured. The overall duration of contouring using LC (LC contouring plus ROs review) was compared to the time required to perform manual contouring, which was used as a reference. In this way, the time difference—absolute and relative—between the two contouring methods was obtained.

2.6. Geometrical Analysis

The manually contoured structures (MC) were compared with those generated by LC by means of three indicators: volume variation, Dice Similarity Coefficient (DSC) and shift of the center of mass. For structures with a volume greater than 15 cm3, the volume percentage variation was considered. Conversely, for smaller structures, the absolute change in volume was analyzed, since the percentage variation was not considered indicative, given that small variations in volume lead to large percentage variations.
DSC [23] is a measure of the overlap of two volumes. Its value is comprised between 0 and 1, where 0 indicates no overlap while 1 stay for complete overlap. If X and Y are the two volumes to be compared, the coefficient DSC (X|Y) is defined as DSC (X|Y) = 2|X∩Y|/(|X| + |Y|). Finally, starting from the coordinates of the center of mass of each structure in latero-lateral (X), cranio-caudal (Y) and antero-posterior (Z) direction, its displacements between manual and auto-segmented contouring were evaluated. All the parameters were obtained from the statistics tool of the contouring module of Eclipse TPS.

2.7. Dosimetric Analysis

A dosimetric analysis was performed to evaluate the effects of unsupervised use of LC on the assessment of dose distribution.
The original treatment plan, optimized and clinically approved with the manually contoured volumes, was recalculated on the LC contoured structure-set using the AAA algorithm (version 15.6.06) of Eclipse TPS, the same as the original plan.
The differences in the Dose Volume Histograms (DVH) between the two structure sets were then evaluated and plans were compared using the metrics reported in Table 1.
For serial organs, metrics associated with maximum dose were used, while for parallel organs the average dose or dose too large volume were considered.

3. Results

3.1. Contouring Time

The absolute and percentage variations of the contouring times are shown in Figure 1. The maximum time saving, both absolute and relative, was obtained for the H&N setting (80 min and 65%, respectively). The minimum changes, both absolute and relative, were found for rectum (3 min and 17%, respectively). Similar variations were found for prostate treatments, while breast cases showed intermediate values.

3.2. Geometrical Analysis

Figure 2 shows the average percentage variations in volumes for structures with a volume greater than 15 cm3. The associated uncertainty is expressed in terms of ±1 standard deviation. The OAR with the minimum variation (1%) is lung; the structures with the greatest percentage variation are bowel and oral cavity, with mean percentage variations of 65% and 32%, respectively.
The absolute volume variations for structures with a volume smaller than 15 cm3 are reported in Figure 3. The associated uncertainty is expressed in terms of ±1 standard deviation. All the structures show values close to or less than 1 cm3.
Figure 4 shows the average Dice Index for the analyzed structures, with the relative uncertainty, expressed as ±1 standard deviation. The lowest DSC value is 0.39 for the penile bulb. The best results were found for lungs, characterized by a Dice Index of 0.99. Furthermore, a good agreement was found for bladder, heart, and femoral heads, with values greater than or close to 0.9. Considering all structures, the average DSC is 0.72.
The absolute value of the three-dimensional displacement of the center of mass is represented, for all the structures, in Figure 5. The lowest values were found for lungs, with values close to 0. The greatest displacement occurred for bowel, with a value equal to 2.4 cm. In Figure 6 the absolute values of the displacements in each direction are reported for bowel.

3.3. Dosimetric Analysis

In Table 1, the metrics used for the dosimetric comparison of the treatment plans are reported. The most relevant difference was found in the bowel for rectal cancer treatments: the mean volume covered by the 45 Gy isodose was 10.4 cm3 for the MC structures versus 289.4 cm3 for the LC ones.

4. Discussion

The present study explores the effects of commercial deep-learning based software for auto-contouring on the clinical workflow of a radiation oncology department at a tertiary cancer hospital. In particular, the focus was on timesaving and on the accuracy of the contoured structures.
To accurately assess the time reduction, we evaluated the clinical settings having the highest impact on the workflow in our radiotherapy department. In addition, for each disease site we focused on, all the OARs included in the clinical routine were considered.
Limbus performance was already analyzed by other authors, who investigated multi-observer variability [1], qualitative evaluations of expert ROs [21] and specific evaluations for lung SBRT [22,23]. Furthermore, Zabel et al. [16] compared the manual contouring workflow with LC and an additional atlas-based automatic contouring algorithm for bladder and rectum contouring. Finally, a recent study by D’Aviero et al. evaluates the geometric accuracy of the contours limited to H&N district [24]. The present study includes 28 OARs and 4 anatomical subsets, resulting in a total of 84 contours analyzed. To the best of our knowledge, there are no data available in the literature on such a comprehensive list of OARs and diseases. Furthermore, this study investigates the entire radiotherapy workflow, focusing on geometrical accuracy, timesaving and dosimetric implications of LC implementation in a radiotherapy department.
The possibility to save time is greater in anatomical districts characterized by a greater number and complexity of OARs. Our data are similar to those reported in the literature. As an example, in the setting of lung cancer, Lustberg et al. [2] showed an average time saving of 61% compared to existing clinical practice and 22% compared to the use of atlas-based contours. Wong et al. [1] also found remarkable decreases in contouring time, although for H&N the absolute time reduction is not comparable to ours due to the smaller number of structures contoured by Wong et al.
LC provides good results, as no gross contouring errors were found. This high-quality performance is highlighted by the average DICE Index of 0.72 which can be considered acceptable in clinical practice [21]. However, some OARs have characteristics that deserve to be discussed.
As can be seen by center of mass and geometrical analyses, there is a difference in bowel manual contouring versus automatic segmentation. LC considers as bowel the entire abdominal cavity, extending the caudal limit including the whole inferior abdomen, regardless of the presence of the intestinal loops. During manual contour, however, bowel was considered as abdominal cavity whose caudal limit is defined by the presence of intestinal loops [14,25,26]. These differences justify the dosimetric variation observed.
Regarding the oral cavity, the differences are due to different approaches in contouring; similar results are found by Zhong et al. [27]. LC considers the extended oral cavity, as Contouring Head and Neck OARs Guidelines suggest [15], including the oral tongue and anterior portion of the oropharynx. In manual contouring, the latter was instead excluded from the oral cavity OAR, since it is part of PTV.
The low DSC for penile bulb is an expected finding, as the anatomical markers or the necessary soft tissue contrast for the penile bulb is generally lacking on CT. To best identify penile bulbs and reduce great contouring variability, some authors have stressed the importance of performing an MRI or CT scan with contrast in the urethra for optimal identification of the penile bulb [14].
About brachial plexuses, the institutional practice is not to contour the complete brachial plexus until, laterally, the thoracic wall because for oropharynx tumors the dose to the brachial plexus axillary trunk is negligible [28]. This choice is due to the necessity, in manual contouring, to reach a compromise between the contouring time and the usefulness of the executed contour. However, this tradeoff is not necessary in the case of automatic contouring.
A disagreement in the cranial limit of plexuses was also found. During manual contouring, the brachial plexuses start from the spinal nerves through the neural foramina from the C4–C5 (C5 nerve roots) to the T1–T2 (T1 nerve roots) level. In LC the cranial level of brachial plexus is often higher, such as C2–C3, probably due to the position of neck. These issues explain differences in DSC values for brachial plexuses (about 0.7) compared to those found by D’Aviero et al. [24] (about 0.95).
Regarding parotid glands, no significant changes in geometric parameters were found. However, there is a non-negligible variation of dosimetric indicator. Although the shape and position of parotid glands are similar in manual contouring and LC, minimal differences could drastically affect dosimetric parameters because of the proximity of parotid glands to PTV and to the steep dose gradients. These results are similar to those reported by Nelms et al. [29].
Good results were found for lungs, femoral heads and bladder. DSC values for these OARs were similar to those found by Wong et al. for bladder, femoral heads [21] and lungs [22]. Furthermore, Zabel et al. [16] bladder DSC value −0.97—confirms our result.
A limitation of the study is the low number of patients analyzed for each setting. However, the analysis considers all OARs involved in the clinical workflow for the considered anatomical regions. This allows for a comprehensive assessment of the impact of LC on radiotherapy routine and considers all the steps of the radiotherapy planning process, from contouring to dosimetric consequences of the unsupervised use of LC. A complete description of LC impact on radiotherapy routine can provide useful information.
As a novelty, this study provides quantitative evidence of the time savings achieved by LC use. These values are realistic, thanks to the number of contoured structures. Furthermore, it is possible to identify the anatomical sites which most benefit from LC. Dosimetric evaluation shows that, although DVH differences are not significant in most cases, LC contoured structures must always be supervised by an expert contourer. Otherwise, especially in regions near to high dose gradients, there may be relevant dosimetric variations.

5. Conclusions

Although an accurate visual review by an expert clinician is still required, LC can significantly reduce the time required for contouring and simplify the workflow leading to treatment planning. Its implementation also allows reducing interobserver variability and improving the interpretation of radiological anatomy. Furthermore, LC can support staff training and the continuous assessment of clinical contouring and structure segmentation.

Author Contributions

Conceptualization, L.R. and S.F.; software, M.R.L.P.; validation, V.C.B.; formal analysis, L.R.; investigation, L.B.; resources, M.P. and D.C.; data curation, C.P.; writing—original draft preparation, L.R.; writing—review and editing, P.F. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of ASL TO-4 (protocol code 05-21 and 12 May 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data are available upon request by contacting the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Wong, J.; Fong, A.; McVicar, N.; Smith, S.; Giambattista, J.; Wells, D.; Kolbeck, C.; Giambattista, J.; Gondara, L.; Alexander, A. Comparing deep learning-based auto-segmentation of organs at risk and clinical target volumes to expert inter-observer variability in radiotherapy planning. Radiother. Oncol. 2020, 144, 152–158. [Google Scholar] [CrossRef] [PubMed]
  2. Lustberg, T.; van Soest, J.; Gooding, M.; Peressutti, D.; Aljabar, P.; van der Stoep, J.; van Elmpt, W.; Dekker, A. Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. Radiother. Oncol. 2018, 126, 312–317. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Schipaanboord, B.; Boukerroui, D.; Peressutti, D.; van Soest, J.; Lustberg, T.; Dekker, A.; van Elmpt, W.; Gooding, M.J. An Evaluation of Atlas Selection Methods for Atlas-Based Automatic Segmentation in Radiotherapy Treatment Planning. IEEE Trans. Med. Imaging 2019, 38, 2654–2664. [Google Scholar] [CrossRef] [Green Version]
  4. Ibragimov, B.; Xing, L. Segmentation of organs-at-risks in head and neck CT images using convolutional neural networks. Med. Phys. 2017, 44, 547–557. [Google Scholar] [CrossRef] [Green Version]
  5. Men, K.; Dai, J.; Li, Y. Automatic segmentation of the clinical target volume and organs at risk in the planning CT for rectal cancer using deep dilated convolutional neural networks. Med. Phys. 2017, 44, 6377–6389. [Google Scholar] [CrossRef] [PubMed]
  6. Clark, K.; Vendt, B.; Smith, K.; Freymann, J.; Kirby, J.; Koppel, P.; Moore, S.; Phillips, S.; Maffitt, D.; Pringle, M.; et al. The Cancer Imaging Archive (TCIA): Maintaining and operating a public information repository. J. Digit. Imaging 2013, 26, 1045–1057. [Google Scholar] [CrossRef] [Green Version]
  7. Zuley, M.L.; Jarosz, R.; Kirk, S.; Lee, Y.; Colen, R.; Garcia, K.; Delbeke, D.; Pham, M.; Nagy, P.; Sevinc, G.; et al. Radiology data from the cancer genome atlas head-neck squamous cell carcinoma [TCGA-HNSC] collection. Cancer Imaging Arch. 2016, 10. [Google Scholar] [CrossRef]
  8. Grossberg, A.; Mohamed, A.; Elhalawani, H.; Bennett, W.; Smith, K.; Nolan, T.; Chamchod, S.; Kantor, M.; Browne, T.; Hutcheson, K.; et al. Data from head and neck cancer CT atlas. Cancer Imaging Arch. 2017, 10, K9. [Google Scholar] [CrossRef]
  9. Zuley, M.L.; Jarosz, R.; Drake, B.F.; Rancilio, D.; Klim, A.; Rieger-Christ, K.; Lemmerman, J. Radiology data from the cancer genome atlas prostate adenocarcinoma [tcga-prad] collection. Cancer Imaging Arch. 2016, 9. [Google Scholar] [CrossRef]
  10. Rezaei, M.; Mohammadbeigi, A.; Khoshgard, K.; Haghparast, A. CT images and radiotherapy treatment planning of patients with breast cancer: A dataset. Data Brief 2017, 13, 390–395. [Google Scholar] [CrossRef]
  11. Brouwer, C.L.; Steenbakkers, R.J.; van den Heuvel, E.; Duppen, J.C.; Navran, A.; Bijl, H.P.; Chouvalova, O.; Burlage, F.R.; Meertens, H.; Langendijk, J.A.; et al. 3D Variation in delineation of head and neck organs at risk. Radiat. Oncol. 2012, 7, 32. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Grégoire, V.; Ang, K.; Budach, W.; Grau, C.; Hamoir, M.; Langendijk, J.A.; Lee, A.; Le, Q.T.; Maingon, P.; Nutting, C.; et al. Delineation of the neck node levels for head and neck tumors: A 2013 update. DAHANCA, EORTC, HKNPCSG, NCIC CTG, NCRI, RTOG, TROG consensus guidelines. Radiother. Oncol. 2014, 110, 172–181. [Google Scholar] [CrossRef]
  13. Sun, Y.; Yu, X.L.; Luo, W.; Lee, A.W.; Wee, J.T.; Lee, N.; Zhou, G.Q.; Tang, L.L.; Tao, C.J.; Guo, R.; et al. Recommendation for a contouring method and atlas of organs at risk in nasopharyngeal carcinoma patients receiving intensity-modulated radiotherapy. Radiother. Oncol. 2014, 110, 390–397. [Google Scholar] [CrossRef] [Green Version]
  14. Gay, H.A.; Barthold, H.J.; O’Meara, E.; Bosch, W.R.; El Naqa, I.; Al-Lozi, R.; Rosenthal, S.A.; Lawton, C.; Lee, W.R.; Sandler, H.; et al. Pelvic normal tissue contouring guidelines for radiation therapy: A Radiation Therapy Oncology Group consensus panel atlas. Int. J. Radiat. Oncol. Biol. Phys. 2012, 83, e353–e362, Erratum in Int. J. Radiat. Oncol. Biol. Phys. 2012, 84, 7. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Brouwer, C.L.; Steenbakkers, R.J.; Bourhis, J.; Budach, W.; Grau, C.; Grégoire, V.; van Herk, M.; Lee, A.; Maingon, P.; Nutting, C.; et al. CT-based delineation of organs at risk in the head and neck region: DAHANCA, EORTC, GORTEC, HKNPCSG, NCIC CTG, NCRI, NRG Oncology and TROG consensus guidelines. Radiother. Oncol. 2015, 117, 83–90. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Wong, J.; Huang, V.; Giambattista, J.A.; Teke, T.; Kolbeck, C.; Giambattista, J.; Atrchian, S. Training and Validation of Deep Learning-Based Auto-Segmentation Models for Lung Stereotactic Ablative Radiotherapy Using Retrospective Radiotherapy Planning Contours. Front. Oncol. 2021, 11, 626499. [Google Scholar] [CrossRef]
  17. Wong, J.; Huang, V.; Wells, D.; Giambattista, J.; Giambattista, J.; Kolbeck, C.; Otto, K.; Saibishkumar, E.P.; Alexander, A. Implementation of deep learning-based auto-segmentation for radiotherapy planning structures: A workflow study at two cancer centers. Radiat. Oncol. 2021, 16, 101. [Google Scholar] [CrossRef]
  18. Zabel, W.J.; Conway, J.L.; Gladwish, A.; Skliarenko, J.; Didiodato, G.; Goorts-Matthews, L.; Michalak, A.; Reistetter, S.; King, J.; Nakonechny, K.; et al. Clinical Evaluation of Deep Learning and Atlas-Based Auto-Contouring of Bladder and Rectum for Prostate Radiation Therapy. Pract. Radiat. Oncol. 2021, 11, e80–e89. [Google Scholar] [CrossRef]
  19. Cefaro, G.A.; Genovesi, D.; Perez, C.A. Delineating Organs at Risk in Radiation Therapy; Springer: London, UK, 2013. [Google Scholar]
  20. Nielsen, M.H.; Berg, M.; Pedersen, A.N.; Andersen, K.; Glavicic, V.; Jakobsen, E.H.; Jensen, I.; Josipovic, M.; Lorenzen, E.L.; Nielsen, H.M.; et al. Delineation of target volumes and organs at risk in adjuvant radiotherapy of early breast cancer: National guidelines and contouring atlas by the Danish Breast Cancer Cooperative Group. Acta Oncol. 2013, 52, 703–710. [Google Scholar] [CrossRef] [Green Version]
  21. Gambacorta, M.A.; Chiloiro, G.; Valentini, V. Should We Tailor the Delineation of Pelvic Structures According to Tumor Presentation? In Multidisciplinary Management of Rectal Cancer; Springer: Cham, Switzerland, 2018; pp. 165–179. [Google Scholar]
  22. Merlotti, A.; Alterio, D.; Vigna-Taglianti, R.; Muraglia, A.; Lastrucci, L.; Manzo, R.; Gambaro, G.; Caspiani, O.; Miccichè, F.; Deodato, F.; et al. Technical guidelines for head and neck cancer IMRT on behalf of the Italian association of radiation oncology—Head and neck working group. Radiat. Oncol. 2014, 9, 264. [Google Scholar] [CrossRef]
  23. Dice, L.R. Measures of the amount of ecologic association between species. Ecology 1945, 26, 297–302. [Google Scholar] [CrossRef]
  24. D’Aviero, A.; Re, A.; Catucci, F.; Piccari, D.; Votta, C.; Piro, D.; Piras, A.; Di Dio, C.; Iezzi, M.; Preziosi, F.; et al. Clinical Validation of a Deep-Learning Segmentation Software in Head and Neck: An Early Analysis in a Developing Radiation Oncology Center. Int. J. Environ. Res. Public Health 2022, 19, 9057. [Google Scholar] [CrossRef] [PubMed]
  25. Banerjee, R.; Chakraborty, S.; Nygren, I.; Sinha, R. Small bowel dose parameters predicting grade ≥ 3 acute toxicity in rectal cancer patients treated with neoadjuvant chemoradiation: An independent validation study comparing peritoneal space versus small bowel loop contouring techniques. Int. J. Radiat. Oncol. Biol. Phys. 2013, 85, 1225–1231. [Google Scholar] [CrossRef] [PubMed]
  26. Li, S.; Gong, Y.; Yang, Y.; Guo, Q.; Qian, J.; Tian, Y. Evaluation of small bowel motion and feasibility of using the peritoneal space to replace bowel loops for dose constraints during intensity-modulated radiotherapy for rectal cancer. Radiat. Oncol. 2020, 15, 211. [Google Scholar] [CrossRef] [PubMed]
  27. Zhong, Y.; Yang, Y.; Fang, Y.; Wang, J.; Hu, W. A Preliminary Experience of Implementing Deep-Learning Based Auto-Segmentation in Head and Neck Cancer: A Study on Real-World Clinical Cases. Front. Oncol. 2021, 11, 638197. [Google Scholar] [CrossRef]
  28. Hall, W.H.; Guiou, M.; Lee, N.Y.; Dublin, A.; Narayan, S.; Vijayakumar, S.; Purdy, J.A.; Chen, A.M. Development and validation of a standardized method for contouring the brachial plexus: Preliminary dosimetric analysis among patients treated with IMRT for head-and-neck cancer. Int. J. Radiat. Oncol. Biol. Phys. 2008, 72, 1362–1367. [Google Scholar] [CrossRef] [PubMed]
  29. Nelms, B.E.; Tomé, W.A.; Robinson, G.; Wheeler, J. Variations in the contouring of organs at risk: Test case from a patient with oropharyngeal cancer. Int. J. Radiat. Oncol. Biol. Phys. 2012, 82, 368–378. [Google Scholar] [CrossRef]
Figure 1. Absolute (right) and percentage (left) time reduction obtained with Limbus auto-segmentation software.
Figure 1. Absolute (right) and percentage (left) time reduction obtained with Limbus auto-segmentation software.
Life 12 02088 g001
Figure 2. Percentage volume variation for OARs with volume greater than 15 cm3.
Figure 2. Percentage volume variation for OARs with volume greater than 15 cm3.
Life 12 02088 g002
Figure 3. Absolute volume variation for OARs with volume smaller than 15 cm3.
Figure 3. Absolute volume variation for OARs with volume smaller than 15 cm3.
Life 12 02088 g003
Figure 4. Dice Index for the evaluated structures.
Figure 4. Dice Index for the evaluated structures.
Life 12 02088 g004
Figure 5. 3D absolute shift of the OARs center of mass.
Figure 5. 3D absolute shift of the OARs center of mass.
Life 12 02088 g005
Figure 6. Displacement of the center of mass of the bowel, as absolute value in each direction.
Figure 6. Displacement of the center of mass of the bowel, as absolute value in each direction.
Life 12 02088 g006
Table 1. Metrics used for DVH. The results of Limbus Contour (LC) and Manual contour (MC) are reported for different treatment sites and OAR ± one standard deviation.
Table 1. Metrics used for DVH. The results of Limbus Contour (LC) and Manual contour (MC) are reported for different treatment sites and OAR ± one standard deviation.
Treatment SiteOARDVH MetricMC Mean ± SDLC Mean ± SD
H&NOral cavityMean dose42.8 ± 4.847.9 ± 3.6
EsophagusD1cc44.6 ± 3.445.7 ± 5.2
LarynxD1cc 49.2 ± 2.449.3 ± 1.4
LarynxMean dose 34.8 ± 3.635.9 ± 2.1
MandibleD1 cc 63.3 ± 6.662.9 ± 7
Spinal CordD0.03 cc 28.6 ± 1.728.4 ± 1.8
Spinal CordDmax 29.5 ± 1.329.1 ± 1.5
Inner Ear RMean dose 11.3 ± 6.810.2 ± 6.2
Inner Ear LMean dose 10.5 ± 6.19.3 ± 5.5
Parotid RMean dose 32.4 ± 7.736.9 ± 8.7
Parotid RV30Gy 48.8 ± 10.956.2 ± 11
Parotid LMean dose 29 ± 6.133.3 ± 6.6
Parotid LV30Gy43.6 ± 9.853 ± 12.4
Lung RD30% 5.6 ± 85.5 ± 7.9
Lung LD30%4.6 ± 6.44.5 ± 6.2
ThyroidV45Gy77.7 ± 25.477.7 ± 25.4
Brain StemD0.03cc35.1 ± 2.934.6 ± 2.5
Brachial Plexus RD0.03 cc56.6 ± 1.858.7 ± 4.6
Brachial Plexus LD0.03 cc56.8 ± 1.858.8 ± 3.7
Left BreastLung RV5Gy0 ± 00 ± 0
Lung LV10Gy12.1 ± 2.312.1 ± 2.3
Lung LV20Gy8.5 ± 1.88.5 ± 1.8
Lung LV5Gy19.9 ± 3.319.9 ± 3.4
HeartV25Gy1.2 ± 11 ± 0.9
BreastD1cc0.7 ± 0.11.4 ± 0.2
ProstatePenile bulbDmean14.8 ± 3.222.2 ± 15
Femoral Head RDmax35.3 ± 334.9 ± 3
Femoral Head LDmax37.5 ± 6.137.4 ± 5.8
RectumV50Gy17.4 ± 4.319.7 ± 3.6
RectumV60Gy7.2 ± 28.4 ± 1.9
RectumV65Gy3.8 ± 0.84.6 ± 1.7
RectumV68Gy2 ± 0.72.7 ± 1.6
BladderV60Gy14 ± 2.316.6 ± 4.5
RectumFemoral Head RV30Gy27.6 ± 4.827.6 ± 4.5
Femoral Head RV40Gy1.1 ± 0.80.8 ± 0.8
Femoral Head RV45Gy0 ± 00 ± 0
Femoral Head LV30Gy 24.3 ± 9.722.7 ± 11.7
Femoral Head LV40Gy0.8 ± 1.30.5 ± 0.9
Femoral Head LV45Gy0 ± 00 ± 0
BladderV35Gy30.1 ± 2232.7 ± 24.8
BladderV40Gy19.5 ± 16.820.1 ± 17.3
BladderV50Gy0.6 ± 1.11.3 ± 1.1
BowelV45Gy10.4 ± 15.7289.4 ± 34
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Radici, L.; Ferrario, S.; Borca, V.C.; Cante, D.; Paolini, M.; Piva, C.; Baratto, L.; Franco, P.; La Porta, M.R. Implementation of a Commercial Deep Learning-Based Auto Segmentation Software in Radiotherapy: Evaluation of Effectiveness and Impact on Workflow. Life 2022, 12, 2088.

AMA Style

Radici L, Ferrario S, Borca VC, Cante D, Paolini M, Piva C, Baratto L, Franco P, La Porta MR. Implementation of a Commercial Deep Learning-Based Auto Segmentation Software in Radiotherapy: Evaluation of Effectiveness and Impact on Workflow. Life. 2022; 12(12):2088.

Chicago/Turabian Style

Radici, Lorenzo, Silvia Ferrario, Valeria Casanova Borca, Domenico Cante, Marina Paolini, Cristina Piva, Laura Baratto, Pierfrancesco Franco, and Maria Rosa La Porta. 2022. "Implementation of a Commercial Deep Learning-Based Auto Segmentation Software in Radiotherapy: Evaluation of Effectiveness and Impact on Workflow" Life 12, no. 12: 2088.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop