Next Article in Journal
Investigating the Role of Heparanase in Breast Cancer Development Utilising the MMTV-PyMT Murine Model of Mammary Carcinoma
Next Article in Special Issue
Ultrasound Imaging with Flexible Array Transducer for Pancreatic Cancer Radiation Therapy
Previous Article in Journal
Arterial Thrombosis in Patients with Acute Myeloid Leukemia: Incidence and Risk Factors
Previous Article in Special Issue
Ablative vs. Non-Ablative Radiotherapy in Palliating Locally Advanced Pancreatic Cancer: A Single Institution Experience and a Systematic Review of the Literature
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 15
Issue 11
10.3390/cancers15113061
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

deepPERFECT: Novel Deep Learning CT Synthesis Method for Expeditious Pancreatic Cancer Radiotherapy

by
Hamed Hooshangnejad
1,2,3,
Quan Chen
4,
Xue Feng
5,
Rui Zhang
6 and
Kai Ding
2,3,*
1
Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, MD 21287, USA
2
Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins School of Medicine, Baltimore, MD 21287, USA
3
Carnegie Center of Surgical Innovation, Johns Hopkins School of Medicine, Baltimore, MD 21287, USA
4
City of Hope Comprehensive Cancer Center, Duarte, CA 91010, USA
5
Carina Medical LLC, Lexington, KY 40513, USA
6
Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, USA
*
Author to whom correspondence should be addressed.
Cancers 2023, 15(11), 3061; https://doi.org/10.3390/cancers15113061
Submission received: 6 April 2023 / Revised: 22 May 2023 / Accepted: 25 May 2023 / Published: 5 June 2023
(This article belongs to the Special Issue Radiation Therapy for Pancreatic Cancer)
Browse Figures
Versions Notes

Abstract

:

Simple Summary

Pancreatic cancer is a devastating disease with more than 60,000 new cases each year and a less than 10 percent 3-year overall survival rate. Radiation therapy is an effective treatment for locally advanced pancreatic cancer. The current clinical RT workflow, however, is lengthy and involves separate image acquisition for diagnostic CT and planning CT, which imposes a huge burden on patients and their caretakers. Moreover, studies have shown a reduction in mortality rate from expeditious radiotherapy treatment courses. Here, we proposed an innovative deep learning solution to adapt the shape of a patient’s body in diagnostic CT to the treatment delivery setup, and consequently, reduce the time to treatment initiation by half. As a result, our method also reduces the time to surgery and greatly decreases the risk of progression for pancreatic cancer.

Abstract

Major sources of delay in the standard of care RT workflow are the need for multiple appointments and separate image acquisition. In this work, we addressed the question of how we can expedite the workflow by synthesizing planning CT from diagnostic CT. This idea is based on the theory that diagnostic CT can be used for RT planning, but in practice, due to the differences in patient setup and acquisition techniques, separate planning CT is required. We developed a generative deep learning model, deepPERFECT, that is trained to capture these differences and generate deformation vector fields to transform diagnostic CT into preliminary planning CT. We performed detailed analysis both from an image quality and a dosimetric point of view, and showed that deepPERFECT enabled the preliminary RT planning to be used for preliminary and early plan dosimetric assessment and evaluation.

1. Introduction

Pancreatic cancer is a devastating disease with more than 60,000 new cases each year and less than 10% 5-year overall survival rate [1]. Pancreatic cancer patients are at great risk of distant progression, so achieving local control (LC) is critical for these patients [1,2,3,4,5,6]. Radiation therapy (RT) is an effective treatment for achieving LC [7,8,9,10]. Rapid treatment planning and delivery are critical for aggressive pancreatic cancer and a week delay increases the possibility of spreading or recurring [11,12], yet the current RT workflow is considerably time-consuming.
Figure 1 shows the current RT workflow which consists of numerous steps. Multiple appointments and image acquisition result in significant wait time, and a huge burden on patients and caretakers [13,14]. One major source of delay in the workflow is due to the several appointments and separate image acquisitions, namely acquiring a diagnostic computed tomography (dCT) scan and the planning CT (pCT) scan. This results in a median of 15 days delay for patient diagnosis and 15 more from diagnosis to treatment initiation [15].
Reducing the delays to treatment initiation has a significant clinical impact. It has been shown that shorter therapy initiation is associated with improved survival [16]. Amongst more than 29,000 patients, starting any treatment within 6 weeks improved median overall survival [16]. Moreover, the data from more than 70,000 pancreatic cancer patients shows the most substantial associations with worsened mortality were seen among other cancer types with a 3.2% increase in mortality per week of delay [17]. Moreover, reducing the time before surgery to 32 days reduces the risk of the progression of the tumor to the unresectable stage by half compared with a longer waiting time.
Because of the dCT’s superior resolution, it is used for delineating the tumor. It is shown that the dCT can be used for RT planning [13,14,18,19]. However, in practice, because of the differences in image acquisition settings, dCT is not used for RT planning and a separate pCT is acquired, which results in a considerable delay between diagnosis and treatment delivery. In the case of the patients included in this study, there was a median of 11 days, ranging from 2 to 30 days delay between diagnostic scan and simulation scan. The previous attempts such as STAT RT [20] for using the day 1 onboard imaging for treatment planning before subsequent treatment delivery for palliative cases are not suitable for pancreatic cancer SBRT. The physicians need time to decide the trade-off between target coverage and organ at risk (OAR) sparing, and then the dose prescription.
Figure 2 shows a typical pair of dCT and pCT acquired in our institution. The use of a different couch in the radiology department and radiation oncology department results in a clear difference in the shape of the patient’s back. In radiology CT scans, the curved couch top is used for patient comfort. By contrast, radiation oncology flat couch top focuses on the daily reproducibility of patient position. Also, to reduce patient movement, the active breath-hold technique is used as the standard acquisition procedure for pCT and treatment delivery in our institution, which causes a difference in patient anatomy on the two scans [21,22,23]. Thus, the design of the initial RT plan is only possible until after the acquisition of the pCT scan which results in a considerable stall in the treatment workflow.
Radiation oncology has been more optimized and automated as a result of the development of machine learning (ML) and deep learning (DL) approaches [24]. Moreover, DL methods can be added to different components of the oncology pathway from screening and diagnosis to treatment design and delivery [25], and it can shorten the treatment course, improve efficiency, consistency, and most importantly treatment outcomes [26,27]. Specifically, in RT DL techniques have been applied to many tasks to improve the efficacy and speed like dose calculation, segmentation, auto-treatment planning, image guidance, and image registration [28]. The short runtime of DL models has resulted in speeding up time-consuming tasks [29]. For instance, MCDNet has been introduced to accelerate Monte Carlo dose calculation [30]. VoxelMorph is a well-known DL model for image registration [31], and numerous DL methods have been proposed for image segmentation [29]. These models can implement precision medicine by, for instance, helping with initial treatment modality decisions [32], and personalizing treatment design and delivery [33,34].
In this study, we addressed the delay between diagnosis and treatment delivery, by developing deepPERFECT; a generative adversarial network (GAN) model. The adversarial loss that encourages the generation of data indistinguishable from real data, resulted in the huge success of GAN models in image synthesis applications [35], such as synthesizing CT from cone-beam CT [36,37], and magnetic resonance image [38]. deepPERFECT synthesizes a four-dimensional (4D) output, including three channels of three- dimensional deformation fields (DFs) that transform dCT into synthesized pCT (sCT), which can be used for initial verification of the RT treatment plan, thus expediting the treatment course.
In our feasibility study, we showed that deepPERFECT can expedite the current clinical workflow and treatment course by removing the need for acquiring pCT before designing the initial treatment RT plan. Given that there is a significant relationship between the increase in mortality and, OAR toxicity and delay in treatment delivery [39,40,41,42], we believe deepPERFECT improves the quality of life for much-needed pancreatic cancer patients. More importantly, it allows the physicians to evaluate the potential RT prognosis ahead of time, and verify the plan on the treatment day-one CT. On Day 1 treatment, the sCT plan will be adapted to the treatment delivery patient setup using on/off table imaging for same-day online adaptive radiation therapy (ART) such as on-table adaptive therapy methods like Ethos or same-day off-table online ART using Raystation adaptive treatment planning (RaySearch Laboratories, Sweden). In the expedited workflow, deepPERFECT removes the need for a separate CT simulation session for preliminary RT planning, the preliminary plan is designed on the synthesized CT. Same day (treatment day 1) on-board or off-treatment table sim CT is then acquired on the same day as treatment or a day before for treatment delivery and adaptation. As a result, we will save considerable time, as the design of the initial RT plan and OAR-target trade-offs which is delayed until after the CT simulation session in the standard of care, can be done on synthesized preliminary CT.

2. Materials and Methods

2.1. Data Preparation

We included data from 25 pancreatic cancer patients treated with stereotactic body radiation therapy (SBRT) in our institution under Internal Review Board (IRB) approval (15 cases for training and validation, and 10 cases for testing and RT planning). dCT was conducted with 120 KVp, a 50 cm field of view, and automatic exposure control, and pCT scans were acquired with 120 KVp, 200 mA, and a 50 cm field of view. The dCT scans had slice thickness ranging from 0.5 to 1.25 mm, and the pCT scans’ slice thickness was 2 mm. To achieve a uniform 3D physical dimension, in practice, all scans can be resampled to a 1 mm slice thickness, but due to GPU limitation, we resampled to 2.5 mm. The pCT image was contoured by physicians as part of the standard of care in our institution, and the dCT and sCT contours were generated by applying deformation vector fields to the dCT contours, and then, were verified by a physician.
Using an in-house robust couch removal algorithm, we first removed the couch from the scans, only keeping the patient’s body. Because the treatment couch was identical, it was later digitally added to the synthesized scans. pCT and dCT were first aligned with the spine. Because a pCT image had not yet been acquired at the time of dCT acquisition, we chose dCT as the physical origin for rigid registration and used ROI-restricted rigid registration using an in-house automatic spine segmentation algorithm to further align the spines.
Next, the dCT was deformably registered to pCT. To avoid the unreal deformation of the spine, such as elongation of vertebrae and disk, we used the spine mask to define a rigidity penalty term. Because the high contrast lung region dominated the result of the registration, we performed a two-step sequential registration, first, the whole body was registered, and automatically segmented lungs were excluded for further abdominal-focused registration. The abdominal mask was determined by using the lowest point on lung contours while keeping the spine rigid penalty to avoid unreal deformation. All registrations were done by the state-of-the-art Elastix image registration algorithm [43,44,45,46,47,48,49,50,51,52].
As 15 cases were too few for the successful training of a DL model, we augmented the data by applying random shifts of between −20 mm and 20 mm and random rotations of −10 to 10 degrees along the z-axis (depth); lastly, we used randomly generated patches of the CT images as the inputs to the model. Overall, in each epoch, the DL model was trained on more than 6000 augmented data.

2.2. Deep Learning Model

We used four network configurations: (1) a 3D U-Net convolutional network with a patch size of 128 × 128 × 128, (2) a 2.5D Pix2Pix generative adversarial network (GAN) with three adjacent slices of 128 × 128, (3) a 3D Pix2Pix GAN with a small patch size (32 × 32 × 32), and (4) a 3D Pix2Pix GAN with a large patch size (128 × 128 × 128). The output of the DL model was the DFs. We chose to generate DFs rather than CT scans to keep the CT intensity calibration intact. The intensity of dCT and pCT is based on careful calibration and quality assurance of the CT scanner, and is crucial for accurate dose calculation. Because DFs are real numbers with positive and negative values, we used the leaky rectified linear unit activation function f(x) = max (0.01x, x). Batch normalization was applied to the output of the convolutional layers. deepPERFECT was trained with three loss functions: (1) DF identity loss, the L1-norm of the difference between the true and generated DFs; (2) abdominal identity loss, the L1-norm of the difference in the intensity of the abdominal portion of the pCT and intermediate sCT scans, created using the generated DFs applied to dCT in-place; (3) the adversarial loss for GAN architecture. A weighted Adam optimizer was used to optimize the loss functions. Figure 3 shows an overview of the DL model and loss used for training.

2.3. Training and Testing of the Model

Of the 15 cases used for training, we used data from 13 patients for training and from 2 for validation. After the hyperparameters were set, we trained the model on all 15 cases and tested it on the 10 leave-out cases. We used the following cost function for deepPERFECT training. The model was trained using NVIDIA Titan XP with 12 GB RAM.
l o s s = λ 1 L G A N G , D + λ 2 L L 1 G + λ 3 L L 1 I + λ 4 R s m o o t h ( G )
λ is the hyparameter that indicates the effect of each loss on the final loss function value. In this equation, G denotes the generator output, which represents deformation vector fields, and D denotes the discriminator. I denotes the final deformed image and R represents the regulatory term. L G A N G , D is the adversarial loss, defined as:
L G A N G , D = E x , y log ⁡ D x , y + E x ( log ⁡ 1 − D x , G x )
in which x represents the input and y is the target deformation vector field.
L L 1 G is the L1 norm of the differences between target DVFs and network generator DVFs, defined as:
L L 1 G = E x , y ( y − G ( x ) )
L L 1 I is the L 1 norm between target image I and the deformed image, created by applying G ( x ) or G x to the input image; it is defined as:
L L 1 I = E x , y ( I − G x ( x ) )
Finally, to enforce smooth deformation fields, we used the second-order curvature regulatory term, which is widely used in the registration literature, which is given as follows:
R s m o o t h G = ∫ ∑ j = 1 3 ∆ G i x 2

2.4. Radiation Therapy Planning

We planned the ten test cases with volumetric modulated arc therapy (VMAT) SBRT (33 Gy in 5 fx), according to the pancreatic SBRT planning protocol of our institution. The planning target volume (PTV) was created using a 2 mm expansion of the mock multiple active breath-hold (GTV-multabh), which itself, is a 3 mm uniform expansion of gross tumor volume (GTV). For further details, please refer to our previous studies [5,21,22]. The target objectives for RT planning were as follows: 100% of the PTV received 25 Gy, at least 95% of the PTV volume received 33 Gy, less than a 1 cc volume of PTV received more than 42.9 Gy, 100% of the GTV received 33 Gy, and at least 95% of the GTV-multabh received 33 Gy. The organs at risk (OAR) constraints were as follows: less than 20 ccs of the bowel, duodenum, and stomach received 20 Gy, less than 1 cc of the bowel, duodenum, and stomach received 33 Gy, less than 25% of kidney received 12 Gy, less than 50% of liver received 12 Gy, and less than 1 cc of the spinal cord received 8 Gy. We used the Raystation (RaySearch Laboratories, Stockholm, Sweden) treatment planning system for plan optimization and dose distribution calculation.
As shown in Figure 4, for each patient, we designed and optimized two VMAT plans for pCT and sCT. The dose distribution was calculated for four scenarios: (1) The dose distribution of the pCT plan on the pCT scan (the ground truth dose distribution). (2) The dose distribution of the sCT plan on the sCT scan. (3) The “Plan Recalculation” scenario, in which we recalculated the dose distribution of the sCT plan on the pCT scan using the pCT isocenter by shifting the iso-center of the sCT plan beams to the pCT iso-center. Then, we evaluated the dosimetric indices using the pCT contours. (4) The “Synthesized ROIs on Planning CT” scenario, in which we recalculated the dose distribution of the sCT plan on the pCT scan using the sCT isocenter (no shift in isocenter). Then, we evaluated the dosimetric indices using the sCT contours mapped to the pCT. Using this scenario, we assured that the observed difference between the ground truth and the “Plan Recalculation” scenario was due to unpredictable and patient-specific abdominal anatomical variation and not CT quality.

2.5. Evaluation Metrics

First, we evaluated deepPERFECT by comparing the intensity of pCT scans with sCT scans using the root averaged squared sum of differences (RASSD). Secondly, the body contours on the two scans were compared using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). No center of mass alignment was performed for the DSC and HD calculations. Lastly, we reported dose volume histogram point measurements, the V22 Gy and V33 Gy, defined as the volume of the ROI receiving 22 Gy and 33 Gy, for OARs (duodenum, stomach, and bowel), and the percentage of target coverage with the prescribed dose (V33 Gy) for target volumes (GTV and PTV) for the four dose distribution scenarios, as explained in the previous section.

2.6. Statistical Analysis

Using the pairwise permutation test (n = 10,000), we tested the equivalency of dosimetric indices. The normality assumption was circumvented by using a non-parametric permutation test.

2.7. Optimization Parameters

deepPERFECT was trained using Adam optimizer [53] with starting learning rate of 0.01, which decayed exponentially with a rate of 0.97. We choose β 1 = 0.5 , β 2 = 0.9 , and e p s i l o n = 10 − 8 . The training was performed for 150 epochs, and the learning rate was updated after each epoch. We determined the cost function weights via trial and error and multiple tests on the training and validation sets.

2.8. Placement of the Virtual Couch

To synthesize sCT, the couch was first removed from the dCT scans, but the treatment couch was required for accurate dose calculation. Thus, later in the process, we developed an algorithm that augments the sCT scan with the couch by placing a virtual couch with its surface tangent to the back of the patient.

3. Results

Figure 5 shows the results of an example test case. Figure 5A shows the dCT scan (input to the model), Figure 5B shows the pCT scan (ground truth), and Figure 5C–F show the sCT scans for 3D Pix2Pix with large patches, U-Net, 2.5D Pix2Pix, and 3D Pix2Pix with small patches. As seen, the back of the patient and the overall shape of the abdominal area have the most similarity to the sCT scan generated by 3D Pix2Pix with large patches (Figure 5C).
We evaluated the quality of the sCT scan using the ASSD, DSC, and HD for the entire body contour. The results are summarized in Table 1. The 3D Pix2Pix trained on the large patches showed the best performance among the other configurations. We used this configuration to generate sCT scans for the RT planning part of the study.
For the best model, Pix2Pix 3D with large patches, the average ± standard deviation difference in GTV volume in pCT and sCT was 1.1 ± 1.8 cc. The GTV minimum distance to the main OARs, namely the duodenum, stomach, and bowel, was also measured. The average ± standard deviation of the difference in minimum distances between GTV–duodenum, GTV–stomach, and GTV–bowel were 0.37 ± 1.1 mm, 0.52 ± 1.4 mm, and 0.61 ± 1.5 mm, respectively. There was a median of 11 days’ delay, ranging from 2 to 30 days, between the diagnostic scan and simulation scan, and a median of 8 days to treatment initiation. The average DSC between dCT and pCT was 0.62 ± 0.1, which increased to 0.82 ± 0.12 for pCT and sCT. Moreover, the DSC in Figure 6 shows the diagnostic CT (Figure 6A), planning CT (Figure 6B), and synthesized CT (Figure 6C) at the abdominal level/in the abdominal window (40/400) on the first row. The second row illustrates an HU intensity difference map (Figure 6D) between the planning CT and diagnostic CT (pCT–dCT) and (Figure 6E) between the planning CT and synthesized CT (pCT–sCT). deepPERFECT’s predicted synthesized CT from diagnostic CT shows a high similarity to the planning CT.
Next, we compared the SBRT plans using the OARs (duodenum, stomach, and bowel) V20 Gy and V33 Gy, Dmax (max dose), and V100% and V95% for the target volumes (GTV and PTV). The duodenal V33 Gy showed a marginally significant difference (p-value = 0.049) between ground truth and plan recalculation. No other statistically significant differences were found in V20 Gy and V33 Gy. As expected, due to the unpredictable and patient-specific abdominal anatomical variations the “plan recalculation” scenario, in which the isocenter is only shifted, there was, on average, a 2% reduction in PTV coverage with the prescribed dose (33 Gy). PTV V95% and GTV V100%, and V95%, were comparable. Figure S1 of the Supplementary Materials shows the V95% coverage for GTV and PTV, as well as the Dmax for the OARs.

4. Discussion

We presented a novel DL system to reduce the extended delay before treatment delivery due to the separate scan acquisitions. Our method makes the dCT compatible with the treatment room setup, and thus, allows an initial RT plan to be designed. Thus, physicians can evaluate the potential RT prognosis ahead of time, verify the plan on the day-one treatment CT, and apply any online adaptation if needed. This reduces the wait time before the start of the treatment course. Our method can reduce the length of the treatment course by at least one week, and given that the treatment delay increases mortality [40], we believe our method can improve the patients’ quality of life.
The sources of delay from diagnosis to treatment delivery are not only the scheduling, wait times, and the acquisition of planning CT, but also treatment design, RT planning, and verification. Physicians need time to make a decision regarding the trade-off between target coverage and sparing OARs, and dose prescription. On treatment day one, there is insufficient time to make a decision. For example, the patient may have previous RT treatment, or neoadjuvant or other concurrent/adjuvant cancer treatment therapies such as chemotherapy. These are all delayed until pCT acquisition; therefore, synthesizing deepPERFECT-generated sCT can provide enough time for physicians to perform assessments ahead of time. With the implementation of deepPERFECT, as soon as the diagnostic CT is available, physicians can also evaluate the early prognosis of RT treatment, and decide on the prescription and dose constraint, and initiate the treatment.
In this study, we used the 3D UNet GAN architecture for developing the deepPERFECT framework. The 3D UNet GAN architecture has been long used for various medical image synthesis applications. It has been used for synthesizing PET scans from MRI [54,55], and synthesizing the CT scan from MRI [56] and is frequently used in MRI reconstruction [57,58,59,60,61], low dose CT denoising [57,58,62], optimization of the pre-trained network for sharpness detection and highlighting low contrast region in CT image [62], and many other applications. Due to 3D UNet GAN robust performance and versatility, we also based deepPERFECT design upon 3D UNet architecture. More advanced DL methods have also been used for similar applications that warrants future study to determine if they can be used to improve the performance of deepPERFECT framework.
Although we used a DL model in this study, other methods including analytical and physical-based models like the finite element method could potentially be used. Previously, these methods have been used to predict the deformation of the body and organs because of surgical procedures and physiological deformation [4,5,6,14]. The downside of the DL models is that they require high computational resources, a large amount of data, and a long training time, however, their main advantage is very short run-time. Our DL model generates the DFs in less than a second, but a finite element model may take hours to do the analysis
Table 1 shows the results of the quantified image synthesis evaluation. The 3D Pix2Pix with large patches has the best performance among the other structures. Although the 3D Pix2Pix with small patches and 2.5D Pix2Pix require much less GPU memory, they have lower performance. We believe this is because the differences between dCT and pCT are clearer in a large field of view. The U-Net structure has the worst performance compared to the GAN models, which demonstrates the superiority of GAN models in the synthesis task. Here, we generated DFs; therefore, the CT intensity of the sCT remains undistorted and calibrated, and as a result, sCT can be directly used for planning. The average DSC for the GTV contour was 0.82. The lowest GTV DSC was observed for cases with the maximum delay. We also observed a trend between DSCs and delay, whereby the longer the delay, the lower the DSC. In addition, the GTV location is subjected to abdominal day-to-day movement and filling, which makes the exact prediction of the GTV location nearly impossible. In daily clinical practice, this challenge is addressed by aligning the GTV or fiducial markers inside the GTV under the guidance of onboard cone beam CT and using adaptive radiation therapy using the treatment Day1 CT. [63,64]
Figure 7 shows bar plots for V22 Gy and V33 Gy of the duodenum, stomach, and bowel, and the V33 Gy target coverage. The reason V22 Gy and V33 Gy were chosen is that, as seen in the planning protocol, these indices are the main clinical constraints for the OARs. Moreover, because the prescribed dose of the SBRT plans was 33 Gy, the V33 Gy for target volumes is of great interest to physicians. Our results suggested that the RT plan on the sCT scan had no statistically significant difference from the ground truth. However, when the dose was recalculated on the planning CT, using the planning CT contours, there was a significant difference between the ground truth and recalculated V22 Gy and V33 Gy. This difference is due to the natural, unpredictable, and patient-specific variation in the abdominal organs. When the isocenter of dose distribution was shifted to the pCT plan isocenter, we achieved full GTV coverage, and although the average PTV coverage was reduced, the change in PTV coverage was not statistically significant from the ground truth.
One major consideration with deepPERFECT is how easy it is to implement this method in the clinic. In this study, we showed the feasibility of using a DL-based model to shorten the time to treatment initiation in pancreatic cancer RT. However, at this moment, deepPERFECT is not ready to use in the clinic due to limited resources, such as GPU, for high-resolution and large-dataset training. Currently, we are working on acquiring resources to enhance our model for clinical use. deepPERFECT can easily be embedded in treatment planning systems (TPS), similarly to the auto-segmentation or auto-planning features of TPS. As soon as the diagnostic scan of the patient is available, a synthesized CT can be generated with one click and an RT plan can be designed for early treatment decision-making and outcome assessment.
We are aware that our study may have a few shortcomings. First, here, we used a small set of training data. We tried to overcome this issue by using multiple data augmentation methods. As explained in the Section 2, we could increase the amount of data from 15 cases to more than 6000 cases. Another shortcoming is that due to GPU limitations, we had to reduce the resolution of the CT images. The loss of resolution in our current model may have resulted in some uncertainty in tumor delineation. A high-quality dCT scan is ideal for tumor delineation due to the high resolution and contrast. As a result, the performance of our current model was degraded due to this loss of quality and resolution. The current resolution may not be enough for clinical treatment planning and delivery, as well; however, here, our goal was to demonstrate the feasibility of the model’s rapid workflow, and as part of our future studies, we will train the model on high-end GPUs with no degradation in image quality; therefore, since deepPERFECT generates the sCT sccan directly from dCT, the sCT image will be a high-resolution scan, as well.
Another limitation of our study is that in our current model, we do not include any breathing information. Here, we used diagnostic CT to generate a breath-hold planning CT scan. Considering that patients may have different breath-hold levels, the current model may not be able to capture patient-specific patterns. A potential remedy is to incorporate a breathing measuring device to measure the patient’s breath-hold level via a quick breathing test. Based on this, we can scale the deformation vector fields to match the breathing level on the sCT image to the patient-specific breath-hold level. However, at the moment, these data are not available to us; therefore, as part of our future work, to implement this model in our clinical practice, we will measure patients’ breath-hold levels using a spirometer to create a breathing level-aware system. The use of conditional-GAN, as used in this study, can, to some extent, incorporate patient-specific information into preliminary CT generation.
An important consideration regarding deepPERFECT is its reliance on systems with on-line and off-line adaptive RT (ART) capabilities. ART can prolong the treatment time comparing to the conventional RT, however, recent advances in artificial intelligence (AI) applications [65] and graphical processing units (GPU) based dose calculation engines [66,67,68], have enabled the many steps of the online ART to be done in a reasonable timeframe [69]. For instance, the recently commercially available ART system Varian Ethos system (Varian Medical Systems, Palo Alto, CA, USA) can perform online ART, where new RT plan is adapted from initial RT plan based on simulation/planning CT while patient is lying on the RT treatment couch, in less than 20 minutes [70,71,72]. Güngör, et al. [73] did a thorough time analysis of online ART using ViewRay’s MRIdian A3i (ViewRay, Oakwood Village, OH, USA) and reported that 83.2% of adaptive fractions were completed in less than 60 minutes. Although the treatment time may increase, using deepPERFECT, the time-to-treatment initiation will be greatly reduced.
New-onset of diabetes and weight loss are common features of pancreatic cancer [74]. The severity, extent to which patients are affected, and which type of patients in most affected are not well understood. Studies have reported that patients with high body mass index (BMI), and obesity show the most weight loss [75]. Although we only used data from 25 patients, this result is consistent with what we have seen in our data. More importantly, another important factor is the time delay between dCT and pCT scans. Following a dCT scan, patients with a low BMI, even with a long delay between the two scans, do not show noticeable weight loss on a pCT scan. However, for patients with similar delays, the higher the BMI, the more noticeable the change in the patient’s body shape. Finally, for one patient with a high BMI and 2 days’ delay between the two scans, there was no weight loss seen. As a result, although the current model does not account for weight loss, which is reflected in a body contour DSC of 0.95, ultimately, with the implementation of deepPERFECT, which results in reducing the delay to a few days, the effect of weight loss will be negligible.
Finally, in the current study, we only applied our DL model to pancreatic cancer cases. However, deepPERFECT can be used for other anatomical sites, as well. Our preliminary evaluations showed that our model can successfully be applied to liver and lung cancer patients, but further training and testing are required. We also applied our model to a prostate cancer patient, but due to the considerable differences in the CT field of view, the model showed inferior performance. Nevertheless, the concept of planning a CT-free workflow can still be applied to prostate cancer. Therefore, future studies will aim to improve the performance of the model by using more data and increasing the resolution of the model using higher-performance GPU resources, and extending the application of deepPERFECT to more anatomical sites.

5. Conclusions

We demonstrated the feasibility of planning a CT-free rapid pancreatic RT workflow using our deepPERFECT method. We used a fully convolutional 3D/4D GAN DL model to synthesize planning CT from the initial diagnostic CT. The synthesized CT is compatible with the treatment room setup and mimics the patient’s shape on the treatment couch. Using this method, we showed that a comparable RT plan to the planning CT plan can be designed using synthesized CT, which, in turn, considerably expedites the workflow and reduces the highly undesirable wait time before RT treatment delivery. This allows physicians to assess the potential RT outcome well ahead of the RT treatment course and verify and adapt the initial plan, if needed, to the day-one treatment CT.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cancers15113061/s1, Figure S1: Comparison of dosimetric indices for SBRT plans: the comparison was done for V95% for target volumes (GTV and PTV) and Dmax for the proximal OARs (duodenum, stomach, and bowel). Each figure consists of four box plots for RT plans designed on pCT (ground truth), sCT, and plan recalculation (dose recalculation by shifting the isocenter of beams to pCT isocenter), Synthesized ROIs on planning CT (dose recalculation on pCT using sCT ROIs).

Author Contributions

Conceptualization, H.H. and K.D.; methodology, H.H. and K.D.; software, H.H. and K.D.; validation, H.H. and K.D.; formal analysis, H.H. and K.D.; investigation, H.H. and K.D.; resources, K.D.; data curation, H.H. and K.D.; writing—original draft preparation, H.H. and K.D.; writing—review and editing, H.H., K.D., Q.C., X.F. and R.Z.; visualization, H.H. and K.D.; supervision, K.D.; project administration, K.D.; funding acquisition, K.D. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Cancer Institute of the National Institutes of Health (award number: R37CA229417). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Institutional Review Board Statement

The study was approved by the JHU IRB, Approval Code: IRB-1, Approval Date: 18 March 2020.

Informed Consent Statement

Written informed consent has been obtained from the patient(s) to publish this paper.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors upon request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Siegel, R.L.; Miller, K.D.; Fuchs, H.E.; Jemal, A. Cancer Statistics, 2022. CA Cancer J. Clin. 2022, 72, 7–33. [Google Scholar] [CrossRef]
  2. Sener, S.F.; Fremgen, A.; Menck, H.R.; Winchester, D.P. Pancreatic Cancer: A Report of Treatment and Survival Trends for 100,313 Patients Diagnosed from 1985–1995, Using the National Cancer Database. J. Am. Coll. Surg. 1999, 189, 1–7. [Google Scholar] [CrossRef] [PubMed]
  3. Shin, S.H.; Kim, H.J.; Hwang, D.W.; Lee, J.H.; Song, K.B.; Jun, E.; Shim, I.K.; Hong, S.M.; Kim, H.J.; Park, K.M.; et al. The DPC4/SMAD4 Genetic Status Determines Recurrence Patterns and Treatment Outcomes in Resected Pancreatic Ductal Adenocarcinoma: A Prospective Cohort Study. Oncotarget 2017, 8, 17945–17959. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Hooshangnejad, H.; Han-Oh, S.; Shin, E.J.; Narang, A.; Rao, A.D.; Lee, J.; McNutt, T.; Hu, C.; Wong, J.; Ding, K. Demonstrating the Benefits of Corrective Intra-Operative Feedback in Improving the Quality of Duodenal Hydrogel Spacer Placement. Med. Phys. 2022, 49, 4794–4803. [Google Scholar] [CrossRef] [PubMed]
  5. Hooshangnejad, H.; Youssefian, S.; Narang, A.; Shin, E.J.; Rao, A.D.; Han-Oh, S.; McNutt, T.; Lee, J.; Hu, C.; Wong, J.; et al. Finite Element-Based Personalized Simulation of Duodenal Hydrogel Spacer: Spacer Location Dependent Duodenal Sparing and a Decision Support System for Spacer-Enabled Pancreatic Cancer Radiation Therapy. Front. Oncol. 2022, 12, 833231. [Google Scholar] [CrossRef] [PubMed]
  6. Hooshangnejad, H.; Youssefian, S.; Guest, J.K.; Ding, K. FEMOSSA: Patient-Specific Finite Element Simulation of the Prostate–Rectum Spacer Placement, a Predictive Model for Prostate Cancer Radiotherapy. Med. Phys. 2021, 48, 3438–3452. [Google Scholar] [CrossRef] [PubMed]
  7. Bruynzeel, A.M.E.; Lagerwaard, F.J. The Role of Biological Dose-Escalation for Pancreatic Cancer. Clin. Transl. Radiat. Oncol. 2019, 18, 128–130. [Google Scholar] [CrossRef] [Green Version]
  8. Ben-Josef, E.; Shields, A.F.; Vaishampayan, U.; Vaitkevicius, V.; El-Rayes, B.F.; McDermott, P.; Burmeister, J.; Bossenberger, T.; Philip, P.A. Intensity-Modulated Radiotherapy (IMRT) and Concurrent Capecitabine for Pancreatic Cancer. Int. J. Radiat. Oncol. Biol. Phys. 2004, 59, 454–459. [Google Scholar] [CrossRef]
  9. Li, J.; Ng, J.; Allendorf, J.; Saif, M.W. Locally Advanced Pancreatic Adenocarcinoma: Are We Making Progress? Highlights from the “2011 ASCO Annual Meeting”. Chicago, IL, USA, 3–7 June 2011. JOP J. Pancreas 2011, 12, 347–350. [Google Scholar]
  10. Machtay, M.; Paulus, R.; Moughan, J.; Komaki, R.; Bradley, J.E.; Choy, H.; Albain, K.; Movsas, B.; Sause, W.T.; Curran, W.J. Defining Local-Regional Control and Its Importance in Locally Advanced Non-Small Cell Lung Carcinoma. J. Thorac. Oncol. 2012, 7, 716–722. [Google Scholar] [CrossRef] [Green Version]
  11. International Atomic Energy Agency. Radiotherapy in Palliative Cancer Care: Development and Implementation; Human Health Reports; International Atomic Energy Agency: Vienna, Austria, 2012; ISBN 978-92-0-109009-6. [Google Scholar]
  12. Buss, E.J.; Kachnic, L.A.; Horowitz, D.P. Radiotherapy for Locally Advanced Pancreatic Ductal Adenocarcinoma. Semin. Oncol. 2021, 48, 106–110. [Google Scholar] [CrossRef] [PubMed]
  13. Wong, S.; Roderick, S.; Kejda, A.; Atyeo, J.; Grimberg, K.; Porter, B.; Booth, J.; Hruby, G.; Eade, T. Diagnostic Computed Tomography Enabled Planning for Palliative Radiation Therapy: Removing the Need for a Planning Computed Tomography Scan. Pract. Radiat. Oncol. 2021, 11, e146–e153. [Google Scholar] [CrossRef] [PubMed]
  14. Hooshangnejad, H.; Ding, K. Feasibility of Planning-CT-Free Rapid Workflow for Stereotactic Body Radiotherapy: Removing the Need for Planning CT by AI-Driven, Intelligent Prediction of Body Deformation. In Proceedings of the Medical Imaging 2022: Image-Guided Procedures, Robotic Interventions, and Modeling, Virtual, 21–27 March 2022; Volume 12034. [Google Scholar]
  15. Salomaa, E.-R.; Sällinen, S.; Hiekkanen, H.; Liippo, K. Delays in the Diagnosis and Treatment of Lung Cancer. Chest 2005, 128, 2282–2288. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Gamboa, A.C.; Rupji, M.; Switchenko, J.M.; Lee, R.M.; Turgeon, M.K.; Meyer, B.I.; Russell, M.C.; Cardona, K.; Kooby, D.A.; Maithel, S.K.; et al. Optimal Timing and Treatment Strategy for Pancreatic Cancer. J. Surg. Oncol. 2020, 122, 457–468. [Google Scholar] [CrossRef]
  17. Khorana, A.A.; Tullio, K.; Elson, P.; Pennell, N.A.; Grobmyer, S.R.; Kalady, M.F.; Raymond, D.; Abraham, J.; Klein, E.A.; Walsh, R.M.; et al. Time to Initial Cancer Treatment in the United States and Association with Survival over Time: An Observational Study. PLoS ONE 2019, 14, e0213209. [Google Scholar]
  18. Kenny, L.; Lehman, M. Sequential Audits of Unacceptable Delays in Radiation Therapy in Australia and New Zealand. Australas Radiol. 2004, 48, 29–34. [Google Scholar] [CrossRef]
  19. Schuler, T.; Back, M.; Hruby, G.; Carroll, S.; Jayamanne, D.; Kneebone, A.; Stevens, M.; Lamoury, G.; Morgia, M.; Wong, S.; et al. Introducing Computed Tomography Simulation—Free and Electronic Patient-Reported Outcomes—Monitored Palliative Radiation Therapy into Routine Care: Clinical Outcomes and Implementation Experience. Adv. Radiat. Oncol. 2021, 6, 100632. [Google Scholar] [CrossRef]
  20. Wilson, D.D.; Alonso, C.E.; Sim, A.J.; Peck, T.; Handsfield, L.L.; Chen, Q.; Blackhall, L.; Showalter, T.N.; Reardon, K.A.; Read, P.W. STAT RT: A Prospective Pilot Clinical Trial of Scan-Plan-QA-Treat Stereotactic Body Radiation Therapy for Painful Osseous Metastases. Ann. Palliat. Med. 2019, 8, 221–230. [Google Scholar] [CrossRef]
  21. Han-Oh, S.; Hill, C.; Wang, K.K.-H.; Ding, K.; Wright, J.L.; Alcorn, S.; Meyer, J.; Herman, J.; Narang, A. Geometric Reproducibility of Fiducial Markers and Efficacy of a Patient-Specific Margin Design Using Deep Inspiration Breath Hold for Stereotactic Body Radiation Therapy for Pancreatic Cancer. Adv. Radiat. Oncol. 2021, 6, 100655. [Google Scholar] [CrossRef]
  22. Su, L.; Iordachita, I.; Zhang, Y.; Lee, J.; Ng, S.K.; Jackson, J.; Hooker, T.; Wong, J.; Herman, J.M.; Sen, H.T.; et al. Feasibility Study of Ultrasound Imaging for Stereotactic Body Radiation Therapy with Active Breathing Coordinator in Pancreatic Cancer. J. Appl. Clin. Med. Phys. 2017, 18, 84–96. [Google Scholar] [CrossRef] [Green Version]
  23. Hooshangnejad, H.; Ding, K. Predicting the Efficacy of Spacer Placement for Pancreatic Cancer Using a Novel Patient-Specific Virtual Spacer. In Medical Physics; Wiley: Hoboken, NJ, USA, 2021; Volume 48. [Google Scholar]
  24. Jarrett, D.; Stride, E.; Vallis, K.; Gooding, M.J. Applications and Limitations of Machine Learning in Radiation Oncology. Br. J. Radiol. 2019, 92, 20190001. [Google Scholar] [CrossRef]
  25. Hooshangnejad, H.; Chen, Q.; Feng, X.; Zhang, R.; Ding, K. DAART: A Deep Learning Platform for Deeply Accelerated Adaptive Radiation Therapy for Lung Cancer. Johns Hopkins Med. 2023; in press. [Google Scholar]
  26. Lustberg, T.; van Soest, J.; Gooding, M.; Peressutti, D.; Aljabar, P.; van der Stoep, J.; van Elmpt, W.; Dekker, A. Clinical Evaluation of Atlas and Deep Learning Based Automatic Contouring for Lung Cancer. Radiother. Oncol. 2018, 126, 312–317. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Oberije, C.; Nalbantov, G.; Dekker, A.; Boersma, L.; Borger, J.; Reymen, B.; van Baardwijk, A.; Wanders, R.; De Ruysscher, D.; Steyerberg, E. A Prospective Study Comparing the Predictions of Doctors versus Models for Treatment Outcome of Lung Cancer Patients: A Step toward Individualized Care and Shared Decision Making. Radiother. Oncol. 2014, 112, 37–43. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Field, M.; Hardcastle, N.; Jameson, M.; Aherne, N.; Holloway, L. Machine Learning Applications in Radiation Oncology. Phys. Imaging Radiat. Oncol. 2021, 19, 13–24. [Google Scholar] [CrossRef] [PubMed]
  29. Sahiner, B.; Pezeshk, A.; Hadjiiski, L.M.; Wang, X.; Drukker, K.; Cha, K.H.; Summers, R.M.; Giger, M.L. Deep Learning in Medical Imaging and Radiation Therapy. Med. Phys. 2019, 46, e1–e36. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Peng, Z.; Shan, H.; Liu, T.; Pei, X.; Wang, G.; Xu, X.G. MCDNet—A Denoising Convolutional Neural Network to Accelerate Monte Carlo Radiation Transport Simulations: A Proof of Principle with Patient Dose From X-Ray CT Imaging. IEEE Access 2019, 7, 76680–76689. [Google Scholar] [CrossRef]
  31. Balakrishnan, G.; Zhao, A.; Sabuncu, M.R.; Dalca, A.V.; Guttag, J. An Unsupervised Learning Model for Deformable Medical Image Registration. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 9252–9260. [Google Scholar]
  32. Markham, M.J.; Wachter, K.; Agarwal, N.; Bertagnolli, M.M.; Chang, S.M.; Dale, W.; Diefenbach, C.S.M.; Rodriguez-Galindo, C.; George, D.J.; Gilligan, T.D. Clinical Cancer Advances 2020: Annual Report on Progress against Cancer from the American Society of Clinical Oncology. J. Clin. Oncol. 2020, 38, 1081. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Yang, Y.; Ford, E.C.; Wu, B.; Pinkawa, M.; van Triest, B.; Campbell, P.; Song, D.Y.; McNutt, T.R. An Overlap-Volume-Histogram Based Method for Rectal Dose Prediction and Automated Treatment Planning in the External Beam Prostate Radiotherapy Following Hydrogel Injection. Med. Phys. 2013, 40, 11709. [Google Scholar] [CrossRef]
  34. Cilla, S.; Ianiro, A.; Romano, C.; Deodato, F.; Macchia, G.; Buwenge, M.; Dinapoli, N.; Boldrini, L.; Morganti, A.G.; Valentini, V. Template-Based Automation of Treatment Planning in Advanced Radiotherapy: A Comprehensive Dosimetric and Clinical Evaluation. Sci. Rep. 2020, 10, 423. [Google Scholar] [CrossRef] [Green Version]
  35. Greenspan, H.; van Ginneken, B.; Summers, R.M. Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique. IEEE Trans. Med. Imaging 2016, 35, 1153–1159. [Google Scholar] [CrossRef]
  36. Liu, Y.; Lei, Y.; Wang, T.; Fu, Y.; Tang, X.; Curran, W.J.; Liu, T.; Patel, P.; Yang, X. CBCT-Based Synthetic CT Generation Using Deep-Attention CycleGAN for Pancreatic Adaptive Radiotherapy. Med. Phys. 2020, 47, 2472–2483. [Google Scholar] [CrossRef] [PubMed]
  37. Chen, L.; Liang, X.; Shen, C.; Jiang, S.; Wang, J. Synthetic CT Generation from CBCT Images via Deep Learning. Med. Phys. 2020, 47, 1115–1125. [Google Scholar] [CrossRef] [PubMed]
  38. Liu, Y.; Chen, A.; Shi, H.; Huang, S.; Zheng, W.; Liu, Z.; Zhang, Q.; Yang, X. CT Synthesis from MRI Using Multi-Cycle GAN for Head-and-Neck Radiation Therapy. Comput. Med. Imaging Graph. 2021, 91, 101953. [Google Scholar] [CrossRef] [PubMed]
  39. Hanna, T.P.; King, W.D.; Thibodeau, S.; Jalink, M.; Paulin, G.A.; Harvey-Jones, E.; O’Sullivan, D.E.; Booth, C.M.; Sullivan, R.; Aggarwal, A. Mortality Due to Cancer Treatment Delay: Systematic Review and Meta-Analysis. BMJ 2020, 371, m4087. [Google Scholar] [CrossRef] [PubMed]
  40. Han, D.; Hooshangnejad, H.; Chen, C.-C.; Ding, K. A Beam-Specific Optimization Target Volume for Stereotactic Proton Pencil Beam Scanning Therapy for Locally Advanced Pancreatic Cancer. Adv. Radiat. Oncol. 2021, 6, 100757. [Google Scholar] [CrossRef] [PubMed]
  41. Hooshangnejad, H.; Han, D.; Feng, Z.; Dong, L.; Sun, E.; Du, K.; Ding, K. Systematic Study of the Iodinated Rectal Hydrogel Spacer Material Discrepancy on Accuracy of Proton Dosimetry. J. Appl. Clin. Med. Phys. 2022, 23, e13774. [Google Scholar] [CrossRef]
  42. Feng, Z.; Hooshangnejad, H.; Shin, E.J.; Narang, A.; Lediju Bell, M.A.; Ding, K. The Feasibility of Haar Feature-Based Endoscopic Ultrasound Probe Tracking for Implanting Hydrogel Spacer in Radiation Therapy for Pancreatic Cancer. Front. Oncol. 2021, 11, 759811. [Google Scholar] [CrossRef] [PubMed]
  43. Shamonin, D.P.; Bron, E.E.; Lelieveldt, B.P.F.; Smits, M.; Klein, S.; Staring, M.; Initiative, A.D.N. Fast Parallel Image Registration on CPU and GPU for Diagnostic Classification of Alzheimer’s Disease. Front. Neuroinform. 2014, 7, 50. [Google Scholar] [CrossRef] [Green Version]
  44. Klein, S.; Staring, M.; Murphy, K.; Viergever, M.A.; Pluim, J.P.W. Elastix: A Toolbox for Intensity-Based Medical Image Registration. IEEE Trans Med. Imaging 2009, 29, 196–205. [Google Scholar] [CrossRef]
  45. Reinhardt, J.M.; Christensen, G.E.; Hoffman, E.A.; Ding, K.; Cao, K. Registration-Derived Estimates of Local Lung Expansion as Surrogates for Regional Ventilation. Lect. Notes Comput. Sci. 2007, 4584, 763. [Google Scholar]
  46. Du, K.; Reinhardt, J.M.; Christensen, G.E.; Ding, K.; Bayouth, J.E. Respiratory Effort Correction Strategies to Improve the Reproducibility of Lung Expansion Measurements. Med. Phys. 2013, 40, 123504. [Google Scholar] [CrossRef] [Green Version]
  47. Ding, K.; Cao, K.; Christensen, G.E.; Hoffman, E.A.; Reinhardt, J.M. Registration-Based Regional Lung Mechanical Analysis: Retrospectively Reconstructed Dynamic Imaging versus Static Breath-Hold Image Acquisition. In Medical Imaging 2009: Biomedical Applications in Molecular, Structural, and Functional Imaging; SPIE: Bellingham, WA, USA, 2009; pp. 101–109. [Google Scholar]
  48. Yin, Y.; Hoffman, E.A.; Ding, K.; Reinhardt, J.M.; Lin, C.-L. A Cubic B-Spline-Based Hybrid Registration of Lung CT Images for a Dynamic Airway Geometric Model with Large Deformation. Phys. Med. Biol. 2010, 56, 203. [Google Scholar] [CrossRef] [PubMed]
  49. Ding, K.; Bayouth, J.E.; Buatti, J.M.; Christensen, G.E.; Reinhardt, J.M. 4DCT-based Measurement of Changes in Pulmonary Function Following a Course of Radiation Therapy. Med. Phys. 2010, 37, 1261–1272. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  50. Du, K.; Ding, K.; Cao, K.; Bayouth, J.E.; Christensen, G.E.; Reinhardt, J.M. Registration-Based Measurement of Regional Expiration Volume Ratio Using Dynamic 4DCT Imaging. In Proceedings of the 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Chicago, IL, USA, 30 March–2 April 2011; pp. 424–428. [Google Scholar]
  51. Du, K.; Bayouth, J.E.; Cao, K.; Christensen, G.E.; Ding, K.; Reinhardt, J.M. Reproducibility of Registration-Based Measures of Lung Tissue Expansion. Med. Phys. 2012, 39, 1595–1608. [Google Scholar] [CrossRef] [Green Version]
  52. Reinhardt, J.M.; Ding, K.; Cao, K.; Christensen, G.E.; Hoffman, E.A.; Bodas, S. V Registration-Based Estimates of Local Lung Tissue Expansion Compared to Xenon CT Measures of Specific Ventilation. Med. Image Anal. 2008, 12, 752–763. [Google Scholar] [CrossRef] [Green Version]
  53. Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
  54. Yan, Y.; Lee, H.; Somer, E.; Grau, V. Generation of Amyloid PET Images via Conditional Adversarial Training for Predicting Progression to Alzheimer’s Disease. In PRedictive Intelligence in MEdicine; Rekik, I., Unal, G., Adeli, E., Park, S.H., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 26–33. [Google Scholar]
  55. Shin, H.-C.; Ihsani, A.; Mandava, S.; Sreenivas, S.T.; Forster, C.; Cha, J.; Initiative, A.D.N. Ganbert: Generative Adversarial Networks with Bidirectional Encoder Representations from Transformers for Mri to Pet Synthesis. arXiv 2020, arXiv:2008.04393. [Google Scholar]
  56. Ranjan, A.; Lalwani, D.; Misra, R. GAN for Synthesizing CT from T2-Weighted MRI Data towards MR-Guided Radiation Treatment. Magn. Reson. Mater. Phys. Biol. Med. 2022, 35, 449–457. [Google Scholar] [CrossRef] [PubMed]
  57. Shitrit, O.; Riklin Raviv, T. Accelerated Magnetic Resonance Imaging by Adversarial Neural Network. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support; Springer: Cham, Switzerland, 2017; pp. 30–38. [Google Scholar]
  58. Seitzer, M.; Yang, G.; Schlemper, J.; Oktay, O.; Würfl, T.; Christlein, V.; Wong, T.; Mohiaddin, R.; Firmin, D.; Keegan, J.; et al. Adversarial and Perceptual Refinement for Compressed Sensing MRI Reconstruction. In Proceedings of the Medical Image Computing and Computer Assisted Intervention—MICCAI 2018, Granada, Spain, 16–20 September 2018; Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 232–240. [Google Scholar]
  59. Ran, M.; Hu, J.; Chen, Y.; Chen, H.; Sun, H.; Zhou, J.; Zhang, Y. Denoising of 3D Magnetic Resonance Images Using a Residual Encoder–Decoder Wasserstein Generative Adversarial Network. Med. Image Anal. 2019, 55, 165–180. [Google Scholar] [CrossRef] [Green Version]
  60. Kim, K.H.; Do, W.-J.; Park, S.-H. Improving Resolution of MR Images with an Adversarial Network Incorporating Images with Different Contrast. Med. Phys. 2018, 45, 3120–3131. [Google Scholar] [CrossRef]
  61. Wani, N.; Raza, K. Chapter 3—Multiple Kernel-Learning Approach for Medical Image Analysis. In Soft Computing Based Medical Image Analysis; Dey, N., Ashour, A.S., Shi, F., Balas, V.E., Eds.; Academic Press: Cambridge, MA, USA, 2018; pp. 31–47. ISBN 9780128130872. [Google Scholar]
  62. Yi, X.; Babyn, P. Sharpness-Aware Low-Dose CT Denoising Using Conditional Generative Adversarial Network. J. Digit. Imaging 2018, 31, 655–669. [Google Scholar] [CrossRef]
  63. Jaffray, D.A.; Siewerdsen, J.H.; Wong, J.W.; Martinez, A.A. Flat-Panel Cone-Beam Computed Tomography for Image-Guided Radiation Therapy. Int. J. Radiat. Oncol. Biol. Phys. 2002, 53, 1337–1349. [Google Scholar] [CrossRef] [PubMed]
  64. Wong, J.W. How Good Are You? In True Tales of Medical Physics; Springer: Cham, Switzerland, 2022; pp. 153–171. [Google Scholar]
  65. Netherton, T.J.; Cardenas, C.E.; Rhee, D.J.; Court, L.E.; Beadle, B.M. The Emergence of Artificial Intelligence within Radiation Oncology Treatment Planning. Oncology 2021, 99, 124–134. [Google Scholar] [CrossRef]
  66. Jia, X.; Ziegenhein, P.; Jiang, S.B. GPU-Based High-Performance Computing for Radiation Therapy. Phys. Med. Biol. 2014, 59, R151. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  67. Spalding, M.; Walsh, A.; Aland, T. Evaluation of a New GPU-Enabled VMAT Multi-Criteria Optimisation Plan Generation Algorithm. Med. Dosim. 2020, 45, 368–373. [Google Scholar] [CrossRef]
  68. Aland, T.; Walsh, A.; Jones, M.; Piccini, A.; Devlin, A. Accuracy and Efficiency of Graphics Processing Unit (GPU) Based Acuros XB Dose Calculation within the Varian Eclipse Treatment Planning System. Med. Dosim. 2019, 44, 219–225. [Google Scholar] [CrossRef] [PubMed]
  69. Byrne, M.; Archibald-Heeren, B.; Hu, Y.; Teh, A.; Beserminji, R.; Cai, E.; Liu, G.; Yates, A.; Rijken, J.; Collett, N.; et al. Varian Ethos Online Adaptive Radiotherapy for Prostate Cancer: Early Results of Contouring Accuracy, Treatment Plan Quality, and Treatment Time. J. Appl. Clin. Med. Phys. 2022, 23, e13479. [Google Scholar] [CrossRef] [PubMed]
  70. Sibolt, P.; Andersson, L.M.; Calmels, L.; Sjöström, D.; Bjelkengren, U.; Geertsen, P.; Behrens, C.F. Clinical Implementation of Artificial Intelligence-Driven Cone-Beam Computed Tomography-Guided Online Adaptive Radiotherapy in the Pelvic Region. Phys. Imaging Radiat. Oncol. 2021, 17, 4. [Google Scholar] [CrossRef]
  71. Yoon, S.W.; Lin, H.; Alonso-Basanta, M.; Anderson, N.; Apinorasethkul, O.; Cooper, K.; Dong, L.; Kempsey, B.; Marcel, J.; Metz, J. Initial Evaluation of a Novel Cone-Beam CT-Based Semi-Automated Online Adaptive Radiotherapy System for Head and Neck Cancer Treatment—A Timing and Automation Quality Study. Cureus 2020, 12, e9660. [Google Scholar] [CrossRef] [PubMed]
  72. Archambault, Y.; Boylan, C.; Bullock, D.; Morgas, T.; Peltola, J.; Ruokokoski, E.; Genghi, A.; Haas, B.; Suhonen, P.; Thompson, S. Making On-Line Adaptive Radiotherapy Possible Using Artificial Intelligence and Machine Learning for Efficient Daily Re-Planning. Med. Phys. Int. J. 2020, 8, 77–86. [Google Scholar]
  73. Güngör, G.; Serbez, İ.; Temur, B.; Gür, G.; Kayalılar, N.; Mustafayev, T.Z.; Korkmaz, L.; Aydın, G.; Yapıcı, B.; Atalar, B.; et al. Time Analysis of Online Adaptive Magnetic Resonance–Guided Radiation Therapy Workflow According to Anatomical Sites. Pract. Radiat. Oncol. 2021, 11, e11–e21. [Google Scholar] [CrossRef] [PubMed]
  74. Olson, S.H.; Xu, Y.; Herzog, K.; Saldia, A.; DeFilippis, E.M.; Li, P.; Allen, P.J.; O’Reilly, E.M.; Kurtz, R.C. Weight Loss, Diabetes, Fatigue, and Depression Preceding Pancreatic Cancer. Pancreas 2016, 45, 986–991. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  75. Hendifar, A.E.; Petzel, M.Q.B.; Zimmers, T.A.; Denlinger, C.S.; Matrisian, L.M.; Picozzi, V.J.; Rahib, L.; on behalf of the Precision Promise Consortium. Pancreas Cancer-Associated Weight Loss. Oncologist 2019, 24, 691–701. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Cancers 15 03061 g001 550
Figure 1. An illustration of the current clinical workflow that, due to the many steps before treatment, results in considerable treatment delay. On the contrary, using synthesized planning CT, we proposed a rapid workflow that significantly reduces treatment delay.
Figure 1. An illustration of the current clinical workflow that, due to the many steps before treatment, results in considerable treatment delay. On the contrary, using synthesized planning CT, we proposed a rapid workflow that significantly reduces treatment delay.
Cancers 15 03061 g001
Cancers 15 03061 g002 550
Figure 2. A typical dCT (A) and pCT (B) pair from a patient. (A) The difference in the shape of the couch results in a clear change in the patient’s back curvature. Additionally, the overall shape of the body is different due to the active breath-hold motion management procedure that is used for pCT acquisition and treatment delivery. (C,D) The differences in the patient’s body shape result in a difference in dose distribution (the couches were already removed for planning).
Figure 2. A typical dCT (A) and pCT (B) pair from a patient. (A) The difference in the shape of the couch results in a clear change in the patient’s back curvature. Additionally, the overall shape of the body is different due to the active breath-hold motion management procedure that is used for pCT acquisition and treatment delivery. (C,D) The differences in the patient’s body shape result in a difference in dose distribution (the couches were already removed for planning).
Cancers 15 03061 g002
Cancers 15 03061 g003 550
Figure 3. An overview of the 3D GAN model and losses used to train the model.
Figure 3. An overview of the 3D GAN model and losses used to train the model.
Cancers 15 03061 g003
Cancers 15 03061 g004 550
Figure 4. Illustration of the RT planning scenarios.
Figure 4. Illustration of the RT planning scenarios.
Cancers 15 03061 g004
Cancers 15 03061 g005 550
Figure 5. An illustration of planning CT synthesis result: (A) the dCT, the input to the model; (B) the pCT, the ground truth; and (C–F) the sCT generated by Pix2Pix 3D with large patches, U-Net, Pix2Pix 2.5., and Pix2Pix with small patches, respectively.
Figure 5. An illustration of planning CT synthesis result: (A) the dCT, the input to the model; (B) the pCT, the ground truth; and (C–F) the sCT generated by Pix2Pix 3D with large patches, U-Net, Pix2Pix 2.5., and Pix2Pix with small patches, respectively.
Cancers 15 03061 g005
Cancers 15 03061 g006 550
Figure 6. First row: The diagnostic CT (A), planning CT (B), and synthesized CT (C) at abdominal level/in abdominal window (40/400). Second row: (D) the HU intensity difference map between planning CT and diagnostic CT (pCT–dCT) and (E) between planning CT and synthesized CT (pCT–sCT). deepPERFECT’s predicted synthesized CT from diagnostic CT shows a high similarity to planning CT.
Figure 6. First row: The diagnostic CT (A), planning CT (B), and synthesized CT (C) at abdominal level/in abdominal window (40/400). Second row: (D) the HU intensity difference map between planning CT and diagnostic CT (pCT–dCT) and (E) between planning CT and synthesized CT (pCT–sCT). deepPERFECT’s predicted synthesized CT from diagnostic CT shows a high similarity to planning CT.
Cancers 15 03061 g006
Cancers 15 03061 g007 550
Figure 7. Comparison of dosimetric indices for SBRT plans: the comparison was performed for V33 Gy for the target volumes (GTV (A) and PTV (B)), and for V20 Gy and V33 Gy for the proximal OARs (duodenum (C,D), stomach (E,F), and bowel (G,H)). Each figure consists of four box plots for RT plans designed using pCT (ground truth), sCT, plan recalculation (dose recalculation by shifting the isocenter of beams to pCT isocenter), and synthesized ROIs on planning CT (dose recalculation based on pCT using sCT ROIs).
Figure 7. Comparison of dosimetric indices for SBRT plans: the comparison was performed for V33 Gy for the target volumes (GTV (A) and PTV (B)), and for V20 Gy and V33 Gy for the proximal OARs (duodenum (C,D), stomach (E,F), and bowel (G,H)). Each figure consists of four box plots for RT plans designed using pCT (ground truth), sCT, plan recalculation (dose recalculation by shifting the isocenter of beams to pCT isocenter), and synthesized ROIs on planning CT (dose recalculation based on pCT using sCT ROIs).
Cancers 15 03061 g007
Table
Table 1. The results of the image quality evaluation. Bold values show the best performance among all configurations.
Table 1. The results of the image quality evaluation. Bold values show the best performance among all configurations.
Model ArchitecturePix2Pix 3D Large PatchesPix2Pix 3D Small PatchesPix2Pix 2.5DU-Net
MetricAverage ± STD
RASSD (HU)334 ± 65541 ± 83874 ± 1561242 ± 132
DSC body contour0.93 ± 0.040.82 ± 0.080.60 ± 0.090.56 ± 0.03
HD body contour (mm)4.6 ± 2.114.6 ± 6.129.8 ± 5.737.2 ± 8.1
DSC GTV0.82 ± 0.120.71 ± 0.160.61 ± 0.190.59 ± 0.09
HD GTV (mm)7.12 ± 3.113.1 ± 8.420.4 ± 7.828.4 ± 6.1
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Hooshangnejad, H.; Chen, Q.; Feng, X.; Zhang, R.; Ding, K. deepPERFECT: Novel Deep Learning CT Synthesis Method for Expeditious Pancreatic Cancer Radiotherapy. Cancers 2023, 15, 3061. https://doi.org/10.3390/cancers15113061

AMA Style

Hooshangnejad H, Chen Q, Feng X, Zhang R, Ding K. deepPERFECT: Novel Deep Learning CT Synthesis Method for Expeditious Pancreatic Cancer Radiotherapy. Cancers. 2023; 15(11):3061. https://doi.org/10.3390/cancers15113061

Chicago/Turabian Style

Hooshangnejad, Hamed, Quan Chen, Xue Feng, Rui Zhang, and Kai Ding. 2023. "deepPERFECT: Novel Deep Learning CT Synthesis Method for Expeditious Pancreatic Cancer Radiotherapy" Cancers 15, no. 11: 3061. https://doi.org/10.3390/cancers15113061

APA Style

Hooshangnejad, H., Chen, Q., Feng, X., Zhang, R., & Ding, K. (2023). deepPERFECT: Novel Deep Learning CT Synthesis Method for Expeditious Pancreatic Cancer Radiotherapy. Cancers, 15(11), 3061. https://doi.org/10.3390/cancers15113061

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop