Usefulness of a Metal Artifact Reduction Algorithm in Digital Tomosynthesis Using a Combination of Hybrid Generative Adversarial Networks

In this study, a novel combination of hybrid generative adversarial networks (GANs) comprising cycle-consistent GAN, pix2pix, and (mask pyramid network) MPN (CGpM-metal artifact reduction [MAR]), was developed using projection data to reduce metal artifacts and the radiation dose during digital tomosynthesis. The CGpM-MAR algorithm was compared with the conventional filtered back projection (FBP) without MAR, FBP with MAR, and convolutional neural network MAR. The MAR rates were compared using the artifact index (AI) and Gumbel distribution of the largest variation analysis using a prosthesis phantom at various radiation doses. The novel CGpM-MAR yielded an adequately effective overall performance in terms of AI. The resulting images yielded good results independently of the type of metal used in the prosthesis phantom (p < 0.05) and good artifact removal at 55% radiation-dose reduction. Furthermore, the CGpM-MAR represented the minimum in the model with the largest variation at 55% radiation-dose reduction. Regarding the AI and Gumbel distribution analysis, the novel CGpM-MAR yielded superior MAR when compared with the conventional reconstruction algorithms with and without MAR at 55% radiation-dose reduction and presented features most similar to the reference FBP. CGpM-MAR presents a promising method for metal artifact and radiation-dose reduction in clinical practice.


Introduction
The causes of metal artifacts are quite complicated. Depending on the shape and density of the metal objects, the appearance of metal artifacts can vary significantly. In medical applications, metal objects can include metallic orthopedic hardware (e.g., surgical pins and clips) or equipment attached to the patient's body (e.g., biopsy needles) [1]. A metal object can produce beam hardening, partial volume, aliasing, small-angle scatter, under-range data acquisition electrons, or overflow of the dynamic range in the reconstruction process. A previous study on digital tomosynthesis (DT) proposed several artifact compensation approaches to minimizing metal artifacts [2][3][4].
Dual-energy DT (DE-DT) has recently become available. One of the inherent capabilities of DE-DT is the generation of synthesized monochromatic images obtained at different energy levels (keV) from a single data acquisition [3,4]. By generating monochromatic images at higher energy levels (e.g., 140 keV), beam-hardening artifacts can be suppressed. However, DT devices that enable dual-energy acquisition in clinical practice are rare; generally, DT acquisition is performed through single-energy acquisition. Therefore, from the viewpoint of versatility, an improved method must be developed to achieve metal artifact reduction (MAR) in polychromatic radiography.
Based on highly attenuating metal objects, nearly all X-ray photons are attenuated, and few reach the detector, which results in under-range data acquisition electrons. Combined learning by standardizing the low-dose projection of the prior image. Moreover, we designed a novel mask pyramid projection learning strategy that would fully utilize the prior projection guidance to improve the continuity of projection completion, thereby alleviating the new artifacts in the reconstructed DT images. The final DT image was then reconstructed from the completed projection using the conventional filtered back projection (FBP) algorithm [32]. Compared with the previous MAR approaches for DT [2][3][4], the whole framework was trained efficiently for the complementary learning of prior processed images according to the method of each network so that the prior image generation and deep projection completion procedures can be learned in a collaborative manner and benefit from each other. Our recommended MAR algorithm (combination of hybrid GAN: cycle-GAN_pix2pix_MPN [CGpM-MAR]) is described in the Methods section.
In addition, we investigated the causal relationship between dose reduction and quality of MAR. Our findings suggest the possibility of reducing exposure dose and improving image quality using the CGpM-MAR algorithm. The developmental process and basic evaluation of the method are presented in this study.

Phantom Specifications
To evaluate image quality (with the implant and artificial bone introducing the artifacts and contrast, respectively), we immersed a prosthesis phantom containing an implant in the center of a polymethyl methacrylate case (custom-made product, Kyoto Kagaku Co., Tokyo, Japan) filled with water (case dimensions: ϕ 200 × 300 mm). Given that water is often used as a substitute for soft tissue in phantom experiments, the area of the phantom filled with water thus simulated soft tissue. The phantom was an artificial bone (orthopedic humeral model: normal anatomy; foam cortical shell; canal diameter: 9 mm; overall length: 300 mm, Pacific Research Laboratories Inc., Vashon, WA, USA). The TRIGEN Humeral Nails Proximal Straight System (Model: 38153000; titanium alloy; diameter: 8 mm; overall length: 160 mm, Smith & Nephew Orthopaedics KK Inc., Tokyo, Japan) was used. In the prosthesis phantom, we used an internal fracture fixation (intramedullary fracture fixation) to simulate a humeral proximal fracture.

Overview of CGpM-MAR
The novel CGpM-MAR algorithm was implemented in the case of projection-space data to reduce metal artifacts while reducing the radiation dose during DT. This method is based on a combination of multi-training networks (cycle-GAN and pix2pix) with MPN and a mask pyramid learning strategy to fully utilize prior projection guidance completion, and thus, alleviate new artifacts in a projection space involving hybrid and subjectively reconstructed MAR images.
Cycle-GAN allows translations between domains (reference and low doses) that are not fully associated and can solve problems of image quality deterioration due to low-dose acquisition. By applying another style to the image during the pix2pix translation process to get prior projection images, final MPN learning could be introduced to recover more structural and anatomically plausible information from the metallic component.
The overview of the recommended CGpM-MAR) algorithm is as follows: Step 1 [Cycle-GAN]: Translates low-dose images to reference-dose images.
Step 2 [Prior images for linear interpolation (LI) [8] processing and pix2pix GAN]: Generates a metal-free image by extracting the metal region and by LI interpolation. A prior image with a deep neural network (pix2pix) is generated to facilitate the projection completion.

CGpM-MAR
The variable definitions required for the outline of the algorithm are shown below:

CGpM-MAR
The variable definitions required for the outline of the algorithm are shown below: The purpose of cycle-GAN is to translate low-dose projections into reference-dose projections. In the learning process, a low-dose projection can be effectively translated into a reference-dose projection using two discriminators and a generator. Cycle-GAN is used to learn mapping functions between two domains A and B that were given training samples. The network objective contains three types of terms: Adversarial loss, cyclic consistency loss, and identity loss. The training dataset included 148 projection images and each corresponding image related to the input image pair (A (74), B (74)) were randomly selected from the generated acquisition data as the training set.
The adversarial loss is given as The cyclic consistency loss is given as The identity loss is given as The total loss is given as where λ controls the relative importance of the two objectives. In this study, λ was set to 10 at an initial learning rate of 0.0002 [28]. cycle-GAN was developed to solve the following problem: Diagnostics 2021, 11, 1629 6 of 20 In cycle-GAN, the Adam optimization algorithm [34] was used with a batch size of 1. The architecture of the building components is shown in Appendix A (Table A1).

Linear Interpolation
We generated a prior image with a deep neural network to facilitate the projection completion procedure [23], as the metal-free prior image would provide a good estimation for the original projection. We first applied the LI [8] to produce an initial estimation of the metal trace region and acquire the LI-corrected projection P temp_LI for the following procedures: Step_1: extract metal from AB † ∈ {0, 1} i×j (i: high, j: width) → M sk Step_2: AB † − M sk → P temp Step_3: LI processing for P temp → P temp_LI

Pix2pix
In this procedure, the original projection with metal artifacts was used as the input and to train a neural network to generate the prior image with fewer metal artifacts. In particular, with relatively large metal objects, the metal artifacts in the original projection would be strong, and the neural network would have difficulty reducing the metal artifacts. Therefore, besides the original projection, we also used the LI-corrected image into the prior image generation procedure [23] and used the pix2pix neural network, to refine the LI-corrected image by backpropagation learning.
G and D trained adversarially with GAN are expressed as follows: The training dataset included 148 projection images, and each corresponding images related to the input image pair (AB † (74), P temp_LI (74)) were randomly selected as the training set from the cycle-GAN processed data: where z is the random noise vector (Gaussian noise). The pix2pix was developed to solve the following problem: where γ controls the relative importance of the two objectives L GAN_c and L GAN_L1 . In this study, γ was set to 100, the initial learning rate was set to 0.0002, and the momentum parameters were set to β 1 = 0.5, β 2 = 0.999 [25]. In pix2pix, we used the Adam optimization algorithm [34] with a batch size of 1. The architecture of the building components is shown in Appendix A (Table A2).

MPN
We used a novel mask pyramid projection learning strategy to fully utilize the prior projection guidance to improve the continuity of projection completion and thus alleviate the new artifacts in the reconstructed DT images. We used the MPN to retain the metal trace information in each layer explicitly so that the network can extract more discriminative features to restore missing information in the metal trace region. The training dataset included 222 projection images and each corresponding related to the input image (P temp (74), AB † † (74), M sk (74)) were randomly selected as the training set from the data processed in metal extract, LI, and pix2pix.
In computing the loss function, the network only considered the content of the metal mask. The content loss is given as follows: The output score matrix from the discriminator was modulated by the metal mask M sk so that the discriminator can selectively ignore the unmasked regions. The adversarial part of the mask fusion loss is given as: where S takes metal mask M sk as the input, and each block of S is coupled with an encoding block in D.
The total mask fusion loss is given as where η is the balance the between L GAN_M and η. In this study, η was set to 100, the initial learning rate was set to 0.0002, and the momentum parameters were set to β 1 = 0.5, β 2 = 0.999 [30].
In MPN, we used the Adam optimization algorithm [34] with a batch size of 1. The architecture of the building components are shown in Appendix A (Table A3).

Optimization Parameters of the Epochs
The optimizations epochs in each network (cycle-GAN, pix2pix, and MPN) were evaluated based on the mean square error (MSE) [35] and structural similarity (SSIM) [36] for the projection image (straightforward on the detector). The MSE of the identified projection image can be obtained as follows: where re f (i, j) is the (i, j)th entry of the reference-dose projection image, and low (i, j) is the (i, j)th entry of the low-dose projection image in each epoch. The SSIM index between pixel values i and j was calculated as follows: where l is the luminance, c is the contrast, and s is the structure (ω = ξ = ψ = 1.0). The mean SSIM (MSSIM) was then used to evaluate the overall image quality as follows: where i q and j q are the image contents at the qth pixel, and K is the number of pixels in the image. Optimization was evaluated based on the MSE and MSSIM. The lowest MSE, highest MSSIM, and epochs were selected as the optimum parameters.

Evaluation of Image Quality
The DT system-derived real projection data were used for image reconstruction. MATLAB (version 9.7.0.1216025, MathWorks, Natick, MA, USA) was used to reconstruct and process images. The artifact index (AI) values [37] for metal artifacts containing low-frequency components in the CGpM-MAR and the conventional algorithms (FBP (reference-dose FBP image; reference FBP, low-dose FBP image; original FBP), DT-MAR [2], and CNNMAR [15] with reconstruction from the original projections) were compared to assess the decrease in metal artifacts on each in-focus plane image. A weighting factor of 0.6 was used in DT-MAR [(FBP * 0.4) + (BP * 0.6)] (FBP; FBP reconstructed images and BP; back projection imgaes) [2]. We further ascertained the metal artifacts containing highfrequency components. Gumbel distributions [3,38] are statistical models for determining the influence of high-frequency artifacts. The characteristics of the CGpM-MAR and conventional algorithms were evaluated based on the MAR. CGpM-MAR was evaluated using the optimized parameters generated based on the application image.

AI
The AI of the identified metal artifacts was calculated as follows: where n = 1, 2, . . . , 6, or 8 defines the formula for ROI arti f act_n (regions of interest (ROI)_location_1; ROI arti f act_1 , ROI arti f act_2 , . . . , ROI arti f act_6 , ROI_location_2; ROI arti f act_1 , ROI arti f act_2 , . . . , ROI arti f act_8 ; Figure 2a) that represents the corresponding ROI for the relative standard deviations (SDs) of real features (metal artifacts) in the in-focus plane. ROI BG is the relative SD of the background in the in-focus plane. To evaluate each feature (metal artifacts) and background, the ROI was set at 3 × 9 pixels (ROI_location_1; streak artifact area) and 3 × 7 pixels (ROI_location_2; dark artifact area).
In this study, we compared the AI values of the reconstruction algorithms at different radiation doses between two groups (ROI_location_1 and ROI_location_2). The numbers of samples in the groups were 18 (ROI_location_1) and 24 (ROI_location_2). Statistical analyses were performed using IBM SPSS Statistics for Windows (version 24.0, SPSS Inc., Chicago, IL, USA). Probability (p) values < 0.05 were considered statistically significant.

Statistical Model with Gumbel Distributions
The analysis method is outlined as follows: First, a rectangular window with a width of 8 pixels and a length (X-ray sweep direction) of 24 pixels was placed on each in-focus plane image to reduce metal artifacts, as shown in Figure 2b. Second, the parallel-line profiles of the pixel values at 1-pixel intervals resulted in 23 parallel-line pixel-value profiles (each sampling size: 23). Third, the pixel-value profiles were graphed, and the maximal variations between the adjacent pixel values were determined and analyzed based on the Gumbel distribution. Finally, the cumulative probability function was measured using the symmetry rank method with order statistics: where q is the sampling size with order statistics.
To evaluate linearity, the Pearson correlation coefficient was determined and analyzed (p values < 0.01) using IBM SPSS Statistics for Windows (version 24.0, SPSS Inc., Chicago, IL, USA). In this study, we compared the AI values of the reconstruction algorithms at different radiation doses between two groups (ROI_location_1 and ROI_location_2). The numbers of samples in the groups were 18 (ROI_location_1) and 24 (ROI_location_2). Statistical analyses were performed using IBM SPSS Statistics for Windows (version 24.0, SPSS Inc., Chicago, IL, USA). Probability (p) values < 0.05 were considered statistically significant.

Statistical Model with Gumbel Distributions
The analysis method is outlined as follows: First, a rectangular window with a width of 8 pixels and a length (X-ray sweep direction) of 24 pixels was placed on each in-focus plane image to reduce metal artifacts, as shown in Figure 2b. Second, the parallel-line profiles of the pixel values at 1-pixel intervals resulted in 23 parallel-line pixel-value profiles (each sampling size: 23). Third, the pixel-value profiles were graphed, and the maximal variations between the adjacent pixel values were determined and analyzed based on the Gumbel distribution. Finally, the cumulative probability function was measured using the symmetry rank method with order statistics: where q is the sampling size with order statistics.
To evaluate linearity, the Pearson correlation coefficient was determined and analyzed (p values < 0.01) using IBM SPSS Statistics for Windows (version 24.0, SPSS Inc., Chicago, IL, USA).  Figure 4 shows the reconstructed images of the prosthesis phantom acquired with the CGpM-MAR algorithm and each of the established algorithms for reconstruction with and without MAR processing at 55% reduction of the radiation dose of up to 0.31 mSv. Remarkably, the DT images produced using the CGpM-MAR algorithm showed decreased metal artifacts in the radiographic sweep direction (i.e., vertical direction), specifically in the peripheral regions of the prosthesis phantom. On the other hand, images produced with the help of DT-MAR demonstrated noise and metal artifacts. Comparison of the difference between CGpM-MAR and the conventional algorithm resulted in the smallest DT-MAR. CNNMAR showed a certain reduction in artifacts, but the streak artifacts generated from around the metal were remarkable. pix2pix, 250 epochs; and MPN, 400 epochs). Using the optimization verification results, each training network image was generated by setting epochs for CGpM-MAR and then evaluated and compared with those of the images obtained using conventional algorithms (Figure 3). The training was performed on a GPU (Geforce RTX 2080 Ti; 11 GB of memory, NVIDIA Co., Santa Clara, CA, USA). The total calculation time required to process the   Figure 4 shows the reconstructed images of the prosthesis phantom acquired with the CGpM-MAR algorithm and each of the established algorithms for reconstruction with and without MAR processing at 55% reduction of the radiation dose of up to 0.31 mSv. Remarkably, the DT images produced using the CGpM-MAR algorithm showed decreased metal artifacts in the radiographic sweep direction (i.e., vertical direction), specifically in the peripheral regions of the prosthesis phantom. On the other hand, images produced with the help of DT-MAR demonstrated noise and metal artifacts. Comparison of the difference between CGpM-MAR and the conventional algorithm resulted in the smallest DT-MAR. CNNMAR showed a certain reduction in artifacts, but the streak artifacts generated from around the metal were remarkable.     Figure 5 presents the placement of the ROI in the prosthesis phantom and a plot of the AI results. CGpM-MAR yielded the lowest metal artifact characteristic values and features most similar to the reference FB,P regardless of the MAR processing status at 55% radiation-dose reduction, which is up to 0.31 mSv (ROI_location_1; 0.43 mSv: average AI ± standard error: 0.1020 ± 0.0179, 0.  In ROI_location_1, the differences in the metal artifacts, except for CGpM-MAR compared with the conventional low-dose FBP without MAR processing, were statistically significant (p < 0.05; Table 1). In ROI_location_2, the differences in metal artifacts between the CGpM-MAR and low-dose with and without MAR processing reconstruction algorithms were statistically significant (p < 0.05; Table 2). These AI results showed that the CGpM-MAR algorithm most effectively reduced metal artifacts in areas with a large number of artifacts (ROI_locations_2). In addition, CGpM-MAR had the closest AI value to the reference FBP.  In ROI_location_1, the differences in the metal artifacts, except for CGpM-MAR compared with the conventional low-dose FBP without MAR processing, were statistically significant (p < 0.05; Table 1). In ROI_location_2, the differences in metal artifacts between the CGpM-MAR and low-dose with and without MAR processing reconstruction algorithms were statistically significant (p < 0.05; Table 2). These AI results showed that the CGpM-MAR algorithm most effectively reduced metal artifacts in areas with a large number of artifacts (ROI_locations_2). In addition, CGpM-MAR had the closest AI value to the reference FBP.

Discussion
This study revealed that the CGpM-MAR algorithm yielded an adequate overall performance, reducing the radiation dose by 55%. The combination of multi-training network images produced using this algorithm yielded good results independently of the type of metal present in the prosthesis phantom. In addition, this algorithm successfully removed low-and high-frequency artifacts from the images. CGpM-MAR was particularly useful in reducing a large number of artifacts. Therefore, this algorithm is a promising new option for prosthetic imaging, as it generated artifact-reduced images and reduced radiation doses that were far superior to those obtained from images processed using conventional algorithms. The flexibility of CGpM-MAR in the choice of imaging parameters, which is based on the desired final images and prosthetic imaging conditions, promises increased usability.

Discussion
This study revealed that the CGpM-MAR algorithm yielded an adequate overall performance, reducing the radiation dose by 55%. The combination of multi-training network images produced using this algorithm yielded good results independently of the type of metal present in the prosthesis phantom. In addition, this algorithm successfully removed low-and high-frequency artifacts from the images. CGpM-MAR was particularly useful in reducing a large number of artifacts. Therefore, this algorithm is a promising new option for prosthetic imaging, as it generated artifact-reduced images and reduced radiation doses that were far superior to those obtained from images processed using conventional algorithms. The flexibility of CGpM-MAR in the choice of imaging parameters, which is based on the desired final images and prosthetic imaging conditions, promises increased usability.
The projection-space combination of multi-training approaches described here can be used to generate images to formulate the MAR as a deep learning algorithm for projection completion problems to improve the generalization and robustness of the framework. Since directly regressing accurate missing projection data is difficult to undertake [23], we propose to incorporate the prior projection image generation procedure and adopt a combination of multiple networks and a projection completion strategy. This method can improve the continuity of the projection values at the boundary of metal traces and alleviate the new artifacts, which are common drawbacks of projection completion-based MAR methods. Therefore, we believe that our novel CGpM-MAR could effectively reduce metal artifacts in actual practice.
The ability of CGpM-MAR to obtain MAR images and reduce the radiation dose by approximately 55% (Figures 5 and 6) may be due to the benefits of the first process, cycle-GAN. Training an image-to-image translation framework requires fully associated images, which is often difficult to learn. Cycle-GAN allows the translation between domains that are not fully associated and can therefore solve this problem. Cycle-GAN has three types of losses: First, the cycle consistency loss calculates the difference between the original image and the original domain after translation into another domain and the original domain. Second, adversarial loss guarantees that the image is real. Third, identity loss preserves the quantization of the pixel space of the image. The two generators use a U-Net [14] structure, and the two discriminators have a Patch-GAN-based structure [39] for learning. By applying another style to the image during the translation process, the low-dose projection image can then be applied to the reference-dose projection image.
The MPN used as the final learning process in CGpM-MAR contributed to the MAR. The reason was that adversarial learning could be introduced into the projection to recover more structural and anatomically plausible information from the metallic domain. In addition, a new MPN has been developed, which extracts geometric information of different scales and mask fusion loss that penalizes premature saturation, making learning more robust for different shapes of metal implants, were introduced.
In DT-MAR processing without deep learning methods [2], voxels containing artifacts tended to have higher values than their neighboring artifact-free voxels, which affected the prosthetic appearance, wherein former voxels stood out against the background of the latter. Accordingly, these residual artifacts are conspicuous when images subjected to the FBP method are compared with non-artifact-reduced images. Therefore, DT-MAR processing based on the polychromatic radiographic imaging method has a limited ability to reduce metal artifacts.
The usefulness of image quality improvement for reducing noise and metal artifacts in DT using deep learning has recently been reported [4,31]. Although noise and radiationdose reductions using deep learning in the DT of the breast are possible, no studies have reported on the reduction of radiation dose related to MAR. The radiation-dose reduction is approximately 20% in MAR without deep learning [6], thus, applying deep learning can further improve MAR and radiation-dose reduction.
Our CGpM-MAR algorithm has some limitations. First, this study was to reduce DT metal artifacts by single-energy (polychromatic) acquisition depending on the versatility of the processing. When DE-DT acquisition becomes widespread in the future, we would like to apply CGpM-MAR in monochromatic radiographic imaging. Second, because CGpM-MAR combines multiple deep learning processes, generating the final image takes time. Therefore, improvements in hardware processing are desired to speed up the process. Third, the model used for learning used a phantom. By acquiring data, such as prosthesis size and the positional relationship between prosthesis and normal structure, under various conditions according to clinical use, image quality can possibly be improved further. Moreover, further knowledge of various other structural patterns is needed in processing complicated structures.
Although the results of this study is limited to prosthesis phantom, the evaluations were performed in a state closest to actual biological composition in vivo. Furthermore, this approach will accelerate clinical application in terms of radiation-dose reduction and image quality improvement, which are issues in X-ray imaging. We believe that the CGpM-MAR algorithm will optimize the acquisition protocol in future X-ray imaging and radiation-dose reduction technology and improve the accuracy of medical images.
Recently, some works studied GAN applications with and without MAR [40,41]. These studies applied GAN technology to the reconstructed image to improve the accuracy of the tomographic image in the in-plane and longitudinal directions. Although the images processed were projection images, we predict that further improvements to the image quality (MAR) of the reconstructed image can be made by theoretically using GAN in the three-dimensional direction or in multiple slice generation from the input slice images. In addition, investigating the simultaneous reduction of GAN and metal artifacts in threedimensional data would be an interesting future research direction.

Conclusions
This prosthesis phantom study revealed that a 55% reduction in radiation dose is feasible with our novel CGpM-MAR algorithm. Our algorithm was particularly useful in reducing dark artifacts and yielded relatively better statistical results (P < 0.05) in terms of metal artifact reduction than conventional reconstruction algorithms. Therefore, the CGpM-MAR algorithm can be integrated into the clinical application workflow to accelerate image acquisition and reconstruction, and reduce metal artifacts while maintaining excellent image quality in radiologic imaging with prostheses.