Fine-Tuning a Denoiser for Low-Dose CT Image Reconstruction ^†

Abstract

A commonly employed imaging modality that relies on ionizing radiation is Computed Tomography (CT). While lowering the radiation dose is beneficial for patient health, it can result in reduced image quality. Therefore, improving low-dose CT (LDCT) reconstruction is a significant area of research. The LoDoPaB-CT benchmark evaluates LDCT reconstruction methods, where many top methods use UNet-type architectures. We explore a two-stage approach for LDCT reconstruction: the first stage employs traditional filtered backprojection (FBP), while the second stage performs CT image enhancement. Our training strategy involves pretraining a neural network to denoise natural grayscale images, which are corrupted by Gaussian noise, followed by fine-tuning the network for CT image enhancement using LDCT and normal-dose CT (NDCT) pairs. Experiments on various small subsets of the LoDoPaB-CT dataset demonstrate the effectiveness of our method, showing that less task-specific data are required for training.

1. Introduction

To provide non-invasive visualization of internal structures, Computed Tomography (CT) is essential in modern medical imaging. Its widespread use is partly due to its versatility, serving various diagnostic purposes [1,2]. However, minimizing ionizing radiation while maintaining image quality remains a critical challenge, particularly for low-dose CT (LDCT) reconstruction [3]. Recent research focuses on LDCT reconstruction, leveraging advanced algorithms and deep learning techniques [4,5,6,7,8,9,10].

Historically, iterative variational regularization methods, like total variation [11], total generalized variation [12], and Hessian Schatten norms [13], dominated CT image reconstruction. Recently, deep learning, especially convolutional neural networks (CNNs), has revolutionized medical image reconstruction, effectively reducing artifacts and noise [14]. Approaches range from fully learned methods, which use sinograms or raw projection data, to iterative and multi-stage methods integrating traditional techniques with neural networks. For instance, ISTA UNet [15], the Learned Primal-Dual algorithm [16], and ItNet [17] are notable for their success in LDCT reconstruction, particularly in the LoDoPaB-CT challenge [18].

Building on recent trends of pretraining models on large, diverse datasets from various domains, such as those used in Natural Language Processing (e.g., [19]) and Computer Vision (e.g., [20]), we introduce a two-stage approach for LDCT reconstruction. The first stage involves traditional filtered backprojection (FBP), while the second stage focuses on CT image enhancement. Two-stage methods using UNets or other neural networks are prevalent in CT image processing [9,10,15,21,22,23]. In the present work, we use the Gaussian denoiser DRUNet [24], which has been pretrained on several natural image datasets, including the Berkeley segmentation, DIV2K, Flick2K, and the Waterloo Exploration dataset. For the subsequent task of LDCT image enhancement, fine-tuning is performed using CT-specific images. We emphasize that our distinctive training strategy involves pretraining on non-CT images with different noise characteristics than LDCT images.

Furthermore, our method demonstrates robustness through experiments on various small subsets of the LoDoPaB-CT dataset, highlighting the advantage of our pretraining strategy over training the network from scratch. These experiments also show that our approach requires less task-specific data to achieve similar results.

2. Problem Formulation

CT images are reconstructed from X-Ray projection data measured at various angles. The forward model defines how the desired image relates to the collected projection data. Let y be the projection data, $x\in {\mathbb{R}}^{n\times n}$ the true CT image, and R the Radon transform. With these definitions, the forward model can be simplified as follows:

$$\begin{array}{cc}\hfill y& =R\left(x\right)+ϵ,\hfill \end{array}$$

where $ϵ$ represents measurement errors, such as quantum noise, electronic noise, scatter, and artifacts from patient motion. For a detailed model description, we refer to [21]. CT image reconstruction aims to estimate x from the projection data y, utilizing the system matrix R.

This study addresses the issue of reconstructing LDCT images with a reduced radiation dose. The primary challenge is the increased noise level in the projection data resulting from the lower radiation dose. Additionally, the inversion of R is ill-posed, which further amplifies the noise.

2.1. The Two-Stage Approach

The methodology employed for the reconstruction of CT images comprises two stages. In the initial stage, reconstruction is performed utilizing the FBP, which is a conventional analytical method, but the noise is amplified in the measurements. In the subsequent stage, a neural network, designated as $G_{\theta }$, is employed to enhance the LDCT image. Here, the low-dose FBP-reconstructed image is mapped to an enhanced version. Mathematically, the two-stage method can be expressed in the following way:

$$\begin{array}{cc}\hfill \widehat{x}& =G_{\theta }\circ \mathrm{FBP}\left(y\right),\hfill \end{array}$$

(1)

where $G_{\theta }$ denotes a neural network with learnable parameters $\theta$, $\mathrm{FBP}$ denotes the filtered backprojection, and $\widehat{x}$ is the final reconstruction result.

To enhance the performance of $G_{\theta }$ for CT image enhancement, we adopt a fine-tuning strategy. Initially, $G_{\theta }$ is pretrained on a large dataset to denoise natural grayscale images corrupted by Gaussian noise. Subsequently, we fine-tune $G_{\theta }$ for the specific task of LDCT image enhancement using paired data consisting of LDCT and normal-dose CT (NDCT) images. Fine-tuning adapts $G_{\theta }$ to the characteristics of LDCT image enhancement, aiming to improve the quality and fidelity of the FBP-reconstructed images.

2.2. DRUNet: Pretraining for Robust Image Restoration

The DRUNet [24] model is trained for various image restoration tasks, particularly for denoising natural grayscale images corrupted by Gaussian noise. It combines the effectiveness of UNets in image translation tasks, combined with the enhanced modeling capacity of ResNets through stacked residual blocks. The DRUNet model was trained on a dataset comprising 8794 natural images from the Berkeley Segmentation, DIV2K, Flick2K, and Waterloo Exploration datasets. The optimization process utilizes the ADAM algorithm with the objective of minimizing the mean absolute error (MAE) loss between the processed image and the ground truth. This pretrained network is then fine-tuned for the specific purpose of improving CT images using paired LDCT and NDCT images.

3. Implementation Details

Following the methodologies outlined in [10], we adopted a composite loss function to train our neural network architecture. Initially, we employed the MAE loss function, denoted as $L_{\mathrm{MAE}}$, which focuses on ensuring the accuracy of pixel intensity and is known for its robustness in handling outliers [25]. The MAE loss for a single sample is defined as

$$L_{\mathrm{MAE}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}|{\widehat{x}}_i-y_i|,$$

where N denotes the amount of pixels in the image, ${\widehat{x}}_i$ refers to the i-th pixel of the neural network output, and $y_i$ represents the i-th pixel from the ground truth (NDCT image).

To further enhance reconstruction quality, we integrated Structural Similarity (SSIM) [26] in our loss function. The SSIM metric evaluates the structural similarity of images by analyzing the statistical attributes of local regions, rather than focusing on the precision of individual pixels.

This composite loss function L is formulated as

$$\begin{array}{cc}\hfill L& =L_{\mathrm{MAE}}+\alpha (1-\mathrm{SSIM})+\lambda {\parallel \theta \parallel }_2^2,\hfill \end{array}$$

where $\mathrm{SSIM}$ is calculated according to the method described in [26], $\alpha >0$ adjusts the weight of the $(1-\mathrm{SSIM})$ term, $\lambda >0$ is the regularization hyperparameter, and ${\parallel \theta \parallel }_2^2$ represents the squared $L_2$ norm of the parameters of the model.

For optimization, we utilized the ADAM algorithm [27] with default parameters ${\beta }_1=0.9$ and ${\beta }_2=0.999$. The initial learning rate was set to ${10}^{-4}$. In order to enhance the stability of our network, we employed two data augmentation techniques: rotational augmentation and Gaussian noise augmentation.

4. The LoDoPaB-CT Dataset

The LoDoPaB-CT dataset [18] consists of 46,573 pairs of chest CT images obtained from patients and their respective simulated low-dose measurements. The original data were obtained from the LIDC/IDRI dataset [28] and the ground truth images were cropped to $362\times 362$ pixels. This dataset serves as a benchmark for evaluating LDCT reconstruction algorithms, offering 35,820 training, 3522 validation, 3553 test, and 3678 challenge images.

5. Evaluation Metrics

We adopted the evaluation metrics established in the LoDoPaB-CT challenge [18], which include the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). Additionally, we utilize their respective fixed-range (FR) versions of the PSNR and SSIM algorithms, namely, PSNR-FR and SSIM-FR. These metrics standardize the range of values rather than use the full dynamic range of the ground truth image. The PSNR measures the relationship between the highest achievable signal strength and the level of noise distortion [29]. On the other hand, SSIM, introduced by Wang et al. [26], evaluates the structural similarity of two images, as elaborated in the section on implementation details. Higher values for both the PSNR and SSIM indicate better reconstruction quality in our evaluations.

6. Experimental Results

To explore the impact of fine-tuning a pretrained network on our results, we evaluated the performance of the DRUNet in several scenarios. We compared the effectiveness of pretraining the DRUNet on the Gaussian denoising of natural grayscale images against the random initialization of the model weights. In both cases, the models are fine-tuned on various subsets of the LoDoPaB-CT dataset for comparison. We utilized ‘drunet_gray.pth’ as the pretrained model weights, which is available for download from a GitHub repository (https://github.com/cszn/DPIR/tree/master/model_zoo (accessed on 18 June 2024)).

In the LoDoPaB-CT dataset, we randomly selected 0.2%, 1%, and 5% of the training and validation datasets. Specifically, this selection resulted in 71 training and 7 validation images for the 0.2% case, 358 training and 35 validation images for the 1% case, and 1791 training and 176 validation images for the 5% case. When testing the trained networks, we selected 300 random images from the test set of the LoDoPaB-CT dataset.

Table 1 outlines the results across various scenarios, while Figure 1 provides visual representations of an individual test samples for each scenario. A direct comparison reveals that fine-tuning the pretrained network consistently yields better metrics than training from scratch across all three cases. Additionally, the effectiveness of pretraining becomes more pronounced as the number of images for fine-tuning decreases. Notably, training the network from scratch with 358 images yields similar results to fine-tuning the pretrained network with only 71 images. Similarly, training from scratch with 1791 images shows comparable results to fine-tuning with 358 images. This demonstrates that less task-specific data are needed to achieve similar outcomes.

Table 1. Performance comparison of the DRUNet trained from scratch and fine-tuned using the pretrained weights for Gaussian noise removal. The models are evaluated in terms of PSNR and SSIM across three subsets of the LoDoPaB-CT dataset.

LoDoPaB-CT Dataset	71 Training Images		358 Training Images		1791 Training Images
	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM
DRUNet From Scratch	$34.39\pm 4.10$	$0.8185\pm 0.1508$	$35.35\pm 4.43$	$0.8416\pm 0.1505$	$35.90\pm 4.70$	$0.8499\pm 0.1485$
DRUNet Fine-Tuned	$35.30\pm 4.34$	$0.8374\pm 0.1506$	$35.87\pm 4.64$	$0.8468\pm 0.1499$	$36.14\pm 4.78$	$0.8516\pm 0.1482$

Figure 1. Reconstruction results for a test sample from the test set of the LoDoPaB-CT dataset. The initial column presents the ground truth, which is followed by the corresponding FBP, which represents the initial stage of our methodology. In the second stage, the DRUNet is employed to enhance the FBP result. Subsequent columns illustrate different amounts of training images (71, 358, and 1791). Each column compares results achieved by training the DRUNet from scratch versus fine-tuning the pretrained DRUNet. PSNR and SSIM metrics quantify the quality improvements, demonstrating better performance when leveraging the pretrained DRUNet.

7. Conclusions

In this study, we presented a two-stage method for reconstructing LDCT images by integrating classical FBP with deep learning techniques, aimed at enhancing image quality. Our training strategy involves the initial pretraining of a neural network on natural grayscale images to remove Gaussian noise, followed by fine-tuning specifically for LDCT image enhancement. This strategy diverges from conventional methods that typically rely on pretraining with solely domain-specific CT data.

Experiments on subsets of the LoDoPaB-CT dataset validated our approach, revealing improvements in both PSNR and SSIM metrics compared to training the network from scratch. Our findings illustrate the benefit of pretraining strategies, highlighting the potential for practical applications in medical imaging where minimizing data requirements is crucial.

For more details, the preprint of our paper can be found on arXiv [30]. Additional results using diverse architectures and datasets will be presented in future work.