Multi-Intensity Optimization-Based CT and Cone Beam CT Image Registration

Abstract

Cancer is a highly lethal disease that is mainly treated by image-guided radiotherapy. Because the low dose of cone beam CT is less harmful to patients, cone beam CT images are often used for target delineation in image-guided radiotherapy of various cancers, especially in breast and lung cancer. However, breathing and heartbeat can cause position errors in images taken during different periods, and the low dose of cone beam CT also results in insufficient imaging clarity, rendering existing registration methods unable to meet the CT and cone beam CT registration tasks. In this paper, we propose a novel multi-intensity optimization-based CT and cone beam CT registration method. First, we use a multi-weighted mean curvature filtering algorithm to preserve the multi-intensity details of the input image pairs. Then, the strong edge retention results are registered using and intensity-based method to obtain the multi-intensity registration results. Next, a novel evaluation method called intersection mutual information is proposed to evaluate the registration accuracy of the different multi-intensity registration results. Finally, we determine the optimal registration transformation by intersection mutual information and apply it to the input image pairs to obtain the final registration results. The experimental results demonstrate the excellent performance of the proposed method, meeting the requirements of image-guided radiotherapy.

1. Introduction

Cancer, with remarkable incidence and mortality, is one of the most concerning diseases worldwide, especially for a country like China, where the population base is undoubtedly huge. According to the latest statistics published by the National Cancer Center (NCC) of China [1], more than four million new cancer cases and nearly two million new cancer deaths happened in China in 2016. For this reason, radiotherapy, together with surgery and chemotherapy as major or complementary approaches, plays an important role as one of the three major strategies of cancer management [2,3,4].

Radiotherapy kills cancer cells by feeding high-energy dose rays to the target area. During image-guided radiotherapy (IGRT), images taken at different periods (before and after radiotherapy) need to be registered for target delineation and postoperative curative effectiveness analysis [5]. IGRT represents a great breakthrough in modern radiation oncology, in which linear accelerators are equipped with imaging devices, thus providing verification images prior to and during treatment [6,7,8,9]. The extensively adopted IGRT methods can be classified into two categories: 1000 V CT imaging using a cone X-ray beam (KVCT/CBCT) and an electronic portal image device (EPID) using a MV X-ray beam. Typically, considering the difference in beam energy, CBCT is widely applied in the clinical IGRT routine [10,11,12,13]. The development of IGRT has promoted the research and application of related new technologies for cancer treatment [14,15,16]. For CBCT-based guided radiotherapy, it is meaningful that, with the help of imaging registration techniques, the shift data generated according to the comparison of on-site images can correct the treatment table per fraction, subsequently resulting in more precise patient positioning. Specifically, in the process of IGRT, CT images (with a high imaging dose more harmful to patients) are first captured, and the cancerous areas are analyzed using the clear CT images before developing the radiotherapy plan. Then, CBCT images (with a low imaging dose less harmful to patients) serve as a guide to the location of the patient’s cancerous areas during the radiotherapy sessions. However, the current methods mainly rely on manual matching; although many registration methods have since been proposed, most of them cannot meet the requirements of CT and cone beam CT (CBCT) image registration tasks. Some typical imaging styles of CT and CBCT in IGRT are shown in Figure 1.

As can be seen from Figure 1, unlike most existing image registration tasks, the CT and CBCT image registration task faces the following difficulties: (i) poor imaging quality. As shown in Pair 1 of Figure 1, CBCT images generally have poor imaging quality (low-dose imaging), and most existing intensity-based registration methods cannot find sufficient feature regions; (ii) imaging deformation. As shown in Pair 2, the length of the yellow lines is consistent, meaning that the CBCT image is deformed compared with the CT image, indicating that the rigidity-based registration method is not suitable to solve the CT–CBCT registration task; (iii) incomplete imaging. As shown in Pair 3 and Pair 4, the imaging region of the CBCT is incomplete compared with the corresponding CT image, and most visual features and similarity-based registration methods cannot be applied to the CT–CBCT image registration task; (iv) inconsistent visual features. As shown in Pair 4, the poor quality of the CBCT image can cause some messy feature points that are not strictly consistent with the CT image. It should be noted that these four issues can occur randomly or even simultaneously during CBCT imaging, which poses a great challenge to the existing registration methods.

To address the above problems, we propose a novel multi-intensity optimization-based CT–CBCT registration method for image-guided radiotherapy. First, we propose a novel multi-weighted mean curvature filtering algorithm (MWMC) algorithm to process the input image pairs to obtain a series of multi-intensity image pairs of CT and CBCT. Then, the multi-intensity image pairs are registered using an intensity-based method. Next, we propose a novel evaluation index called intersection mutual information (IMI) to determine the optimal registration transformation for different intensity image pairs. The final registration result can be produced by applying the optimal registration transformation to the source input image pairs. The experimental results demonstrate the excellent performance of the proposed method in CT–CBCT image registration tasks, with the registration accuracy and time cost meeting the requirements of IGRT.

The remainder of this paper is organized as follows: Section 2 introduces the proposed multi-intensity optimization-based CT–CBCT registration method; Section 3 provides the experimental results and their analysis; Section 4 draws the conclusion of our work and discusses prospects for future applications.

2. Materials and Methods

In this section, the proposed CT and CBCT registration method is introduced in detail. As shown in Figure 2, the proposed method has two main stages. First, the input CT–CBCT pairs are processed using the proposed multi-weighted mean curvature filtering algorithm (MWMC) to produce the multi-intensity edge preserving results. Then, the multi-intensity image pairs are registered using the intensity-based method. In addition, we propose a novel evaluation metric for incomplete image registration to measure the registration performance, called intersection mutual information (IMI). Using IMI, we can obtain the optimal registration transformation with different intensity image pairs, before finally applying this transformation to register the source input pairs.

2.1. Details of the Proposed Model

The proposed model in this paper mainly contains four steps: multi-weighted mean curvature filter, intensity-based registration, multi-intensity-based registration, and intersection mutual information-based optimal transformation selection. Below, these four steps are introduced in detail.

2.1.1. Multi-Weighted Mean Curvature Filter

Weighted mean curvature (WMC) [17] originates from mean curvature (MC) [18,19]. MC has been widely used in the field of image processing, and the MC of an image U is defined as

$$H(U)=\frac{1}{n}\nabla \cdot \frac{\nabla U}{{‖\nabla U‖}_2},$$

(1)

where $\nabla$ and $\nabla \cdot$ are the gradient and divergence operators, respectively. For 2D images, $n=2$, MC can be rewritten as

$$H=\frac{U_x^2{U}_{yy}-2U_x{U}_y{U}_{xy}+U_y^2{U}_{xx}}{2{(U_x^2+U_y^2)}^{\frac{3}{2}}}.$$

(2)

MC is independent of image contrast, which is suitable for medical images. WMC is defined as

$$H^w(U)=n{‖\nabla U‖}_{2}H(U).$$

(3)

For 2D images, Equation (3) can be rewritten as

$$H^w(U)={‖\nabla U‖}_2(\nabla \cdot \frac{\nabla U}{{‖\nabla U‖}_2})=\Delta U-\frac{U_y^2{U}_{yy}+2U_x{U}_y{U}_{xy}+U_x^2{U}_{xx}}{U_x^2+U_y^2},$$

(4)

where $\Delta$ denotes the isotropic Laplace operator. The WMC regularization term is defined as

$$\mathcal{R}{H}^w(U)={{\displaystyle \int \left|H^w(U)\right|}}^{q}d\overrightarrow{x},$$

(5)

where $q$ is the scalar parameter defining the norm, $q>0$. $\overrightarrow{x}\in R^n$ is the spatial coordinate, and n is the dimension of the input image ($n=2$ in this paper).

The proposed multi-weighted mean curvature (M_WMC) is defined as

$$M_{WMC}{(I)}^m=\left\{\begin{array}{c}I,\hspace{1em}m=0\\ M_{WMC}{(I)}^{m-1}+\delta H^w(M_{WMC}{(I)}^{m-1},m\ge 1\end{array}\right.,$$

(6)

where m is the number of filtered image pairs obtained by M_WMC, and $\delta$ is the size of the discrete timestep. Figure 3 shows a set of multi-intensity edge preserving results using M_WMC with different m.

It can be seen from Figure 3 that edge preserving results with different intensity can be obtained using MWMC with different m. Figure 4 gives the results of color images using MWMC, where it can be clearly observed that MWMC can effectively retain the strong edge information of different intensities. The main motivation of the proposed method based on MWMC is that, by filtering out the chaotic textures and noisy points in CBCT and CT images, the registration accuracy can be improved. When m is large enough, only the significant information is retained in the input image, which helps us to improve the registration accuracy and efficiency.

2.1.2. Intensity-Based Registration

The registration problem can be defined as follows: find a suitable transformation such that a transformed version of a moving image is similar to a fixed image [20,21,22]; that is, given images $s(m,n)$ and $t(m,n)$, find the spatial transformation $f$ such that $s(f_x(m,n),f_y(m,n))$, and $t(m,n)$ are similar or consistent by optimizing

$$\underset{f\in c}{\min }\psi (s(f),t,f),$$

(7)

where $\psi (\cdot ,\cdot ,\cdot )$ denotes the difference measure. The spatial transformation can be expressed as

$$s(f_x(m,n),f_y(m,n))={\displaystyle \sum _{p\in Z}{\displaystyle \sum _{q\in Z}c[p,q]\phi (}}{f}_x(m,n)-p)\phi (f_y(m,n)-q),$$

(8)

where c is the coefficient, and $\phi (x)$ is the image representation function. For our registration tasks, CT and CBCTs are 2D images; hence, the optimization function using the least squares method can be defined as

$$\psi (s(f),t,f)={\displaystyle \sum _m{\displaystyle \sum _n(s(}}{f}_x(m,n),f_y(m,n)-t(m,n){)}^2.$$

(9)

As the CT and CBCT images do not use the same imaging modality, there is no linear relationship between the intensity values. Accordingly, when using the sum of squared differences, cross-correlation may not work. Thus, we can use the mutual information to measure the similarity of CT and CBCT images. In this paper, we used CT as the fixed image and CBCT as the moving image.

Let $P_{s(f),t}(s(f),t)$ denote the joint probability distribution function of image $s(f)$ and t. The mutual information [23] can be defined as

$$MI(s(f),t)={\displaystyle \sum _{s(f),p}{P}_{s(f),p}(s(f),t)\log }(\frac{P_{s(f),t}(s(f),t)}{P_{s(f)}(s(f))P_t(t)}).$$

(10)

Thus, the optimization function can be defined as

$$MI(s(f),t)\to \underset{f}{\min }.$$

(11)

The process of intensity-based image registration is shown in Figure 5. Intensity-based image registration is an iterative process, which begins with a random initial transformation matrix. Then, the transformation matrix is applied to the input CBCT image with bilinear interpolation. When the transformation is finished, a similarity metric is used to compare the transformed CBCT with the input CT image. Next, the optimizer checks for a stop condition. The process stops when the similarity measure matrix is large enough or the maximum number of iterations is reached.

2.1.3. Multi-Intensity Based Registration

The intensity-based registration method introduced in Section 2.2 lacks robustness for CT–CBCT registration tasks. Figure 6 shows some intensity-based registration results using different similarity metrics. It can be found that CT–CBCT image registration cannot be achieved using the similarity metrics of phase correlation and mean squares. Mutual information outperformed the other two similarity metrics. However, due to the imaging quality of CBCT images, the registration result based on mutual information still could not obtain ideal results (as shown by the white arrows in Figure 6).

The reason for the poor registration performance in Figure 6 is the poor imaging quality of CT and CBCT introduced in Section 1. Because intensity-based registration methods are more sensitive to local intensity information, noisy points and messy textures directly affect the registration performance. Therefore, we propose a multi-intensity-based registration method.

First, we generate a series of multi-intensity edge-preserving CT and CBCT image pairs using the MWMC proposed in Section 2.1. Then, the multi-intensity edge-preserving results are registered using the intensity-based method, allowing us to obtain transformation matrices of the input images with different intensities. Some results are given in Figure 7. It can be seen that excellent registration performance could be achieved with certain intensity edge-preserving results, such as m = 80 and m = 100. Therefore, after determining the optimal registration transformation, it can be directly applied to the input CT and CBCT images, thus obtaining an ideal registration result.

2.1.4. Intersection Mutual Information-Based Optimal Transformation Selection

Due to the incomplete imaging and poor imaging quality of CBCT, the calculation of mutual information can be vulnerable to noisy points and inconsistent imaging regions. In order to overcome this problem, we propose a novel similarity metric, i.e., intersection mutual information, which can be defined as follows:

$$IMI=MI\{(R_{moved\_CBCT}\cap R_{CT})·I_{moved\_CBCT},(R_{moved\_CBCT}\cap R_{CT})·I_{CT}\},$$

(12)

where $MI\{,\}$ can be computed using Equation (10), and $R_I$ denotes the imaging regions of image I. In our paper, the imaging region was segmented by the active contours (snakes) technique, which is a region growing algorithm [24,25]. To illustrate the validity of the proposed IMI, Figure 8 presents a set of experiments. As can be seen from Figure 8b, the MI values of CT and moving CBCT were greater than the MI values of CT and moved CBCT, which is obviously unreasonable. In other words, we cannot directly use MI to measure the registration degree of the input image pairs. Using Equation (12), the obtained IMI value after registration is significantly higher than that before registration, indicating that we can measure the registration degree of the input image pairs using IMI.

Finally, the optimal transformation $f_{opt}$ with different multi-intensity registration results can be determined by ranking their IMI.

$$f_{opt}=\underset{i\le m}{\max }IMI(I_{cbct}(f_i),I_{ct}),$$

(13)

where m is the number of multi-intensity edge preserving image pairs obtained by MWMC. The optimal transformation can be found from the m transformations using Equation (13). Furthermore, the final registration result can be obtained by applying $f_{opt}$ to the input CBCT.

2.2. Sample and Data

The datasets used in our experiments were obtained from our cooperative medical facility, Affiliated Hospital of Yunnan University. The samples are shown in Figure 9. All datasets were anonymized. For each patient, a pair of treatment planning CT and CBCT images was obtained. All planning CT images were acquired using the same CT simulation system (Philips Brilliance Big Bore), with a slice thickness of 3 mm. Likewise, CBCT images were all obtained using the same imaging device integrated on linac (XVI, Elekta Solutions AB, Stockholm, Sweden) prior to treatment. The size of all images was uniformly set to 256 × 256 pixels in our experiments.

2.3. Measures of Parameters

The parameters required in the proposed method mainly included the multi-intensity retention parameter m and the intensity-based registration parameters. After extensive testing using the CT and CBCT images, when the multi-intensity parameters exceeded 100, the processing results remained stable. Therefore, we set m to 100 in this paper. For the CT and CBCT images exported from medical devices, their size was uniformly normalized to 250 × 250. In practical applications, the size can be adjusted as needed.

3. Experiments and Analysis

To illustrate the effectiveness of the proposed CT–CBCT registration method, we compare the proposed method with some typical registration methods using several datasets.

3.1. Experimental Settings

In our experiments, we compared the proposed method with the Advanced Normalization Tools (ANTs) [26,27,28,29,30,31]. ANTs is the most used medical image registration toolkit with very good performance. An overview of the compared methods and similarity terms is given in

Table 1. Compared methods. Similarity measure acronyms: CC = neighborhood cross-correlation, Mean Squares = mean squared difference, MI = mutual information.

Method	Transformation	Similarity Measures
Affine	Affine registration	MI, Mean Squares, GC
Rigid	Rigid registration	MI, Mean Squares, GC
Similarity	Rotation + uniform scaling	MI, Mean Squares, GC
SyN	Symmetric diffeomorphic	CC, MI, Mean Squares, Demons

. Among them, Affine denotes the affine transformation-based method, Rigid denotes the rigid transformation-based method, Similarity denotes the rotation and uniform scaling transformation-based method, and SyN denotes the symmetric diffeomorphic transformation-based method.

3.2. Experimental Results and Analysis

The experimental results using different medical image registration methods are shown in Figure 10. As can be seen from Pair 4, when the imaging regions of CT and CBCT images were relatively consistent, i.e., the CBCT image was relatively complete, the Affine, Similarity, SyN-based, and our proposed method could achieve better registration results. For CBCT images with partially missing regions, the Affine and SyN-based methods also achieved good registration performance, as shown in Pair 3. However, it can be seen from the enlarged region (yellow arrow) that the registration accuracy of the proposed method was superior to that of the Affine and SyN-based methods.

As shown in Figure 11 (Pair 1 and Pair 2), the compared methods were completely ineffective for severely incomplete imaging CBCT and CT image pairs. However, the proposed method also achieved good registration accuracy in these cases. These experiments illustrate that the proposed method can overcome the challenges in CT and CBCT image registration tasks introduced in Section 1.

To further validate the performance of the proposed CT–CBCT registration method, we performed a quantitative comparison of different methods using the IMI and SSIM metrics [32,33]; the results are shown in

Table 2. Quantitative comparison of different methods according to IMI and SSIM metrics. The best results are shown in bold.

	Pair 1		Pair 2		Pair 3		Pair 4
Method	IMI	SSIM	IMI	SSIM	IMI	SSIM	IMI	SSIM
Affine	0.4746	0.3183	0.5032	0.2705	0.6904	0.3030	0.8766	0.3134
Rigid	0.5711	0.2542	0.5035	0.2973	0.6923	0.2636	0.8351	0.3515
Similarity	0.4727	0.2569	0.5549	0.3885	0.6912	0.2667	0.8767	0.3138
SyN	0.4742	0.2712	0.4813	0.2283	0.7236	0.2906	0.8835	0.3188
Our method	0.5940	0.3898	0.5826	0.3847	0.7676	0.3066	0.8926	0.3159

. It can be seen that the values of IMI obtained using were superior to those using other methods. in particular, our method exhibited a great advantage when using incomplete images, such as Pair 1 and Pair 2. This is consistent with the visual comparison shown in Figure 9. For relatively complete CBCT images, our method also achieved good performance (Pair 3 and Pair 4).

Table 3. The average IMI and SSIM metrics using different methods for all datasets. The best results are shown in bold.

Method	Affine	Rigid	Similarity	SyN	Our Method
IMI	0.6362	0.6505	0.6488	0.6406	0.7092
SSIM	0.3013	0.2916	0.3064	0.2772	0.3492

presents the average values of IMI and SSIM using different registration methods on our datasets. It can be seen that our method achieved the best IMI and SSIM values, indicating its higher registration accuracy compared to other methods. More importantly, the proposed method is a promising candidate for application in IGRT.

4. Conclusions

In this paper, we proposed a novel multi-intensity optimization-based CT–CBCT registration method for IGRT tasks. The proposed method can overcome the problems caused by poor imaging quality, imaging deformation, incomplete imaging, and inconsistent visual features of CBCT and CT image pairs. The experimental results demonstrated the excellent performance of the proposed method in CT–CBCT image registration tasks and showed its potential application in IGRT. At present, this method can be applied to the evaluation of postoperative treatment effectiveness, as well as other medical applications. However, because the proposed method involves an optimization process, it is difficult to guide patient positioning during IGRT in real time. Thus, in the future, we will consider how to reduce the complexity of the proposed method and make it more suitable for IGRT.