This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).
Nonrigid multimodal image registration plays an important role in medical image processing and analysis. Existing image registration methods based on similarity metrics such as mutual information (MI) and sum of squared differences (SSD) cannot achieve either high registration accuracy or high registration efficiency. To address this problem, we propose a novel two phase nonrigid multimodal image registration method by combining Weber local descriptor (WLD) based similarity metrics with the normalized mutual information (NMI) using the diffeomorphic freeform deformation (FFD) model. The first phase aims at recovering the large deformation component using the WLD based nonlocal SSD (wldNSSD) or weighted structural similarity (wldWSSIM). Based on the output of the former phase, the second phase is focused on getting accurate transformation parameters related to the small deformation using the NMI. Extensive experiments on T1, T2 and PD weighted MR images demonstrate that the proposed wldNSSDNMI or wldWSSIMNMI method outperforms the registration methods based on the NMI, the conditional mutual information (CMI), the SSD on entropy images (ESSD) and the ESSDNMI in terms of registration accuracy and computation efficiency.
Nonrigid image registration is one of the most challenging problems in medical image processing. Given two medical images, the objective of the registration process is to find a reasonable nonrigid transformation, such that a transformed version of the float image is similar to the reference one. Despite phenomenal progress in medical image resolution, one modality is often not sufficient to produce a precise diagnosis since different imaging modalities differ in interpreting the anatomy, tissue and organ that they may capture, so multimodal medical image registration is useful for relating clinically significant information from different images. For example, it can be used to improve the diagnostic tasks and imageguided interventions. However, accurate nonrigid multimodal registration is highly challenging because of intensity variations and nonrigid transformations between images.
In general, image registration involves three main components: deformation model, similarity metric and optimization strategy. In the nonrigid image registration, the deformation model can be divided into two main categories [
As regards optimization strategy, numerous optimization methods have been proposed to optimize the parameters of the deformation model in the nonrigid image registration. Examples include gradient descent, Newton's method, Powell's method and discrete optimization [
Apart from deformation model and optimization strategy, similarity metrics have received much attention in the field of image registration. Many similarity metrics have been proposed for different applications [
In addition to information theory measuresbased registration methods, structural representation methods have been investigated for multimodal medical image registration. Pluim
To achieve both accuracy and efficiency of the nonrigid multimodal registration method, a similarity metric based on the Weber local descriptor (WLD) proposed in [
This paper is structured as follows: Section 2 presents the novel two phase nonrigid multimodal image registration method. Section 3 provides discussions of key parameters in the proposed method and comparisons of registration accuracy and efficiency among our method and the NMI, CMI, ESSD and ESSDNMI methods. Finally, the conclusion is given in Section 4.
In general, image registration is stated as the following minimization problem:
In
Different from abovementioned coarsetofine deformation registration methods, the proposed two phase image registration method aims at resolving the above minimization problem by using different similarity metrics in two registration phases as shown in
In our method, we used the diffeomorphic FFD model as the deformation model which uses a multiresolution way by concatenating the FFDs with different grid sizes and limiting the control points displacement less than 0.4 × the grid sizes [
Ernst Weber observed that the ratio of the increment threshold to the background intensity is a constant in human perception [
Although Weber's Law describes fundamental relationships of the human perception,
The WLD has two components: differential excitation (
By applying the arctangent function which can limit the output to prevent from increasing or decreasing too quickly when the input becomes larger or smaller [
To tailor the WLD to our nonrigid multimodal registration method, it is expressed as:
Actually, WLD features can be extracted from a square symmetric neighborhood of size (2
To demonstrate the performance of the WLD with different
From
To describe the difference between the extracted WLD feature of
As discussed in Section 2.2.1, by combining the WLD using the neighborhoods of different radii
Based on the wldNSSD, we define the objective function
Given two images
The SSIM index is not a metric. However, the distance
Similar to the wldNSSD, the similarity metric wldWSSIM is defined as:
Based on the wldWSSIM, the objective function
To demonstrate the advantage of the wldNSSD and the wldWSSIM over such metrics as the NMI and the ESSD, we compute the four distance measures on ten pairs of T1 and T2 weighted MR images rotated around the domain center with different angles and translated in
To obtain more accurate registration results, the small deformation phase is needed for the refined registration. In this phase, the float image
In this section, to determine the key parameters in the proposed method and make comparisons of registration performance among our method, the NMI method, the CMI method, the ESSD method and the ESSDNMI method, extensive experiments have been performed on thirty T1, T2 and PDweighted MR images of size 256 × 212 pixels from the BrainWeb database [
As regards the simulated deformation
When we use expert landmark annotations as ground truth, for a estimated deformation
The patch sizes
The weighting term γ is specific to the processed images. To quantitatively determine γ, we make some tests on γ with fifteen T1, T2 and PDweighted MR images.
The comparison results of registration precision with different γ for similarity measures wldWSSIM and wldNSSD are shown in
To demonstrate the advantage of the wldNSSD and wldWSSIM, wldNSSDNMI and wldWSSIMNMI methods, they are compared with other evaluated methods in terms of registration accuracy and efficiency.
The mean and standard deviation (std) of TRE
In this paper, we have proposed a two phase nonrigid multimodal medical image registration method using the Weber local descriptor based similarity metrics and the normalized mutual information. In the first phase, the parameters relevant to the large deformation are obtained by minimizing the objective function defined by the novel similarity metric wldNSSD or wldWSSIM which is focused not on intensities of individual pixels but on structural information of images. With the good initial deformation value provided by the output of the large deformation phase, the parameters related to the small deformation can be accurately obtained using the NMI in the second phase. The nonrigid image registration experiments on the T1, T2 and PD weighted MR images demonstrate that compared with the NMI, CMI, ESSD and ESSDNMI methods, our method can obtain smaller registration errors and higher computational efficiency. Future work will be focused on extending our method to multimodal 3D medical image registration.
This work was partly supported by the National 973 project (Grant No.: 2011CB933103), the Project of the National 12thFive Year Research Program of China (Grant No.: 2012BAI13B02), the National Natural Science Foundation of China (NSFC) (Grant No.: 30911120497) and the Biology and Bioinformatics Ph.D. Programs Construction Project of Guangxi University, China.
The authors declare no conflict of interest.
Two phase nonrigid multimodal image registration.
Differential excitation of Weber local descriptor.
The square symmetric neighborhoods with different
Entropy image and WLD with different
Distance measures for T1 and T2 weighted MR images. (
Landmarks in the deformed T1, deformed PD and original T2 images. (
TRE
Time (in seconds) for the wldNSSD and wldWSSIM using various patch sizes. (
Comparison of registration precision with different γ for wldWSSIM and wldNSSD. (
Nonrigid multimodal registration results for the wldWSSIMNMI, wldNSSDNMI and ESSDNMI methods operating on T1T2 and PDT2 weighted MR images. (
TRE
 



 
2.8  2.2  3.0  2.4  2.9  2.4  
1.5  1.1  1.7  1.2  1.7  1.3  
1.4  1.0  1.5  1.2  1.6  1.2  
1.4  0.9  1.6  1.1  1.5  1.1  
1.4  0.9  1.5  1.1  1.5  1.1  
1.2  0.7  1.3  0.8  1.3  0.9  
0.9  0.5  1.0  0.7  1.0  0.6  
1.0  0.6  1.1  0.7  1.1  0.7  
0.8  0.4  0.9  0.5  0.9  0.5 
TRE
 



 
0.9  0.7  1.0  0.8  1.0  0.8  
4.9  4.2  5.0  4.4  4.9  4.2  
15.4  9.2  15.7  9.3  15.6  9.5  
9.5  6.2  9.8  6.3  9.6  6.8  
8.9  5.8  8.6  5.6  9.1  6.2  
8.6  5.7  8.6  5.6  8.9  6.2  
8.4  5.6  8.4  5.6  8.7  5.8  
7.4  4.9  7.7  5.0  7.6  5.0  
6.7  3.8  6.8  3.9  6.8  3.9  
7.1  4.0  7.1  4.1  7.2  4.2  
6.1  3.3  6.2  3.4  6.2  3.5 
Computation time (in seconds) for all the evaluated methods.
 

248 ± 20  268 ± 22  280 ± 31  
350 ± 28  346 ± 26  366 ± 26  
224 ± 16  230 ± 15  246 ± 16  
284 ± 24  292 ± 21  294 ± 28  
86 ± 7  88 ± 9  84 ± 9  
119 ± 12  118 ± 12  123 ± 14  
118 ± 11  122 ± 10  125 ± 11  
148 ± 14  152 ± 15  157 ± 15 