# Exponential-Distance Weights for Reducing Grid-like Artifacts in Patch-Based Medical Image Registration

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- We calculate the relative weight of each point prediction using an exponential function on each patch, according to the distance from each point to the center point. This allows fusion of the predictions from all overlapping patches, while giving lower weight to predictions that are made by the patches near their edges.
- The proposed patch fusion method can be used together with a patch-based deep learning model for registration without any modification to significantly improve network predictions.

## 2. Methods

#### 2.1. Grid-like Artifacts

#### 2.2. Distance Functions

#### 2.2.1. Euclidean Distance

#### 2.2.2. Manhattan Distance

#### 2.2.3. Chebyshev Distance

#### 2.3. The Process of Patch Fusion

## 3. Experiments and Results

#### 3.1. Dataset Description

^{3}voxel resolution. The dataset also contained segmentation label images of cerebrospinal fluid (CSF), gray matter (GM), and white matter (WM).

#### 3.2. Experimental Details

#### 3.3. Experimental Results

#### 3.4. Comparisons of the Results with Different Strides

#### 3.5. Comparisons of the Results with Different Distance Functions

#### 3.6. Comparisons of the Results with Different Weighting Methods

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Hu, Y.; Modat, M.; Gibson, E.; Li, W.; Ghavami, N.; Bonmati, E.; Wang, G.; Bandula, S.; Moore, M.; Emberton, C.; et al. Weakly-supervised convolutional neural networks for multimodal image registration. Med. Image Anal.
**2018**, 49, 1–13. [Google Scholar] [CrossRef] [PubMed] - Hering, A.; Häger, S.; Moltz, J.; Lessmann, N.; Heldmann, S.; van Ginneken, B. CNN-based Lung CT Registration with Multiple Anatomical Constraints. Med. Image Anal.
**2021**, 72, 102139. [Google Scholar] [CrossRef] [PubMed] - Balakrishnan, G.; Zhao, A.; Sabuncu, M.R.; Guttag, J.; Dalca, A.V. Voxelmorph: A learning framework for deformable medical image registration. IEEE Trans. Med. Imaging
**2019**, 38, 1788–1800. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Fu, Y.; Lei, Y.; Wang, T.; Curran, W.J.; Liu, T.; Yang, X. Deep learning in medical image registration: A review. Phys. Med. Biol.
**2020**, 65, 20TR01. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Yang, X.; Kwitt, R.; Styner, M.; Niethammer, M. Quicksilver: Fast predictive image registration–a deep learning approach. NeuroImage
**2017**, 158, 378–396. [Google Scholar] [CrossRef] [PubMed] - Cao, X.; Yang, J.; Zhang, J.; Wang, Q.; Yap, P.T.; Shen, D. Deformable image registration using a cue-aware deep regression network. IEEE Trans. Med. Imaging
**2018**, 65, 1900–1911. [Google Scholar] [CrossRef] [PubMed] - Fan, J.; Cao, X.; Yap, P.T.; Shen, D. BIRNet: Brain image registration using dual-supervised fully convolutional networks. Med. Image Anal.
**2019**, 54, 193–206. [Google Scholar] [CrossRef] [PubMed] - Hu, S.; Zhang, L.; Li, G.; Liu, M.; Fu, D.; Zhang, W. Brain Deformable Registration Using Global and Local Label-Driven Deep Regression Learning in the First Year of Life. IEEE Access
**2019**, 8, 25691–25705. [Google Scholar] [CrossRef] - Malkauthekar, M.D. Analysis of Euclidean distance and Manhattan distance measure in Face recognition. In Proceedings of the Third International Conference on Computational Intelligence and Information Technology (CIIT 2013), Mumbai, India, 18–19 October 2013; pp. 503–507. [Google Scholar]
- Coghetto, R. Chebyshev distance. Formaliz. Math.
**2016**, 24, 121–141. [Google Scholar] [CrossRef] [Green Version] - Wang, L.; Xie, C.; Lin, Y.; Zhou, H.; Chen, K.; Cheng, D.; Dubost, F.; Collery, B.; Khanal, B.; Khanal, B.; et al. Evaluation and Comparison of Accurate Automated Spinal Curvature Estimation Algorithms with Spinal Anterior-posterior X-Ray Images: The AASCE2019 Challenge. Med. Image Anal.
**2021**, 27, 102115. [Google Scholar] [CrossRef] [PubMed] - Ma, X.; Xi, B.; Zhang, Y.; Zhu, L.; Sui, X.; Tian, G.; Yang, J.A. Machine Learning-based Diagnosis of Thyroid Cancer Using Thyroid Nodules Ultrasound Images. Curr. Bioinform.
**2020**, 15, 349–358. [Google Scholar] [CrossRef] - LaMontagne, P.J.; Benzinger, T.L.; Morris, J.C.; Keefe, S.; Hornbeck, R.; Xiong, C.; Grant, E.; Hassenstab, J.; Moulder, K.G.; Vlassenko, A.; et al. OASIS-3: Longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and Alzheimer disease. MedRxiv
**2019**. [Google Scholar] [CrossRef] [Green Version] - Zhang, S.; Liu, P.X.; Zheng, M.; Shi, W. A diffeomorphic unsupervised method for deformable soft tissue image registration. Comput. Biol. Med.
**2020**, 120, 103708. [Google Scholar] [CrossRef] [PubMed] - Müller, D.; Kramer, F. MIScnn: A framework for medical image segmentation with convolutional neural networks and deep learning. BMC Med. Imaging
**2021**, 21, 1–11. [Google Scholar] [CrossRef] [PubMed] - Wu, W. Patchify. Available online: https://github.com/dovahcrow/patchify.py (accessed on 10 August 2021).

**Figure 4.**The heat map of the weights used by the AAW method, and the distances obtained by using different distance functions in the 33rd slice of a 64 × 64 × 64 voxels patch. (

**a**) AAW method; (

**b**) Chebyshev distance; (

**c**) Manhattan distance; (

**d**) Euclidean distance.

**Figure 5.**Plot of the weights of the AAW method, the exponential weighted distance (

**a**) and the magnitude of the corresponding Fourier transformation (

**b**) for the 32nd row of the 33rd slice. From left to right: AAW, exponential Chebyshev-distance-weighted (ECDW), exponential Manhattan-distance-weighted (EMDW), and exponential Euclidean-distance-weighted (EEDW). From the amplitude of FFT, we can find that the AAW method and ECDW retain some high-frequency information and cannot remove the grid-like artifacts to a greater degree.

**Figure 6.**The pipeline of patch fusion in the test phase. The size of both the moving image and the fixed image is $H\times W\times C$, and the overlapping patches of size $h\times w\times c$ are extracted using the sliding window with stride $s$. The DDF patch of $h\times w\times c\times 3$ is obtained by the trained registration model. The output DDF patch is located at the same location as the input patch. Finally, the whole DDF is obtained by the EDW method.

**Figure 7.**A schematic diagram of the overlapping patches. The shaded regions represent overlap. Points A and B are the center points of patch 1 and patch 2, respectively, and point C is a point in the overlap region. The predicted value of point C in patch 1 and patch 2 are ${\varphi}_{A}$ and ${\varphi}_{B}$, respectively. ${d}_{1}$ is the distance from point C to point A when point C is in patch 1, ${d}_{2}$ is the distance from point C to point B when point C is in patch 2. The final predicted value is ${\varphi}_{C}$. According to Equation (6), the weights of point C in patch 1 and patch 2 are as follows: ${\omega}_{A}=\frac{{e}^{-{d}_{1}}}{{e}^{-{d}_{1}}+{e}^{-{d}_{2}}}$ and ${\omega}_{B}=\frac{{e}^{-{d}_{2}}}{{e}^{-{d}_{1}}+{e}^{-{d}_{2}}}$. Finally, we can get ${\varphi}_{C}={\omega}_{A}{\varphi}_{A}+{\omega}_{B}{\varphi}_{B}$ from Equation (7).

**Figure 8.**The fusion results for different methods. The positions of $\mathrm{det}(D{\varphi}^{-1})<0$ are marked in red. From left to right: AAW method; MIScnn; patchify and our proposed.

**Figure 9.**Moving image, fixed image and DDFs obtained for different models. (

**a**) moving image; (

**b**) fixed image; (

**c**) VoxelMorph; (

**d**) VoxelMorph (JD); (

**e**) Label-reg.

**Figure 10.**The results of different stride for AAW (

**a**) and proposed method (

**b**). The strides from left to right are 4 × 4 × 4, 8 × 8 × 8, 16 × 16 × 16 and 32 × 32 × 32.

**Figure 11.**DDFs of three distance functions at 100 slices (

**a**) and 160 slices (

**b**) of the axial plane. The 1st column is the Chebyshev distance, the 2nd column is the Manhattan distance, and the 3rd column is the Euclidean distance.

**Figure 12.**DDF results of different weighting methods and their corresponding function curves. (

**a**) IDW; (

**b**) proposed method; (

**c**) weighting function curves.

Stride | 4 × 4 × 4 | 8 × 8 × 8 | 16 × 16 × 16 | 32 × 32 × 32 |

Number of Per Patch | 20,625 | 2873 | 441 | 80 |

Method | DSC | $\mathit{\rho}$ | ||
---|---|---|---|---|

CSF | GM | WM | ||

AAW | 0.7542 | 0.7253 | 0.8188 | 0.0026 |

MIScnn | 0.7512 | 0.7223 | 0.8162 | 0.0028 |

Patchify | 0.7573 | 0.7239 | 0.8192 | 0.0050 |

Proposed | 0.7621 | 0.7407 | 0.8325 | 0.0042 |

Stride | Method | DSC | $\mathit{\rho}$ | Times (s) | ||
---|---|---|---|---|---|---|

CSF | GM | WM | ||||

4 × 4 × 4 | AAW | 0.7491 | 0.7231 | 0.8161 | 0.0018 | $\approx $600 |

Proposed | 0.7578 | 0.7375 | 0.8293 | 0.0037 | $\approx $1000 | |

8 × 8 × 8 | AAW | 0.7512 | 0.7241 | 0.8173 | 0.0022 | 91.3156 |

Proposed | 0.7593 | 0.7390 | 0.8307 | 0.0040 | 97.0658 | |

16 × 16 × 16 | AAW | 0.7542 | 0.7253 | 0.8188 | 0.0026 | 14.1332 |

Proposed | 0.7621 | 0.7407 | 0.8325 | 0.0042 | 14.7074 | |

32 × 32 × 32 | AAW | 0.7556 | 0.7243 | 0.8183 | 0.0034 | 2.6318 |

Proposed | 0.7627 | 0.7406 | 0.8326 | 0.0042 | 2.7107 |

**Table 4.**Registration results of different distance functions and the best results are shown in bold.

Distance Function | DSC | $\mathit{\rho}$ | ||
---|---|---|---|---|

CSF | GM | WM | ||

Chebyshev | 0.7628 | 0.7405 | 0.8326 | 0.0042 |

Manhattan | 0.7626 | 0.7406 | 0.8326 | 0.0042 |

Euclidean | 0.7627 | 0.7406 | 0.8326 | 0.0042 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wu, L.; Hu, S.; Liu, C.
Exponential-Distance Weights for Reducing Grid-like Artifacts in Patch-Based Medical Image Registration. *Sensors* **2021**, *21*, 7112.
https://doi.org/10.3390/s21217112

**AMA Style**

Wu L, Hu S, Liu C.
Exponential-Distance Weights for Reducing Grid-like Artifacts in Patch-Based Medical Image Registration. *Sensors*. 2021; 21(21):7112.
https://doi.org/10.3390/s21217112

**Chicago/Turabian Style**

Wu, Liang, Shunbo Hu, and Changchun Liu.
2021. "Exponential-Distance Weights for Reducing Grid-like Artifacts in Patch-Based Medical Image Registration" *Sensors* 21, no. 21: 7112.
https://doi.org/10.3390/s21217112