# Nonlocal Total Variation Using the First and Second Order Derivatives and Its Application to CT image Reconstruction

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. Problem Definition

#### 2.2. Accelerated Algorithm Using the Proximal Splitting with Passty’s Framework

**[Passty’s framework]**Let us consider the case where $J\left(\overrightarrow{x}\right)$ can be divided into a sum of subfunctions as:

#### 2.3. Optimization

#### 2.3.1. Update the Data-Fidelity Term

#### 2.3.2. Update the Regularization Term

**[Update the TV term**

**]**First, we consider updating the pixel $j$.

**[Update the TKV term**

**]**We update the pixel $j$, ${j}_{k}$, ${j}^{\prime}$, ${j}_{k}^{\prime}$ simultaneously. For updating the pixel $j$, ${j}_{k}$, ${j}^{\prime}$, ${j}_{k}^{\prime}$, we further divide a subfunction ${u}_{g}^{TKV}\left(\overrightarrow{x}\right)$ into four subfunctions as below

#### 2.3.3. The Weight

## 3. Experimental Results

^{2}). We compressed the range showing the reconstructed images to [7.82, 62.30] HU, where this contrast range was determined based on the contrast range used in clinical brain CT imaging. To evaluate image quality, standard RMSE, PSNR, SSIM values were used as metrics. The number of iterations in image reconstructions was 20 for nonlocal TV, TKV, and TV+TKV, which was determined by the fact that changes in image were small enough with this iteration number. We also showed the reconstructed images by the standard Filtered Back-Projection (FBP), and differences in image quality by changing values of the hyper-parameter $t$ (i.e., the trade-off parameter between nonlocal TV (first derivative) term and the TKV (second derivative) term). The ground truth image and the FBP reconstructions are shown in Figure 5. The reconstructed images in the case of sparse-view CT are shown in Figure 6. In Figure 7, we show the used brain CT image with three display gray-scale ranges, from which we observe that the staircase artifacts are severe when the range of display gray-scale range is small. The reconstructed images in the case of low-dose CT are shown in Figure 8. In Figure 9, Figure 10 and Figure 11, we show convergence properties of our iterative algorithm based on Passty’s proximal splitting framework. In Figure 12 and Figure 13, to show the effect of acceleration by Passty’s proximal splitting, we incorporated the TV+TKV term into SIRT (simultaneous iterative reconstruction technique) which is a non-row-action method (a type of the standard iterative algorithm) and compared this with row-action based on our proposed nonlocal TV+TKV. From these figures, it can be observed that our algorithm converged very quickly. It is well-known that the standard iterative algorithms such as Chambolle–Pock [26] and proximal gradient algorithms require several hundreds of iteration up to the convergence. The benefit of our iterative algorithm mainly originates from the fact that our algorithm is of row-action type.

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Kim, Y.; Kudo, H.; Chigita, K.; Lian, S. Image reconstruction in sparse-view CT using improved nonlocal total variation regularization. In Proceedings of the SPIE Optical Engineering + Applications, San Diego, CA, USA, 10 September 2019. [Google Scholar]
- Kim, H.; Chen, J.; Wang, A.; Chuang, C.; Held, M.; Pouliot, J. Non-local total-variation (NLTV) minimization combined with reweighted L1-norm for compressed sensing CT reconstruction. Phys. Med. Biol.
**2016**, 61, 6878–6891. [Google Scholar] [CrossRef] - Kim, K.; El Fakhri, G.; Li, Q. Low-dose CT reconstruction using spatially encoded nonlocal penalty. Med. Phys.
**2017**, 44, e376–e390. [Google Scholar] [CrossRef] [PubMed] - Lv, D.; Zhou, Q.; Choi, J.K.; Li, J.; Zhang, X. NLTG Priors in Medical Image: Nonlocal TV-Gaussian (NLTG) prior for Bayesian inverse problems with applications to Limited CT Reconstruction. Inverse Prob. Imaging
**2019**, 14, 117–132. [Google Scholar] [CrossRef] [Green Version] - Gilboa, G.; Osher, S. Nonlocal Operators with Applications to Image Processing. Multiscale Model. Simul.
**2009**, 7, 1005–1028. [Google Scholar] [CrossRef] - Bresson, X. A Short Note for Nonlocal TV Minimization. Technical Report. Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.210.471&rep=rep1&type=pdf (accessed on 15 June 2020).
- Chambolle, A. An algorithm for Total variation Regularization and Denoising. J. Math. Imaging.
**2004**, 20, 89–97. [Google Scholar] - Rudin, L.; Osher, S.; Fatemi, E. Nonlinear total variation based noise removal algorithms. Physica D
**1992**, 60, 259–268. [Google Scholar] [CrossRef] - Zhang, X.; Xing, L. Sequentially reweighted TV minimization for CT metal artifact reduction. Med. Phys.
**2013**, 40, 1–12. [Google Scholar] [CrossRef] [Green Version] - Bredies, K.; Kunisch, K.; Pock, T. Total generalized variation. SIAM J. Imag. Sci.
**2010**, 3, 492–526. [Google Scholar] [CrossRef] - Ranftl, R.; Bredies, K.; Pock, T. Non-local total generalized variation for optical flow estimation. Lect. Notes Comput. Sci.
**2014**, 8689, 439–454. [Google Scholar] - Parikh, N.; Boyd, S. Proximal Algorithms. Available online: https://web.stanford.edu/~boyd/papers/pdf/prox_algs.pdf (accessed on 15 June 2020).
- Combettes, P.L.; Pesquet, J.C. Proximal splitting methods in signal processing. In Fixed-Point Algorithms for Inverse Problems in Science and Engineering; Springer: New York, NY, USA, 2011. [Google Scholar]
- Passty, G.B. Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. J. Math. Anal. Appl.
**1979**, 72, 383–390. [Google Scholar] [CrossRef] [Green Version] - Herman, G.T.; Meyer, L.B. Algebraic reconstruction techniques can be made computationally efficient (positron emission tomography application). IEEE Trans. Med. Imaging
**1993**, 12, 600–609. [Google Scholar] [CrossRef] [Green Version] - Tanaka, E.; Kudo, H. Subset-dependent relaxation in block-iterative algorithms for image reconstruction in emission tomography. Phys. Med. Biol.
**2003**, 48, 1405–1422. [Google Scholar] [CrossRef] [PubMed] - Wang, G.; Jiang, M. Ordered subset simultaneous algebraic reconstruction techniques (OS SART). J. of X Ray Sci. Technol.
**2004**, 12, 169–177. [Google Scholar] - Dong, J.; Kudo, H. Proposal of Compressed Sensing Using Nonlinear Sparsifying Transform for CT Image Reconstruction. Med. Imaging Technol.
**2016**, 34, 235–244. [Google Scholar] - Dong, J.; Kudo, H. Accelerated Algorithm for Compressed Sensing Using Nonlinear Sparsifying Transform in CT Image Reconstruction. Med. Imaging Technol.
**2017**, 35, 63–73. [Google Scholar] - Dong, J.; Kudo, H.; Kim, Y. Accelerated Algorithm for the Classical SIRT Method in CT Image Reconstruction. In Proceedings of the 5th International Conference on Multimedia and Image Processing, Nanjing, China, 10–12 January 2020; pp. 49–55. [Google Scholar]
- Kudo, H.; Takaki, K.; Yamazaki, F.; Nemoto, T. Proposal of fault-tolerant tomographic image reconstruction. In Proceedings of the SPIE Optical Engineering + Applications, San Diego, CA, USA, 3 October 2016. [Google Scholar]
- Kudo, H.; Dong, J.; Chigita, K.; Kim, Y. Metal artifact reduction in CT using fault-tolerant image reconstruction. In Proceedings of the SPIE Optical Engineering + Applications, San Diego, CA, USA, 10 September 2019. [Google Scholar]
- Li, M.; Yang, H.; Kudo, H. An accurate iterative reconstruction algorithm for sparse objects: Application to 3 D blood vessel reconstruction from a limited number of projections. Phys. Med. Biol.
**2002**, 47, 2599–2609. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Kudo, H.; Suzuki, T.; Rashed, E.A. Image reconstruction for sparse-view CT and interior CT-introduction to compressed sensing and differentiated backprojection. Quant. Imaging Med. Surg.
**2013**, 3. [Google Scholar] [CrossRef] - Buades, A.; Coll, B.; Morel, J.M. On image denoising methods. Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.100.81&rep=rep1&type=pdf (accessed on 15 June 2020).
- Sidky, E.Y.; Jørgensen, J.H.; Pan, X. Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle–Pock algorithm. Phys. Med. Biol.
**2012**, 57, 3065–3091. [Google Scholar] [CrossRef] - Han, Y.; Ye, J.C. Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT. IEEE Trans. Med. Imaging
**2018**, 37, 1418–1429. [Google Scholar] [CrossRef] [Green Version] - Jin, K.H.; McCann, M.T.; Froustey, E.; Unser, M. Deep Convolutional Neural Network for Inverse Problems in Imaging. IEEE Trans. Image Process.
**2017**, 26, 4509–4522. [Google Scholar] [CrossRef] [Green Version] - Shi, F.; Cheng, J.; Wang, L.; Yap, P.-T.; Shen, D. Low-Rank Total Variation for Image Super-Resolution. Med. Image Comput. Comput. Assist. Interv.
**2013**, 16, 155–162. [Google Scholar] [PubMed] [Green Version] - Niu, S.; Yu, G.; Ma, J.; Wang, J. Nonlocal low-rank and sparse matrix decomposition for spectral CT reconstruction. Inverse Probl.
**2018**, 34. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Definition of the pixel location in the proposed regularization term corresponding to $k=1,\text{}2,\text{}3,\cdots ,\text{}7,\text{}8$.

**Figure 2.**Raster scanning during the update (${j}^{\prime}=1,\text{}2,\text{}3,\text{}\dots \text{}$).

**Figure 3.**Cost performance of changing the size of weight. (

**a**) Image quality improvement (PSNR), (

**b**) Computation time [ms].

**Figure 4.**Demonstration of how the size of weight ($\mathsf{\Omega}$) influences image quality and computation time.

**Figure 5.**(

**a**) Ground truth, (

**b**) FBP with 64 projection data with no noise, (

**c**) FBP with 256 projection data with the number of photon counts $3\times {10}^{6}$. All images are displayed with the same window of [7.82, 62.30] HU.

**Figure 6.**The reconstructed images of sparse-view CT (64 projection data with no noise). (

**a**) Ground truth, (

**b**) Nonlocal TV ($t=1.0$), (

**c**) Nonlocal TKV ($t=0.0$), (

**d**) Nonlocal TV+TKV ($t=0.3$) were compared. All images are displayed with the same window of [7.82, 62.30] HU.

**Figure 7.**Demonstration of appearance of the staircase artifacts with various gray-scale ranges in displaying the brain CT image. (

**a**) Window Width [−346.27, 416.40] HU, (

**b**) Window Width [−128.36, 198.49] HU, (

**c**) Window Width [7.82, 62.30] HU.

**Figure 8.**The reconstructed images of low-dose CT (256 projection data and the number of photon counts 3$\times {10}^{6}$). (

**a**) Ground truth, (

**b**) Nonlocal TV ($t=1.0$), (

**c**) Nonlocal TKV ($t=0.0$), (

**d**) Nonlocal TV+TKV ($t=0.3$) were compared. All images are displayed with the same window of [7.82, 62.30] HU.

**Figure 12.**The reconstructed images of sparse-view CT (64 projection data with no noise). SIRT nonlocal TV+TKV and row-action accelerated nonlocal TV+TKV (our proposed method) were compared. All images are displayed with the same window of [7.82, 62.30] HU.

**Figure 13.**The reconstructed images of low-dose CT (256 projection data and the number of photon counts $3\times {10}^{6}$). SIRT nonlocal TV+TKV and row-action accelerated nonlocal TV+TKV (our proposed method) were compared. All images are displayed with the same window of [7.82, 62.30] HU.

Nonlocal TV | Nonlocal TKV | Nonlocal TV + TKV | |
---|---|---|---|

Convergence | Good | Not bad | Good |

High contrast | Yes | No | Yes |

Smooth intensity change | No | Yes | Yes |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kim, Y.; Kudo, H.
Nonlocal Total Variation Using the First and Second Order Derivatives and Its Application to CT image Reconstruction. *Sensors* **2020**, *20*, 3494.
https://doi.org/10.3390/s20123494

**AMA Style**

Kim Y, Kudo H.
Nonlocal Total Variation Using the First and Second Order Derivatives and Its Application to CT image Reconstruction. *Sensors*. 2020; 20(12):3494.
https://doi.org/10.3390/s20123494

**Chicago/Turabian Style**

Kim, Yongchae, and Hiroyuki Kudo.
2020. "Nonlocal Total Variation Using the First and Second Order Derivatives and Its Application to CT image Reconstruction" *Sensors* 20, no. 12: 3494.
https://doi.org/10.3390/s20123494