3D Tensor Based Nonlocal Low Rank Approximation in Dynamic PET Reconstruction

Abstract

Reconstructing images from multi-view projections is a crucial task both in the computer vision community and in the medical imaging community, and dynamic positron emission tomography (PET) is no exception. Unfortunately, image quality is inevitably degraded by the limitations of photon emissions and the trade-off between temporal and spatial resolution. In this paper, we develop a novel tensor based nonlocal low-rank framework for dynamic PET reconstruction. Spatial structures are effectively enhanced not only by nonlocal and sparse features, but momentarily by tensor-formed low-rank approximations in the temporal realm. Moreover, the total variation is well regularized as a complementation for denoising. These regularizations are efficiently combined into a Poisson PET model and jointly solved by distributed optimization. The experiments demonstrated in this paper validate the excellent performance of the proposed method in dynamic PET.

1. Introduction

Positron emission tomography (PET) seeks to obtain radioactivity distributions by collecting numerous photons emitted by the annihilations of positrons that come from the isotope-labeled tracer injected in living tissue. Correspondently, considering the unique traits of physiological and molecular imaging, PET is universally and irreplaceably required in clinic imaging, especially in cancer diagnoses [1], lesion detection [2], and functional supervision in vivo. However, despite its predominance for functional imaging, PET is dwarfed in resolution due to limitations either from acquisition time or the injection dose when compared with other structural medical imaging systems such as magnetic resonance imaging (MRI) and computed tomography (CT). Moreover, in dynamic PET imaging, where the time-varying radioactivity concentration at each spatial location is obtained, the structural information and denoising performance are more critical, along with increasing demands for time activity curves (TAC) for different regions of interest (ROI). Under this framework, reliable algorithms for dynamic PET reconstruction have been discussed and debated for decades.

Initial attempts at PET reconstruction included the famous analytic filtered back-projected (FBP) [3] based algorithms, least squares (LS) [4], and the maximum likelihood-expectation maximization (ML-EM) [5] method. Although milestones, problematic conditions [6] always accompany the optimization of the mentioned algorithms (i.e., the solution might be sensitive to trivial fluctuation and thus consequently increase noise along the iterations). Thus, much effort has been devoted to this topic, among which the maximum a posteriori (MAP) [7] strategy, or penalized ML method [6], have become the key to the solution. The main idea for this strategy is to integrate regularization terms, or image priors from a Bayesian perspective, into the reconstruction model. There are various representative regularizations, including the weighted quadratic penalty [8], Gibbs prior [9], Gauss–Markov prior [10], Huber prior [11], total variation [12], and so on. More recently, especially for dynamic imaging, temporal priors are starting to be introduced in reconstruction (e.g., the tracer kinetics model [13,14,15], kernel model [16], and dose estimation [17]).

Nevertheless, in spite of the significant progress achieved by researchers, traditional reconstruction algorithms are still open to question. For one thing, traditional methods, to a large extent, only focus on denoising behavior while ignoring the structural information within the images. This has undoubtedly created a greater divide within the increasing clinical demand for resolution. The other problem, which is of greater significance, is that the trade-off between the temporal and the spatial resolution constantly degrades the quality of the dynamic reconstruction (i.e., a better temporal resolution requires more frequent sampling within a given duration, which causes fewer photon counts for each frame, thus undermining the spatial resolution, and vice versa).

In this paper, we explore a potential solution for the mentioned dilemma by providing a novel tensor based nonlocal low-rank framework in dynamic PET reconstruction. Several efforts have been devoted to related topics. Low rank decomposition [18] and non-negative matrix factorization [19,20,21], on the one hand, have been found to be able to capture the inner temporal correlation in a dynamic PET. On the other hand, the nonlocal feature, which refers to the abundant self-similar structures within an image, are excellently adopted in image denoising (e.g., the nonlocal means (NLM) [22], weighted nuclear norm minimization (WNNM) [23], block-matching 3D denoising (BM3D) [24], nonlocal restoration [25], and nonlocal representation [26]). Moreover, in [27], Dong et al. demonstrate the existence of nonlocal self-similarities and sparse features in MRI images. As shown in Figure 1, nonlocal low-rank [28,29] features exist not only in ordinary natural images but also in PET reconstructed images. Furthermore, as will be illustrated in Section 3, we promote the use of matrix based nonlocal low-rank features in a novel tensor form and successfully apply it to dynamic PET reconstruction. In this way, the structures are not only enhanced spatially by the image itself but simultaneously completed by relevant frames across the temporal dimension.

The main contributions are listed as follows:

(1)

An innovative form of a nonlocal low rank tensor constraint is adopted in the Poisson’s model, which captures data correlation in multiple dimensions in dynamic PET, beyond just spatiotemporal correlation. For one thing, without any additional information, Poisson’s model exploits the temporal information among the frames themselves, effectively complementing the structures in low-active frames and recovering severely corrupted data. For the other, it exploits the spatial information from nonlocal self-similarities within each frame, thereby enhancing the structured sparsity for each image.

(2)

As an additional regularization, the total variation (TV) constraint is employed to extract local structure and further complement the denoising function. On the other hand, the expectation–maximization method is employed as a fidelity term to incorporate hidden data in the objective function and thus increase efficiency in optimization.

(3)

In the optimization procedure, we develop a distributed optimization framework inspired by the alternative direction method of multipliers (ADMM) [30]. In this way, the mentioned terms can be explicitly organized in a united objective function and effectively handled as three subproblems during the iterations.

2. Background

2.1. Dynamic PET Imaging Model

Typically, during the detection procedure, the PET scanner collects the emitted photons and then pre-processes them into so-called sinogram data as the input of the reconstruction. For dynamic PET, let the sinogram matrix $\mathit{Y}=\left[{\mathit{y}}_1,{\mathit{y}}_2,\dots ,{\mathit{y}}_t,\dots ,{\mathit{y}}_T\right]\in {ℝ}^{M \times T}$ denote the collection of sinogram data vectors, where $t=1,2,\dots ,T$ denotes the index of the frames; and sinogram vector ${\mathit{y}}_t$ = $\{y_{tq }$, $q=1,\dots ,M\}\in {ℝ}^M$ denotes the sum of the photons collected in the t-th frame, where q represents the index of the total M pairs of detectors. Correspondently, $\mathit{X}=\left[{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_t,\dots {\mathit{x}}_T\right]$ $\in {ℝ}^{N \times T}$ denotes the collection of the images that are supposed to be recovered, where vector ${\mathit{x}}_t\in {ℝ}^N$ represents the t-th frame. Since Poisson distribution uses inherited PET systems, the reconstruction of each frame can be successfully modeled by the affine transformation:

$${\mathit{y}}_t\sim \mathrm{Poisson}\left\{{\overline{\mathbf{y}}}_t\right\}\mathrm{s}.\mathrm{t}.{\overline{\mathbf{y}}}_t=E\left({\mathit{y}}_t\right)=\mathit{G}{\mathit{x}}_t+{\mathit{r}}_t+{\mathit{s}}_t,$$

(1)

where ${\overline{\mathit{y}}}_t$ is the expectation of ${\mathit{y}}_t$; $\mathit{G}\in {ℝ}^{M\times N}$ is the system matrix; and ${\mathit{r}}_t$ and ${\mathit{s}}_t$ represent the random coincidence and scatter coincidence, which inevitably contain heavy noise. In this way, we can obtain the likelihood function of ${\mathit{y}}_t$ as

$$\mathrm{Pr}({\mathit{y}}_t|{\mathit{x}}_t)=\prod _q^M{e}^{-{\overline{y}}_{tq}}\frac{{\overline{y}}_{tq}{}^{y_{tq}}}{y_{tq}!}.$$

(2)

Instead of maximizing Label (2), we estimate ${\mathit{x}}_t$ by minimizing the negative log-likelihood version for the convenience of optimization:

$$\underset{{\mathit{x}}_t}{\min }\mathrm{P}\left({\mathit{x}}_t\right)=\underset{{\mathit{x}}_t}{\min }-\log (\mathrm{Pr}({\mathit{y}}_t|{\mathit{x}}_t))=\underset{{\mathit{x}}_t}{\min }\sum _q^M{\overline{y}}_{tq}-y_{tq}\log ({\overline{y}}_{tq})s.t.{\overline{\mathit{y}}}_t=\mathit{G}{\mathit{x}}_t+{\mathit{r}}_t+{\mathit{s}}_t,$$

(3)

where the constant term $\log (y_{tq}!)$ is left out.

Therefore, in the scale of dynamic reconstruction, Equation (3) can be transformed into

$$\underset{\mathit{X}}{\min }\mathrm{P}\left(\mathit{X}\right)=\underset{\mathit{X}}{\min }\sum _t^T\sum _q^M{\overline{y}}_{tq}-y_{tq}\log ({\overline{y}}_{tq})s.t.{\overline{\mathit{y}}}_t=\mathit{G}{\mathit{x}}_t+{\mathit{r}}_t+{\mathit{s}}_t.$$

(4)

However, optimization merely by Equation (4) is ill-conditioning—i.e., the reconstruction is vulnerable to accumulated iterative noise accompanying the iteration. The predominant solution to this shortcoming is to include the regularization terms in Equation (4) as the image prior. Thus, the object function of PET reconstruction can be written as

$$\underset{\mathit{X}}{\min }\mathsf{\Psi }\left(\mathit{X}\right)=\underset{\mathit{X}}{\min }\mathrm{P}\left(\mathit{X}\right)+\mathrm{R}\left(\mathit{X}\right),$$

(5)

where $\mathrm{P}\left(\mathit{X}\right)$ denotes the fidelity term defined in Equation (4), and $\mathrm{R}\left(\mathit{X}\right)$ denotes the regularization term.

2.2. Tensor Decomposition

Just like the matrix, the low-rank approximation of the tensor is also inevitably based on tensor decomposition. At present, the strategies for tensor decomposition mainly fall into three groups: the CANDECOMP/PARAFAC [31] (CP) decomposition methods, Tucker decomposition [32,33] methods, and tensor-singular value decomposition [34,35,36] (t-SVD) methods. Because of the stability and efficiency inherited in their optimization, t-SVD has aroused increasing interest among the community.

As shown in Figure 2, for a three-way tensor $𝓐\in {ℝ}^{n_1\times n_2\times n_3}$, t-SVD shares the same form as the matrix SVD:

$$𝓐=𝓤*𝓢*{𝓥}^T,$$

(6)

where $𝓤\in {ℝ}^{n_1\times n_1\times n_3}$ and $𝓥\in {ℝ}^{n_2\times n_2\times n_3}$ are orthogonal tensors [37]; ${𝓥}^T$ represents the conjugate transpose [34] of $𝓥$; $𝓢\in {ℝ}^{n_1\times n_2\times n_3}$ is a f-diagonal tensor in which each frontal slice ${𝓢}^{\left(i\right)}$ is a diagonal matrix [35]; and $*$ denotes the t-product [37,38].

In this paper, we proposed a nonlocal tensor low-rank framework for dynamic PET reconstruction, where the tensor low-rank approximation is efficiently and effectively conducted by a t-SVD based method. The details of this method will be illustrated in Section 3.

3. Method

In this paper, we proposed a novel reconstruction framework that can jointly recover, denoise, and (mostly) critically complete the structures in the dynamic PET imaging system. The overall procedure is illustrated in Algorithm 1.

Algorithm 1: Dynamic PET reconstruction via Nonlocal Low-rank Tensor Approximation and Total Variation

Input: Sinogram $Y$ and system matrix $\mathit{G}$, weighting parameters $\alpha ,\beta ,\lambda ,\eta$, and the reference frame index $t_r$.

1: Initialization: $k=0,{\mathit{X}}^0=\mathrm{FBP}\left(\mathbf{Y}\right)$.

2: Repeat:

3:  Compute the patch sets $S_{tri}{}^{\left(k\right)}, \forall i$ based on the $t_r$-th frame ${\mathit{x}}_{tr}{}^{\left(k\right)}$ using (7).                         $⊳𝓛-\mathrm{subproblem}$

4:  Construct the nonlocal featured tensor ${𝓧}_i{}^{\left(k+1\right)},\forall i$ by (8) and (9). 5:  Approximate the low-rank tensor ${𝓛}_i{}^{\left(k+1\right)},\forall i$ by adopting t-SVT method in (13) and (21).

6:  Construct $\mathit{\Omega }$ by updating the differential vector ${\mathit{\omega }}_t{}^{\left(k+1\right)},\forall t$ via (22).                         $⊳\mathit{\omega }-\mathrm{subproblem}$ 7:  Update the Lagrangian multiplier ${\mathit{v}}_t{}^{\left(k+1\right)},\forall t$ via (23).

8:  Repeat:                   $⊳\mathit{X}-\mathrm{subproblem}$

9:    E-step: Introduce the expectation variable ${\widehat{c}}_{tqj}$ and construct the $\mathit{X}-$relevant objective function $\mathsf{\Psi }\left(\mathit{X}\right)$ in (24).

10:    M-step: Update $x_{tj}^{\left(k+1\right)},\forall t,j$ using (25) and (26).

11:  Until: Inner Relative change $\frac{({\mathit{X}}^{\left(k+1\right)}-{\mathit{X}}^{\left(k\right)})}{{\mathit{X}}^{\left(k+1\right)}}<{10}^{-5}$

12:$k\leftarrow k+1$

13: Until: Relative change $\frac{({\mathit{X}}^{\left(k+1\right)}-{\mathit{X}}^{\left(k\right)})}{{\mathit{X}}^{\left(k+1\right)}}<{10}^{-6}$

14: Output: Reconstructed image sequence ${\mathit{X}}^{\left(k\right)}$.

3.1. Nonlocal Low Rank Tensor Approximation

The nonlocal tensor regularization consists of two parts: forming the nonlocal tensor within the recovered frames and formulating the low-rank property in the formed tensors.

3.1.1. Tensor Formulation

During the optimizing procedure, a temporary estimated image sequence $\mathit{X}=\left[{\mathit{x}}_1, {\mathit{x}}_2, \dots ,{\mathit{x}}_t,\dots {\mathit{x}}_T\right]\in {ℝ}^{N\times T}$ will be obtained after each iteration, where ${\mathit{x}}_t\in {ℝ}^N$ represents the t-th frame of image vector as mentioned in Section 2.1. Similar to Figure 1, numerous W $\times$ W sized overlapping patches ${\tilde{\mathit{x}}}_{ti}\in {ℝ}^n$ (n = W $\times$ W) can be extracted from each ${\mathit{x}}_t$, where i denotes the index of the image patch. According to the nonlocal self-similar properties, within the image, there are plentiful patches that share the same structure with each ${\tilde{\mathit{x}}}_{ti}$. Based on the Euclidean distance, we choose the m nearest patches for each ${\tilde{\mathit{x}}}_{ti}$:

$$S_{ti}=\{s|\Vert {\tilde{\mathit{x}}}_{ti}-{\tilde{\mathit{x}}}_{ti,s}{\Vert }_2<{\rho }_{ti}\},$$

(7)

where $S_{ti}$ is the index set of similar patches for the i-th positioned patch ${\tilde{\mathit{x}}}_{ti}$; and ${\rho }_{ti}$ is the threshold value defined by the distance between ${\tilde{\mathit{x}}}_{ti}$ and its m-th nearest patch. Thus, for each exemplar ${\tilde{\mathit{x}}}_{ti}$, we formulate a matrix:

$${\mathit{X}}_{ti}=\left[{\tilde{\mathit{x}}}_{ti},{\tilde{\mathit{x}}}_{ti,1},\dots ,{\tilde{\mathit{x}}}_{ti,s},\dots ,{\tilde{\mathit{x}}}_{ti,m-1}\right],{\mathit{X}}_{ti}\in {ℝ}^{n \times m}.$$

(8)

As shown in Figure 1, due to the nonlocal self-similarity, it is fairly assumed that each ${\mathit{X}}_{ti}$ is low-rank.

From a dynamic recovery perspective, sets $S_{ti}$ for $t=1,2,\dots ,T$ are identical, since the structure in dynamic PET is unchanged. Therefore, based on the patch position i, we can construct a 3D tensor ${𝓧}_i\in {ℝ}^{n \times m \times T}$, whose frontal slices are calculated as

$${𝓧}_i\left(:,:,t\right)={\mathit{X}}_{ti}, t=1,2,\dots ,T.$$

(9)

Figure 3 illustrates the overall procedure for forming a nonlocal tensor. Within each iteration, numerous ${𝓧}_i$s for various i positions will be constructed and then approximated by a low-rank property.

3.1.2. Low Rank Tensor Approximation

The next step is the low rank approximation for constructed tensors ${𝓧}_i,\forall i$. Traditionally, the primal model of rank regularization searches for the tensor ${𝓛}_i$, where

$${𝓛}_i=\underset{{𝓛}_i}{\arg }minrank\left({𝓛}_i\right)\mathrm{s}.\mathrm{t}.{𝓛}_i={𝓧}_i.$$

(10)

Nevertheless, this model is non-deterministic polynomial-time (NP) hard, and its direct optimization is nonconvex. Taking this situation into account, we choose a surrogate form of Equation (10) and turn it into a convex optimization issue:

$${𝓛}_i=\arg \underset{{𝓛}_i}{\min }\frac{1}{2}\Vert {𝓛}_i-{𝓧}_i{\Vert }_F^2+\lambda \Vert {𝓛}_i{\Vert }_{*}.$$

(11)

Here, ${\Vert \cdot \Vert }_{*}$ denotes the tensor nuclear norm [39] and ${\Vert \cdot \Vert }_F$ denotes the Frobenius norm. Under this circumstance, it is feasible to get a closed-form solution by adopting tensor singular value thresholding (t-SVT) [39]:

$${𝓛}_i={𝓤}_i*{\mathcal{D}}_{\lambda }\left({𝓢}_i\right)*{𝓥}_i{}^T,$$

(12)

with ${𝓧}_i={𝓤}_i*{𝓢}_i*{𝓥}_i{}^T$ representing the mentioned t-SVD of ${𝓧}_i$ and

$${\mathcal{D}}_{\lambda }({𝓢}_i)=IFF{T}_{\left(3\right)}({({\overline{𝓢}}_i-\lambda )}_{+}),$$

(13)

where $(X{)}_{+}=\max (X,0)$; $IFF{T}_{\left(3\right)}\left(X\right)$ denotes the fast Inverse Fast Fourier Transform (IFFT) of X across dimension 3; and ${\overline{𝓢}}_i=FF{T}_{\left(3\right)}\left({𝓢}_i\right)$ denotes the Fast Fourier Transform (FFT) of ${𝓢}_i$ across dimension 3.

3.2. Total Variation Regularization in Dynamic PET

Apart from nonlocal low-rank tensor regularization, we also incorporate the total variation [40] (TV) as a complementary constraint into the dynamic PET reconstruction framework. Unlike the nonlocal tensor, the adopted TV regularization focuses on inherited local information within each frame. Moreover, this pixel-based regularization compromises patch-based regularization and thus improves the denoising performance of the proposed algorithm.

For each image frame ${\mathit{x}}_t\in {ℝ}^N$ in the estimated sequence $\mathit{X}=\left[{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_t,\dots {\mathit{x}}_T\right]\in {ℝ}^{N \times T}$, we adopt an l₂ formed TV regularization, which is properly formulated in accordance with augmented Lagrangian optimization [41,42]:

$$\underset{x_{tj}}{min}\sum _j\Vert {\omega }_{tj}{\Vert }_2\mathrm{s}.\mathrm{t}.D_j{\mathit{x}}_t={\omega }_{tj}\mathrm{for}\mathrm{all}j,$$

(14)

where ${\omega }_{tj}=D_j{\mathit{x}}_t$ denotes the discrete gradient of ${\mathit{x}}_t$ at position j; and $D_j$ denotes the j-th element of the corresponding differential operator $\mathit{D}$. Correspondingly, the augmented Lagrangian function can be written as:

$$\underset{\mathit{\Omega }}{\min }\mathrm{DTV}\left(\mathit{\Omega }|\mathit{X}\right) =\sum _t^T\Vert {\mathit{\omega }}_t\Vert +\frac{\eta }{2}\Vert {\mathit{\omega }}_t-\mathit{D}{\mathit{x}}_t{\Vert }_2^2-{\mathit{\nu }}_t\left({\mathit{\omega }}_t-\mathit{D}{\mathit{x}}_t\right),$$

(15)

where $\mathit{\Omega }=\left[{\mathit{\omega }}_1,{\mathit{\omega }}_2,\dots ,{\mathit{\omega }}_T\right]\in {ℝ}^{N \times T}$ represents the collection of discrete gradient vectors ${\mathit{\omega }}_t=\mathit{D}{\mathit{x}}_t$ for each recovered frame; $\eta$ represents the tunable weighting parameters of the quadratic term; and ${\mathit{\nu }}_t$ represents the updatable multiplier vector. Consequently, this method guarantees the convexity of TV regularization, and hence equips the proposed method with global convergence.

3.3. Expectation Maximization for Fidelity Term

Indisputably, solving the fidelity term Equation (4), to a large extent, is an essential mission in estimating the reconstruction images $X$. Regardless of regularization, Equation (4) can be further written as:

$$\underset{\mathit{X}}{\min }\mathrm{P}\left(\mathit{X}\right)=\underset{\mathit{X}}{\min }\sum _{t=1}^T\sum _j^N\sum _q^M\left(g_{qj}{x}_{tj}-c_{tqj}\log (g_{qj}{x}_{tj}\right)) \mathrm{s}.\mathrm{t}.y_{tq}=\sum _q{c}_{tqj},$$

(16)

where $x_{tj}$ denotes the j-th pixel in image ${\mathit{x}}_t$; $g_{qj}$ is the qj-th entry in system matrix G, representing the contribution of the j-th pixel given to the q-th detector; and the hidden variable $c_{tqj}$ represents the photon count from the j-th pixel to the q-th detector pair in the t-th frame.

The main challenge lies in the solving procedure, especially handling the hidden variable $c_{tqj}$. In this work, we adopt the well-known expectation maximization (EM) [5,43], which introduces ‘complete data’ into the model and thus facilitates optimization. In order to solve Equation (18), there are two essential steps:

• E-step: This step employs the expectation ${\widehat{c}}_{tqj}$ = $E\left(c_{tqj}|{\mathit{x}}_t,y_{tq}\right)$ as the substitute for the hidden variable $c_{tqj}$ in Equation (18):

  $${\widehat{c}}_{tqj}=\frac{g_{qj}{x}_{tj}}{{\sum }_j^N{g}_{qj}{x}_{tj}+r_{tq}+s_{tq}}{y}_{tq .}$$

  (17)
• M-step: This step maximizes the likelihood by zeroing the derivative of Equation (19):

  $$\frac{\partial \mathrm{P}\left(\mathit{X}\right)}{\partial x_{tj}}=0.$$

  (18)

The EM algorithm, which will be illustrated in greater detail in the next section, makes our proposed framework readily and efficiently solvable.

3.4. The Overall Optimization Framework

Based on Equation (5), the overall reconstruction model can be represented as

$$\underset{\mathit{X}}{\min }\mathsf{\Psi }\left(\mathit{X}\right)=\underset{\mathit{X}}{\min }\alpha \mathrm{TNL}\left(\mathit{X}\right)+\beta \mathrm{DTV}\left(\mathit{X}\right)+\mathrm{P}\left(\mathit{X}\right),$$

(19)

where $\mathrm{P}\left(\mathit{X}\right)$ is the fidelity term in the reconstruction model Equation (16); $\mathrm{TNL}\left(\mathit{X}\right)$ represents the tensor formed nonlocal low rank constraint in Section 3.1; $\mathrm{DTV}\left(\mathit{X}\right)$ represents the dynamic PET adopted total variation term in Section 3.2; and $\alpha$ and $\beta$ denote the weighting parameters. By taking the mentioned terms into account, the objective function can be formulated as:

$$\begin{gathered} \mathsf{\Psi }\left(\mathit{X},𝓛,\mathit{\omega }\right)=\alpha ({\sum }_i\frac{1}{2}\Vert {𝓛}_i-{𝓧}_i{\Vert }_F^2+\lambda \Vert {𝓛}_i{\Vert }_{*})+\beta ({\sum }_t^T\Vert {\mathit{\omega }}_t\Vert \\ +\frac{\eta }{2}\Vert {\mathit{\omega }}_t-\mathit{D}{\mathit{x}}_t{\Vert }_2^2-{\mathit{\nu }}_t\left({\mathit{\omega }}_t-\mathit{D}{\mathit{x}}_t\right))+{\sum }_t^T{\sum }_q^M{\overline{y}}_{tq}-y_{tq}\log ({\overline{y}}_{tq}), \\ s.t.{\overline{\mathit{y}}}_t=\mathit{G}{\mathit{x}}_t+{\mathit{r}}_t+{\mathit{s}}_t. \end{gathered}$$

(20)

Normally, recovering $\mathit{X}=\left[{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_t,\dots {\mathit{x}}_T\right]\in {ℝ}^{N\times T}$ directly from Equation (20) is a complex process. For this process, we refer to the alternative direction method of multipliers (ADMM) [30] and divide the model into three subproblems of $𝓛, \mathit{\omega }$ and $\mathit{X}$ in a distributed optimization way.

(1) 

$𝓛$-subproblem. Let ${(\cdot )}^{\left(k\right)}$ denote the updated variable after the k-th iteration; e.g., ${\mathit{X}}^{\left(k\right)}$ represents the computed image sequence after the k-th iteration. In the (k + 1)-th iteration, nonlocal low-rank approximation is first implemented. After the formulation of nonlocal feature tensors ${\mathcal{X}}_i{}^{\left(k+1\right)},\forall i$, as illustrated in Section 3.1, we can obtain the function related to each low rank tensor:

$${𝓛}_i{}^{\left(k+1\right)}={𝓤}_i{}^{\left(k+1\right)}*{\mathcal{D}}_{\lambda }({𝓢}_i{}^{\left(k+1\right)})*{𝓥}_i^{\left(k+1\right)}{}^T,$$

(21)

with ${𝓧}_i{}^{\left(k+1\right)}={𝓤}_i{}^{\left(k+1\right)}*{𝓢}_i{}^{\left(k+1\right)}*{𝓥}_i^{\left(k+1\right)}{}^T$

(2) 

$\mathit{\omega }$-subproblem. Unlike updating ${𝓛}_i$, the discrete gradient is updated frame by frame. As shown in Equation (15), we update $\mathit{\Omega }=\left[{\mathit{\omega }}_1,{\mathit{\omega }}_2,\dots ,{\mathit{\omega }}_T\right]\in {ℝ}^{N\times T}$ by employing the shrinkage operator [44]:

$${\mathit{\omega }}_t{}^{\left(k+1\right)}=\mathit{max}\left\{\Vert \mathit{D}{\mathit{x}}_t{}^{\left(k\right)}-\frac{{\mathit{\nu }}_t{}^{\left(k\right)}}{\eta }{\Vert }_2-\frac{1}{\eta },0\right\}\frac{\mathit{D}{\mathit{x}}_t{}^{\left(k\right)}-{\mathit{\nu }}_t{}^{\left(k\right)}/\eta }{\Vert \mathit{D}{\mathit{x}}_t{}^{\left(k\right)}-{\mathit{\nu }}_t{}^{\left(k\right)}/\eta {\Vert }_2} .$$

(22)

Correspondingly, the multiplier vector updates by:

$${\mathit{v}}_t{}^{\left(k+1\right)}={\mathit{v}}_t{}^{\left(k\right)}-\eta (\mathit{D}{\mathit{x}}_t{}^{\left(k\right)}-{\mathit{\omega }}_t{}^{\left(k+1\right)}).$$

(23)

(3) 

$\mathit{X}$-subproblem. After the update of $𝓛$ and $\mathit{\omega }$, the last critical process is to update $\mathit{X}=\left[{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_t,\dots {\mathit{x}}_T\right]\in {ℝ}^{N\times T}$. In addition to the fidelity term in Equation (18), the former mentioned regularizations must be considered. In this procedure, we adopt the EM algorithm and hence reform Equation (20) into a joint function relevant to $\mathit{X}$ (or $x_{tj}$):

$$\begin{gathered} \mathsf{\Psi }\left(\mathit{X}\right)=\sum _t^T\sum _j^N\sum _q^M\left(g_{qj}{x}_{tj}-{\widehat{c}}_{tqj}\log (g_{qj}{x}_{tj}\right))+\alpha \sum _i\sum _t^T\sum _j^N\frac{1}{2}\Vert {𝓛}_i{}_{\left(t\right)}^{\left(k+1\right)}-{\mathit{\Gamma }}_{itj}{x}_{tj}{\Vert }_F^2 \\ +\beta \sum _t^T\sum _j^N\frac{\eta }{2}{({\mathit{D}}_j{x}_{tj}-{\omega }_{tj}{}^{\left(k+1\right)})}^2+{\nu }_{tj}{}^{\left(k+1\right)}\left({\omega }_{tj}{}^{\left(k+1\right)}-{\mathit{D}}_j{x}_{tj}\right) \\ s.t.{\widehat{c}}_{tqj}=\frac{g_{qj}{x}_{tj}{}^{\left(k\right)}}{{\sum }_j^N{g}_{qj}{x}_{tj}{}^{\left(k\right)}+r_{tq}+s_{tq}}{y}_{tq}. \end{gathered}$$

(24)

Here, ${𝓛}_{i\left(t\right)}$ (or ${𝓛}_i\left(:,:,t\right)$) denotes the t-th frontal slice of tensor ${𝓛}_i$; ${\mathit{\Gamma }}_{itj}$ denotes the contribution weight for pixel ${\mathit{x}}_{tj}$ to tensor ${𝓧}_i$; ${\omega }_{tj}$ denotes the j-th element of ${\mathit{\omega }}_t$, and $D_j$ denotes the j-th column of the differential operator $\mathit{D}$; and ${\nu }_{tj}$ is the j-th element of multiplier ${\mathit{v}}_t$.

According to the M-step in Equation (18), we can harvest a unitary quadratic equation:

$$\begin{gathered} A_{tj}{(x_j)}^2+B_{tj}{x}_j+C_{tj}=0 \\ s.t.{\mathrm{A}}_{tj}=\alpha \sum _i\sum _t^T{\mathit{\Gamma }}_{itj}^T{\mathit{\Gamma }}_{itj}+\beta \eta \sum _t^T{\mathit{D}}_j^T{\mathit{D}}_j , C_{tj}=-\sum _t^T\sum _q^M{\widehat{c}}_{tqj} \\ {\mathrm{B}}_{tj}=\sum _t^T\sum _q^M{g}_{qj}-\alpha \sum _i\sum _t^T{\mathit{\Gamma }}_{itj}^T{𝓛}_i{}_{\left(t\right)}^{\left(k+1\right)} -\sum _t^T\beta \eta {\mathit{D}}_j^T{\omega }_{tj}{}^{\left(k+1\right)}-{\mathit{D}}_j^T{\nu }_{tj}{}^{\left(k+1\right)} . \end{gathered}$$

(25)

Therefore, $x_{tj}{}^{\left(k+1\right)}$ can be readily solved as the positive root of (25):

$$x_{tj}^{\left(k+1\right)}=\frac{(-B_{tj}+\sqrt{B_{tj}{}^2-4A_{tj}{C}_{tj}})}{2A_{tj}}.$$

(26)

In this X sub-problem, given that we do not have a close-formed solution for the PET reconstruction model [5], the X-subproblem is not fully solved by the employment of EM. However, due to the implementation of ADMM based optimization, the overall framework will converge, even if its sub-problems are not carried out exactly [30].

Algorithm 1 demonstrates the overall procedure of our proposed algorithm. It is noteworthy that, in the initialization step, we employed the filtered backward projection (FBP) [3] method to produce a ‘warm start’. By doing so, as we will show in the next section, iterative performance is notably improved.

4. Experiments and Results

In this section, we perform various experiments, in order to validate the qualitative and quantitative measures of the proposed method. Data on diversified photon counts, sizes, tracers, and structures are recovered by our proposed method and compared with the results of representative and state-of-the-art algorithms.

4.1. Implementations

4.1.1. Evaluation Criteria

For the qualitative evaluation, randomly selected reconstructions are shown in this part, where the structural detail, noise level, and so on will be intuitively illustrated. For the quantitative evaluation, other than the peak signal-to-noise ratio (PSNR)), we also employ the relative bias and variance [45] as the indicators of the resolution and smoothness, respectively:

$$\mathrm{Bias}=(\frac{1}{N_t})\sum _j^{N_t}\frac{|x_j-{\widehat{x}}_j|}{{\widehat{x}}_j},$$

(27)

$$\mathrm{Variance}=(\frac{1}{(N_t-1)})\sum _j^{N_t}{(\frac{|x_j-\overline{x}|}{x_j})}^2,$$

(28)

where ${\widehat{x}}_j$ denotes the ground truth in the j-th pixel; $\overline{x}$ denotes the mean value of the ROI; and $N_t$ denotes the total number of pixels in the given ROI. Unlike the PSNR, the smaller the relative bias and variance are, the better the reconstruction is.

We also conduct the multiple simulations experiment. In this study, we employ the contrast recovery coefficient (CRC) and the standard deviation (STD) [16,46]:

$$\mathrm{CRC}=\frac{1}{R}\sum _{r=1}^R\frac{|{\overline{S}}_r-{\overline{B}}_r|}{{\overline{B}}_r},$$

(29)

$$\mathrm{STD}=\frac{1}{N_B}\sum _{j=1}^{N_B}\frac{\sqrt{\frac{1}{R-1}{\sum }_{r=1}^R(B_{r,j}-{\overline{B}}_j)}}{{\overline{B}}_j}.$$

(30)

Here, $R=50$ represents the number of realizations in the simulation. In Equation (29), ${\overline{S}}_r$ represents the mean value of the ROI in r-th realization and ${\overline{B}}_r$ represents the mean value of the background region in r-th realization. In Equation (30), $N_B$ denotes the total number of pixels in the background region; ${\overline{B}}_j=\left(1/R\right){\sum }_{r=1}^R{B}_{r,j}$ denotes the mean value of j-th pixel in background region across R realizations.

For the real data, we adopt the contrast to noise ratio (CNR) [47]:

$$CNR=\left(m_{ROI}-m_{background}\right)/S{D}_{background},$$

(31)

where the $m_{ROI}$ and $m_{background}$ represent the intensity of the ROI and background region respectively, and $S{D}_{background}$ is the standard deviation of the background region.

4.1.2. Dataset

As shown in Figure 4, we mainly adopt a 64 $\times$ 64 sized Zubal brain phantom as the template and employ the 11C-dihydrotetrabenazine (denoted as DTBZ) as the tracer. The scanning procedure in simulated into 18 frames for a duration of 20 min with the corresponding TAC presented in Figure 1. Moreover, to validate the performance of the algorithms under a diversified tracer dose, the data are generated in 3 $\times$ 10⁶, 10⁷ and 3 $\times$ 10⁷ total photon counts over 18 frames. Furthermore, in the multiple realizations experiment, we generate total 100 realizations for 3 $\times$ 10⁶ and 3 $\times$ 10⁷ counted data (R = 50 for each simulation). In addition, all the simulated settings correspond to real cases.

Furthermore, 111 $\times$ 111 sized Zubal head phantoms [16] are recovered to validate the effectiveness of the proposed method on different tracers (18F-FDG) and image sizes. Moreover, real cardiac data are tested in this section. The data are scanned over 60 min by a Hamamatsu SHR-22000 (Hamamatsu Photonics K.K., Hamamatsu City, Japan). There are, overall, 19 frames, and each are scanned by 130 detector pairs from 192 angles.

4.1.3. Comparative Algorithms

To evaluate the performance of the proposed algorithm, we introduce five representative algorithms in comparison (the maximum likelihood-expectation maximization (ML–EM) algorithm [5], the penalized weighted least square (PWLS) method [8], the total variation optimized by augmented Lagrangian (TV-AL) method [48], the penalized likelihood incremental optimization method regularized by hyperbolic potential function [45,49] (denoted as PLH-IO), and the spatial-temporal total variation (ST-TV) method [50]) proposed for dynamic PET reconstruction.

4.1.4. Parameters Setting

After deliberate examination, we set the weighting parameters as follows: the nonlocal tensor weight parameter $\alpha =1.7$; TV weight parameter $\beta =0.9$; tensor thresholding parameter $\lambda =2.5$; and and parameter $\eta =50$. Another critical issue is the tensors’ sizes. As shown in Figure 5, the optimal patch size is 3 $\times$ 3. Thus, if the number of feature patches is set to 10 in the 18-frame sequence, each selected tensor’s size is 9 $\times$ 10 $\times$ 18. We set the maximum iteration to 500 for all methods in comparison.

4.1.5. Experiment Description

We evaluate our method both qualitatively and quantitatively on simulation and real PET data. We firstly compare the reconstructions for 64 $\times$ 64 sized Zubal brain data under 3 $\times$ 10⁷ total photon counts, and demonstrate 5th, 11th, and 17th frame in Figure 6. In Figure 7, we further compare the 11th reconstructed frames under lower-counted sequences: 3 $\times$ 10⁶ and 1 $\times$ 10⁷ photon counts. In addition, we recover the 111 $\times$ 111 sized Zubal head phantom to validate the performance under different sizes and TACs, as shown in Figure 8. For the real patient study, we demonstrate the first frame of dynamic cardiac PET reconstructions in Figure 9, and compute the CNR for each method.

In the quantitative evaluations, we firstly present the PSNR, relative bias and variance for data under 3 $\times$ 10⁷, 3 $\times$ 10⁶ and 1 $\times$ 10⁷ photon counts in

Table 1. Average value of statistical evaluation terms for reconstructed Brain Phantom sequences under different photon counts.

Algorithm	PSNR (dB)			Relative Bias			Relative Variance
	3 × 106	1 × 107	3 × 107	3 × 106	1 × 107	3 × 107	3 × 106	1 × 107	3 × 107
ML-EM	13.00	14.39	15.75	0.2374	0.2024	0.1742	0.0252	0.0147	0.0138
PWLS	15.20	16.25	16.81	0.2054	0.1747	0.1627	0.0217	0.0162	0.0154
TV-AL	15.91	17.13	18.76	0.1804	0.1548	0.1286	0.0303	0.0161	0.0152
PLH-IO	16.48	17.92	19.87	0.1883	0.1539	0.1225	0.0205	0.0131	0.0115
ST-TV	16.85	19.07	20.21	0.1725	0.1359	0.1139	0.0161	0.0125	0.0097
Ours	17.29	19.54	21.03	0.1609	0.1172	0.0983	0.0148	0.0111	0.0083

; furthermore, we compute the relative bias and variance in each ROI for 3 $\times$ 10⁶ counted data and demonstrate them in

Table 2. Average value of relative bias and variance in each region of interest (ROI). 3 × 10⁶ photon counted data are tested and shown.

Algorithm	Relative Bias					Relative Variance
	Whole	ROI1	ROI2	ROI3	ROI4	Whole	ROI1	ROI2	ROI3	ROI4
ML-EM	0.2374	0.2353	0.3909	0.4407	0.2654	0.0252	0.0260	0.0105	0.0292	0.0410
PWLS	0.2054	0.1955	0.1853	0.2381	0.1801	0.0217	0.0377	0.0054	0.0176	0.0217
TV-AL	0.1804	0.1905	0.1758	0.2114	0.1522	0.0303	0.0464	0.0056	0.0215	0.0303
PLH-IO	0.1883	0.2601	0.1560	0.1680	0.2077	0.0205	0.0285	0.0035	0.0205	0.0205
ST-TV	0.1725	0.2001	0.1388	0.1700	0.1791	0.0161	0.0210	0.0042	0.0151	0.0190
Ours	0.1609	0.1864	0.1235	0.1591	0.1671	0.0148	0.0203	0.0035	0.0118	0.0186

. In Figure 10a–c, we compute the PSNR, relative bias, and variance for each frame in 1 $\times$ 10⁷ counted data. In Figure 10d, we demonstrate the performance of the trade-off between resolution and smoothness, by altering the parameters in each algorithm and plotting the bias and variance in each reconstruction.

Moreover, we conduct the experiment on multiple simulations. In our experiment, we run 100 realizations for 3 $\times$ 10⁷ and 3 $\times$ 10⁶ counted data, and draw the CRC-STD curves, as shown in Figure 11a–b. Accordingly, the CRC is computed from ROI 2, and the background STD is computed from ROI 4, which represents the white matter. In addition, we run the Wilcoxon rank sum test on the multiple realization test, as demonstrated in Figure 11c–d. In addition, we further examine the universality of the proposed method by reconstructing data under wider range of photon counts, as demonstrated in Figure 11a, from 1 $\times$ 10⁶ to 1 $\times$ 10⁸. Further discussions on convergence are illustrated in Figure 11b and the discussion section.

4.2. Results

4.2.1. Qualitative Evaluation

The initial experiment focuses on the resolution and the denoising performance throughout the temporal dimension. According to the TAC in Figure 4, the distinction of activity between different ROIs is inconspicuous in the early imaging stage, which consequently hampers the recovery of the structural information in the corresponding image frames. As demonstrated in Figure 6, the reconstructions of ML-EM and PWLS suffer from severe iterative noise and fail to recover a clear boundary between regions. On the other hand, the TV-AL and PLH-IO show more acceptable results for the 17th frame. However, when it comes to former frames, neither of these two methods are able to recover clear structures. Moreover, the TV-AL suffers from the staircase effect and artifacts, and PLH-IO tends to over-smooth the image. Although ST-TV improves the resolution by incorporating temporal information, it is still limited by recovering more detailed structures. In contrast, our proposed method manages to recover more detailed structures and less noise in reconstructing the brain phantom sequence under $3\times {10}^7$ photon counts. This contrast is more distinctive in simulated low-dose images. As we can see in Figure 7, when recovering early frames in the low-count data, our proposed method is able to recover substantially clearer structures than those of other methods under similar noise levels.

Meanwhile, our method also shows its universality in recovering sequences under different sizes and TACs. In this experiment, 111 $\times$ 111 sized Zubal head phantom data were tested in an 18F-FDG environment. In Figure 8, the 15th frame is randomly selected out of 24 frames. In addition, real patient data are tested. In Figure 9, the second sequence is shown, and the photon counts are around 2.2 $\times$ 10⁵. Obviously, our proposed method yields clearer boundaries and more conspicuous contrast between ROIs and the background.

4.2.2. Quantitative Evaluation

The quantitative measurements are also meticulously implemented. Table 1 demonstrates the average statistical values for the dynamic image sequences under diversified photon counts. According to the table, the proposed method enjoys a higher PSNR and lower relative bias and variance, revealing solid and substantial merits in structural enhancement, resolution improvement, and image denoising. Table 2 provides more detailed measurements for each ROI reconstruction. As shown in the table, the proposed method reconstructs images at a lower bias and variance in each ROI to vary the count level, which demonstrates better resolution and smoothness than those of comparable methods.

Other than spatial information, temporal trends are also considered in Figure 10, where Figure 10a–c presents the PSNR, bias, and variance for each frame. It can be easily observed that the proposed method, overall, has better results for each frame, which will facilitate the exploitation of temporal information. In addition, given the fact that regular reconstruction methods are largely based on the trade-off between the resolution and the noise level, we also implemented an experiment of this trade-off, correspondingly represented by the relative bias and the relative variance in Figure 10d. As we can see, apart from the relatively poorer performances of ML-EM and PWLS, both TV-AL and PLH-IO show a negative correlation between these two indexes. In contrast, the marks of our proposed method are densely concentrated in the bottom-left of this graph, which shows a better image quality and better compromise for the mentioned trade-off.

To better validate the proposed method, we further implement the experiments on multiple realizations. As demonstrated in Figure 11a–b, we run the experiments under high-count and low-count scenarios (3 $\times$ 10⁷ and 3 $\times$ 10⁶ counted data) and draw the correspondent CRC-STD curves, which illustrate the performance of compared methods. In these curves, each point corresponds to a certain setting for parameters in the relative method. According to the figures, the proposed method manages to recover higher CRC while keeping the background STD in a low level, which validates the stability of our method in reducing the noise while keeping the contrast distinctive. Moreover, we analyze the statistical performance of the Wilcoxon rank sum test on the multiple realizations data. As shown in Figure 11c–d, we compute the p-value between the proposed method and compared methods, in terms of PSNR on multi-simulation. In this study, the proposed method significantly outperforms the ST-TV in 3 $\times$ 10⁶ dataset, at p-value < 0.01; more distinctively, the proposed method outperforms other methods in 3 $\times$ 10⁶ dataset and all methods in 3 $\times$ 10⁷ dataset, at a p-value < 0.001.

4.2.3. Robustness and Convergence Analysis

The robustness and convergence experiments are demonstrated in this section. As we can see from Figure 12a, the proposed method presents better performance and robustness than other methods under a broad range of photon counts.

For the convergence, Figure 12b demonstrates the PSNR for iterations of all tested methods. Specifically, we individually test our methods with and without a warmstart, which is mentioned in Section 3. Our warmstart-equipped method surpasses other methods in its convergence performance.

5. Discussion

Other than merely denoising, the proposed method simultaneously provides enhancement and completion of structural sparsity by introducing a 3D tensor based nonlocal low-rank constraint. Unlike the tracer kinetics based dynamic reconstruction method, the proposed method spontaneously exploits the inner temporal correlation within the sequence without the need for tracer information and model fitting. In fact, our proposed method not only managed to suppress noise while recovering at a high resolution, it also enhanced, and even completed, the structural information in the reconstruction sequence.

This result is attributed to the tensor based low rank approximation. As presented in Figure 3, the third dimension of the ${𝓧}_i\in {ℝ}^{n\times m\times T}$ exists alongside the temporal information. By implementing the tensor based low rank constraint on each ${𝓧}_i$, the spatial information is spontaneously infiltrated from high-counted frames to low-counted frames, while keeping the voxels’ relative intensities and edge arrangements fixed. Considering this feature, a dynamic PET sequence is considered ideal for this framework, given the unchanged boundary and structures along the dynamic sequences. On the other hand, since noise is randomly and sparsely arranged in the PET sequence, it is ruled out as the sparse component in the low rank approximation.

Furthermore, we demonstrate the contribution of different regularization components in Figure 13 by setting various hyper-parameters. According to the figure, tensor constraints can successfully recover detailed structures but are slightly limited in smoothing an image under low counts. Fortunately, the employment of a TV constraint compensates for this issue, as demonstrated in Figure 13c.

Nevertheless, there are still several concerns in this study. Firstly, for the data in unwilling but conspicuous motion, the proposed method is limited in harvesting temporal correlation, though other dynamic PET algorithms also suffer from this issue, to the best of our knowledge. Currently, the optimal solution is to conduct motion correction before reconstruction. Secondly, since 3D structural nonlocal features are not well proven in the computer vision community, this method is not guaranteed to function well in 3D PET reconstruction. However, we still provide two solutions for applying the proposed method in 3D data at the current stage: (1) Conduct the method slice by slice; and (2) rebin the data into a 2D form before reconstruction.

The last issue focuses on the computational cost. For simulated data in Figure 4, the computational time for each method is demonstrated in

Table 3. Computational time for each method.

Method	ML-EM	PWLS	TV-AL	PLH-IO	ST-TV	Ours
Computational time (s/iteration)	0.04375	0.3163	0.07935	0.3507	0.1639	4.379

. Here, the computational experiments are implemented under Matlab R2014a (Mathworks, Natick, MA., USA), on the same desktop with an Intel Core i7-4720HQ CPU (Santa Clara, CA, USA) @2.60 GHz and 8 GB RAM. We have to concede that, compared with the traditional pixel-based algorithms, the computational cost is inevitable in the proposed method, due to the multiple decompositions for feature tensors generated by feature cubes (or patches in 2D situation [23,24]). To address this issue, we will continue optimizing the proposed algorithm and employing other algorithms, such as TCTF [51].

In addition, the choice for the tensor decomposition model is still open in our future work. The T-SVD based method is proved effective in our work and [34,35,36], yet, strictly speaking, its tubal based rank is the analogous rank extended from SVD. In our future work, we will further analyze the data-structures, explore the feasibilities of other potential models, i.e., CP and Tucker rank [31], and testify the applicabilities of the latest proposed CP rank based methods [52,53] as well as Tucker rank based methods [54,55]).

6. Conclusions

In this paper, we provide a novel tensor based nonlocal low-rank framework for dynamic PET reconstruction. By introducing a nonlocal featured tensor and applying the t-SVT in low-rank tensor approximation, the image structures are efficiently enhanced while effectively depressing the noise. More significantly, structural information is further completed by other frames in an interactive way, thereby compromising the conflict between spatial and temporal resolution. On the other hand, accompanied by the TV term denoising (from a local and pixel-based perspective), the regularizations are firstly integrated in the Poisson reconstruction model and efficaciously optimized in a distributed framework.