Bayesian 3D X-ray Computed Tomography with a Hierarchical Prior Model for Sparsity in Haar Transform Domain

Abstract

In this paper, a hierarchical prior model based on the Haar transformation and an appropriate Bayesian computational method for X-ray CT reconstruction are presented. Given the piece-wise continuous property of the object, a multilevel Haar transformation is used to associate a sparse representation for the object. The sparse structure is enforced via a generalized Student-t distribution ($\mathcal{S}{t}_g$), expressed as the marginal of a normal-inverse Gamma distribution. The proposed model and corresponding algorithm are designed to adapt to specific 3D data sizes and to be used in both medical and industrial Non-Destructive Testing (NDT) applications. In the proposed Bayesian method, a hierarchical structured prior model is proposed, and the parameters are iteratively estimated. The initialization of the iterative algorithm uses the parameters of the prior distributions. A novel strategy for the initialization is presented and proven experimentally. We compare the proposed method with two state-of-the-art approaches, showing that our method has better reconstruction performance when fewer projections are considered and when projections are acquired from limited angles.

1. Introduction

Computed Tomography (CT) has been developed and widely used in medical diagnosis [1] and industrial Non-Destructive Testing (NDT) [2] in recent decades. In CT, objects are observed using different techniques, for example X-rays [3], ultrasound [4], microwaves [5], or infra-red [6]. X-ray CT employs the absorption of X-rays by the organs in a body or by the materials in industrial components to reconstruct the internal structure of the imaged object. When performing X-ray CT, a set of X-ray images of the measured parts is acquired. The intensity measured by the X-ray images corresponds to the intensity of the radiation passing through and attenuated by the object. CT reconstruction is typically treated as an inverse problem.

The conventional analytical techniques for CT reconstruction are based on the Radon transform [7]. Filtered Back-Projection (FBP) [8] is the most frequently-used analytical method in practical applications. FBP performs well when reconstructing from sufficient data with a high signal-to-noise ratio (SNR), but it suffers from artifacts when reconstructing from insufficient data or with noise.

Owing to considerations regarding patients’ health in medical CT and in order to reduce acquisition time in industrial applications, reconstruction with insufficient datasets is increasingly attracting the attention of researchers. Reconstruction from fewer projections is an ill-posed inverse problem [9,10]. In this case, conventional analytical reconstruction methods provide unsatisfactory results, and iterative methods can be used to improve the reconstruction performance. The Algebraic Reconstruction Technique (ART) [11,12], the Simultaneous Algebraic Reconstruction Technique (SART) [13], and the Simultaneous Iterative Reconstruction Technique (SIRT) [14,15] are some of the iterative methods proposed initially. These methods consider the discretized forward system model: $\mathit{g}=\mathit{H}\mathit{f}$, where $\mathit{f}\in {\mathbb{R}}^{N\times 1}$ represents the object, $\mathit{g}\in {\mathbb{R}}^{M\times 1}$ represents the observed dataset, and matrix $\mathit{H}\in {\mathbb{R}}^{M\times N}$ is the linear projection operator, mainly based on the geometry of acquisition (e.g., parallel beam, cone beam, etc.) [16,17,18]. Typically, the system of equations is under-determined, i.e., $N>M$. In this context, regularization methods are frequently used, and the forward system is modeled as:

$$\mathit{g}=\mathit{H}\mathit{f}+\mathit{ϵ},$$

(1)

where $\mathit{ϵ}\in {\mathbb{R}}^{M\times 1}$ represents the additive noise applied to the projection system. The regularization methods estimate the unknowns by minimizing a penalty criterion, which generally consists of two terms:

$$J\left(\mathit{f}\right)=Q(\mathit{g},\mathit{f})+\lambda R\left(\mathit{f}\right).$$

(2)

The loss function $Q(\mathit{g},\mathit{f})$ describes discrepancies in the observed data, such as the quadratic ($L2$) loss $Q(\mathit{g},\mathit{f})={∥\mathit{g}-\mathit{H}\mathit{f}∥}_2^2$ or $L_q$ loss $Q(\mathit{g},\mathit{f})={∥\mathit{g}-\mathit{H}\mathit{f}∥}_q^q$ with $1\le q<2$. Other expressions such as the Huber function are also reported. The regularization term $R\left(\mathit{f}\right)$ is a penalty on the complement criterion of $\mathit{f}$, such as restriction for smoothness ${∥\mathsf{\Phi }\left(\mathit{f}\right)∥}_2^2$ or for sparsity ${∥\mathsf{\Phi }\left(\mathit{f}\right)∥}_1$, where $\mathsf{\Phi }\left(\mathit{f}\right)$ represents a linear function of $\mathit{f}$. The parameter $\lambda$ is known as the regularization parameter, which controls the trade-off between the forward model discrepancy and the penalty term.

By choosing different regularization functions $R\left(\mathit{f}\right)$, different regularization methods can be implemented. $R\left(\mathit{f}\right)=0$ refers to the Least-Squares (LS) method [19], with the drawback that the reconstruction is sensitive to the noise due to the ill-posedness of the problem and the ill-conditioning of the operator $\mathit{H}$. Quadratic Regularization (QR), also known as the Tikhonov method [20], is given by $R\left(\mathit{f}\right)={∥\mathsf{\Phi }\left(\mathit{f}\right)∥}_2^2$, where the linear operator $\mathsf{\Phi }(·)$ is the derivation operator in most cases. The well-known Total Variation (TV) method [21,22,23] is defined by $R\left(\mathit{f}\right)={∥{\mathit{D}}_{TV}\mathit{f}∥}_{TV}$ where ${\mathit{D}}_{TV}$ is the gradient operator. ${\mathit{D}}_{TV}$ is equal to ${∥{\mathit{D}}_x\mathit{f}∥}_1+{∥{\mathit{D}}_y\mathit{f}∥}_1+{∥{\mathit{D}}_z\mathit{f}∥}_1$ for a 3D object in an anisotropic form, where ${\mathit{D}}_x$, ${\mathit{D}}_y$ and ${\mathit{D}}_z$ are respectively the gradient operators in the x, y and z directions. The $L_1$ norm is used in TV for sparse estimations, which enforces the sparsity of ${\mathit{D}}_{TV}\mathit{f}$. The appearance of the non-differentiable $L_1$ term leads to difficulties for the implementation of optimization algorithms. Many optimization algorithms have been proposed to solve this $L_1$ norm optimization problem, for example the primal-dual method [24], the split Bregman method [22], etc. In regularization optimization, due to the large projection data size and the great number of voxels, the explicit expression of the solution cannot be used directly because of the impossibility of inversing the large size matrix such as ${\left({\mathit{H}}^T\mathit{H}+\lambda {\mathit{D}}^T\mathit{D}\right)}^{-1}$. Hence, optimization algorithms such as gradient descent or conjugate gradient are often used.

More general regularization methods have been developed based on the constrained and dual-variable regularization method:

$$J(\mathit{f},\mathit{z})=Q_1(\mathit{g},\mathit{f})+\eta Q_2(\mathit{f},\mathit{z})+\lambda R\left(\mathit{z}\right),$$

(3)

which corresponds to the maximum a posterior optimization of a hierarchical structured model where both $\mathit{f}$ and $\mathit{z}$ are unknown variables. In such a model, the penalty regularization term is set on $\mathit{z}$, which is associated with $\mathit{f}$ via a linear transformation. The loss functions $Q_1(\mathit{g},\mathit{f})$ and $Q_2(\mathit{f},\mathit{z})$ are for example quadratic ($L2$), i.e., $Q_1(\mathit{g},\mathit{f})={∥\mathit{g}-\mathit{H}\mathit{f}∥}_2^2$ and $Q_2(\mathit{f},\mathit{z})={∥\mathit{f}-\mathit{D}\mathit{z}∥}_2^2$ where $\mathit{D}$ is a linear transform operator such as a wavelet transformation.

Among the methods treating this type of regularization problem, we mention here the Alternating Direction Method of Multipliers (ADMM) [25]. It minimizes $\mathsf{\Phi }\left(\mathit{f}\right)+\mathsf{\Psi }\left(\mathit{z}\right)$ subject to $\mathit{A}\mathit{f}+\mathit{B}\mathit{z}=\mathit{C}$, and it covers a large number of estimation forms. One example is when $\mathsf{\Phi }\left(\mathit{f}\right)={∥\mathit{g}-\mathit{H}\mathit{f}∥}^2$, $\mathsf{\Psi }\left(\mathit{z}\right)=\mathit{R}\left(\mathit{z}\right)$, $\mathit{A}=\mathit{I}$, $\mathit{B}=-\mathit{D}$, and $\mathit{C}=\mathbf{0}$ and refers to the above-mentioned bi-variable regularization method corresponding to Equation (3).

In the above-mentioned regularization methods, there is always a regularization parameter $\lambda$ to be fixed. Sometimes, the regularization term consists of more than one parts, and each of them are weighted by a regularization parameter, for example the elastic-net regularizer [26]. In these cases, two or even more regularization parameters need to be fixed. Cross Validation (CV) and the L-curve method [20,27,28] are conventional methods used to determine suitable values for these parameters. However, this work must be repeated for different simulated datasets and is therefore very costly. Statistical methods, therefore, have been developed and used to solve this problem.

From the probabilistic point of view, a Gaussian model for the additive noise in the forward model, Equation (1), leads to the quadratic expression ${∥\mathit{g}-\mathit{H}\mathit{f}∥}^2$ in the corresponding regularization criterion. However, in some types of tomography, for example Positron Emission Tomography (PET) or X-ray tomography with a very low number of phantoms, the noise is modeled by a Poisson distribution. In order to account for a more precise modeling of the noise and the other variables and parameters, statistical methods are used [29]. The Maximum Likelihood (ML) methods [30] and different estimation algorithms such as the Expectation Maximization (EM) algorithms [31], the Stochastic EM (SEM) [32], or the Ordered Subsets-EM (OS-EM) [33] are commonly used in PET-CT reconstruction problems.

Another widely-used type of probabilistic method for PET or X-ray CT reconstruction is the Bayesian inference [34,35,36,37]. The prior knowledge is translated by the prior probability model and is used to obtain the expression of the posterior distribution. The basic Bayesian formula is:

$$p(\mathit{f}|\mathit{g},\mathit{\theta })=\frac{p(\mathit{g}|\mathit{f},{\theta }_1)p(\mathit{f}|{\theta }_2)}{p(\mathit{g}|\mathit{\theta })},\mathrm{with}p(\mathit{g}|\mathit{\theta })=\int p(\mathit{g}|\mathit{f},{\theta }_1)p(\mathit{f}|{\theta }_2)\mathrm{d}\mathit{f},$$

(4)

where $p(\mathit{g}|\mathit{f},{\theta }_1)$ is the likelihood, $p(\mathit{f}|{\theta }_2)$ is the prior distribution, $p(\mathit{f}|\mathit{g},\mathit{\theta })$ is the posterior distribution, $\mathit{\theta }=\left({\theta }_1,{\theta }_2\right)$ are the parameters of these different distributions, and $p\left(\mathit{g}|\mathit{\theta }\right)$ is the evidence of the parameters in the data $\mathit{g}$. By using Maximum A Posterior (MAP) estimator $\widehat{\mathit{f}}={\arg \max }_{\mathit{f}}\left\{p(\mathit{f}|\mathit{g},\mathit{\theta })\right\}={\arg \min }_{\mathit{f}}\left\{-\ln p(\mathit{f}|\mathit{g},\mathit{\theta })\right\}$, links between the Bayesian method and almost all the regularization methods can be illustrated. A Gaussian prior for $p\left(\mathit{f}\right)$ in Equation (4) leads to the quadratic ($L2$) regularization method, while a Laplacian prior in Equation (4) leads to the $L1$ (LASSO or TV) regularization method. The regularization parameter can be related to ${\theta }_1$ and ${\theta }_2$. One advantage of the Bayesian method is having some explanation for the regularization parameter via its link with ${\theta }_1$ and ${\theta }_2$. For example, when $p(\mathit{g}|\mathit{f},{\theta }_1)$ and $p(\mathit{f}|{\theta }_2)$ are Gaussian with ${\theta }_1$ and ${\theta }_2$, respectively the variances of the noise and the variance of the prior, then the regularization parameter is $\lambda ={\theta }_1/{\theta }_2$. Another advantage of the Bayesian method is that these parameters can also be estimated to achieve unsupervised or semi-supervised methods. This is achieved by obtaining the expression of the joint posterior probability law:

$$p(\mathit{f},\mathit{\theta }|\mathit{g})=\frac{p(\mathit{g}|\mathit{f},{\theta }_1)p(\mathit{f}|{\theta }_2)p\left(\mathit{\theta }\right)}{p\left(\mathit{g}\right)},$$

(5)

where $p\left(\mathit{\theta }\right)$ is an appropriate prior on $\mathit{\theta }$. For a hierarchical structured model where a hidden variable $\mathit{z}$ appears in the prior model, we have:

$$p(\mathit{f},\mathit{z},\mathit{\theta }|\mathit{g})=\frac{p(\mathit{g}|\mathit{f},{\theta }_1)p(\mathit{f}|\mathit{z},{\theta }_2)p(\mathit{z}|{\theta }_3)p\left(\mathit{\theta }\right)}{p\left(\mathit{g}\right)},$$

(6)

where $\mathit{\theta }=\left[{\theta }_1,{\theta }_2,{\theta }_3\right]$.

With the posterior distribution obtained from an unsupervised Bayesian inference as in Equation (5), we distinguish three estimation methods. The first method consists of integrating out $\mathit{\theta }$ from $p(\mathit{f},\mathit{\theta }|\mathit{g})$ to obtain $p(\mathit{f}|\mathit{g})$ and then using $p(\mathit{f}|\mathit{g})$ to infer on $\mathit{f}$. The second approach is firstly to marginalize $p(\mathit{f},\mathit{\theta }|\mathit{g})$ with respect to $\mathit{f}$ to obtain $p(\mathit{\theta }|\mathit{g})=\int p(\mathit{f},\mathit{\theta }|\mathit{g})\mathrm{d}\mathit{f}$ and estimate $\widehat{\mathit{\theta }}={\arg \max }_{\mathit{\theta }}\left\{p(\mathit{\theta }|\mathit{g})\right\}$, then use $\widehat{\mathit{\theta }}$ as it was known. Unfortunately, these approaches do not often give explicit expressions for $p(\mathit{f}|\mathit{g})$ or $p(\mathit{\theta }|\mathit{g})$. The third and easiest algorithm to implement is the joint optimization, which estimates variable $\mathit{f}$ and parameter $\mathit{\theta }$ iteratively and alternately. Bayesian point estimators such as Joint Maximum A Posteriori (JMAP) [38] and Posterior Mean (PM) [39] via Variational Bayesian Approximation (VBA) methods [40,41,42] are often used.

In order to distinguish the details of a reconstructed object, a high-resolution image is expected. In industrial applications, especially for the NDT of a large-size object, the size of the projection (1000 images of ${1000}^2$ pixels) and the number of voxels (${1000}^3$ voxels) become critical, and so does the projection and back-projection operators in CT. It is necessary to account for some computational aspects, for example the GPU processor [43,44].

In our previous work [45], we proposed to use the Bayesian method via a synthesis model, in which the multilevel Haar transformation coefficient $\mathit{z}$ of the image is first estimated, and then, the final image reconstruction result is obtained from post-processing: $\widehat{\mathit{f}}=\mathit{D}\widehat{\mathit{z}}$. In this case, when using a Laplacian prior model and the MAP estimator, the problem becomes equivalent to the optimization of $J\left(\mathit{z}\right)={∥\mathit{g}-\mathit{H}\mathit{D}\mathit{z}∥}^2+\lambda {∥\mathit{z}∥}_1$, which is a typical $L_1$ regularization method. The particularity of our work was to use a generalized Student-t ($S{t}_g$) prior model [46] in place of the Laplacian model.

In this paper, we present a Hierarchical Haar transform-based Bayesian Method (HHBM), first proposed in [47], in which the object to be reconstructed, $\mathit{f}$, is related to the Haar transformation coefficient $\mathit{z}$ by $\mathit{f}=\mathit{D}\mathit{z}+\mathit{\xi }$ where $\mathit{\xi }$ represents the modelization uncertainties. $\mathit{f}$ and $\mathit{z}$ are estimated simultaneously. Wavelets provide an optimal representation for a piecewise continuous function consisting of homogeneous blocs separated by jump discontinuities (the contours), as the wavelet representation is sparse for such signals. The transformations used are, for example, the Haar transformation [48], the Curvelet Transformation (CVT) [49], the Contourlet Transformation (CT) [50], Dual-Tree Complex Wavelet Transform (DT-CWT) [51], etc. As long as the object under consideration, $\mathit{f}$, is piecewise continuous or constant, the Haar transform is appropriate, with the advantage that: (1) the transform coefficients are sparse; (2) the transformation operator is orthogonal so that the inverse operator and the transpose are identical; and (3) the computation of this transformation consists of only additions and subtractions, while the cost of computation is only $O\left(\sqrt{N}\right)$ where N is the size of the object $\mathit{f}$.

The sparsity of the transformation coefficient is generally defined by three classes of distributions: the generalized Gaussian distributions [52], the mixture distributions [53], and the heavy-tailed distributions [54]. In this paper, we use a generalization of the Student-t distribution ($S{t}_g$), which belongs to the heavy-tailed family and has many advantages when enforcing the sparsity of variables [46].

In this paper, we extend extensively the previous work by: (1) adapting the forward model and prior models to the 3D case, which is more appropriate for real 3D large data size applications; (2) comparing the RMSE of the phantom reconstructed by the HHBM method with those by the conventional QR and TV methods, we show the advantages of the semi-supervised property of the HHBM method and that the HHBM method outperforms the TV method when insufficient data are estimated; (3) proposing new ideas for fixing the hyper-parameters in the proposed model; and (4) evaluating the performance of the proposed method in the situations when the number or the angle distribution of the projections is limited.

The rest of this paper is organized as follows: Section 2 presents the proposed hierarchically-structured Bayesian method; Section 3 gives the details of the implementations and the choice of hyperparameters, as well as the simulation results; some points on the initialization of hyper-parameters are discussed in Section 4. Conclusions are drawn and prospective future research is presented in Section 5.

2. The Semi-Supervised Hierarchical Model

The Hierarchical Haar-based Bayesian Method (HHBM) is presented, in which the object $\mathit{f}$ and its multilevel Haar transform coefficient $\mathit{z}$ are considered jointly. A sparse enforcing prior is defined on $\mathit{z}$. The wavelet transformation has been used for the reconstruction of tomography images in some state-of-the-art works [55,56,57,58], using both regularization and Bayesian methods. In these state-of-the-art methods, the phantom $\mathit{f}$ is obtained by a post-processing from reconstructed coefficient $\mathit{z}$. In this paper, the phantom $\mathit{f}$ and the coefficient $\mathit{z}$ are simultaneously estimated.

2.1. Forward System Model and Likelihood

In the proposed method, the forward model introduced in Equation (1) is considered. Generally, the noise in tomography is modeled by a Poisson distribution [59], but in X-ray CT, the Gaussian approximation is often used. We adopt the Gaussian approximation and propose to use a zero mean and non-stationary model where the variance is considered to be unknown, belonging to an inverse Gamma distribution given that this distribution provides a good adaptation of the positivity property of the variances $v_{{ϵ}_i}$:

$$\begin{array}{c}p(\mathit{ϵ}|{\mathit{v}}_{ϵ})=\mathcal{N}\left(\mathit{ϵ}|\mathbf{0},{\mathit{V}}_{ϵ}\right),{\mathit{V}}_{ϵ}=\mathrm{diag}\left[{\mathit{v}}_{ϵ}\right],\mathrm{where}{\mathit{v}}_{ϵ}={\left[v_{{ϵ}_1},\cdots ,v_{{ϵ}_M}\right]}^{\prime }\in {\mathbb{R}}^{M\times 1}\hfill \end{array}$$

(7)

$$\begin{array}{c}p({\mathit{v}}_{ϵ}|{\alpha }_{{ϵ}_0},{\beta }_{{ϵ}_0})=\prod _{i=1}^M\mathcal{I}\mathcal{G}(v_{{ϵ}_i}|{\alpha }_{{ϵ}_0},{\beta }_{{ϵ}_0})\hfill \end{array}$$

(8)

The vector $v_{ϵ}$ is considered in order to account for the difference of sensitivity to noise for each detector in each projection direction.

According to the forward model of the linear system, Equation (1), and the prior model of the noise, Equation (7), the likelihood of this model system is:

$$p(\mathit{g}|\mathit{f},{\mathit{v}}_{ϵ})=\mathcal{N}(\mathit{g}|\mathit{H}\mathit{f},{\mathit{V}}_{ϵ}).$$

(9)

In Bayesian inference, the likelihood is combined with the prior distributions to determine the posterior distribution.

2.2. Hierarchical Prior Model and Prior Distributions

Typically, the objects considered in medical and industrial X-ray CT are piecewise continuous. In this paper, a hierarchical prior model is used to define the piecewise continuous property. In this hierarchical prior model, a sparsity enforcing model is defined for the wavelet transformation coefficients of the image. A large number of methods accounting for the sparse structure of the solution have been proposed in the literature. Among them, the L1 regularization method is most frequently used, which minimizes the criterion $J\left(\mathit{f}\right)={∥\mathit{g}-\mathit{H}\mathit{f}∥}_2^2+\lambda {∥\mathsf{\Phi }\mathit{f}∥}_1$ where $\mathsf{\Phi }$ is a linear operator, for example the gradient in the TV method. Another class of methods, known as “synthesis” [60], minimizes $J\left(\mathit{z}\right)={∥\mathit{g}-\mathit{H}\mathit{D}\mathit{z}∥}_2^2+\lambda {∥\mathit{z}∥}_1$ where $\mathit{f}=\mathit{D}\mathit{z}$, and $\mathit{z}={\mathit{D}}^{-1}\mathit{f}$ is for example a wavelet transform.

In this paper, we propose to use the multilevel Haar transformation as the sparse dictionary. The transformation is modeled using a discretized forward model:

$$\mathit{f}=\mathit{D}\mathit{z}+\mathit{\xi },$$

(10)

where $\mathit{D}\in {\mathbb{R}}^{N\times N}$ represents the inverse multilevel Haar transformation and $\mathit{\xi }\in {\mathbb{R}}^{N\times 1}$ represents the uncertainties of the transformation, which is introduced to relax the exact relation of the transform operator $\mathit{D}$. $\mathit{\xi }$ is supposed to be sparse. Unlike the gradient operator used in the TV method, the multilevel Haar transformation is orthogonal, i.e., ${\mathit{D}}^{-1}={\mathit{D}}^T$. This property provides certain advantages during optimization, especially for the big data size problems, as the inversion and transpose of the operator are identical and can be replaced by each other for different types of $\mathit{D}$.

$\mathit{\xi }$ is modeled by a Gaussian distribution, $p\left(\mathit{\xi }\right)=\mathcal{N}(\mathit{\xi }|0,{\mathit{V}}_{\xi })$, where ${\mathit{V}}_{\xi }=\mathrm{diag}\left[{\mathit{v}}_{\xi }\right]$ and ${\mathit{v}}_{\xi }={[v_{{\xi }_1},\cdots ,v_{{\xi }_N}]}^{\prime }$. ${\mathit{v}}_{\xi }$ is considered an unknown variance. It is modeled in order to realize a semi-supervised system where the variance is estimated. Here, ${\mathit{v}}_{\xi }$ is modeled by an inverse Gamma distribution, with the same consideration as the model of ${\mathit{v}}_{ϵ}$. The Gaussian model with an inverse Gamma distributed variance, $p({\mathit{v}}_{\xi }|{\alpha }_{{\xi }_0},{\beta }_{{\xi }_0})={\prod }_{j=1}^N\mathcal{I}\mathcal{G}(v_{{\xi }_j}|{\alpha }_{{\xi }_0},{\beta }_{{\xi }_0})$, leads to a generalized Student-t ($\mathcal{S}{t}_g$) distribution [46]. Consequently, a $\mathcal{S}{t}_g$ distribution is derived for $\mathit{\xi }$, and the sparse property of $\mathit{\xi }$ can be guaranteed. From Equation (10), the conditional distribution $p(\mathit{f}|\mathit{z},{\mathit{v}}_{\xi })$ is derived:

$$p(\mathit{f}|\mathit{z},{\mathit{v}}_{\xi })=\mathcal{N}(\mathit{f}|\mathit{D}\mathit{z},{\mathit{V}}_{\xi }),$$

(11)

with:

$$p({\mathit{v}}_{\xi }|{\alpha }_{{\xi }_0},{\beta }_{{\xi }_0})=\prod _j^N\mathcal{I}\mathcal{G}(v_{{\xi }_j}|{\alpha }_{{\xi }_0},{\beta }_{{\xi }_0}).$$

(12)

For practical applications where these parameters are not known or difficult to obtain, we use a semi-supervised method in which the variances of noises, ${\mathit{v}}_{ϵ}$ and ${\mathit{v}}_{\xi }$, are unknown. In HHBM, the inverse Gamma distribution is used to model ${\mathit{v}}_{ϵ}$ and ${\mathit{v}}_{\xi }$, $p\left({\mathit{v}}_{ϵ}\right)=\mathcal{I}\mathcal{G}({\mathit{v}}_{ϵ}|{\alpha }_{{ϵ}_0},{\beta }_{{ϵ}_0})$ and $p\left({\mathit{v}}_{\xi }\right)=\mathcal{I}\mathcal{G}({\mathit{v}}_{\xi }|{\alpha }_{{\xi }_0},{\beta }_{{\xi }_0})$. Consequently, both $\mathit{\xi }$ and $\mathit{ϵ}$ are modeled by a $\mathcal{S}{t}_g$ distribution.

Vector $\mathit{z}={\left[z_1,\cdots ,z_N\right]}^{\prime }$ represents the multilevel Haar transformation coefficient of piece-wise continuous $\mathit{f}$. As mentioned above, $\mathit{z}$ is sparse. In this paper, the generalized Student-t distribution ($\mathcal{S}{t}_g$) [46] is used to enforce the sparsity structure of $\mathit{z}$. The $\mathcal{S}{t}_g$ distribution can be expressed as the marginal of a bivariate normal-inverse Gamma distribution:

$$\mathcal{S}{t}_g(\mathit{z}|\alpha ,\beta )=\int \mathcal{N}(\mathit{z}|0,\mathit{v})\mathcal{I}\mathcal{G}(\mathit{v}|\alpha ,\beta )\mathrm{d}\mathit{v}.$$

(13)

Thanks to the fact that normal and inverse Gamma are conjugate distributions, the use of the $\mathcal{S}{t}_g$ via Equation (13) simplifies the computations when using the Bayesian point estimators such as the posterior mean via the Variational Bayesian Approximation (VBA) method [42].

From Equation (13), the $\mathcal{S}{t}_g$ prior distribution modeling $\mathit{z}$ is expressed as the following model:

$$\begin{array}{c}p(\mathit{z}|{\mathit{v}}_z)=\mathcal{N}(\mathit{z}|0,{\mathit{V}}_z),\mathrm{where}{\mathit{V}}_z=\mathrm{diag}\left[{\mathit{v}}_z\right],{\mathit{v}}_z=\left[v_{z_1},v_{z_2},\cdots ,v_{z_N}\right]\hfill \end{array}$$

(14)

$$\begin{array}{c}p({\mathit{v}}_z|{\alpha }_{z_0},{\beta }_{z_0})=\prod _j^N\mathcal{I}\mathcal{G}(v_{z_j}|{\alpha }_{z_0},{\beta }_{z_0}),\hfill \end{array}$$

(15)

where $v_{z_j},\forall j=1:N$ are i.i.d. distributed. The difference between the standard $\mathcal{S}t$ distribution, $\mathcal{S}t(z|\nu )=\int \mathcal{N}(z|0,v)\mathcal{I}\mathcal{G}(v|\frac{\nu }{2},\frac{\nu }{2})\mathrm{d}v$, and the generalized $\mathcal{S}{t}_g$, given in Equation (13), is that $\mathcal{S}t(z|\nu )$ is governed by one parameter $\nu$, but $\mathcal{S}{t}_g(z|\alpha ,\beta )$ is governed by two parameters $(\alpha ,\beta )$. With these two parameters, the $\mathcal{S}{t}_g$ does not only enforce the sparsity of the variable, but also controls the sparsity rate [46]. By changing the values of the two hyper-parameters ${\alpha }_{z_0}$ and ${\beta }_{z_0}$, we can obtain either a heavy-tailed distribution with a narrow peak or a distribution approaching a Gaussian distribution.

2.3. The HHBM Method

The prior models of the proposed Bayesian method based on the forward model of Equation (1) and the prior model of Equation (10) are:

$$\begin{array}{c}p(\mathit{g}|\mathit{f},{\mathit{v}}_{ϵ})\propto {\left|{\mathit{V}}_{ϵ}\right|}^{-\frac{1}{2}}\exp \left[-\frac{1}{2}{\left(\mathit{g}-\mathit{H}\mathit{f}\right)}^T{\mathit{V}}_{ϵ}^{-1}\left(\mathit{g}-\mathit{H}\mathit{f}\right)\right],\hfill \end{array}$$

(16)

$$\begin{array}{c}p(\mathit{f}|\mathit{z},{\mathit{v}}_{\xi })\propto {\left|{\mathit{V}}_{\xi }\right|}^{-\frac{1}{2}}\exp \left[-\frac{1}{2}{\left(\mathit{f}-\mathit{D}\mathit{z}\right)}^T{\mathit{V}}_{\xi }^{-1}\left(\mathit{f}-\mathit{D}\mathit{z}\right)\right],\hfill \end{array}$$

(17)

$$\begin{array}{c}p(\mathit{z}|{\mathit{v}}_z)\propto {\left|{\mathit{V}}_z\right|}^{-\frac{1}{2}}\exp \left[-\frac{1}{2}{\mathit{z}}^T{\mathit{V}}_z^{-1}\mathit{z}\right],\hfill \end{array}$$

(18)

$$\begin{array}{c}p({\mathit{v}}_z|{\alpha }_{z_0},{\beta }_{z_0})\propto \prod _j^N{v}_{z_j}^{-({\alpha }_{z_0}+1)}\exp \left[-{\beta }_{z_0}{v}_{z_j}^{-1}\right],\hfill \end{array}$$

(19)

$$\begin{array}{c}p({\mathit{v}}_{ϵ}|{\alpha }_{{ϵ}_0},{\beta }_{{ϵ}_0})\propto \prod _i^M{v}_{{ϵ}_i}^{-({\alpha }_{{ϵ}_0}+1)}\exp \left[-{\beta }_{{ϵ}_0}{v}_{{ϵ}_i}^{-1}\right],\hfill \end{array}$$

(20)

$$\begin{array}{c}p({\mathit{v}}_{\xi }|{\alpha }_{{\xi }_0},{\beta }_{{\xi }_0})\propto \prod _j^N{v}_{{\xi }_j}^{-({\alpha }_{{\xi }_0}+1)}\exp \left[-{\beta }_{{\xi }_0}{v}_{{\xi }_j}^{-1}\right].\hfill \end{array}$$

(21)

Figure 1 shows the generative graph of the proposed model in which the hyperparameters in the rectangles need to be initialized:

Via the Bayes rule, Equation (6), the joint posterior distribution of all the unknowns given in the data is derived:

$$\begin{array}{cc}\hfill p& (\mathit{f},\mathit{z},{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi },{\mathit{v}}_z|\mathit{g})=\frac{p(\mathit{g},\mathit{f},\mathit{z},{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi },{\mathit{v}}_z)}{p\left(\mathit{g}\right)}\hfill \\ & =\frac{p(\mathit{g}|\mathit{f},{\mathit{v}}_{ϵ})p(\mathit{f}|\mathit{z},{\mathit{v}}_{\xi })p(\mathit{z}|{\mathit{v}}_z)p\left({\mathit{v}}_z\right)p\left({\mathit{v}}_{ϵ}\right)p\left({\mathit{v}}_{\xi }\right)}{p\left(\mathit{g}\right)}\hfill \\ & \propto p(\mathit{g}|\mathit{f},{\mathit{v}}_{ϵ})p(\mathit{f}|\mathit{z},{\mathit{v}}_{\xi })p(\mathit{z}|{\mathit{v}}_z)p\left({\mathit{v}}_z\right)p\left({\mathit{v}}_{ϵ}\right)p\left({\mathit{v}}_{\xi }\right).\hfill \end{array}$$

(22)

Bayesian point estimators are often used for estimation via the a posteriori distribution. In this paper, we focus on the JMAP estimation, given that in the case of the large data size of the 3D object, the computational costs for the VBA algorithm is too expensive.

2.4. Joint Maximum a Posteriori Estimation

The negative logarithm of the posterior distribution is used as the criterion of optimization in order to simplify the exponential terms. The maximization of the posterior distribution becomes a minimization of the criterion:

$$\begin{array}{cc}\hfill (\mathit{f},\mathit{z},{\mathit{v}}_z,{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi })& =\arg \max \left\{p(\mathit{f},\mathit{z},{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi },{\mathit{v}}_z|\mathit{g})\right\}\hfill \\ & =\arg \min \left\{-\ln p(\mathit{f},\mathit{z},{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi },{\mathit{v}}_z|\mathit{g})\right\}\hfill \\ & =\arg \min J(\mathit{f},\mathit{z},{\mathit{v}}_z,{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi }).\hfill \end{array}$$

(23)

We substitute the distribution formulas and obtain:

$$\begin{array}{c}J(\mathit{f},\mathit{z},{\mathit{v}}_z,{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi })=-\ln p(\mathit{f},\mathit{z},{\mathit{v}}_{ϵ},{\mathit{v}}_{\xi },{\mathit{v}}_z|\mathit{g})\hfill \\ =\frac{1}{2}\sum _i^M\ln v_{{ϵ}_i}+\frac{1}{2}{(\mathit{g}-\mathit{H}\mathit{f})}^T{\mathit{V}}_{ϵ}^{-1}(\mathit{g}-\mathit{H}\mathit{f})\hfill \\ +\frac{1}{2}\sum _j^N\ln v_{{\xi }_j}+\frac{1}{2}{(\mathit{f}-\mathit{D}\mathit{z})}^T{\mathit{V}}_{\xi }^{-1}(\mathit{f}-\mathit{D}\mathit{z})\hfill \\ +\frac{1}{2}\sum _j^N\ln v_{z_j}+\frac{1}{2}{\mathit{z}}^T{\mathit{V}}_z^{-1}\mathit{z}+({\alpha }_{z_0}+1)\sum _j^N\ln v_{z_j}\hfill \\ +{\beta }_{z_0}\sum _j^N{v}_{z_j}^{-1}+({\alpha }_{{ϵ}_0}+1)\sum _i^M\ln v_{{ϵ}_i}+{\beta }_{{ϵ}_0}\sum _i^M{v}_{{ϵ}_i}^{-1}\hfill \\ +({\alpha }_{{\xi }_0}+1)\sum _j^N\ln v_{{\xi }_j}+{\beta }_{{\xi }_0}\sum _j^N{v}_{{\xi }_j}^{-1}.\hfill \end{array}$$

(24)

The unknown variables are determined by obtaining the expressions of the alternate minimization in Equation (24):

$$\begin{array}{c}\widehat{\mathit{f}}={\left({\mathit{H}}^T{\widehat{\mathit{V}}}_{ϵ}^{-1}\mathit{H}+{\widehat{\mathit{V}}}_{\xi }^{-1}\right)}^{-1}\left({\mathit{H}}^t{\widehat{\mathit{V}}}_{ϵ}^{-1}\mathit{g}+{\widehat{\mathit{V}}}_{\xi }^{-1}\mathit{D}\widehat{\mathit{z}}\right),\hfill \end{array}$$

(25)

$$\begin{array}{c}\widehat{\mathit{z}}={\left({\mathit{D}}^T{\widehat{\mathit{V}}}_{\xi }^{-1}\mathit{D}+{\widehat{\mathit{V}}}_z^{-1}\right)}^{-1}{\mathit{D}}^T{\widehat{\mathit{V}}}_{\xi }^{-1}\widehat{\mathit{f}},\hfill \end{array}$$

(26)

$$\begin{array}{c}\widehat{v_{z_j}}=\left({\beta }_{z_0}+\frac{1}{2}{\widehat{z}}_j^2\right)/\left({\alpha }_{z_0}+3/2\right),\hfill \end{array}$$

(27)

$$\begin{array}{c}\widehat{v_{{ϵ}_i}}=\left({\beta }_{{ϵ}_0}+\frac{1}{2}{\left(g_i-{\left[\mathit{H}\widehat{\mathit{f}}\right]}_i\right)}^2\right)/\left({\alpha }_{{ϵ}_0}+3/2\right),\hfill \end{array}$$

(28)

$$\begin{array}{c}\widehat{v_{{\xi }_j}}=\left({\beta }_{{\xi }_0}+\frac{1}{2}{\left({\widehat{f}}_j-{\left[\mathit{D}\widehat{\mathit{z}}\right]}_j\right)}^2\right)/\left({\alpha }_{{\xi }_0}+3/2\right),\hfill \end{array}$$

(29)

$\forall i\in [1,M]$ and $\forall j\in [1,N]$.

In 3D X-ray CT, the inversion of matrix ${\left({\mathit{H}}^T{\widehat{\mathit{V}}}_{ϵ}^{-1}\mathit{H}+{\widehat{\mathit{V}}}_{\xi }^{-1}\right)}^{-1}$ and ${\left({\mathit{D}}^T{\widehat{\mathit{V}}}_{\xi }^{-1}\mathit{D}+{\widehat{\mathit{V}}}_z^{-1}\right)}^{-1}$ in Equations (25) and (26) is impossible due to the large data size. First-order optimization methods are generally used in this case. In this paper, we use the gradient descent algorithm:

$$\begin{array}{c}\mathrm{for}k=1\to I_G:{\widehat{\mathit{f}}}^{(k+1)}={\widehat{\mathit{f}}}^{\left(k\right)}-{\widehat{\gamma }}_f^{\left(k\right)}\nabla {\mathit{J}}_f({\widehat{\mathit{f}}}^{\left(k\right)}),\hfill \end{array}$$

(30)

$$\begin{array}{c}\mathrm{for}k=1\to I_G:{\widehat{\mathit{z}}}^{(k+1)}={\widehat{\mathit{z}}}^{\left(k\right)}-{\widehat{\gamma }}_z^{\left(k\right)}\nabla {\mathit{J}}_z({\widehat{\mathit{z}}}^{\left(k\right)}),\hfill \end{array}$$

(31)

where $I_G$ is the number of iterations for the gradient descent estimation and $\nabla {\mathit{J}}_f(·)$ and $\nabla {\mathit{J}}_z(·)$ are the derivatives of the criterion (24) regarding $\mathit{f}$ and $\mathit{z}$, respectively. ${\widehat{\gamma }}_f(·)$ and ${\widehat{\gamma }}_z(·)$ are the corresponding descent step lengths, which are obtained by using an optimized step length strategy [61]:

$$\begin{array}{c}\nabla \mathit{J}\left(\mathit{f}\right)=-{\mathit{H}}^T{\mathit{V}}_{ϵ}^{-1}\left(\mathit{g}-\mathit{H}\mathit{f}\right)+{\mathit{V}}_{\xi }^{-1}\left(\mathit{f}-\mathit{D}\mathit{z}\right),\hfill \end{array}$$

(32)

$$\begin{array}{c}\nabla \mathit{J}\left(\mathit{z}\right)=-{\mathit{D}}^T{\mathit{V}}_{\xi }^{-1}\left(\mathit{f}-\mathit{D}\mathit{z}\right)+{\mathit{V}}_z^{-1}\mathit{z},\hfill \end{array}$$

(33)

$$\begin{array}{c}{\widehat{\gamma }}_f^{\left(k\right)}=\frac{{∥\nabla \mathit{J}({\widehat{\mathit{f}}}^{\left(k\right)})∥}^2}{{∥{\widehat{\mathit{Y}}}_{ϵ}\mathit{H}\nabla \mathit{J}({\widehat{\mathit{f}}}^{\left(k\right)})∥}^2+{∥{\widehat{\mathit{Y}}}_{\xi }\nabla \mathit{J}({\widehat{\mathit{f}}}^{\left(k\right)})∥}^2},\hfill \end{array}$$

(34)

$$\begin{array}{c}{\widehat{\gamma }}_z^{\left(k\right)}=\frac{{∥\nabla \mathit{J}({\widehat{\mathit{z}}}^{\left(k\right)})∥}^2}{{∥{\widehat{\mathit{Y}}}_{\xi }\mathit{D}\nabla \mathit{J}({\widehat{\mathit{z}}}^{\left(k\right)})∥}^2+{∥{\widehat{\mathit{Y}}}_z\nabla \mathit{J}({\widehat{\mathit{z}}}^{\left(k\right)})∥}^2},\hfill \end{array}$$

(35)

where ${\mathit{Y}}_{ϵ}={\mathit{V}}_{ϵ}^{-\frac{1}{2}}$, ${\mathit{Y}}_{\xi }={\mathit{V}}_{\xi }^{-\frac{1}{2}}$, and ${\mathit{Y}}_z={\mathit{V}}_z^{-\frac{1}{2}}$.

The algorithm concerning the optimization of all the unknowns is given in Algorithm 1.

Algorithm 1 The JMAP Algorithm for the HHBM Method

1: Fix parameters ${\alpha }_{z_0}$, ${\beta }_{z_0}$, ${\alpha }_{{ϵ}_0}$, ${\beta }_{{ϵ}_0}$, ${\alpha }_{{\xi }_0}$, ${\beta }_{{\xi }_0}$, l   2: Input:   $\mathit{H},\mathit{D},\mathit{g}$   3: Output:   $\widehat{\mathit{f}},\widehat{\mathit{z}},{\widehat{\mathit{v}}}_z,{\widehat{\mathit{v}}}_{ϵ},{\widehat{\mathit{v}}}_{\xi }$   4: Initialization:     5: $\widehat{\mathit{f}}\leftarrow$ normalized FBP   6: $\widehat{\mathit{z}}\leftarrow l-\mathrm{level}\mathrm{Haar}\mathrm{transformation}\mathrm{of}\widehat{\mathit{f}}$   7: for$k=1$ to $I_{max}$ do   8:         ${\widehat{\mathit{f}}}^{(0)}=\widehat{\mathit{f}}$   9:         for $k=1$ to $I_G$ do 10:                 Calculate $\nabla \mathit{J}({\widehat{\mathit{f}}}^{(k-1)})$ according to Equation (32) 11:                 Update ${\widehat{\gamma }}_f^{(k)}$ according to Equation (34) 12:                 Update ${\widehat{\mathit{f}}}^{(k)}={\widehat{\mathit{f}}}^{(k-1)}-{\widehat{\gamma }}_f^{(k)}\nabla \mathit{J}({\widehat{\mathit{f}}}^{(k-1)})$ 13:         end for 14:         $\widehat{\mathit{f}}={\widehat{\mathit{f}}}^{(I_G)}$ 15:         ${\widehat{\mathit{z}}}^{(0)}=\widehat{\mathit{z}}$ 16:         for $k=1$ to $I_G$ do 17:                 Calculate $\nabla \mathit{J}({\widehat{\mathit{z}}}^{(k-1)})$ according to Equation (33) 18:                 Update ${\widehat{\gamma }}_z^{(k)}$ according to Equation (35) 19:                 Update ${\widehat{\mathit{z}}}^{(k)}={\widehat{\mathit{z}}}^{(k-1)}-{\widehat{\gamma }}_z^{(k)}\nabla \mathit{J}({\widehat{\mathit{z}}}^{(k-1)})$ 20:         end for 21:         $\widehat{\mathit{z}}={\widehat{\mathit{z}}}^{(I_G)}$ 22:         Optimize ${\mathit{v}}_z$ according to Equation (27) 23:         Optimize ${\mathit{v}}_{ϵ}$ according to Equation (28) 24:         Optimize ${\mathit{v}}_{\xi }$ according to Equation (29) 25: end for

3. Initialization and Experimental Results

For the simulations, the 3D simulated “Shepp–Logan” phantom, shown in Figure 2 on the left, Figure 3 on the top, and the 3D real “head” object, shown in Figure 2 on the right, Figure 3 on the bottom, both of size ${256}^3$, are used as the objects of interest to compare the performance of the proposed method to the performance of the other state-of-the-art methods. Both the Shepp–Logan and head phantoms consist of several different homogeneous areas, so both are piecewise continuous. The voxel values of the original objects are normalized to $[0,1]$. The projection directions are uniformly distributed, and each projection consists of ${256}^2$ detectors corresponding to a ${256}^2$ sized image.

The proposed HHBM method is compared with the conventional Quadratic Regularization (QR) and Total Variation (TV) methods. For the QR method, the gradient descent algorithm is used for the 3D large data size problem. For the TV method, the split Bregman method [22] is used to solve the $L_1$ norm minimization problem.

To evaluate the proposed method and compare it with the state-of-the-art methods, four different metrics are used:

• the Relative Mean Squared Error (RMSE), RMSE = ${∥\mathit{f}-\widehat{\mathit{f}}∥}^2/{∥\mathit{f}∥}^2$, which shows a relative error of the results;
• the Improvement of the Signal-to-Noise Ratio (ISNR), which measures the improvement during iterations;
• the Peak Signal-to-Noise-Ratio (PSNR), which presents the SNR relative to the peak data value;
• the Structural Similarity of IMage (SSIM) [62], which evaluates the quality of the result approaching human vision.

In 3D X-ray CT, the projection matrix $\mathit{H}$ is very large and is not accessible. For the simulations, only the projection operator $\mathit{H}\mathit{f}$ and the back-projection operator ${\mathit{H}}^T\mathit{g}$ are used. Considering that the costly projection and back-projection operators are computed in every iteration, the GPU processor is used via the ASTRA toolbox [63] to accelerate the computation.

3.1. Initializations

The initialization for the variables $\mathit{f}$ and $\mathit{z}$, as well as the hyperparameters ${\alpha }_{{ϵ}_0}$, ${\beta }_{{ϵ}_0}$, ${\alpha }_{{\xi }_0}$, ${\beta }_{{\xi }_0}$, ${\alpha }_{z_0}$, and ${\beta }_{z_0}$ are discussed in this section.

The reconstructed phantom obtained by using the Filtered Back Projection (FBP) method is considered to be initial value ${\widehat{\mathit{f}}}_{ini}$. The initialization of coefficient ${\widehat{\mathit{z}}}_{ini}$ is the multilevel Haar transformation of ${\widehat{\mathit{f}}}_{ini}$: ${\widehat{\mathit{z}}}_{ini}={\mathit{D}}^{-1}{\widehat{\mathit{f}}}_{ini}$. In this article, we choose the level of transformation such that $\mathit{z}$ has a sparse structure. As shown in Figure 4, when the transform level is small, for example two levels, the coefficient $\mathit{z}$ is not sparse; when the transform level is sufficiently large, the coefficient is sparse. In this paper, we set $\mathit{z}$ as a five-level Haar transform coefficient.

The initialization for ${\alpha }_{z_0}$ and ${\beta }_{z_0}$ is based on the sparse structure of the variable $\mathit{z}$. In Figure 4, we can see that the sparsity rate depends on the rank of transform coefficient r, where $r\in \left[1,l+1\right]$. For example, when $l=2$, shown in Figure 4, the coefficient has three ranks, $r\in \left[1,2,3\right]$. The first rank $r=1$ corresponds to the low frequency components in the transform coefficient.

The variable $\mathit{z}$ is modeled by a $\mathcal{S}{t}_g$ distribution, with variance equal to $\mathrm{Var}[z_j|{\alpha }_{z_0},{\beta }_{z_0}]={\beta }_{z_0}/({\alpha }_{z_0}-1),\forall j\in [0,N]$. In this article, we fix the value ${\alpha }_{z_0}=2.1$ in order to have $\mathrm{Var}[z_j|{\alpha }_{z_0}=2.1,{\beta }_{z_0}]\approx {\beta }_{z_0},\forall j\in [0,N]$. The sparsity rate of $\mathit{z}$ is defined by initializing different values for ${\beta }_{z_0}$, with a sparser structure when ${\beta }_{z_0}$ is smaller. ${\beta }_{z_0}$ is initialized as ${\beta }_{z_0}={10}^{-r+1}$. When $l=5$, we have $r=[1,2,3,4,5,6]$, and the hyperparameters ${\beta }_{z_0}=[{10}^0,{10}^{-1},{10}^{-2},{10}^{-3},{10}^{-4},{10}^{-5}]$, respectively.

The initialization for the hyperparameters, ${\alpha }_{{ϵ}_0},{\beta }_{{ϵ}_0},{\alpha }_{{\xi }_0}$, and ${\beta }_{{\xi }_0}$, is based on the prior models of the variances ${\mathit{v}}_{ϵ}$ and ${\mathit{v}}_{\xi }$ we have chosen. In the proposed method, we consider the background of the generalized Student-t distribution, in which both $\mathit{ϵ}$ and $\mathit{\xi }$ are modeled by a Gaussian distribution with inverse Gamma distributed variance, i.e., the $\mathcal{S}{t}_g$ distribution according to Equation (13).

The noise $\mathit{ϵ}$ depends on the SNR of the dataset. In order to exploit this information in the initialization, we express the biased dataset as the sum of uncontaminated dataset ${\mathit{g}}_0$ and the additive noise $\mathit{ϵ}$:

$$\mathit{g}={\mathit{g}}_0+\mathit{ϵ}.$$

(36)

As the noise $\mathit{ϵ}$ and the uncontaminated data ${\mathit{g}}_0$ are supposed to be independent, we have:

$${∥\mathit{g}∥}^2={∥{\mathit{g}}_0∥}^2+{∥\mathit{ϵ}∥}^2.$$

(37)

The SNR of the dataset is:

$$\mathrm{SNR}=10\log \frac{{∥{\mathit{g}}_0∥}^2}{{∥ϵ∥}^2}=10\log \frac{{∥\mathit{g}∥}^2-{∥ϵ∥}^2}{{∥\mathit{ϵ}∥}^2}.$$

(38)

With $\mathbb{E}\left[\mathit{ϵ}\right]=0$, we have:

$$v_{ϵ}=\mathbb{E}\left[{\mathit{ϵ}}^2\right]\approx \frac{{∥\mathit{ϵ}∥}^2}{M}=\frac{{∥\mathit{g}∥}^2}{M}\times \frac{1}{1+{10}^{SNR/10}}.$$

(39)

The mean of variance ${\mathit{v}}_{ϵ}$ of the noise $\mathit{ϵ}$ is $\mathbb{E}\left[v_{{ϵ}_i}|{\alpha }_{{ϵ}_0},{\beta }_{{ϵ}_0}\right]={\beta }_{{ϵ}_0}/({\alpha }_{{ϵ}_0}-1),\forall i\in [0,M]$, so we obtain:

$${\beta }_{{ϵ}_0}=\frac{{∥\mathit{g}∥}^2}{M}\times \frac{1}{1+{10}^{SNR/10}}\times \left({\alpha }_{{ϵ}_0}-1\right).$$

(40)

The two hyperparameters ${\alpha }_{{ϵ}_0}$ and ${\beta }_{{ϵ}_0}$ are combined according to Equation (40); hence, initialization for one of them is sufficient. In real applications, the SNR of the dataset is unknown, but we can use the projection of an empty object, i.e., $\mathit{f}=0$, to obtain a rough value of the variance of noise ${\mathit{v}}_{ϵ}$.

Figure 5 shows the influence of the value of ${\alpha }_{{ϵ}_0}$ on the reconstruction. According to the results, a bigger value for ${\alpha }_{{ϵ}_0}$ results in a smaller value on RMSE for different numbers of projections and the SNR of the dataset. This monotonous property facilitates the initialization of this hyperparameter, as a large value for ${\alpha }_{{ϵ}_0}$ satisfies all cases. When ${\alpha }_{{ϵ}_0}$ is greater than a threshold value, the RMSE does not change with different initialization values for ${\alpha }_{{ϵ}_0}$.

For $\mathit{\xi }$, both ${\alpha }_{{\xi }_0}$ and ${\beta }_{{\xi }_0}$ are analyzed for the influence of the reconstruction results.

Figure 6 shows the influence of the hyperparameter ${\alpha }_{{\xi }_0}$. Different colors represent an initialization with a different value of ${\beta }_{{\xi }_0}$. As here we focus on the analysis of ${\alpha }_{{\xi }_0}$ for all cases of ${\beta }_{{\xi }_0}$ values, we do not show the corresponding ${\beta }_{{\xi }_0}$ value for each different color. For different noise levels, different numbers of projections, and different ${\beta }_{{\xi }_0}$ values, the RMSE has an upward trend when the value of ${\alpha }_{{\xi }_0}$ becomes larger.

Figure 7 shows the influence of the hyperparameter ${\beta }_{{\xi }_0}$. Different colors represent an initialization with a different hyperparameter ${\alpha }_{{\xi }_0}$. For different noise levels, different numbers of projections, and different values for ${\alpha }_{{\xi }_0}$, when the value of ${\beta }_{{\xi }_0}$ increases, the RMSE decreases, then, after a slight increase, levels out.

In [46], it is pointed out that when $\alpha$ and $\beta$ of the $\mathcal{S}{t}_g$ distribution are both large, the $\mathcal{S}{t}_g$ distribution approaches a Gaussian distribution, which is the case for the additive noise $\mathit{ϵ}$. If $\alpha$ and $\beta$ are very small (approaching zero), the $\mathcal{S}{t}_g$ distribution becomes a non-informative distribution (Jeffreys distribution); when $\alpha$ and $\beta$ are both small, the $\mathcal{S}{t}_g$ has the sparsity enforcing property, which is the case for the sparse $\mathit{\xi }$. Consequently, the initialization of the hyperparameters is theoretically supported, and they can be initialized with respect to these properties in other simulations.

3.2. Simulation Results with a Limited Number of Projections

We apply $180,90,60,45,36$, and 18 projections evenly distributed in $\left[0,180\right]$ degrees for the reconstruction of the 3D Shepp–Logan phantom of size ${256}^3$; each projection contains ${256}^2$ detectors. The number of projections is chosen such that there is respectively one projection every $1,2,3,4,5$, and 10 degrees.

In

Table 1. RMSE, ISNR, PSNR, and SSIM of the reconstructed phantom with 50 global iterations (10 gradient descent iterations in each global iteration). The values of the regularization parameters are respectively ${\lambda }_{QR}=10$ and ${\lambda }_{TV}=50$ for SNR = 40 dB, ${\lambda }_{QR}=600$ and ${\lambda }_{TV}=100$ for SNR = 20 dB. TV, Total Variation; HHBM, Hierarchical Haar transform-based Bayesian Method; QR, Quadratic Regularization.

	$256\times 256\times 256$Shepp–Logan Phantom
	180 Projections						90 Projections
	40 dB			20 dB			40 dB			20 dB
	QR	TV	HHBM	QR	TV	HHBM	QR	TV	HHBM	QR	TV	HHBM
$\mathbf{RMSE}$	$0.0236$	$0.0114$	$\mathbf{0.0069}$	$0.1309$	$\mathbf{0.0209}$	$0.0755$	$0.0401$	$0.0212$	$\mathbf{0.0092}$	$0.1558$	$\mathbf{0.0491}$	$0.1117$
$\mathbf{ISNR}$	$5.5584$	$8.7217$	$\mathbf{10.9346}$	$7.2024$	$\mathbf{15.1775}$	$10.2162$	$6.6136$	$9.3832$	$\mathbf{12.9973}$	$8.4583$	$\mathbf{13.4765}$	$9.9056$
$\mathbf{PSNR}$	$30.0675$	$33.2308$	$\mathbf{35.4437}$	$22.6318$	$\mathbf{30.6069}$	$25.0209$	$27.7743$	$30.5439$	$\mathbf{34.1579}$	$21.8754$	$\mathbf{26.8937}$	$23.3227$
$\mathbf{SSIM}$	$0.9999$	$0.9999$	$\mathbf{1.0000}$	$0.9992$	$\mathbf{0.9999}$	$0.9995$	$0.9997$	$0.9999$	$\mathbf{0.9999}$	$0.9990$	$\mathbf{0.9997}$	$0.9993$
	60 Projections						45 Projections
	40 dB			20 dB			40 dB			20 dB
	QR	TV	HHBM	QR	TV	HHBM	QR	TV	HHBM	QR	TV	HHBM
$\mathbf{RMSE}$	$0.0636$	$0.0321$	$\mathbf{0.0107}$	$0.1656$	$\mathbf{0.0753}$	$0.1293$	$0.0904$	$0.0474$	$\mathbf{0.0132}$	$0.1854$	$0.0901$	$\mathbf{0.1414}$
$\mathbf{ISNR}$	$9.3826$	$12.3480$	$\mathbf{17.1346}$	$9.1492$	$\mathbf{12.5701}$	$10.2226$	$10.3301$	$13.1308$	$\mathbf{18.6839}$	$10.0137$	$13.1476$	$\mathbf{11.1916}$
$\mathbf{PSNR}$	$25.7693$	$28.7347$	$\mathbf{33.5214}$	$21.6116$	$\mathbf{25.0325}$	$22.6849$	$24.2404$	$27.0412$	$\mathbf{32.5942}$	$21.1195$	$24.2535$	$\mathbf{22.2974}$
$\mathbf{SSIM}$	$0.9996$	$0.9995$	$\mathbf{0.9999}$	$0.9990$	$\mathbf{0.9995}$	$0.9992$	$0.9994$	$0.9997$	$\mathbf{0.9999}$	$0.9988$	$0.9994$	$\mathbf{0.9991}$
	36 Projections						18 Projections
	40 dB			20 dB			40 dB			20 dB
	QR	TV	HHBM	QR	TV	HHBM	QR	TV	HHBM	QR	TV	HHBM
$\mathbf{RMSE}$	$0.1177$	$0.0680$	$\mathbf{0.0169}$	$0.1957$	$\mathbf{0.1116}$	$0.1500$	$0.2581$	$0.2104$	$\mathbf{0.0574}$	$0.2907$	$0.2313$	$\mathbf{0.2014}$
$\mathbf{ISNR}$	$10.6591$	$13.0424$	$\mathbf{19.0933}$	$10.8633$	$\mathbf{13.3032}$	$12.0187$	$10.7122$	$11.5992$	$\mathbf{17.2373}$	$10.8088$	$11.8022$	$\mathbf{12.4036}$
$\mathbf{PSNR}$	$23.0949$	$25.4783$	$\mathbf{31.5292}$	$20.8865$	$\mathbf{23.3264}$	$22.0420$	$19.6263$	$20.5133$	$\mathbf{26.1514}$	$19.1085$	$20.1020$	$\mathbf{20.7033}$
$\mathbf{SSIM}$	$0.9993$	$0.9996$	$\mathbf{0.9999}$	$0.9988$	$\mathbf{0.9993}$	$0.9990$	$0.9983$	$0.9987$	$\mathbf{0.9996}$	$0.9981$	$0.9985$	$\mathbf{0.9987}$

, different evaluation metrics of the reconstructed ${256}^3$ Shepp–Logan phantom are compared. It is shown that the HHBM method does not always perform better than the TV method, especially when there are sufficient numbers of projections. However, when there is insufficient projection data, the HHBM method is more robust than the TV method. On the other hand, as it is known that the choice of regularization parameter plays an important role in the regularization methods like QR or TV and the value for the regularization parameter should be selected for each different system settings, the HHBM method is much more robust on the initialization of hyper-parameters. As we can see from Figure 5, Figure 6 and Figure 7, once we have chosen the hyperparameters in a certain interval, which is not difficult to fix according to the properties of the prior model, we can obtain the appropriate reconstruction results. More importantly, in the Bayesian approach, the prior model can be chosen from a variety of other suitable distributions, which gives more possibilities for the models than the conventional regularization methods. We may also choose different point estimators from the posterior distribution, for example the posterior mean, etc.

Figure 8 and Figure 9 show the reconstructed middle slice of the “Shepp–Logan” phantom and “head” object by using the TV and HHBM methods from 36 projections with SNR = 40 dB and SNR = 20 dB. The red curve illustrates the profile of the blue line position. In the reconstructed Shepp–Logan phantom obtained using the TV method, the three small circles on the top of the slice are not evident. By using the HHBM method, we can distinguish these three small circles. By comparing the profiles of the slice of the reconstructed Shepp–Logan phantom, we can see that by using the HHBM method, the contour positions on the profile are closer to the original profile than those obtained using the TV method. In the reconstructed head object, there are more details than the simulated Shepp–Logan phantom, especially in the zoom area in the second line in Figure 8. By comparing the results, we can see that for the type of object that contains some small details, the TV method derives a result with smoother homogeneous areas, but with fewer details in the contour areas than the HHBM method. Some of the white material, which is dispersed into discontinuous small blocks in the head object, is connected in the results of the TV method. From these images, we conclude that with an insufficient number of projections, the proposed method gives results with clearer contours and details.

Figure 10 shows the reconstructed Shepp–Logan phantom from 18 projections of SNR = 40 dB and 20 dB. In this very underdetermined case, the HHBM method can still obtain a result that is clear enough to distinguish the primary zones and contours of the object.

Figure 11 and Figure 12 show the comparison between the QR, TV, and HHBM methods with a high SNR = 40 dB and a low SNR = 20 dB dataset, respectively. The abscissa corresponds to the number of projections evenly distributed from $0^{\circ }$–${180}^{\circ }$, and the ordinate is the RMSE after 50 iterations. In the simulations, we used an SNR = 40 dB to represent a weak noise case and SNR = 20 dB for a strong noise case. When SNR = 40 dB, the HHBM method outperforms both quadratic regularization and the TV method. When SNR = 20 dB, the TV method with the optimal regularization parameter outperforms the HHBM method. However, in Figure 12, we show another curve in light green (TV2) showing the TV reconstruction with some random regularization parameters, which are chosen as a value not far from the optimal regularization parameters. From these results, we can see that the HHBM method is more robust than the TV reconstruction method with respect to the regularization parameter, the optimal value of which is, on the other hand, difficult to determine in the real applications where we cannot evaluate the estimation quality.

3.3. Simulation Results with a Limited Angle of Projections

In both medical and industrial X-ray CT, another common challenge is the limit of projection angles. In this part of the simulation, we use evenly-distributed projections in a limited range of angles for the simulated 3D “Shepp–Logan” phantom and the 3D “head” object, both of which have a size of ${256}^3$.

Figure 13 and Figure 14 show the middle slice of the reconstructed Shepp–Logan phantom and the head object, from 90 projections distributed between $0^{\circ }$ and ${90}^{\circ }$, with projection SNR of 40 dB and 20 dB. By using the TV method, the reconstructed object is blurry along the diagonal direction for which there is no projection data, and there is a square corner where the object should have a rounded edge. By using the HHBM method, we get results that are more consistent with the true shape and clearer contours.

Figure 15 and Figure 16 show the comparison of the performance in terms of RMSE of different methods with a high SNR = 40 dB and a low SNR = 20 dB. In this comparison, four cases of limited projection angles are considered, and they are: 45, 90, 135, and 180 projections evenly distributed in $[0^{\circ },{45}^{\circ }]$, $[0^{\circ },{90}^{\circ }]$, $[0^{\circ },{135}^{\circ }]$, and $[0^{\circ },{180}^{\circ }]$, respectively. There is one projection every $1^{\circ }$. From these two figures, we conclude that the proposed HHBM method remains more robust than the other two conventional methods when there are limited projection angles.

3.4. Simulation with a Different Forward Model

To study the robustness of the results with respect to the modeling errors, we apply a slightly different forward model for the one used for the generation of the simulated data and the one used for the reconstruction. During the projection, the projector is applied on the Shepp–Logan phantom of size ${1024}^3$, with the detector size of ${256}^2$ for each projection direction.

In Figure 17, we show the results of the two projection data obtained, as well as their difference ${\delta }_{proj}$, which can be considered as the forward modeling error. The ${\delta }_{proj}$ is rescaled in order to show clearly the details. We then use the data obtained from the ${1024}^3$ phantom to reconstruct an object of size ${256}^3$.

In Figure 18, the middle slices of the reconstructed Shepp–Logan phantom by using the QR, TV, and HHBM methods are presented, by using 180 and 36 projections, respectively. From the figures, we can see that when there are 180 projections, all three methods performs well, and the TV method detects better the contours, while the HHBM method has more noise at the contour areas. When there are insufficient projection numbers (36 projections in this simulation), the HHBM method outperforms the QR and TV methods for reconstructing the details in the phantom, for example the three small circles in the top of the phantom.

In

Table 2. The RMSE of the reconstructed ${256}^3$ Shepp–Logan phantom by using projection obtained from a ${1024}^3$ Shepp–Logan phantom.

RMSE	QR	TV	HHBM
180 proj	0.0581	0.0540	0.0558
90 proj	0.0655	0.0573	0.0610
60 proj	0.0846	0.0675	0.0690
45 proj	0.1079	0.0830	0.0783
36 proj	0.1326	0.1027	0.0882

, the RMSE of the reconstructed phantom by using the different methods are compared. We can conclude that, when the projector model is different than the reconstruction one, all these three methods (QR, TV, and HHBM) have good performance when there are 180 projections. When the projection number decreases, the TV and HHBM methods outperform the QR method. Comparing with the TV method, the HHBM method is more robust to the number of projections. When the projection number is smaller than 60, the HHBM method outperforms the TV method.

All the MATLAB codes for the simulations in this paper can be found on GitHub [64].

4. Discussion

One advantage of the Bayesian approach is the estimation of the parameters along with the estimation of unknown variables of the forward model at each iteration. However, like in regularization methods, the hyper-parameters need to be initialized.

While the parameters in the regularization methods play an important role in the final results and they are costly to fix, the hyper-parameters in HHBM can be initialized based on the prior information (the sparse structure of $\mathit{z}$) and the prior model (the Student-t distribution). In this article, we have shown that once the hyper-parameters are fixed in a certain appropriate interval, which is not difficult to obtain, the corresponding algorithm is robust. In this work, the hyper-parameters are not fixed via the classical approach using non-informative prior laws (i.e., considering the inverse Gamma corresponding parameters such that they approach Jeffreys’) [65].

5. Conclusions

In this paper, we propose a Bayesian method with a hierarchical structured prior model based on multilevel Haar transformation (HHBM) for 3D X-ray CT reconstruction. Simulation results indicate that for a limited number of projections or limited projection angles, the proposed method is more robust to noise and to regularization parameters than the classical QR and TV methods.

Indeed, we observe a relatively weak influence of the hyper-parameters in the behavior of the corresponding iterative algorithm. The interest of this weak dependency is that it offers a practical way to ensure the initialization of the algorithm, which typically is not-trivial.

In this context, as future work, we are investigating the causes of the relatively weak influence of the hyper-parameters and the theoretical foundation of the corresponding robust interval, extending the discussion to the same approach using sparsity-enforcing priors expressed as normal variance mixtures, but for other mixing distributions (Gamma, generalized inverse Gaussian) [66].

Another extension of this work is to consider the posterior mean as an estimator. This can be done via the Variational Bayesian Approach (VBA), but a practical implementation requires a method of accessing the diagonal elements of the large matrix ${\mathit{H}}^T\mathit{H}$, which is being studied by our group.