Image Deblurring under Impulse Noise via Total Generalized Variation and Non-Convex Shrinkage

Abstract

Image deblurring under the background of impulse noise is a typically ill-posed inverse problem which attracted great attention in the fields of image processing and computer vision. The fast total variation deconvolution (FTVd) algorithm proved to be an effective way to solve this problem. However, it only considers sparsity of the first-order total variation, resulting in staircase artefacts. The L1 norm is adopted in the FTVd model to depict the sparsity of the impulse noise, while the L1 norm has limited capacity of depicting it. To overcome this limitation, we present a new algorithm based on the Lp-pseudo-norm and total generalized variation (TGV) regularization. The TGV regularization puts sparse constraints on both the first-order and second-order gradients of the image, effectively preserving the image edge while relieving undesirable artefacts. The Lp-pseudo-norm constraint is employed to replace the L1 norm constraint to depict the sparsity of the impulse noise more precisely. The alternating direction method of multipliers is adopted to solve the proposed model. In the numerical experiments, the proposed algorithm is compared with some state-of-the-art algorithms in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), signal-to-noise ratio (SNR), operation time, and visual effects to verify its superiority.

1. Introduction

Image deblurring under impulse noise is important in digital image processing research. It is widely applied in many aspects of science and engineering such as space detection, photodetection, remote sensing technology, medical image analysis, and geological exploration. In the process of acquiring, storing, and transmitting images, owing to limitations of imaging equipment and conditions, images are prone to quality degradation, such as image blurring, noise pollution, contrast decline, and loss of details. Image restoration of degraded images is an inverse problem whose solution is often ill-posed [1]. For such types of problems, regularization proved to be an effective method. This involves the proper use of some prior information of the degraded image as a fidelity term and an image optimization model that contains an efficient solution algorithm as a regularization term, because recovering a clear image is of great importance for advanced image processing in the later stages.

The total variation (TV) regularization proposed by Rudin, Osher, and Fatemi [2], also known as the ROF model, proved to be one of the most advanced regularizations in the field of image reconstruction. Based on the TV regularization, some extended models and their algorithms were widely applied to image restoration. For example, Wang et al. [3] proposed a fast total variation deconvolution (FTVd) model for image deblurring. The model uses the total variation (TV) regularization based on the assumption that the image is piecewise smooth; the method can remove image noise sufficiently, but it is prone to producing a “staircase effect”. Furthermore, the FTVd model adopted the L1 norm to express the sparsity of impulse noise. However, it cannot depict the statistic characteristics of impulse noise.

Recently, some TV extensions emerged. Sakurai et al. [4] proposed a four-directional total variation model to extend the vertical and horizontal gradient information in the TV regularization into four directions. Ren et al. [5] proposed a fractional-order TV (FTV) model and tried to extend the integer-order TV model to the fractional order. Selesnick and Chen [6] proposed an overlapping group sparsity TV (OGSTV) model, in which the neighborhood information of the image gradient was taken into account and applied in conjunction with an L2-norm fidelity term to the Gaussian-noise denoising of one-dimensional signals. Liu et al. [7] extended the model to a two-dimensional model based on an L1-norm fidelity term and applied the extended model to image deconvolution and impulse noise removal, achieving good results [8]. Bredies et al. [9] proposed a total generalized variation (TGV) model based on total variation, which simultaneously constrains the first-order and second-order gradients of the image and has many excellent mathematical properties such as convexity, lower semi-continuity, and rotation invariance; it is also capable of approximating any number of items. The TGV model proved—in theory and practice—to be able to remove the staircase effect effectively while preserving the edges and details of the image. TGV regularization terms attracted wide attention in the academic community and were applied to the reconstruction of nuclear magnetic resonance images [10] and seismic signal denoising [11,12]. Inspired by the advantages of TGV regularization, we incorporate it here in the image deblurring process.

The aforementioned image deblurring methods mostly employ the L1 norm to constrain the fidelity term. However, the L1 norm is a convex relaxation of the L0 norm, with limited ability to induce sparsity [13]. Recently, Woodworth and Chartrand [14] proved that Lp-pseudo-norm shrinkage is a method with better sparsity compared with soft-threshold shrinkage, which attracted extensive attention in academia. Zheng et al. [15] theoretically compared the advantages of Lp-minimization and L1-minimization, and tested them in a Gaussian noisy setting. Zhang et al. [16] applied Lp shrinkage [17] to the reconstruction of tomographic images to obtain an effect superior to the L1-norm constraint. Chen et al. [18] applied an Lp-pseudo-norm constraint to the sparse time–frequency representation of the seismic signal spectrum to obtain a time–frequency spectrum with higher time–frequency resolution. Wong et al. [19] proposed a model combining an Lp-pseudo-norm fidelity term and OGS regularization, which uses neighborhood structure similarity of image gradients for image denoising.

Considering the superiority of Lp-pseudo-norm and TGV, here, we employ Lp-pseudo-norm to describe the sparsity of impulse noise and adopt the TGV regularization term to fit the image sparsity. On the one hand, the Lp-pseudo-norm constraint increases the degrees of freedom of the fidelity term and better represents the sparsity of the impulse noise. In this way, the deblurring process is more robust to noise pollution. On the other hand, the TGV regularization term simultaneously puts constraints on the information of the first-order and second-order gradients of the image, thereby effectively suppressing the staircase effect.

To solve the problem, the alternating direction method of multipliers (ADMM) [20,21] is employed to decompose the complex problem into a number of sub-problems that are relatively easy to solve. At the same time, to improve the efficiency of the algorithm, it is assumed here that the image satisfies a periodic boundary condition. Following a fast deconvolution method [3,22,23], fast Fourier transform (FFT) is employed based on the convolution law to transform image differences in the time domain to differences in the frequency domain to avoid large matrix multiplication operations. In numerical experiments, the proposed new method is compared with FTVd, TGV, and OGSTV in terms of the following objective indicators: peak signal-to-noise ratio (PSNR), structural similarity (SSIM), signal-to-noise ratio (SNR), and computation time.

The rest of this paper is organized as follows: Section 2 presents prerequisite knowledge related to the algorithms used in this study, including the modeling of the TGV regularization term and the Lp norm. Section 3 firstly presents the model proposed in this study and then elaborates on an efficient method for solving the model in the ADMM framework. Results of the numerical experiment are presented in Section 4, and conclusions are given in Section 5.

2. Prerequisite Knowledge

2.1. FTVd Model

We assumed that the linear mathematical model of an observed degraded image is expressed as follows [24]:

$$\mathit{G}=\mathit{H}\ast \mathit{F}+\mathit{N}.$$

(1)

For the sake of simplicity, it was assumed, in this study, that the processed image is an $N\times N$ square matrix, which can be easily generalized to an image of size $M\times N$. $\mathit{G}\in {ℝ}^{N\times N}$ is a degraded image matrix, $\mathit{F}\in {ℝ}^{N\times N}$ represents an original unknown clear image matrix, $\mathit{H}$ is a blur kernel matrix, $\ast$ represents the two-dimensional convolution operator, and $\mathit{N}\in {ℝ}^{N\times N}$ represents additive noise, such as Gaussian noise and impulse noise. For impulse noise, a discrete ROF model is constructed using the L1 norm, where the FTVd model is expressed as

$$\underset{F}{min}\{{\Vert \mathit{H}\ast \mathit{F}-\mathit{G}\Vert }_1+\mu \mathrm{R}(\mathit{F})\},$$

(2)

where ${\Vert ·\Vert }_1$ represents the Euclidean L1 norm. In Equation (2), ${\Vert \mathit{H}\ast \mathit{F}-\mathit{G}\Vert }_1$ is the fidelity term using the L1-norm, and $\mathrm{R}(\mathit{F})$ is the regularization. In FTVd mode, $\mathrm{R}(\mathit{F})={\Vert {\mathit{K}}_{\mathit{h}}\ast \mathit{F}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast {\mathit{F}}_{\mathit{v}}\Vert }_1$ is the anisotropic total variation (ATV) regularization term, ${\mathit{K}}_{\mathit{h}}=[-1,1]$ and ${\mathit{K}}_{\mathit{v}}=[-{1,1]}^T$ are the horizontal and vertical convolution operators, respectively, and $\mu$ is a regularization parameter higher than zero, which serves to balance the fidelity term and the regularization term.

As is shown in Equation (1), FTVd employs the ATV as the regularization and the L1 norm to depict the sparsity of impulse noise. However, ATV assumes the processed image to be piecewise constant. As a result, staircase artefacts occur. In addition, the L1 norm is the convex relaxation of the L0 norm. It cannot express the sparsity of impulse noise precisely. Considering the disadvantages of FTVd, the TGV regularization is employed to replace the ATV regularization, and the Lp-pseudo-norm is adopted to depict the statistic characteristics of impulse noise.

2.2. Second-Order TGV Regularization

The standard TGV regularization is expressed by Equation (3) [25].

$${\mathrm{TGV}}_2(\mathit{F})={\alpha }_0({\Vert {\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}}\Vert }_1)+{\mathit{\alpha }}_1({\Vert {\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}\Vert }_1),$$

(3)

where ${\alpha }_0$ and ${\alpha }_1$ serve as the balance coefficients between the first-order total variation ${\Vert {\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}}\Vert }_1$ and the second-order total variation ${\Vert {\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}\Vert }_1$, and ${\mathit{V}}_{\mathit{h}},{\mathit{V}}_{\mathit{v}}\in {ℝ}^{N\times N}$ represent the horizontal and vertical gradients of the processed image.

Since the TGV regularization puts sparse constraints on not only the first-order gradients but also the second-order gradients, the staircase artefacts are relieved. Thus, we adopt the TGV regularization instead of ATV regularization in Equation (2) to further improve the quality of the restored image.

2.3. Lp-Pseudo-Norm

This study focused on exploring image restoration in the presence of impulse noise pollution. An impulse noise is often the noise of alternating bright (white) and dark (black) dots resulting from a number of factors, such as a faulty pixel in the imaging sensor, a wrong memory location in the hardware, a transmission channel noise, and a decoding process; it is considered an additive noise. In recent years, with the introduction of optimization methods, sparsity was widely applied to image restoration. However, the non-convexity of the L0 norm makes solution optimization (NP-hard problem) challenging. In 2004, Nikolova [26] discovered that, when the noise is an impulse noise, the use of an L1 norm for constructing a fidelity term model allows better removal of outliers. Later, many studies used an L1-norm fidelity term for image denoising related to impulse noise or outliers. Image deconvolution in the presence of impulse noise is often treated as solving a minimization problem involving an L1-norm fidelity term and a regularization function. In fact, the sparsity of the L1 norm, which is the convex relaxation of the L0 norm, is not as good as that of the L0 norm. In 2016, Woodworth and Chartrand [14] pointed out that the Lp norm is a better convex approximation of the L0 norm than the L1 norm is, and proved that Lp shrinkage has better sparsity than soft-threshold shrinkage. The Lp norm is defined as ${\Vert \mathit{F}\Vert }_p^{}={({\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{\left|{\mathit{F}}_{ij}\right|}^p}})}^{1/p}$, and the Lp-pseudo-norm is defined as ${\Vert \mathit{F}\Vert }_p^p={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{\left|{\mathit{F}}_{ij}\right|}^p}}$.

To better elaborate on this issue, the contours of the Lp-pseudo-norm are shown in Figure 1. Figure 2 presents a group of diagrams, where the contours represent the fidelity terms for image restoration, while the dashed lines represent the regularization terms. As shown in the figure, it is more likely for the regularization term to intersect with the fidelity term of $p<1$ than that of $p=1$ on the coordinate axis, thereby making the likelihood of obtaining a sparse solution greater. Therefore, by using the Lp-pseudo-norm as a constraint, it is possible to better extract the sparse characteristics of signals in the transform domain to more precisely restore the image in the optimization problem.

3. Model Proposal and Solution Method

3.1. Proposed Model

To use the FFT to achieve a faster algorithm computation speed, it is assumed that the image satisfies a periodic boundary condition. Moreover, the model is expressed as a matrix, and the gradient difference is expressed in the form of a convolution to avoid large-scale matrix multiplication operations, thereby facilitating the subsequent discussion. Combining the Lp-pseudo-norm with the TGV regularization, this study proposes a new regularization model for image restoration to more thoroughly extract the structural characteristics of the first-order and second-order gradient matrices of the image, and obtain a sparser solution. The model is named the TGV-Lp model, described below.

$$\begin{array}{l}\underset{\mathit{F},{\mathit{V}}_{\mathit{h}},{\mathit{V}}_{\mathit{v}}}{min}{\Vert \mathit{H}\ast \mathit{F}-\mathit{G}\Vert }_p^p+\mu {\mathrm{TGV}}_2(\mathit{F})\\ =\underset{\mathit{F},{\mathit{V}}_{\mathit{h}},{\mathit{V}}_{\mathit{v}}}{min}{\Vert \mathit{H}\ast \mathit{F}-\mathit{G}\Vert }_p^p+\mu [{\alpha }_0({\Vert {\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}}\Vert }_1)\\ +{\alpha }_1({\Vert {\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}\Vert }_1+{\Vert {\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}\Vert }_1)]\end{array},$$

(4)

where $\mu$ is the regularization parameter, ${\Vert \cdot \Vert }_p^p$ represents the Lp-pseudo-norm, ${\alpha }_0$ and ${\alpha }_1$ serve as the balance coefficients, and ${\mathit{V}}_{\mathit{h}},{\mathit{V}}_{\mathit{v}}$ represent the horizontal and vertical gradients of the processed image.

3.2. Solution Method

To solve the model described by Equation (4), the complex optimization problem is decomposed into several sub-problems that are solved using the ADMM framework. Then, alternating iterations of the solutions of the sub-problems are performed until the solutions converge to the optimal solution of the original optimization problem. To this end, a series of intermediate variables ${\mathit{X}}_i$ are introduced, with ${\mathit{X}}_i\in {ℝ}^{N\times N}\left(i=0\sim 5\right)$, and they are defined as

$$\{\begin{array}{l}{\mathit{X}}_0=\mathit{H}\ast \mathit{F}-\mathit{G}\\ {\mathit{X}}_{\mathbf{1}}={\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}}\\ {\mathit{X}}_{\mathbf{2}}={\mathit{{\rm K}}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}}\\ {\mathit{X}}_{\mathbf{3}}={\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}\\ {\mathit{X}}_{\mathbf{4}}={\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}\\ {\mathit{X}}_{\mathbf{5}}={\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{{\rm K}}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}\end{array}.$$

(5)

The original problem is now transformed into the following constraint problem:

$$\begin{array}{l}\underset{\mathit{F},{\mathit{V}}_{\mathit{h}},{\mathit{V}}_{\mathit{v}}}{min}{\Vert {\mathit{X}}_{\mathbf{0}}\Vert }_p^p+\mu [{\alpha }_0({\Vert {\mathit{X}}_{\mathbf{1}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{2}}\Vert }_1)+{\alpha }_1({\Vert {\mathit{X}}_{\mathbf{3}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{4}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{5}}\Vert }_1)]\\ s.t.{\mathit{X}}_{\mathbf{0}}=\mathit{H}\ast \mathit{F}-\mathit{G},{\mathit{X}}_{\mathbf{1}}={\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}},{\mathit{X}}_{\mathbf{2}}={\mathit{{\rm K}}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}},\\ {\mathit{X}}_{\mathbf{3}}={\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}},{\mathit{X}}_{\mathbf{4}}={\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}},{\mathit{X}}_{\mathbf{5}}={\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{{\rm K}}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}.\end{array}$$

(6)

The augmented Lagrangian multiplier (ALM) method [20] is used to transform the constraint problem described by Equation (6) into an unconstrained augmented Lagrangian function as follows:

$$\begin{array}{l}J=\underset{{\mathbf{\Lambda }}_{\mathbf{0}}~{\mathbf{\Lambda }}_{\mathbf{5}}}{max}\{\underset{\begin{array}{l}\mathit{F},{\mathit{V}}_{\mathit{h}},{\mathit{V}}_{\mathit{v}},\\ {\mathit{X}}_{\mathbf{0}}~{\mathit{X}}_{\mathbf{5}}\end{array}}{min}{\Vert {\mathit{X}}_{\mathbf{0}}\Vert }_p^p+\mu [{\alpha }_0({\Vert {\mathit{X}}_{\mathbf{1}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{2}}\Vert }_1)+{\alpha }_1({\Vert {\mathit{X}}_{\mathbf{3}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{4}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{5}}\Vert }_1)]\\ -\langle {\beta }_0{\mathbf{\Lambda }}_{\mathbf{0}},{\mathit{X}}_{\mathbf{0}}-(\mathit{H}\ast \mathit{F}-\mathit{G})\rangle -\langle {\beta }_1{\mathbf{\Lambda }}_{\mathbf{1}},{\mathit{X}}_{\mathbf{1}}-({\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}})\rangle -\langle {\beta }_1{\mathbf{\Lambda }}_{\mathbf{2}},{\mathit{X}}_{\mathbf{2}}-({\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}})\rangle \\ -\langle {\beta }_2{\mathbf{\Lambda }}_{\mathbf{3}},{\mathit{X}}_{\mathbf{3}}-{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}\rangle -\langle {\beta }_2{\mathbf{\Lambda }}_{\mathbf{4}},{\mathit{X}}_{\mathbf{4}}-{\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}\rangle -\langle {\beta }_2{\mathbf{\Lambda }}_{\mathbf{5}},{\mathit{X}}_{\mathbf{5}}-({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}})\rangle \\ +\frac{{\beta }_0}{2}({\Vert {\mathit{X}}_{\mathbf{0}}-(\mathit{H}\ast \mathit{F}-\mathit{G})\Vert }_2^2+\frac{{\beta }_1}{2}({\Vert {\mathit{X}}_{\mathbf{1}}-({\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}})\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{2}}-({\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}})\Vert }_2^2)\\ +\frac{{\beta }_2}{2}({\Vert {\mathit{X}}_{\mathbf{3}}-{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{4}}-{\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{5}}-({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}})\Vert }_2^2)\}\end{array}$$

(7)

where ${\mu }_0=\mu {\alpha }_0$ and ${\mu }_1=\mu {\alpha }_1$; the dual variable ${\mathit{\Lambda }}_i\in {ℝ}^{N\times N}(i=0\sim 5)$ is introduced as a Lagrangian multiplier, ${\beta }_j>0 (j=0\sim 2)$ is the penalty parameter, and $\langle \mathit{X}\mathit{,}\mathit{Y}\rangle$ represents the inner product of two matrices X and Y. Upon adding the term $\frac{{\beta }_j}{2}({\Lambda }_i{}^2-{\Lambda }_i{}^2)=0 (i=0\sim 5,j=0\sim 2)$ to Equation (7) to complete the formula, Equation (8) is obtained after rearrangement.

$$\begin{array}{ll}J& =\underset{{\mathbf{\Lambda }}_{\mathbf{0}}~{\mathbf{\Lambda }}_{\mathbf{5}}}{max}\{\underset{\begin{array}{l}\mathit{F},{\mathit{V}}_{\mathit{h}},{\mathit{V}}_{\mathit{v}},\\ {\mathit{X}}_0~{\mathit{X}}_5\end{array}}{min}{\Vert {\mathit{X}}_{\mathbf{0}}\Vert }_p^p+\mu [{\alpha }_0({\Vert {\mathit{X}}_{\mathbf{1}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{2}}\Vert }_1)+{\alpha }_1({\Vert {\mathit{X}}_{\mathbf{3}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{4}}\Vert }_1+{\Vert {\mathit{X}}_{\mathbf{5}}\Vert }_1)]\\ & +\frac{{\beta }_0}{2}({\Vert {\mathit{X}}_{\mathbf{0}}-(\mathit{H}\ast \mathit{F}-\mathit{G})-{\mathbf{\Lambda }}_{\mathbf{0}}\Vert }_2^2+\frac{{\beta }_1}{2}({\Vert {\mathit{X}}_{\mathbf{1}}-({\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}})-{\mathbf{\Lambda }}_{\mathbf{1}}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{2}}-({\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}})-{\mathbf{\Lambda }}_{\mathbf{2}}\Vert }_2^2)\\ & +\frac{{\beta }_2}{2}({\Vert {\mathit{X}}_{\mathbf{3}}-{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}-{\mathbf{\Lambda }}_{\mathbf{3}}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{4}}-{\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}-{\mathbf{\Lambda }}_{\mathbf{4}}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{5}}-({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}})-{\mathbf{\Lambda }}_{\mathbf{5}}\Vert }_2^2)\\ & -\frac{{\beta }_0}{2}{\mathbf{\Lambda }}_{\mathbf{0}}^2-\frac{{\beta }_1}{2}({\mathbf{\Lambda }}_{\mathbf{1}}^2+{\mathbf{\Lambda }}_{\mathbf{2}}^2)-\frac{{\beta }_2}{2}({\mathbf{\Lambda }}_{\mathbf{3}}^2+{\mathbf{\Lambda }}_{\mathbf{4}}^2+{\mathbf{\Lambda }}_{\mathbf{5}}^2)\},\end{array}.$$

(8)

3.2.1. Solving Sub-Problems F, V_h, and V_v

According to the ADMM principle, F, V_h, and V_v are decoupled with the intermediate variables and the dual variables, while there is coupling between F, V_h, and V_v. It is necessary to establish Equation (9) to obtain a simultaneous solution.

$$\{\begin{array}{ll}{J}_{\mathit{F}}& =\frac{{\beta }_0}{2}{\Vert {\mathit{X}}_{\mathbf{0}}^{(k)}-(\mathit{H}\ast \mathit{F}-\mathit{G})-{\mathbf{\Lambda }}_{\mathbf{0}}^{(k)}\Vert }_2^2\\ & +\frac{{\beta }_1}{2}[{\Vert {\mathit{X}}_{\mathbf{1}}^{(k)}-({\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}})-{\mathbf{\Lambda }}_{\mathbf{1}}^{(k)}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{2}}^{(k)}-({\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}})-{\mathbf{\Lambda }}_{\mathbf{2}}^{(k)}\Vert }_2^2]\\ J_{{\mathit{V}}_{\mathit{h}}}& =\frac{{\beta }_1}{2}{\Vert {\mathit{X}}_{\mathbf{1}}^{(k)}-({\mathit{K}}_{\mathit{h}}\ast \mathit{F}-{\mathit{V}}_{\mathit{h}})-{\mathbf{\Lambda }}_{\mathbf{1}}^{(k)}\Vert }_2^2\\ & +\frac{{\beta }_2}{2}[{\Vert {\mathit{X}}_{\mathbf{3}}^{(k)}-{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}-{\mathbf{\Lambda }}_{\mathbf{3}}^{(k)}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{5}}^{(k)}-({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}})-{\mathbf{\Lambda }}_{\mathbf{5}}^{(k)}\Vert }_2^2]\\ J_{{\mathit{V}}_{\mathit{v}}}& =\frac{{\beta }_1}{2}{\Vert {\mathit{X}}_{\mathbf{2}}^{(k)}-({\mathit{K}}_{\mathit{v}}\ast \mathit{F}-{\mathit{V}}_{\mathit{v}})-{\mathbf{\Lambda }}_{\mathbf{2}}^{(k)}\Vert }_2^2\\ & +\frac{{\beta }_2}{2}[{\Vert {\mathit{X}}_{\mathbf{4}}^{(k)}-{\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}-{\mathbf{\Lambda }}_{\mathbf{4}}^{(k)}\Vert }_2^2+{\Vert {\mathit{X}}_{\mathbf{5}}^{(k)}-({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}})-{\mathbf{\Lambda }}_{\mathbf{5}}^{(k)}\Vert }_2^2)\end{array}.$$

(9)

It is assumed here that the image to process satisfies a periodic boundary condition. According to the convolution law, the convolution result of two matrix spaces corresponds to the point multiplication of two matrix spectra in the frequency domain. To effectively avoid the computational complexity introduced by large-scale matrix operations, FFT is used for convolutional computation to solve the problem described by Equation (9). To this end, F, V_h, and V_v in Equation (9) are re-expressed in the frequency domain as follows:

$$\{\begin{array}{ll}{\overline{J}}_{\mathit{F}}& =\frac{{\beta }_0}{2}{\Vert {\overline{\mathit{X}}}_{\mathbf{0}}^{(k)}-(\overline{\mathit{H}}\circ \overline{\mathit{F}}-\overline{\mathit{G}})-{\overline{\mathbf{\Lambda }}}_{\mathbf{0}}^{(k)}\Vert }_2^2\\ & +\frac{{\beta }_1}{2}[{\Vert {\overline{\mathit{X}}}_{\mathbf{1}}^{(k)}-({\overline{\mathit{K}}}_{\mathit{h}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{h}})-{\overline{\mathbf{\Lambda }}}_{\mathbf{1}}^{(k)}\Vert }_2^2+{\Vert {\overline{\mathit{X}}}_{\mathbf{2}}^{(k)}-({\overline{\mathit{K}}}_{\mathit{v}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{v}})-{\overline{\mathbf{\Lambda }}}_{\mathbf{2}}^{(k)}\Vert }_2^2]\\ {\overline{J}}_{{\mathit{V}}_{\mathit{h}}}& =\frac{{\beta }_1}{2}{\Vert {\overline{\mathit{X}}}_{\mathbf{1}}^{(k)}-({\overline{\mathit{K}}}_{\mathit{h}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{h}})-{\overline{\mathbf{\Lambda }}}_{\mathbf{1}}^{(k)}\Vert }_2^2\\ & +\frac{{\beta }_2}{2}[{\Vert {\overline{\mathbf{X}}}_{\mathbf{3}}^{(k)}-{\overline{\mathit{K}}}_{\mathit{h}}\circ {\overline{\mathit{V}}}_{\mathit{h}}-{\overline{\mathbf{\Lambda }}}_{\mathbf{3}}^{(k)}\Vert }_2^2+{\Vert {\overline{\mathit{X}}}_{\mathbf{5}}^{(k)}-({\overline{\mathit{K}}}_{\mathit{v}}\circ {\overline{\mathit{V}}}_{\mathit{h}}+{\overline{\mathit{K}}}_{\mathit{h}}\circ {\overline{\mathit{V}}}_{\mathit{v}})-{\overline{\mathbf{\Lambda }}}_{\mathbf{5}}^{(k)}\Vert }_2^2]\\ {\overline{J}}_{{\mathit{V}}_{\mathit{v}}}& =\frac{{\beta }_1}{2}{\Vert {\overline{\mathit{X}}}_{\mathbf{2}}^{(k)}-({\overline{\mathit{K}}}_{\mathit{v}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{v}})-{\overline{\mathbf{\Lambda }}}_{\mathbf{2}}^{(k)}\Vert }_2^2\\ & +\frac{{\beta }_2}{2}[{\Vert {\overline{\mathit{X}}}_{\mathbf{4}}^{(k)}-{\overline{\mathit{K}}}_{\mathit{v}}\circ {\overline{\mathit{V}}}_{\mathit{v}}-{\overline{\mathbf{\Lambda }}}_{\mathbf{4}}^{(k)}\Vert }_2^2+{\Vert {\overline{\mathit{X}}}_{\mathbf{5}}^{(k)}-({\overline{\mathit{K}}}_{\mathit{v}}\circ {\overline{\mathit{V}}}_{\mathit{h}}+{\overline{\mathit{K}}}_{\mathit{h}}\circ {\overline{\mathit{V}}}_{\mathit{v}})-{\overline{\mathbf{\Lambda }}}_{\mathbf{5}}^{(k)}\Vert }_2^2)\end{array},$$

(10)

where $\overline{\mathit{X}}$ denotes the frequency spectrum of $\mathit{X}$, and $\circ$ denotes the element-wise matrix multiplication operator. The right side of the equation is a smooth convex function. To seek the optimal solution of Equation (10), the partial differentiation of ${\overline{J}}_F,{\overline{J}}_{V_h},{\overline{J}}_{V_v}$ with respect to $\overline{\mathit{F}},{\overline{\mathit{V}}}_{\mathit{h}},{\overline{\mathit{V}}}_{\mathit{v}}$ is set to zero as follows:

$$\{\begin{array}{l}\frac{\partial {\overline{J}}_{\mathit{F}}}{\partial \overline{\mathit{F}}}={\beta }_0{(\overline{\mathit{H}})}^{\ast }\circ [\overline{\mathit{H}}\circ \overline{\mathit{F}}-\overline{\mathit{G}}-({\overline{\mathit{X}}}_{\mathbf{0}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{0}}^{(k)})]+{\beta }_1\{{\overline{\mathit{K}}}_{\mathit{h}}{}^{\ast }\circ [{\overline{\mathit{K}}}_{\mathit{h}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{h}}-({\overline{\mathit{X}}}_{\mathbf{1}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{1}}^{(k)})]\\ +{\overline{\mathit{K}}}_{\mathit{v}}{}^{\ast }\circ [{\overline{\mathit{K}}}_{\mathit{v}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{v}}-({\overline{\mathit{X}}}_{\mathbf{2}}^{(\mathit{k})}-{\overline{\mathbf{\Lambda }}}_{\mathbf{2}}^{(k)})]\}=0\\ \frac{\partial {\overline{J}}_{{\mathit{V}}_{\mathit{h}}}}{\partial {\overline{\mathit{V}}}_{\mathit{h}}}=-{\beta }_1[{\overline{\mathit{K}}}_{\mathit{h}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{h}}-({\overline{\mathit{X}}}_{\mathbf{1}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{1}}^{(k)})]+{\beta }_2\{{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ [{\overline{\mathit{K}}}_{\mathit{h}}\circ {\overline{\mathit{V}}}_{\mathit{h}}-({\overline{\mathit{X}}}_{\mathbf{3}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{3}}^{(k)})]\\ +{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ [{\overline{\mathit{K}}}_{\mathit{v}}\circ {\overline{\mathit{V}}}_{\mathit{h}}+{\overline{\mathit{K}}}_{\mathit{h}}\circ {\overline{\mathit{V}}}_{\mathit{v}}-({\overline{\mathit{X}}}_{\mathbf{5}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{5}}^{(k)})]\}=0\\ \frac{\partial {\overline{J}}_{{\mathit{V}}_{\mathit{v}}}}{\partial {\overline{\mathit{V}}}_{\mathit{v}}}=-{\beta }_1[({\overline{\mathit{K}}}_{\mathit{v}}\circ \overline{\mathit{F}}-{\overline{\mathit{V}}}_{\mathit{v}}-({\overline{\mathit{X}}}_{\mathbf{2}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{2}}^{(k)})]+{\beta }_2\{{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ [{\overline{\mathit{K}}}_{\mathit{v}}\circ {\overline{\mathit{V}}}_{\mathit{v}}-({\overline{\mathit{X}}}_{\mathbf{4}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{4}}^{(k)})]\\ +{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ [{\overline{\mathit{K}}}_{\mathit{v}}\circ {\overline{\mathit{V}}}_{\mathit{h}}+{\overline{\mathit{K}}}_{\mathit{h}}\circ {\overline{\mathit{V}}}_{\mathit{v}}-({\overline{\mathit{X}}}_{\mathbf{5}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{5}}^{(k)})]\}=0\end{array}.$$

(11)

For the sake of reading convenience, Equation (12) is re-formulated into simultaneous equations of the three variables $\overline{F},{\overline{V}}_h$, and ${\overline{V}}_v$.

$$\{\begin{array}{l}{\mathit{A}}_{\mathbf{11}}\circ \overline{\mathit{F}}+{\mathit{A}}_{\mathbf{12}}\circ {\overline{\mathit{V}}}_{\mathit{h}}+{\mathit{A}}_{\mathbf{13}}\circ {\overline{\mathit{V}}}_{\mathit{v}}={\mathit{B}}_{\mathbf{1}}\\ {\mathit{A}}_{\mathbf{21}}\circ \overline{\mathit{F}}+{\mathit{A}}_{\mathbf{22}}\circ {\overline{\mathit{V}}}_{\mathit{h}}+{\mathit{A}}_{\mathbf{23}}\circ {\overline{\mathit{V}}}_{\mathit{v}}={\mathit{B}}_{\mathbf{2}}\\ {\mathit{A}}_{\mathbf{31}}\circ \overline{\mathit{F}}+{\mathit{A}}_{\mathbf{32}}\circ {\overline{\mathit{V}}}_{\mathit{h}}+{\mathit{A}}_{\mathbf{33}}\circ {\overline{\mathit{V}}}_{\mathit{v}}={\mathit{B}}_{\mathbf{3}}\end{array}.$$

(12)

By re-organizing Equation (11), we obtain

$$\{\begin{array}{ll}{\mathit{A}}_{\mathbf{11}}& ={\mathit{\beta }}_{\mathbf{0}}{\left(\overline{\mathit{H}}\right)}^{\ast }\circ \overline{\mathit{H}}+ {\beta }_1{\left({\overline{\mathit{K}}}_{\mathit{h}}\right)}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{h}}+{\beta }_1{\left({\overline{\mathit{K}}}_{\mathit{v}}\right)}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{v}}\\ {\mathit{A}}_{\mathbf{12}}& =-{\beta }_1{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\\ {\mathit{A}}_{\mathbf{13}}& =-{\beta }_1{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\\ {\mathit{B}}_{\mathbf{1}}& ={\beta }_0{\left(\overline{\mathit{H}}\right)}^{\ast }\circ ({\overline{\mathit{X}}}_{\mathbf{0}}^{(k)}+\overline{\mathit{G}}-{\overline{\mathbf{\Lambda }}}_{\mathbf{0}}^{(k)})+{\beta }_1{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ ({\overline{\mathit{X}}}_{\mathbf{1}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{1}}^{(k)})\\ & +{\beta }_1{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ ({\overline{\mathit{X}}}_{\mathbf{2}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{2}}^{(k)})\end{array},$$

(13)

$$\{\begin{array}{l}{\mathit{A}}_{\mathbf{21}}=-{\beta }_1{\overline{\mathit{K}}}_{\mathit{h}}\\ {\mathit{A}}_{\mathbf{22}}={\beta }_{\mathbf{1}}1+{\beta }_2{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{h}}+{\beta }_2{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{v}}\\ {\mathit{A}}_{\mathbf{23}}={\beta }_2 {({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{h}}\\ {\mathit{B}}_{\mathbf{2}}={\beta }_1({\overline{\mathbf{\Lambda }}}_{\mathbf{1}}^{(k)}-{\overline{\mathit{X}}}_{\mathbf{1}}^{(k)})+{\beta }_2[{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ ({\overline{\mathit{X}}}_{\mathbf{3}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{3}}^{(k)})+{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ ({\overline{\mathit{X}}}_{\mathbf{5}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{5}}^{(k)})]\end{array},$$

(14)

$$\{\begin{array}{l}{\mathit{A}}_{\mathbf{31}}=-{\beta }_1{\overline{\mathit{K}}}_{\mathit{v}}\\ {\mathit{A}}_{\mathbf{32}}={\beta }_2{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{v}}\\ {\mathit{A}}_{\mathbf{33}}={\beta }_1\mathbf{1}+{\beta }_2{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{h}}+{\beta }_2{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ {\overline{\mathit{K}}}_{\mathit{v}}\\ {\mathit{B}}_{\mathbf{3}}={\beta }_1({\overline{\mathbf{\Lambda }}}_{\mathbf{2}}^{(k)}-{\overline{\mathit{X}}}_{\mathbf{2}}^{(k)})+{\beta }_2[{({\overline{\mathit{K}}}_{\mathit{v}})}^{\ast }\circ ({\overline{\mathit{X}}}_{\mathbf{4}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{4}}^{(k)})+{({\overline{\mathit{K}}}_{\mathit{h}})}^{\ast }\circ ({\overline{\mathit{X}}}_{\mathbf{5}}^{(k)}-{\overline{\mathbf{\Lambda }}}_{\mathbf{5}}^{(k)})]\end{array},$$

(15)

where $\mathbf{1}$ denotes a matrix of ones. Now, F, V_h, and V_v can be solved using FFT based on Cramer’s rule.

$$\{\begin{array}{l}{\mathit{F}}^{(k+1)}=F_{2D}^{-1}\left\{{\left|\begin{array}{lll}{\mathit{B}}_{\mathbf{1}}& {\mathit{A}}_{\mathbf{12}}& {\mathit{A}}_{\mathbf{13}}\\ {\mathit{B}}_{\mathbf{2}}& {\mathit{A}}_{\mathbf{22}}& {\mathit{A}}_{\mathbf{23}}\\ {\mathit{B}}_{\mathbf{3}}& {\mathit{A}}_{\mathbf{32}}& {\mathit{A}}_{\mathbf{33}}\end{array}\right|}_{\ast }/{\left|\begin{array}{lll}{\mathit{A}}_{\mathbf{11}}& {\mathit{A}}_{\mathbf{12}}& {\mathit{A}}_{\mathbf{13}}\\ {\mathit{A}}_{\mathbf{21}}& {\mathit{A}}_{\mathbf{22}}& {\mathit{A}}_{\mathbf{23}}\\ {\mathit{A}}_{\mathbf{31}}& {\mathit{A}}_{\mathbf{32}}& {\mathit{A}}_{\mathbf{33}}\end{array}\right|}_{\ast }\right\}\\ {\mathit{V}}_{\mathit{h}}^{{}^{(k+1)}}=F_{2D}^{-1}\left\{{\left|\begin{array}{lll}{\mathit{A}}_{\mathbf{11}}& {\mathit{B}}_{\mathbf{1}}& {\mathit{A}}_{\mathbf{13}}\\ {\mathit{A}}_{\mathbf{21}}& {\mathit{B}}_{\mathbf{2}}& {\mathit{A}}_{\mathbf{23}}\\ {\mathit{A}}_{\mathbf{31}}& {\mathit{B}}_{\mathbf{3}}& {\mathit{A}}_{\mathbf{33}}\end{array}\right|}_{\ast }/{\left|\begin{array}{lll}{\mathit{A}}_{\mathbf{11}}& {\mathit{A}}_{\mathbf{12}}& {\mathit{A}}_{\mathbf{13}}\\ {\mathit{A}}_{\mathbf{21}}& {\mathit{A}}_{\mathbf{22}}& {\mathit{A}}_{\mathbf{23}}\\ {\mathit{A}}_{\mathbf{31}}& {\mathit{A}}_{\mathbf{32}}& {\mathit{A}}_{\mathbf{33}}\end{array}\right|}_{\ast }\right\}\\ {\mathit{V}}_{\mathit{v}}^{{}^{(k+1)}}=F_{2D}^{-1}\left\{{\left|\begin{array}{lll}{\mathit{A}}_{\mathbf{11}}& {\mathit{A}}_{\mathbf{12}}& {\mathit{B}}_{\mathbf{1}}\\ {\mathit{A}}_{\mathbf{21}}& {\mathit{A}}_{\mathbf{22}}& {\mathit{B}}_{\mathbf{2}}\\ {\mathit{A}}_{\mathbf{31}}& {\mathit{A}}_{\mathbf{32}}& {\mathit{B}}_{\mathbf{3}}\end{array}\right|}_{\ast }/{\left|\begin{array}{lll}{\mathit{A}}_{\mathbf{11}}& {\mathit{A}}_{\mathbf{12}}& {\mathit{A}}_{\mathbf{13}}\\ {\mathit{A}}_{\mathbf{21}}& {\mathit{A}}_{\mathbf{22}}& {\mathit{A}}_{\mathbf{23}}\\ {\mathit{A}}_{\mathbf{31}}& {\mathit{A}}_{\mathbf{32}}& {\mathit{A}}_{\mathbf{33}}\end{array}\right|}_{\ast }\right\}\end{array}.$$

(16)

In Equation (16), $F_{2D}^{-1}$ denotes a two-dimensional inverse Fourier transform, and the division symbol / denotes element-wise matrix division. According to Cramer’s rule, ${\left|\right|}_{\ast }$ in Equation (16) refers to the following: $\begin{array}{ll}{\left|\begin{array}{lll}{\mathit{Y}}_{\mathbf{11}}& {\mathit{Y}}_{\mathbf{12}}& {\mathit{Y}}_{\mathbf{13}}\\ {\mathit{Y}}_{\mathbf{21}}& {\mathit{Y}}_{\mathbf{22}}& {\mathit{Y}}_{\mathbf{23}}\\ {\mathit{Y}}_{\mathbf{31}}& {\mathit{Y}}_{\mathbf{32}}& {\mathit{Y}}_{\mathbf{33}}\end{array}\right|}_{\ast }& ={\mathit{Y}}_{\mathbf{11}}\circ {\mathit{Y}}_{\mathbf{22}}\circ {\mathit{Y}}_{\mathbf{33}}+{\mathit{Y}}_{\mathbf{12}}\circ {\mathit{Y}}_{\mathbf{23}}\circ {\mathit{Y}}_{\mathbf{31}}+{\mathit{Y}}_{\mathbf{13}}\circ {\mathit{Y}}_{\mathbf{21}}\circ {\mathit{Y}}_{\mathbf{32}}\\ & - {\mathit{Y}}_{\mathbf{13}}\circ {\mathit{Y}}_{\mathbf{22}}\circ {\mathit{Y}}_{\mathbf{31}}-{\mathit{Y}}_{\mathbf{12}}\circ {\mathit{Y}}_{\mathbf{21}}\circ {\mathit{Y}}_{\mathbf{33}}- {\mathit{Y}}_{\mathbf{11}}\circ {\mathit{Y}}_{\mathbf{32}}\circ {\mathit{Y}}_{\mathbf{23}}\end{array}$.

3.2.2. Solving Sub-Problems Containing Intermediate Variables ${\mathit{X}}_{\mathit{i}}{}^{}(i=0,1,2,\cdots ,5)$

For the sub-problems containing intermediate variables of ${\mathit{X}}_{\mathit{i}}{}^{}(i=0,1,2,\cdots ,5)$, the objective functions are expressed as

$$\{\begin{array}{l}{J}_{{\mathit{X}}_{\mathbf{0}}}=\underset{{\mathit{X}}_{\mathbf{0}}}{min}{\Vert {\mathit{X}}_{\mathbf{0}}\Vert }_p^p+\frac{{\beta }_0}{2}({\Vert {\mathit{X}}_{\mathbf{0}}-(\mathit{H}\ast {\mathit{F}}^{(k+1)}-\mathit{G})-{\mathit{\Lambda }}_{\mathbf{0}}^{(k)}\Vert }_2^2\\ J_{{\mathit{X}}_{\mathbf{1}}}=\underset{{\mathit{X}}_{\mathbf{1}}}{min}{\mu }_0{\Vert {\mathit{X}}_{\mathbf{1}}\Vert }_1+\frac{{\beta }_1}{2}{\Vert {\mathit{X}}_{\mathbf{1}}-({\mathit{K}}_{\mathit{h}}\ast {\mathit{F}}^{(k+1)}-{\mathit{V}}_{\mathit{h}}^{(k+1)})-{\Lambda }_1^{(k)}\Vert }_2^2\\ {\mathit{J}}_{{\mathit{X}}_{\mathbf{2}}}=\underset{{\mathit{X}}_{\mathbf{2}}}{min}{\mu }_0{\Vert {\mathit{X}}_{\mathbf{2}}\Vert }_1+\frac{{\beta }_1}{2}{\Vert {\mathit{X}}_{\mathbf{2}}-({\mathit{K}}_{\mathit{v}}\ast {\mathit{F}}^{(k+1)}-{\mathit{V}}_{\mathit{v}}^{(k+1)})-{\mathit{\Lambda }}_{\mathbf{2}}^{(k)}\Vert }_2^2\\ J_{{\mathit{X}}_{\mathbf{3}}}=\underset{{\mathit{X}}_{\mathbf{3}}}{min}{\mu }_1{\Vert {\mathit{X}}_{\mathbf{3}}\Vert }_1+\frac{{\beta }_2}{2}{\Vert {\mathit{X}}_{\mathbf{3}}-{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)}-{\mathit{\Lambda }}_{\mathbf{3}}^{(k)}\Vert }_2^2\\ J_{{\mathit{X}}_{\mathbf{4}}}=\underset{{\mathit{X}}_{\mathbf{4}}}{min}{\mu }_1{\Vert {\mathit{X}}_{\mathbf{4}}\Vert }_1+\frac{{\beta }_2}{2}{\Vert {\mathit{X}}_{\mathbf{4}}-{\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}^{(k+1)}-{\mathit{\Lambda }}_{\mathbf{4}}^{(k)}\Vert }_2^2\\ J_{{\mathit{X}}_{\mathbf{5}}}=\underset{{\mathit{X}}_{\mathbf{5}}}{min}{\mu }_1{\Vert {\mathit{X}}_{\mathbf{5}}\Vert }_1+\frac{{\beta }_2}{2}{\Vert {\mathit{X}}_{\mathbf{5}}-({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}^{(k+1)})-{\mathit{\Lambda }}_{\mathbf{5}}^{(k)}\Vert }_2^2\end{array}.$$

(17)

To solve the sub-problem of ${\mathit{X}}_{\mathbf{0}}$ in Equation (17), Lp shrinkage [16,17] is used, with the Lp shrinkage operator defined as $shrin{k}_p(\xi ,\tau )=\mathrm{sgn}(\xi )max\{\left|\xi \right|-{\tau }^{2-p}{\left|\xi \right|}^{p-1},0\}$. In the case of p = 1, the Lp shrinkage operator degenerates into a soft-threshold shrinkage operator. Soft-threshold shrinkage of the sub-problems of $X_1~X_5$ gives

$$\{\begin{array}{l}{\mathit{X}}_{\mathbf{0}}^{(k+1)}=shrin{k}_p(\mathit{H}\ast {\mathit{F}}^{(k+1)}-\mathit{G}+{\mathit{\Lambda }}_{\mathbf{0}}^{(k)},\frac{1}{{\beta }_0})\\ {\mathit{X}}_{\mathbf{1}}^{(k+1)}=shrink({\mathit{K}}_{\mathit{h}}\ast {\mathit{F}}^{(k+1)}-{\mathit{V}}_{\mathit{h}}^{(k+1)}+{\mathit{\Lambda }}_{\mathbf{1}}^{(k)},\frac{{\mu }_0}{{\beta }_1})\\ {\mathit{X}}_{\mathbf{2}}^{(k+1)}=shrink({\mathit{K}}_{\mathit{v}}\ast {\mathit{F}}^{(\mathit{k}+1)}-{\mathit{V}}_{\mathit{v}}^{(k+1)}+{\mathit{\Lambda }}_{\mathbf{2}}^{(k)},\frac{{\mu }_0}{{\beta }_1})\\ {\mathit{X}}_{\mathbf{3}}^{(k+1)}=shrink({\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)}+{\mathit{\Lambda }}_{\mathbf{3}}^{(k)},\frac{{\mu }_1}{{\beta }_2})\\ {\mathit{X}}_{\mathbf{4}}^{(k+1)}=shrink({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}^{(k+1)}+{\mathit{\Lambda }}_{\mathbf{4}}^{(k)},\frac{{\mu }_1}{{\beta }_2})\\ {\mathit{X}}_{\mathbf{5}}^{(k+1)}=shrink({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}^{(k+1)}+{\mathit{\Lambda }}_{\mathbf{5}}^{(k)},\frac{{\mu }_1}{{\beta }_2})\end{array}.$$

(18)

3.2.3. Solving Sub-Problems Containing Dual Variables ${\mathbf{\Lambda }}_{\mathit{i}}(i=0,1,2,\cdots ,5)$

For the sub-problems containing dual variables ${\mathbf{\Lambda }}_{\mathit{i}}{}^{}(i=0,1,2,\cdots ,5)$, the objective functions are expressed as

$$\{\begin{array}{l}{J}_{{\mathbf{\Lambda }}_{\mathbf{0}}}=\underset{{\mathbf{\Lambda }}_{\mathbf{0}}}{max}{\beta }_0\langle {\mathbf{\Lambda }}_{\mathbf{0}},(\mathit{H}\ast {\mathit{F}}^{(k+1)}-\mathit{G})-{\mathit{X}}_{\mathbf{0}}^{(k+1)}\rangle \\ J_{{\mathbf{\Lambda }}_{\mathbf{1}}}=\underset{{\mathbf{\Lambda }}_{\mathbf{1}}}{max}{\beta }_1\langle {\mathbf{\Lambda }}_{\mathbf{1}},({\mathit{K}}_{\mathit{h}}\ast {\mathit{F}}^{(k+1)}-{\mathit{V}}_{\mathit{h}}^{(k+1)})-{\mathit{X}}_{\mathbf{1}}^{(k+1)}\rangle \\ J_{{\mathbf{\Lambda }}_{\mathbf{2}}}=\underset{{\mathbf{\Lambda }}_{\mathbf{2}}}{max}{\beta }_1\langle {\mathbf{\Lambda }}_{\mathbf{2}},({\mathit{K}}_{\mathit{v}}\ast {\mathit{F}}^{(k+1)}-{\mathit{V}}_{\mathit{v}}^{(k+1)})-{\mathit{X}}_{\mathbf{2}}^{(k+1)}\rangle \\ J_{{\mathbf{\Lambda }}_{\mathbf{3}}}=\underset{{\mathbf{\Lambda }}_{\mathbf{3}}}{max}{\beta }_2\langle {\mathbf{\Lambda }}_{\mathbf{3}},({\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)})-{\mathit{X}}_{\mathbf{3}}^{(k+1)}\rangle \\ J_{{\mathbf{\Lambda }}_{\mathbf{4}}}=\underset{{\mathbf{\Lambda }}_{\mathbf{4}}}{max}{\beta }_2\langle {\mathbf{\Lambda }}_{\mathbf{4}},({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}^{(k+1)})-{\mathit{X}}_{\mathbf{4}}^{(k+1)}\rangle \\ J_{{\mathbf{\Lambda }}_{\mathbf{5}}}=\underset{{\mathbf{\Lambda }}_{\mathbf{5}}}{max}{\beta }_2\langle {\mathbf{\Lambda }}_{\mathbf{5}},({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)}+K_h\ast {\mathit{V}}_{\mathit{v}}^{(k+1)})-{\mathit{X}}_{\mathbf{5}}^{(k+1)}\rangle \end{array}.$$

(19)

In the ADMM framework, dual variables can be updated using the ascending gradient method as follows:

$$\{\begin{array}{l}{\mathbf{\Lambda }}_{\mathbf{0}}^{(k+1)}={\mathbf{\Lambda }}_{\mathbf{1}}^{(k)}+\gamma {\beta }_0(\mathit{H}\ast {\mathit{F}}^{(k+1)}-\mathit{G}-{\mathit{X}}_{\mathbf{0}}^{(k+1)})\\ {\mathbf{\Lambda }}_{\mathbf{1}}^{(k+1)}={\mathbf{\Lambda }}_{\mathbf{1}}^{(k)}+\gamma {\beta }_1({\mathit{K}}_{\mathit{h}}\ast {\mathit{F}}^{(k+1)}-{\mathit{V}}_{\mathit{h}}^{(k+1)}-{\mathit{X}}_{\mathbf{1}}^{(k+1)})\\ {\mathbf{\Lambda }}_{\mathbf{2}}^{(k+1)}={\mathbf{\Lambda }}_{\mathbf{2}}^{(k)}+\gamma {\beta }_1({\mathit{K}}_{\mathit{v}}\ast {\mathit{F}}^{(k+1)}-{\mathit{V}}_{\mathit{v}}^{(k+1)}-{\mathit{X}}_{\mathbf{2}}^{(k+1)})\\ {\mathbf{\Lambda }}_{\mathbf{3}}^{(k+1)}={\mathbf{\Lambda }}_{\mathbf{3}}^{(k)}+\gamma {\beta }_2({\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)}-{\mathit{X}}_{\mathbf{3}}^{(k+1)})\\ {\mathbf{\Lambda }}_{\mathbf{4}}^{(k+1)}={\mathbf{\Lambda }}_4^{(k)}+\gamma {\beta }_2({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{v}}^{(k+1)}-{\mathit{X}}_{\mathbf{4}}^{(k+1)})\\ {\mathbf{\Lambda }}_{\mathbf{5}}^{(k+1)}={\mathbf{\Lambda }}_{\mathbf{5}}^{(k)}+\gamma {\beta }_2({\mathit{K}}_{\mathit{v}}\ast {\mathit{V}}_{\mathit{h}}^{(k+1)}+{\mathit{K}}_{\mathit{h}}\ast {\mathit{V}}_{\mathit{v}}^{(k+1)}-{\mathit{X}}_{\mathbf{5}}^{(k+1)})\end{array},$$

(20)

where $\gamma$ denotes the learning rate or the slack variable. The convergence of the algorithm was proven in the case of $\gamma \in (0,\frac{\sqrt{5}+1}{2})$ [21]. Hitherto, all the sub-problems of the model proposed in this study were solved, and the algorithm is herein referred to as TGV_Lp, which is summarized in Algorithm 1.

Algorithm 1. Pseudocode TGV_Lp for image restoration

Input:$\mathit{G}$,$\mathit{H}$ Output:$\mathit{F}$ Initialize: $k=1,{\mathit{X}}_i^{(k)}=0,{\mathbf{\Lambda }}_i^{(k)}=0,(i=0,1,\dots ,5),{\mathit{F}}^{(k)}=0,{\mu }_0,{\mu }_1,{\beta }_i(i=0,1,2),\gamma ,p,tol.$ 1: Set ${\Vert {\mathit{F}}^{(k+1)}-{\mathit{F}}^{(k)}\Vert }_2/{\Vert {\mathit{F}}^{(k)}\Vert }_2$ as 1; 2: While ${\Vert {\mathit{F}}^{(k+1)}-{\mathit{F}}^{(k)}\Vert }_2/{\Vert {\mathit{F}}^{(k)}\Vert }_2>tol$ do 3:   Update ${\mathit{F}}^{(k+1)},{\mathit{V}}_{\mathit{h}}^{(k+1)},{\mathit{V}}_{\mathit{v}}^{(k+1)}$ according to Equation (16); 4:   Update ${\mathit{X}}_i^{(k+1)}(i=0,1,\dots ,5)$ according to Equation (18); 5:   Update ${\mathbf{\Lambda }}_i^{(k+1)}(i=0,1,\dots ,5)$ according to Equation (20); 6:   k = k + 1; 7: End while 8: Return ${\mathit{F}}^{(k)}$ as $\mathit{F}$.

4. Numerical Experiments

To verify the appropriateness and validity of the proposed model, standard images of different styles were selected as experimental objects. The images were degraded with Gaussian blur, average blur, motion blur, and different degrees of impulse noise so that a large number of numerical simulation experiments could be performed to make indicator-based comparisons and evaluations in an objective manner. In particular, the image “Satellite” was downloaded from the Emory University image database [27], and the other images were downloaded from CVG-UGR (Computer Vision Group, University of Granada) image database [28]. These test images were widely used in other relevant publications, thereby allowing for objective and fair comparison of the test results. All calculations in this study were performed using MATLAB R2018a on a Windows 10 system with an Intel ® Core ™ i7-7700 central processing unit (CPU) (4.2 GHz) with 16.0 GB of memory.

4.1. Evaluation Indicators and Stopping Criteria

For objective evaluation of image quality, PSNR, SSIM, and SNR were used. Their definitions are given below [29].

$$PSNR(\mathit{X},\mathit{Y})=10lo{g}_{10}\frac{L^2}{\frac{1}{N^2}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{({\mathit{X}}_{ij}-{\mathit{Y}}_{ij})}^2}}},$$

(21)

where $\mathit{X}$ represents an original image, and $\mathit{Y}$ represents a restored image; $L$ represents the maximum gray level of the image, which was set as 255 in this study. Generally, a larger PSNR value of a restored image denotes less image distortion and greater similarity between the restored image and the original image.

$$SSIM(\mathit{X},\mathit{Y})=\frac{(2{\mu }_X{\mu }_Y+L^2{k}_1^2)(2{\sigma }_X{}_Y+L^2{k}_2^2)}{\left({\mu }_X^2+{\mu }_Y^2+L^2{k}_1^2\right)\left({\sigma }_X^2+{\sigma }_Y^2+L^2{k}_2^2\right)},$$

(22)

where ${\mu }_X$ and ${\mu }_Y$ represent the means of images X and Y, respectively, ${\sigma }_X^2$ and ${\sigma }_Y^2$ represent the variances of X and Y, respectively, ${\sigma }_{XY}$ represents the covariance of X and Y, and the constants $k_1$ and $k_2$ are very small positive numbers introduced to prevent the denominator of Equation (22) from being zero; they were set as 0.01 and 0.03 in this study, respectively. The maximum value of SSIM is 1, which indicates that the original image and the restored image are 100% identical; as the value of SSIM approaches 1, the similarity between the original and restored images increases.

$$SNR(\mathit{X},\mathit{Y})=10lo{g}_{10}\frac{{\Vert \mathit{X}\Vert }_2^2}{{\Vert \mathit{X}-\mathit{Y}\Vert }_2^2}.$$

(23)

Images with low noise have higher SNR values, making SNR an important metric for characterizing the performance of the restoration algorithm.

In the experiments, the iteration stopping criterion for comparing various algorithms was Equation (24), where ${\mathit{F}}^{(k)}$ and ${\mathit{F}}^{(k+1)}$ denote the objective functions after the k-th iteration and (k + 1)-th iteration, respectively.

$${\Vert {\mathit{F}}^{(k+1)}-{\mathit{F}}^{(k)}\Vert }_2\cdot {\Vert {\mathit{F}}^{(k)}\Vert }_2^{-1}<{10}^{-4}.$$

(24)

The blur kernels used in the test were Gaussian blur, average blur, and motion blur, all of which were generated by built-in functions of MATLAB. The function fspecial (‘gaussian’, $S$,$\sigma$) generates an $S\times S$ Gaussian blur kernel standard deviation of $\sigma$, and the kernel is hereinafter referred to as $(G,S,\sigma )$ for convenience; fspecial (‘average’, $S$) generates an $S\times S$ average blur kernel, which is referred to as $(A,S)$; fspecial (‘motion’, $L$,$\theta$) generates a motion blur kernel with a length of $L$ and angle of $\theta$, which is referred to as $(M,L,\theta )$. Noise effects were generated by built-in function imnoise (Ib, ‘salt & pepper’, level), where “Ib” is the blurred image and “level” is the noise density, i.e., the percentage of noise in the total number of pixels.

4.2. Parameter Selection and Sensitivity Analysis

Parameters involved in the algorithm of this study are ${\mu }_0, {\mu }_1, {\beta }_i\left(i=0,1,2\right),p, \mathrm{and} \gamma$. In particular, ${\mu }_0=\mu {\alpha }_0$ and ${\mu }_1=\mu {\alpha }_1$ are regularization parameters with values greater than zero, which serve to balance the fidelity term and regularization term. Based on our previous experience with testing, we assumed here that ${\alpha }_0=2{\alpha }_1$ while gradually increasing the value of $\mu$ between 0.1 and 10 to find the optimal effect of image restoration. Given the characteristics of impulse noise, the value of the Lp-pseudo-norm was gradually increased from 0.1 to 1 by a step of 0.05, and a loop statement was adopted to traverse all paths to find the p value for the optimal PSNR. In all tests, the penalty parameter ${\beta }_i(i=0,1,2)$ of the ALM method was set to have ratios of ${\beta }_0:{\beta }_1:{\beta }_2=50:1:5$, and the slack variable was set to 1, i.e., $\gamma =1$.

4.2.1. Regularization Parameter $\mu$

To test the effect of the regularization parameter $\mu$ on the image restoration results, different levels of blur and impulse noise were introduced into the test images (a)–(i), and PSNR values were recorded by gradually increasing the value of $\mu$ in the range of 0.1–10 while fixing other parameters. The experimental results show that the proposed algorithm is not sensitive to $\mu$; that is, during the recovery process of different images, the value of PSNR does not undergo significant changes with changes in $\mu$, indicating that the proposed algorithm is robust. The test results of the “Boat” image of Figure 3d and “Hill” image of Figure 3f treated with the Gaussian blur kernel at 30%, 40%, 50%, and 60% noise levels are presented in Figure 4. The PSNR value in Figure 4 was not optimal for the proposed algorithm, as other fixed parameters remained to be adjusted optimally. In subsequent experiments, the regularization parameter was set to 1, i.e.,$\mu =1$.

4.2.2. Value of p in Lp-Pseudo-Norm

When it came to selecting the p value of the Lp-pseudo-norm, given the characteristics of the impulse noise, the p value was gradually increased from 0.1 to 1 in steps of 0.05, and a loop statement was adopted to traverse all paths so as to find the p value that could give the optimal PSNR value of image restoration. For restoration of the nine images treated with $(G,7,5)$ at 30–60% noise levels, the best p values were obtained and are listed in

Table 1. Values of p that give optimal peak signal-to-noise ratio (PSNR) values for different test images.

Noise Level	Fingerprint	Butterfly	Baboon	Boat	Bird	Hill	Truck	Plane	Satellite
30%	0.30	0.45	0.50	0.35	0.50	0.55	0.50	0.55	0.75
40%	0.30	0.45	0.50	0.35	0.50	0.55	0.50	0.55	0.75
50%	0.30	0.45	0.50	0.35	0.50	0.55	0.50	0.55	0.75
60%	0.25	0.50	0.50	0.35	0.50	0.55	0.55	0.55	0.75

, with the results showing that the p value was relatively stable for the same image.

In addition, to better show the best selection of the p and $\mu$ values, we used a three-dimensional (3D) figure. The test results of the “Hill” image treated with the Gaussian blur kernel at 30% noise level are presented in Figure 5.

4.3. Numerical Performance Comparison

To test the effectiveness of the proposed algorithm, the following four algorithms were compared: an FTVd model, a TGV model, an OGSTV model, and the proposed frequency-domain TGV-Lp model. The algorithms were adapted from the authors’ web pages or shared by the authors. It is noteworthy that all these algorithms are state-of-the-art algorithms in the field of image restoration. In the experiments, regularization parameters were adjusted until each algorithm achieved its best performance for image restoration to ensure an objective and fair comparison.

4.3.1. Numerical Performance Comparison for Image Recovery in the Presence of Gaussian Blur

The results of the image restoration tests performed on nine test images degraded with Gaussian blur kernel $(G,7,5)$ and 30–60% impulse noise are listed in

Table 2. Numerical results of PSNR (dB), structural similarity (SSIM), and SNR (dB) for the recovery of nine test images degraded with a Gaussian blur kernel $(G,7,5)$ and 30–60% impulse noise. FTVd—fast total variation deconvolution; TGV—total generalized variation; OGSTV—overlapping group sparsity total variation.

Images	Noise Level	FTVd			TGV			OGSTV			Proposed
		PSNR	SSIM	SNR	PSNR	SSIM	SNR	PSNR	SSIM	SNR	PSNR	SSIM	SNR
Finger print	30	30.49	0.950	21.90	36.13	0.980	27.39	36.21	0.980	27.68	38.63	0.989	30.08
	40	29.16	0.913	19.84	35.32	0.976	26.79	35.64	0.981	26.4	37.44	0.983	28.90
	50	28.37	0.866	19.51	33.04	0.959	24.5	33.99	0.957	24.56	35.59	0.970	27.05
	60	25.74	0.831	16.27	28.89	0.915	20.35	28.84	0.906	19.32	30.60	0.923	22.07
Butterfly	30	28.21	0.894	21.96	30.37	0.919	24.17	30.84	0.932	24.59	32.17	0.946	25.92
	40	27.36	0.875	21.11	29.58	0.901	23.33	30.46	0.926	24.21	30.99	0.931	24.74
	50	25.42	0.825	19.17	28.45	0.869	22.2	28.83	0.895	22.45	29.96	0.912	23.71
	60	24.06	0.734	14.47	26.48	0.804	20.24	26.61	0.818	20.27	28.99	0.898	22.75
Baboon	30	21.29	0.628	15.83	22.93	0.756	17.47	23.15	0.765	17.69	24.01	0.806	18.55
	40	21.06	0.608	15.60	22.56	0.730	17.10	22.93	0.756	17.46	23.39	0.779	17.93
	50	20.62	0.572	15.15	22.08	0.693	16.63	21.90	0.702	16.44	22.66	0.738	17.20
	60	20.27	0.525	14.43	21.23	0.627	15.77	20.57	0.546	15.12	22.23	0.710	16.77
Boat	30	27.25	0.840	21.92	29.05	0.894	23.71	29.45	0.906	24.11	31.46	0.932	26.11
	40	26.49	0.802	21.16	28.33	0.873	22.99	29.06	0.898	23.71	30.28	0.914	24.94
	50	25.65	0.789	20.30	27.25	0.839	21.91	27.23	0.836	21.88	28.47	0.884	23.12
	60	23.90	0.756	18.56	25.13	0.766	19.80	26.30	0.780	21.06	27.37	0.849	21.78
Bird	30	31.03	0.914	21.97	32.82	0.934	23.76	33.49	0.947	24.43	34.54	0.957	25.47
	40	29.71	0.890	20.65	31.84	0.915	22.77	33.03	0.941	23.97	33.39	0.943	24.32
	50	26.73	0.831	17.67	29.46	0.874	20.39	30.45	0.882	21.38	32.31	0.930	23.24
	60	22.75	0.722	13.69	26.99	0.797	17.93	27.85	0.813	18.77	29.69	0.898	20.62
Hill	30	28.24	0.859	21.87	30.27	0.898	23.91	30.71	0.908	24.35	31.98	0.925	25.62
	40	27.24	0.831	20.88	29.48	0.878	23.11	30.29	0.899	23.93	30.94	0.909	24.57
	50	25.57	0.802	19.68	28.39	0.845	22.03	28.25	0.865	21.92	29.72	0.887	23.36
	60	24.87	0.760	18.51	26.16	0.773	19.79	27.03	0.845	20.69	28.52	0.855	21.74
Truck	30	29.50	0.887	22.23	32.79	0.926	25.52	33.48	0.939	26.22	34.22	0.946	26.96
	40	28.70	0.828	21.73	31.85	0.908	24.58	33.06	0.932	25.79	33.14	0.933	25.87
	50	27.26	0.794	19.99	30.42	0.873	23.15	31.17	0.905	23.90	32.50	0.923	25.23
	60	26.13	0.620	19.18	29.06	0.860	21.80	30.48	0.865	23.11	31.05	0.903	23.79
Plane	30	32.37	0.902	29.18	33.96	0.948	30.77	36.33	0.975	33.15	36.23	0.974	33.05
	40	31.68	0.898	28.49	33.00	0.929	29.82	35.20	0.961	31.37	35.30	0.965	32.11
	50	30.42	0.831	27.24	31.43	0.892	28.25	32.71	0.901	29.52	33.90	0.951	30.72
	60	27.90	0.730	24.72	29.73	0.764	26.04	30.07	0.805	26.89	31.96	0.927	28.78
Satellite	30	31.80	0.925	18.12	34.17	0.959	20.49	36.97	0.989	23.20	36.54	0.985	23.05
	40	30.38	0.910	16.70	33.39	0.945	19.72	36.00	0.983	22.32	35.76	0.980	22.08
	50	29.08	0.883	15.41	31.37	0.910	17.70	32.32	0.926	18.64	32.61	0.975	18.92
	60	27.99	0.830	14.31	29.54	0.855	15.86	30.05	0.904	16.38	30.21	0.944	16.72

. In the table, the optimal indicators are highlighted in bold.

The results of image restoration tests performed on nine test images degraded with a Gaussian blur kernel $(G,15,5)$ and 30% noise are shown in

Table 3. Numerical results of PSNR (dB), SSIM, and SNR (dB) for the recovery of nine test images degraded with a Gaussian blur kernel $(G,15,5)$ and 30% impulse noise.

Images	Degrada -tion	FTVd			TGV			OGSTV			Proposed
		PSNR	SSIM	SNR	PSNR	SSIM	SNR	PSNR	SSIM	SNR	PSNR	SSIM	SNR
Fingerprint	Blur: $(G,15,5)$ Noise level: 30%	27.31	0.877	18.78	29.89	0.935	21.36	30.09	0.934	21.55	30.72	0.970	22.18
Butterfly		24.91	0.705	18.67	26.96	0.861	20.72	27.05	0.864	20.80	27.47	0.873	21.22
Baboon		19.89	0.481	14.43	20.99	0.597	15.53	21.09	0.603	15.62	21.72	0.645	16.27
Boat		24.73	0.679	19.20	25.63	0.815	20.29	26.37	0.835	21.02	26.86	0.851	21.82
Bird		28.15	0.741	19.08	29.60	0.885	20.54	29.81	0.892	20.75	30.05	0.906	20.91
Hill		25.98	0.690	19.68	27.37	0.830	21.01	27.41	0.832	21.05	28.01	0.840	21.65
Truck		27.56	0.707	20.29	29.42	0.862	22.16	29.57	0.866	22.31	30.30	0.872	23.01
Plane		29.76	0.815	26.58	30.67	0.904	27.65	31.34	0.938	28.15	31.82	0.948	28.64
Satellite		28.20	0.821	14.52	29.66	0.900	15.98	31.47	0.963	17.79	31.39	0.962	17.71

.

An in-depth analysis of the data in Table 2 and Table 3 leads to the following conclusions:

• All four methods can effectively recover images that were degraded by Gaussian blur and different degrees of impulse noise. The PSNR, SSIM, and SNR values of the proposed method are generally higher than those of other competitive methods, which indicates that the proposed method has better deblurring and denoising effects. In particular, for images with many lines and complex textures, such as “Fingerprint” in Figure 3a, “Butterfly” in Figure 3b, and “Baboon” in Figure 3c, the proposed method performed well, especially at high noise levels. For example, for the “Fingerprint” image of Figure 3a treated with a Gaussian blur kernel $(G,7,5)$ and 50% noise, the PSNR value (35.59 dB) of the proposed method was higher by 7.22 dB than that of the FTVd method (28.37 dB), and was higher by 2.55 dB and 1.6 dB than that of the TGV method (33.04 dB) and the OGSTV method (33.99 dB), respectively.
• When restoring the nine degraded images treated with the Gaussian blur kernel $(G,7,5)$ and 30–60% impulse noise, the proposed method achieved 0.58–2.85 dB higher PSNR values compared with the TGV method. When restoring the nine degraded images treated with the Gaussian blur kernel $(G,15,5)$ and impulse noise, the proposed method achieved 0.45–1.73 dB higher PSNR values compared with the TGV method. For example, for the “Bird” image of Figure 3e treated with the Gaussian blur kernel $(G,7,5)$ and two different levels (50% vs. 60%) of noise, the PSNR values of the proposed method (32.31 dB and 29.69 dB) were 2.85 dB and 2.7 dB higher than those of the TGV method (29.46 dB and 26.99 dB), respectively.
• The OGSTV method is an excellent algorithm for image deblurring and impulse noise removal, and it uses the neighborhood gradient information of the image to form a combined gradient for image restoration, which achieves good image restoration performance for smooth image regions but is less satisfactory in recovering image edge regions containing sharp lines and angles. In the test, it was observed that, for smooth images with non-complex textures, such as the “Plane” image of Figure 3h and “Satellite” image of Figure 3i, the OGSTV method had particularly excellent performance, which was slightly better than that of the proposed algorithm in the case of low noise levels. In the case of high noise levels, however, the proposed algorithm was still superior to the OGSTV method. For example, when restoring the “Satellite” image degraded with $(G,7,5)$ and 30% noise, the OGSTV method achieved a slightly higher PSNR value of 36.97 dB compared to the proposed method (36.54 dB), with a difference of only 0.43 dB; however, in the cases of 50% and 60% noise, the PSNR values of the OGSTV method were lower than those of the proposed method.

The above analysis indicates that the use of Lp-pseudo-norm shrinkage in the proposed method increases the degree of freedom by one, making it less difficult to describe image gradient sparsity. The performance of the proposed method in removing Gaussian blur and impulse noise is generally better than that of the three competitive methods, especially for images with complex textures. In addition, the comparison reveals that the image restoration performance of the FTVd algorithm is much worse than that of the other three algorithms; thus, the subsequent experimental results of the FTVd algorithm are not listed in some tables.

4.3.2. Numerical Performance Comparison for Image Recovery in the Presence of Average Blur

The image restoration capability of the proposed method for degraded images treated with average blur and impulse noise was investigated, with

Table 4. Numerical results of PSNR (dB), SSIM, and SNR (dB) for the recovery of nine test images degraded with an average blur kernel $(A,7)$ and 30–60% impulse noise.

Images	Blur Size	Noise Level	TGV			OGSTV			Proposed
			PSNR	SSIM	SNR	PSNR	SSIM	SNR	PSNR	SSIM	SNR
Fingerprint	$(A,7)$	30	36.46	0.983	27.91	36.73	0.980	29.19	38.31	0.989	29.77
		60	28.74	0.905	20.20	29.30	0.907	21.24	30.11	0.910	21.57
Butterfly		30	30.40	0.918	24.15	30.99	0.933	24.74	32.21	0.947	25.97
		60	25.83	0.784	19.59	26.45	0.803	20.66	28.59	0.895	22.35
Baboon		30	23.01	0.761	17.54	23.29	0.778	17.83	24.11	0.814	18.65
		60	21.14	0.622	15.68	21.51	0.657	16.10	22.31	0.715	16.85
Boat		30	29.11	0.894	23.77	29.60	0.907	24.26	31.50	0.933	26.16
		60	25.11	0.755	19.77	26.24	0.776	20.52	27.22	0.855	21.88
Bird		30	32.75	0.933	23.67	33.53	0.948	24.47	34.53	0.957	25.46
		60	27.05	0.785	17.99	29.06	0.853	20.35	30.19	0.899	21.13
Hill		30	30.33	0.899	23.97	30.85	0.910	24.49	32.07	0.927	25.71
		60	25.96	0.757	19.60	26.10	0.762	20.07	27.96	0.852	21.60
Truck		30	32.79	0.925	25.52	33.61	0.940	26.34	34.32	0.947	27.06
		60	27.25	0.780	19.97	30.28	0.848	22.32	31.38	0.904	24.12
Plane		30	33.87	0.945	30.68	36.45	0.975	33.27	36.37	0.969	33.08
		60	28.15	0.793	24.97	31.87	0.922	28.69	32.31	0.926	29.13
Satellite		30	33.95	0.956	20.28	36.85	0.989	23.18	36.52	0.981	23.10
		60	28.06	0.814	14.38	28.52	0.933	14.84	28.54	0.933	14.87

presenting the results of image restoration tests performed on nine degraded test images treated with an average blur kernel $(A,7)$ and two levels (30% vs. 60%) of noise.

With regard to the image restoration performance of the algorithms for different images treated with average blur and different levels of impulse noise, the data in Table 4 lead to conclusions similar to those obtained in the case of Gaussian blur and impulse noise. When restoring the nine degraded test images treated with an average blur kernel $(A,7)$ and impulse noise, the proposed method achieved 0.48–4.16 dB higher PSNR values compared with the TGV method. For images with many lines and complex textures, such as the “Fingerprint” image of Figure 3a, “Butterfly” image of Figure 3b, and “Baboon” image of Figure 3c, the proposed method achieved 0.8–2.14 dB higher PSNR values compared with the OGSTV method. However, the OGSTV performed well in restoring smooth images in the presence of low noise levels.

4.3.3. Numerical Performance Comparison for Image Recovery in the Presence of Motion Blur

Furthermore, the image restoration capability of the proposed method in the case of motion blur and impulse noise was investigated.

Table 5. Numerical results of PSNR (dB) for the image restoration test conducted on four test images degraded with different levels of blur and impulse noise.

Images	Blur Size	Noise Level	FTVd	TGV	OGSTV	Proposed
Baboon	$(M,10,10)$	30	21.7921	24.0100	22.0647	24.8964
		40	21.1785	22.9020	23.2183	24.3687
	$(M,20,20)$	30	21.0863	23.0958	22.8402	23.6167
		40	20.7721	22.4554	22.3522	23.2369
Butterfly	$(M,30,30)$	30	26.0616	28.7070	28.4671	29.3110
		40	25.3602	27.9250	27.8820	28.9013
		50	24.2406	26.8643	27.2269	28.3973
		60	22.4797	25.2250	26.4264	27.7279
Bird	$(M,20,20)$	30	30.0618	31.7805	32.5596	33.3166
		40	28.7341	30.3195	31.8788	32.6383
		50	26.2732	28.2106	31.1538	31.7443
		60	22.9536	25.2543	30.1680	30.3756
Truck	$(M,20,20)$	30	30.3177	31.8087	32.3495	33.3499
		40	29.1381	30.3154	31.7592	32.7189
		50	27.5260	28.4093	31.1104	31.9235
		60	24.7013	25.2220	30.0642	30.7173

presents the PSNR results of image restoration tests performed on the degraded “Butterfly” image of Figure 3b, “Baboon” image of Figure 3c, “Bird” image of Figure 3e, and “Truck” image of Figure 3g treated with different motion blur kernels and 30–60% noise.

For different images treated with varying degrees of motion blur and impulse noise, the data in Table 5 reveal that the proposed method outperformed the TGV and OGSTV methods in removing motion blur and impulse noise. When restoring the four images, the proposed method achieved 0.52–5.50 dB and 0.21–2.83 dB higher PSNR values than the TGV method and the OGSTV method, respectively.

4.4. Comparison of Visual Effects

To better observe the test results, some restored images are presented below for visual comparison purposes.

4.4.1. Comparison of the Visual Effects of Restored Images in the Case of Gaussian Blur

Figure 6 and Figure 7 present image restorations by the four algorithms of the “Boat” image degraded with a Gaussian blur kernel $(G,7,5)$ and 30% impulse noise, and zoomed-in images of the restored images for better comparison, respectively. With respect to the visual effects of the restored images, with the FTVd method (c), a loss of some image details occurred during restoration of the image, and there was an obvious staircase effect. The TGV method (d) and the OGSTV method (e) performed well in suppressing the staircase effect, but slight ringing artefacts were generated when the TGV method was used to restore the image. Furthermore, the OGSTV method performed well in smooth image regions but relatively poorly preserved image edges. The proposed method, however, not only preserved image edges, but also restored smooth image regions well, achieving the best overall visual effects for image restoration.

4.4.2. Visual Effect Comparison for Image Restoration in the Case of Average Blur

Figure 8 and Figure 9 present the “Fingerprint” image restored using the four algorithms from the image degraded with average blur kernel $(A,7)$ and 50% impulse noise, as well as the zoomed-in restored images for better comparison. With respect to the visual effects of the restored images, the FTVd method (c) led to a certain degree of staircase effect in the restored image, the TGV method (d) led to blurred image edges in the restored image, and the OGSTV method (e) needed to be improved in preserving image edges. In contrast, the proposed method performed well in restoring the details of the image lines.

4.4.3. Visual Effect Comparison for Image Restoration of Images Degraded with Motion Blur

Figure 10 compares the zoomed-in details of the “Butterfly” image restored using the four algorithms with those of the image degraded with a motion blur kernel $(M,20,20)$ and 30% impulse noise.

Figure 11 presents a visual effects comparison of the image restorations using the four algorithms for the following images: the “Bird” image degraded with a motion blur kernel $(M,20,20)$ and 30% noise, the “Baboon” image degraded with a motion blur kernel $(M,10,10)$ and 40% noise, the “Butterfly” image degraded with a motion blur kernel $(M,30,30)$ and 50% noise, and the “Truck” image degraded with a motion blur kernel $(M,20,20)$ and 60% noise. As shown in the figure, the proposed method allowed for better preservation of the smooth image regions and restoration of the details of the image lines with fewer artefacts.

4.5. Comparison of Computing Time

Finally, to further test the timeliness of the proposed model, the proposed method was compared with the FTVd, TGV, and OGSTV methods by testing the mean time required to restore the nine images of Figure 3 for different blur kernels at various impulse noise levels. Some test results are shown in

Table 6. Image restoration time (s) of the degraded test images with different levels of blur and impulse noise.

Images	Blur Type	Noise Level	FTVd	TGV	OGSTV	Proposed
Fingerprint 256 × 256	$(G,7,5)$	30	0.86	1.47	0.81	1.19
	$(A,7)$	40	0.92	1.48	0.98	1.17
	$(M,20,20)$	30	0.81	1.47	1.09	1.22
	$(M,20,20)$	60	0.84	1.48	1.47	1.17
Plane 512 × 512	$(G,7,5)$	30	3.28	6.03	6.20	5.31
	$(A,7)$	40	3.42	6.17	5.98	5.36
	$(M,20,20)$	30	3.61	6.28	6.78	5.33
	$(M,20,20)$	60	3.27	6.36	4.16	5.36
Satellite 512 × 512	$(G,7,5)$	30	3.36	7.11	3.11	5.27
	$(A,7)$	40	3.50	6.20	2.66	5.34
	$(M,20,20)$	30	3.31	6.22	3.59	5.16
	$(M,20,20)$	60	3.23	6.23	4.09	5.14

, where the best indicators are highlighted in bold.

The calculation times in the table reveal that FTVd, which is a classic algorithm, is still advantageous in terms of computing speed. The proposed method is comparable to the OGSTV method in terms of image restoration time, while they are both faster than the TGV method. The proposed method uses the Lp-pseudo-norm as a regularization constraint, which increases the method’s computational burden, thereby leading to a certain degree of increase in the image restoration time. However, the gradient difference is re-expressed in the form of a convolution in this study, where it corresponds to the dot multiplication of two matrix spectra in the frequency domain; moreover, FFT is employed for convolution calculation. These strategies effectively avoid large-scale matrix multiplications, thus speeding up the calculation.

5. Conclusions

In this study, a regularized method for image restoration based on the Lp-pseudo-norm and TGV was proposed to de-blur images and remove impulse noise. The method constrains the first-order and second-order gradients of the image and takes advantage of the sparse representation ability of the Lp-pseudo-norm for image gradients. In this way, it effectively removes image blur and noise, and reduces undesirable artefacts (such as staircase and ringing artefacts) in image restoration while better preserving the edges and details of the image. A large number of experimental results revealed that the proposed method is consistently superior to several state-of-the-art methods in terms of numerical indicators and visual effects. The regularization parameters $\mu$ of the Lp-pseudo-norm used in the experiments were relatively invariant, indicating that the proposed algorithm is robust. In follow-up studies, efforts will be made to further improve the computational efficiency of the proposed method. It should be noted that the proposed algorithm uses a generalized regularization term, which can be easily extended to other algorithms and is suitable for other image restoration problems.