The PPADMM Method for Solving Quadratic Programming Problems

Abstract

In this paper, a preconditioned and proximal alternating direction method of multipliers (PPADMM) is established for iteratively solving the equality-constraint quadratic programming problems. Based on strictly matrix analysis, we prove that this method is asymptotically convergent. We also show the connection between this method with some existing methods, so it combines the advantages of the methods. Finally, the numerical examples show that the algorithm proposed is efficient, stable, and flexible for solving the quadratic programming problems with equality constraint.

1. Introduction

This manuscript introduces the following convex optimization model with linear constraints and a separable objective function:

$$\{\begin{array}{l}\min \hspace{0.17em}{f}_1(x)+f_2(y),\\ s.t.\hspace{0.17em}Ax+By=b,\end{array}$$

(1)

where $A\in {ℝ}^{p\times n}$ and $B\in {ℝ}^{p\times m}$ are two matrices, and $b\in {ℝ}^p$ is a known vector. The objective function $f_1:{ℝ}^n\to ℝ$ and $f_2:{ℝ}^m\to ℝ$ are two quadratic functions defined by

$$\{\begin{array}{l}{f}_1(x)={\displaystyle \frac{1}{2}}{x}^{T}Fx+x^{T}f,\\ f_2(y)={\displaystyle \frac{1}{2}}{y}^{T}Gy+y^{T}g,\end{array}$$

(2)

where $F\in {ℝ}^{n\times n},G\in {ℝ}^{m\times m}$ are symmetric positive semidefinite matrices and $f\in {ℝ}^n,g\in {ℝ}^m$ are the known vectors. The class of convex minimization problems arises in many areas of computational science and engineering applications such as compressed sensing [1], financial [2,3], image restoration [4,5,6], network optimization problems [7,8], and traffic planning convex problems [9,10,11,12]. The model (1)–(2) captures many applications in different areas—see the l1-norm regularized least-squares problems in [12,13], the total variation image restoration in [13,14,15,16], and the standard quadratic programming problems in [7,13].

Let $H\in {ℝ}^{p\times p}$ be the symmetric positive definite matrix, $<\cdot ,\cdot {>}_H$ represents the weighted inner product with the weighting matrix H, $\Vert \cdot \Vert$ stands for the Euclidean norm, and ${\Vert \cdot \Vert }_H$ stands for the analogous weighted matrix norm. Note that for the vectors $u,v\in {ℝ}^p$ and the matrix $X\in {ℝ}^{p\times p}$, it holds that $<u,v{>}_H=<Hu,v>$, ${\Vert u\Vert }_H=\Vert H^{{\displaystyle \frac{1}{2}}}u\Vert$ and ${\Vert X\Vert }_H=\Vert H^{{\displaystyle \frac{1}{2}}}X{H}^{-{\displaystyle \frac{1}{2}}}\Vert$.

If $<u,v{>}_H=0$, we say that $u,v\in {ℝ}^p$ are H-orthogonal, which is denoted by $u{\perp }_{H}v$. In particular, if H is the identity matrix, then the vectors $u$ and $v$ are orthogonal, which is simply denoted by $u\perp v$. For $\zeta \in ℂ$, $\overline{\zeta }$ stands for its conjugate complex.

The problem (1)–(2) is mathematically equivalent to the unconstraint optimization problem [7]

$$\underset{\lambda }{\max }\underset{x,y}{\min }\psi (x,y,\lambda )$$

(3)

$\psi (x,y,\lambda )$ is the augmented Lagrangian function defined as

$$\psi (x,y,\lambda )=f_1(x)+f_2(y)-<Ax+By-b,\lambda >+\frac{\beta }{2}{\Vert Ax+By-b\Vert }^2,$$

(4)

where $\lambda$ is the Lagrangian multiplier, and $\beta$ is a regularization parameter. In a word, a point $(x_{*},y_{*})$ is the solution to the problem (1)–(2) if and only if there exists ${\lambda }_{*}\in {ℝ}^p$ such that the point $(x_{*},y_{*},{\lambda }_{*})\in {ℝ}^n\times {ℝ}^m\times {ℝ}^p$ is the solution to the problem (3)–(4) [7].

The most common method to solve the problem (3)–(4) is the alternating direction method of multipliers (ADMM) [7], in which each iteration of the augmented Lagrangian method (ALM) [17,18] has a Gauss–Seidel decomposition. The scheme of the ADMM method for (1)–(2) is

$$\{\begin{array}{l}{x}^{(k+1)}=\arg \min \{\psi (x,y^{(k)},{\lambda }^{(k)})\},\\ y^{(k+1)}=\arg \min \{\psi (x^{(k+1)},y,{\lambda }^{(k)})\},\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\beta (A{x}^{(k+1)}+B{y}^{(k+1)}-b).\end{array}$$

(5)

A significant advantage of ADMM is that each subproblem involves only one of the functions f₁ and f₂ in the original problem. Therefore, the variables x and y are treated separately during iterations, which makes solving the subproblems in (3)–(4) evidently easier than the original problem (1)–(2). The ADMM’s easy implementation and impressive efficiency has recently received extensive attention from many different areas. The ADMM is a very effective method to solve the convex optimization problem. Glowinski and Marrocco [19] were the first batch of scientists to describe it. Gabay [20], Glowinski and Le Tallec [21], as well as Eckstein and Bertsekas [22] studied some convergence results related to ADMM. It is suitable to solve convex programming problems with separable variables [7,23,24], and they are widely used in image processing, statistical learning, and machine learning [4,5,6,7,8,9].

For the convex optimization problems, some methods based on gradient operator are sensitive to the choice of the iteration step. If the parameters are not properly selected, the algorithm may not be convergent. In contrast, the ADMM is robust to the choice of parameters: under some mild conditions, the method can guarantee convergence for any positive parameter of its single parameter. When the objective function is a quadratic function, its global convergence is proved [13,24], and this method is linearly convergent. For example, the ADMM converges linearly to the optimal solution for problems (1) and (2). Although the convergence of the ADMM is perfectly solved, the accurate estimation of its convergence rate is still in its early stages; see, for example, [13,14].

Since the classical ADMM algorithm is inefficient in solving the accuracy of subproblem, Deng and Wo proposed a generalized ADMM in the literature [25], which adds the proximal terms $\frac{1}{2}{\Vert x-x^{(k)}\Vert }_P^2$ and $\frac{1}{2}{\Vert y-y^{(k)}\Vert }_T^2$ to the x- and y-subproblems, respectively. Its scheme for (1)–(2) is

$$\{\begin{array}{l}{x}^{(k+1)}=\arg \min \left\{\psi (x,y^{(k)},{\lambda }^{(k)})+\frac{1}{2}{\Vert x-x^{(k)}\Vert }_P^2\right\},\\ y^{(k+1)}=\arg \min \left\{\psi (x^{(k+1)},y,{\lambda }^{(k)})+\frac{1}{2}{\Vert y-y^{(k)}\Vert }_T^2\right\},\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\alpha \beta \left(A{x}^{(k+1)}+B{y}^{(k+1)}-b\right),\end{array}$$

(6)

where $\alpha \in (0,{\displaystyle \frac{1+\sqrt{5}}{2}})$, and the matrices P and T are symmetric positive semidefinite.

Deng and Wo solved the problem of global and linear convergence of the generalized ADMM and gave the mathematical proof where P and T are symmetric positive semidefinite matrices. What is more, in order to make the subproblems of the generalized ADMM easier to solve and more efficient to run, the ultimate goal is to choose the adequate P and T.

In order to iteratively solve the linear constraint quadratic programming problem (1)–(2), Bai and Tao [18] proposed a class of preconditioned alternating variable minimization with multiplier (PAVMM) methods by the matrix preconditioning strategy and utilizing a parameter accelerating technique, which is based on a weighted inner product and the homologous weighted norm. The iteration scheme is as follows:

$$\{\begin{array}{l}{x}^{(k+1)}=\arg \min \left\{\tilde{\psi }(x,y^{(k)},{\lambda }^{(k)})\right\},\\ y^{(k+1)}=\arg \min \left\{\tilde{\psi }(x^{(k+1)},y,{\lambda }^{(k)})\right\},\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\alpha Q^{-1}{W}^{-1}\left(A{x}^{(k+1)}+B{y}^{(k+1)}-b\right),\end{array}$$

(7)

where $\tilde{\psi }(x,y,\lambda )$ is the following weighted augmented Lagrangian function:

$$\tilde{\psi }(x,y,\lambda )=f_1(x)+f_2(y)-<Ax+By-b,\lambda {>}_{W^{-1}}+\frac{\beta }{2}{\Vert Ax+By-b\Vert }_{W^{-1}}^2,$$

(8)

the matrices W and Q are symmetric positive semidefinite and nonsingular, and $\alpha$ is the relaxation parameter. Actually, the PAVMM method is a class of preconditioned alternating direction method of multipliers (PADMM). Therefore, in this manuscript, the PAVMM method is recorded as PADMM. In particular, the ADMM is a special case of the PADMM. If $W=Q=I$ and $a=\beta$, the PADMM automatically reduces to the ADMM. Later, Bai and Tao [26] also establish a preconditioned and relaxed alternating variable minimization with multiplier (PRAVMM) method based on the PAVMM method, and the scheme is

$$\{\begin{array}{l}{\widehat{x}}^{(k+1)}=\arg \min \{\tilde{\psi }(x,y^{(k)},{\lambda }^{(k)})\},\\ x^{(k+1)}=\varpi {\widehat{x}}^{(k+1)}+(1-\varpi )x^{(k)},\\ {\widehat{y}}^{(k+1)}=\arg \min \{{\tilde{L}}_{\beta }(x^{(k+1)},y,{\lambda }^{(k+{\displaystyle \frac{1}{2}})})\},\\ y^{(k+1)}=\tau {\widehat{y}}^{(k+1)}+(1-\tau )y^{(k)},\\ {\lambda }^{(k+1)}={\lambda }^{(k+{\displaystyle \frac{1}{2}})}-\alpha Q^{-1}{W}^{-1}(A{x}^{(k+1)}+B{y}^{(k+1)}-b)\end{array}$$

(9)

where $\varpi$, $\tau$, and $\alpha$ are positive constants. As above, we also rewrite the PRAVMM into PRADMM. In order to achieve acceleration, it is obvious that the PRADMM adds two relaxation parameters to the iterative process. Hence, the PADMM and ADMM are the special cases of the PRADMM. When $\varpi =\tau =1$, the PRADMM degenerates into the PADMM; when $\varpi =\tau =1$, $W=Q=I$ and $a=\beta$; then, the PRADMM reduces to the ADMM.

To further generalize the PADMM and maximally promote the convergence speed of the PADMM, in this manuscript, we establish a preconditioned and proximal alternating direction of multiplier (PPADMM) to solve the problem (1)–(2) by the iterative methods. Assuming that the matrices $F\in {ℝ}^{n\times n}$ and $G\in {ℝ}^{m\times m}$ are symmetric positive semidefinite, we should choose reasonably the parameters, the weighting, and the preconditioning matrices; the PPADMM is convergent to the unique solution of the problem (1.1)–(1.2) for any initial guess. Obviously, the PPADMM proposed in this paper is an extension of the PADMM. Hence, the ADMM and PADMM are special cases of the PPADMM. In addition, we test the robustness and effectiveness of the PPADMM by using numerical experiments. The experimental results indicate that this method performs better than ADMM, PADMM, and PRADMM when they are employed to solve the convex optimization programming problem (1)–(2).

The paper is organized as follows. In Section 2, the computational properties of PPADMM was established. We introduce some necessary concepts for analyzing the asymptotic convergence of the PPADMM in Section 3. Then, based on the analysis in Section 3, the asymptotic convergence of the PPADMM was demonstrated in Section 4. In Section 5, we have some test by solving the image deblurring problem to illustrate that our method is effective. Finally, we give some concluding remarks in Section 5.

2. The PPADMM Method

In this section, we will introduce the PPADMM proposed in this paper. At the k-th iteration step of the PPADMM, we add the proximal term $\frac{1}{2}{\Vert x-x^{(k)}\Vert }_P^2$ and $\frac{1}{2}{\Vert y-y^{(k)}\Vert }_T^2$ to the x- and y-subproblems during computing minimal points $x^{(k+1)}$ and $y^{(k+1)}$ of the minimization subproblems. In order to solve the problem (1)–(2) by the iterative methods, we are able to establish the preconditioned and proximal alternating direction method of multipliers (PPADMM) as follows:

$$\{\begin{array}{l}{x}^{(k+1)}=\arg \min \left\{\tilde{\psi }(x,y^{(k)},{\lambda }^{(k)})+\frac{1}{2}{\Vert x-x^{(k)}\Vert }_P^2\right\},\\ y^{(k+1)}=\arg \min \left\{\tilde{\psi }(x^{(k+1)},y,{\lambda }^{(k)})+\frac{1}{2}{\Vert y-y^{(k)}\Vert }_T^2\right\},\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\alpha \beta Q^{-1}{W}^{-1}\left(A{x}^{(k+1)}+B{y}^{(k+1)}-b\right),\end{array}$$

(10)

where $P\in {ℝ}^{n\times n}$ and $T\in {ℝ}^{m\times m}$ are two symmetric positive semidefinite matrices, and $\alpha$ and $\beta$ are positive constants. We choose $P={\tau }_1{I}_n-\beta A^{T}A$ with the requirement ${\tau }_1>\beta \Vert A^{T}A\Vert$ and $T={\tau }_2{I}_m-\beta B^{T}B$ with the requirement ${\tau }_2>\beta \Vert B^{T}B\Vert$. In the same way, the matrices W, Q are generated as follows:

$$W={\xi }_1{I}_n-\beta A^{T}A,Q={\xi }_2{I}_m-\beta B^{T}B$$

where ${\xi }_1\in (0,{\displaystyle \frac{1}{{\lambda }_{\max }(A^{T}A)}})$, ${\xi }_2\in (0,{\displaystyle \frac{1}{{\lambda }_{\max }(B^{T}B)}})$.

It is easy to get the derivative of the quadratic functions $f_1(x)$ and $f_2(y)$ in (2) as follows:

$$f_1^{\prime }(x)=Fx+f\hspace{0.17em}\mathrm{and}\hspace{0.17em}{f}_2^{\prime }(y)=Gy+g.$$

(11)

By making full use of Equation (10), the first and second for formulas in Equation (10) are differentiated for x and y, respectively, with a simple manipulation. Afterwards, the iteration scheme of PPADMM can be rewritten as the following mathematically equivalent matrix–vector form:

$$\{\begin{array}{l}F{x}^{(k+1)}+f-A^T{W}^{-1}{\lambda }^{(k)}+\beta A^T{W}^{-1}(A{x}^{(k+1)}+B{y}^{(k)}-b)+P(x^{(k+1)}-x^{(k)})=0,\\ G{y}^{(k+1)}+g-B^T{W}^{-1}{\lambda }^{(k)}+\beta B^T{W}^{-1}(A{x}^{(k+1)}+B{y}^{(k+1)}-b)+T(y^{(k+1)}-y^{(k)})=0\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\alpha \beta Q^{-1}{W}^{-1}(A{x}^{(k+1)}+B{y}^{(k+1)}-b).\end{array},$$

(12)

which can be equivalently reformulated as

$$\{\begin{array}{l}(F+\beta A^T{W}^{-1}A+P)x^{(k+1)}=P{x}^{(k)}+A^T{W}^{-1}[\beta (b-B{y}^{(k)})+{\lambda }^{(k)}]-f,\\ (G+\beta B^T{W}^{-1}B+T)y^{(k+1)}=B^T{W}^{-1}[\beta (b-A{x}^{(k+1)})+{\lambda }^{(k)}]+T{y}^{(k)}-g,\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\alpha \beta Q^{-1}{W}^{-1}(A{x}^{(k+1)}+B{y}^{(k+1)}-b).\end{array}$$

(13)

Since F, G and P, T are symmetric positive semidefinite matrices, W is a symmetric positive definite matrix, and $\alpha >0$, $\beta >0$; thus, we know that the matrices $F+\beta A^T{W}^{-1}A+P$ and $G+\beta B^T{W}^{-1}B+T$ are symmetric positive definite if and only if the following statements are true [18]:

(a)

$null(F)\cap null(A)=\left\{0\right\}$ or $null(F)\cap null(P)=\left\{0\right\}$ or $null(A)\cap null(P)=\left\{0\right\}$.

(b)

$null(G)\cap null(B)=\left\{0\right\}$ or $null(G)\cap null(T)=\left\{0\right\}$ or $null(B)\cap null(T)=\left\{0\right\}$.

Therefore, when these above null-space conditions of PPADMM are well satisfied, measuring the costs of computation in the iteration scheme (13), it is obvious that the main costs of the PPADMM are to solve the linear systems with coefficient matrices $F+\beta A^T{W}^{-1}A+P$ and $G+\beta B^T{W}^{-1}B+T$. When the sizes n, m, and/or p are relatively small, some direct methods such as the Cholesky factorization are able to effectively solve these systems of linear equations. However, when the sizes n, m, and/or p are huge, it will take a lot of time to solve these linear systems by the direct methods. So, the iterative methods are used to solve the problems, e.g., the preconditioned conjugate methods. Of course, the weighted matrices W, P, and T, and penalty parameters $\alpha ,\beta$ should be reasonably selected so that both the matrices $F+\beta A^T{W}^{-1}A+P$ and $G+\beta B^T{W}^{-1}B+T$ have the better conditions than the original matrices F and G, respectively. It will make the linear equations with the coefficient matrices $F+\beta A^T{W}^{-1}A+P$ and $G+\beta B^T{W}^{-1}B+T$ accurate, fast, and robust at each step of the PPADMM.

For the linear systems $Ax=b$, we define

$$A=D-C_L-C_U,$$

where $A=(a_{ij})\in {ℂ}^{m\times n}$ ($m\le n$), $D=diag(A_{11},\cdots ,A_{m\times m})$, $C_L$ and $C_U$ are strictly lower triangular matrix and strictly upper triangular matrix, respectively. Let $L=D^{-1}{C}_L,U=D^{-1}{C}_U$, then, the block SOR iterative method is described as

$$\begin{array}{c}{x}^{(k+1)}=L_w{x}^{(k)}+w{(I-wL)}^{-1}{D}^{-1}b,\\ L_w={(I-wL)}^{-1}[(1-w)I+wU].\end{array}$$

According to the iterative scheme (13), we can see that the PPADMM can be classified as a modified block SOR iterative method for solving the following linear systems.

$$\{\begin{array}{l}(F+\beta A^T{W}^{-1}A)x-\beta A^T{W}^{-1}By-A^T{W}^{-1}z=\beta A^T{W}^{-1}b-f\\ \beta B^T{W}^{-1}Ax+(G+\beta B^T{W}^{-1}B)y-B^T{W}^{-1}z=\beta B^T{W}^{-1}b-g\\ W^{-1}Ax+W^{-1}By=W^{-1}b\end{array}$$

For the properties that a saddle point of the weighted augmented Lagrangian function $\psi (x,y,\lambda )$ defined in (3) possess, we refer to the literature [18] for more details.

Theorem 1.

[7]Let$F\in {ℝ}^{n\times n}$,$G\in {ℝ}^{m\times m}$be symmetric positive semidefinite matrices and$A\in {ℝ}^{p\times n}$,$B\in {ℝ}^{p\times m}$be two arbitrary matrices. Let

$$\mathit{A}(\beta )=\left[\begin{array}{ccc}F+\beta A^T{W}^{-1}A& \beta A^T{W}^{-1}B& -A^T{W}^{-1}\\ \beta B^T{W}^{-1}A& G+\beta B^T{W}^{-1}B& -B^T{W}^{-1}\\ W^{-1}A& W^{-1}B& 0\end{array}\right], \mathit{b}(\beta )=\left(\begin{array}{c}\beta A^T{W}^{-1}b-f\\ \beta B^T{W}^{-1}b-f\\ W^{-1}b\end{array}\right),\mathit{x}=\left(\begin{array}{c}x\\ y\\ \lambda \end{array}\right).$$

Then, the following results are obtained:

(i)

For $x_{*}\in {ℝ}^n,y_{*}\in {ℝ}^m$ and ${\lambda }_{*}\in {ℝ}^p$, then the point $(x_{*},y_{*},{\lambda }_{*})$ is a saddle point of the weighted augmented Lagrangian function $\psi (x,y,\lambda )$ defined in (3) if and only if $x_{*}={(x_{*}^T,y_{*}^T,{\lambda }_{*}^T)}^T$ is a solution of the linear system $\mathrm{A}(\beta )\mathrm{x}=\mathrm{b}(\beta )$;

(ii)

The matrix $\mathrm{A}(\beta )\in {ℝ}^{(n+m+p)\times (n+m+p)}$ is nonsingular if and only if

(a)

$(null(F)\oplus null(G))\cap null((A\hspace{0.17em}B))=\left\{0\right\}$

(b)

$null(A^T)\oplus null(B^T)=\left\{0\right\}$.

In addition, for the complex quadratic polynomial equation, the determinant criterion for locations of its two roots is shown in Lemma 1. It is indispensable to prove the asymptotic convergence of the PPADMM in Section 4.

Lemma 1.

[16,27,28]Assume$\eta$and$\zeta$are two complex constants; then, both roots of the complex quadratic polynomial equation${\lambda }^2+\zeta \lambda +\eta =0$have a modulus of less than one if and only if

$$|\zeta -\overline{\zeta }\eta |+|\eta {|}^2<1.$$

(14)

In particular, if both $\eta$ and $null(G)\cap null(B)=\left\{0\right\}$ are real constants, the condition (14) degrades into $\left|\eta \right|<1$ and $\left|\zeta \right|<1+\eta$.

3. The Asymptotic Convergence of the PPADMM

In this section, we will prove that the PPADMM is globally and asymptotically convergent, and its corresponding asymptotic convergence rate was also estimated.

First, we define the matrices

$$M_{\beta }(\alpha )=\left[\begin{array}{ccc}F+\beta A^T{W}^{-1}A+P& 0& 0\\ \beta B^T{W}^{-1}A& G+\beta B^T{W}^{-1}B+T& 0\\ W^{-1}A& W^{-1}B& {\displaystyle \frac{1}{\alpha \beta }}Q\end{array}\right],N_{\beta }(\alpha )=\left[\begin{array}{ccc}P& -\beta A^T{W}^{-1}B& A^T{W}^{-1}\\ 0& T& B^T{W}^{-1}\\ 0& 0& {\displaystyle \frac{1}{\alpha \beta }}Q\end{array}\right].$$

Obviously, the following matrix equations are true:

$$\mathrm{A}(\beta )=M_{\beta }(\alpha )-N_{\beta }(\alpha )=\left[\begin{array}{ccc}F+\beta A^T{W}^{-1}A& \beta A^T{W}^{-1}B& -A^T{W}^{-1}\\ \beta B^T{W}^{-1}A& G+\beta B^T{W}^{-1}B& -B^T{W}^{-1}\\ W^{-1}A& W^{-1}B& 0\end{array}\right].$$

(15)

Hence, the iteration scheme (13) of the PPADMM can be rewritten as the equivalent matrix-vector form

$$M_{\beta }(\alpha ){\mathrm{x}}^{(k+1)}=N_{\beta }(\alpha ){\mathrm{x}}^{(k)}+\mathit{b}(\beta ).$$

(16)

Multiplying $M_{{}_{\beta }}^{-1}(\alpha )$ by both sides of Equation (16), we obtain

$${\mathrm{x}}^{(k+1)}=M_{{}_{\beta }}^{-1}(\alpha )N_{\beta }(\alpha ){\mathrm{x}}^{(k)}+M_{{}_{\beta }}^{-1}(\alpha )\mathit{b}(\beta ).$$

(17)

It is easy to know that $L_{\beta }(\alpha )=M_{{}_{\beta }}^{-1}(\alpha )N_{\beta }(\alpha )$ is the iterative matrix of (17). For the matrix splitting of (14), the iterative scheme of the PPADMM can be equivalently reconstructed into the matrix splitting iteration method (16), which can solve the linear equation A(β)x = b(β). Therefore, the PPADMM is asymptotically and globally convergent if and only if the spectral radius of the iterative matrix $L_{\beta }(\alpha )$ is less than one, i.e., $\rho (L_{\beta }(\alpha ))<1$. We define the weighted matrices

$$\widehat{A}=W^{-\frac{1}{2}}A,\hspace{0.17em}\widehat{B}=W^{-\frac{1}{2}}B,\hspace{0.17em}\widehat{Q}=W^{\frac{1}{2}}Q{W}^{\frac{1}{2}},$$

(18)

the augmented matrices

$$\widehat{F}=F+\beta A^T{W}^{-1}A+P,\hspace{0.17em}\widehat{G}=G+\beta B^T{W}^{-1}B+T,$$

(19)

and the compounded matrices

$$\widehat{R}=\widehat{A}{(P-\lambda \widehat{F})}^{-1}{\widehat{A}}^T,\hspace{0.17em}\widehat{S}=\widehat{B}{(T-\lambda \widehat{G})}^{-1}{\widehat{B}}^T.$$

(20)

Therefore, $M,N$ can be rewritten as

$$M_{\beta }(\alpha )=\left[\begin{array}{ccc}\widehat{F}& 0& 0\\ \beta {\widehat{B}}^T\widehat{A}& \widehat{G}& 0\\ W^{-\frac{1}{2}}\widehat{A}& W^{-\frac{1}{2}}\widehat{B}& {\displaystyle \frac{1}{\alpha \beta }}Q\end{array}\right],N_{\beta }(\alpha )=\left[\begin{array}{ccc}P& -\beta {\widehat{A}}^T\widehat{B}& {\widehat{A}}^T{W}^{-\frac{1}{2}}\\ 0& T& {\widehat{B}}^T{W}^{-\frac{1}{2}}\\ 0& 0& {\displaystyle \frac{1}{ab}}Q\end{array}\right].$$

Here, we define the block diagonal matrix

$$H=\left[\begin{array}{ccc}I& 0& 0\\ 0& I& 0\\ 0& 0& W^{\frac{1}{2}}\end{array}\right].$$

After preconditioning $M_{\beta }(\alpha )$ and $N_{\beta }(\alpha )$, respectively, we can obtain the block lower-triangular matrix ${\widehat{M}}_{\beta }(\alpha )$ and the block upper-triangular matrix ${\widehat{N}}_{\beta }(\alpha )$ as follows:

$${\widehat{M}}_{\beta }(\alpha )=H{M}_{\beta }(\alpha )H=\left[\begin{array}{ccc}\widehat{F}& 0& 0\\ \beta {\widehat{B}}^T\widehat{A}& \widehat{G}& 0\\ \widehat{A}& \widehat{B}& {\displaystyle \frac{1}{\alpha \beta }}\widehat{Q}\end{array}\right],{\widehat{N}}_{\beta }(\alpha )=H{N}_{\beta }(\alpha )H=\left[\begin{array}{ccc}P& -\beta {\widehat{A}}^T\widehat{B}& {\widehat{A}}^T\\ 0& T& {\widehat{B}}^T\\ 0& 0& {\displaystyle \frac{1}{\alpha \beta }}\widehat{Q}\end{array}\right].$$

It follows that ${\widehat{L}}_{\beta }(\alpha )={\widehat{M}}_{\beta }{(\alpha )}^{-1}{\widehat{N}}_{\beta }(\alpha )=H^{-1}{L}_{\beta }(\alpha )H$. It is obvious that ${\widehat{L}}_{\beta }(\alpha )$ and $L_{\beta }(\alpha )$ are similar matrices, so they have the same eigenvalues. When $\rho (L_{\beta }(\alpha ))<1$, the PPADMM globally converges to the optimal solution of the problem (1)–(2).

Theorem 2.

Suppose that$F\in {ℝ}^{n\times n}$and$G\in {ℝ}^{m\times m}$are symmetric positive semidefinite matrices,$A\in {ℝ}^{p\times n}$and$B\in {ℝ}^{p\times m}$are two matrices such that:

(a)

$(null(F)\oplus null(G))\cap null((A\hspace{0.17em}B))=\left\{0\right\}$and

(b)

$null(A^T)\oplus null(B^T)=\left\{0\right\}$.

Define the scaled matrices $\widehat{A},\widehat{B},\widehat{Q},\widehat{F},\widehat{G}$ and $\widehat{R},\widehat{S}$ as in (18)–(19). Suppose that $\lambda$ is a nonzero eigenvalue of the PPADMM iteration matrix $L_{\beta }(\alpha )$. If the matrices $I+\beta \lambda \widehat{S}$, $P-\lambda \widehat{F}$, and $T-\lambda \widehat{G}$ are nonsingular, then $\lambda$ is an eigenvalue of the following eigenvalue problem $({\lambda }^2{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\lambda E+\tilde{Q})\tilde{w}=0$, where

$$E=\alpha \beta \tilde{S}-\tilde{Q}+\alpha {\beta }^2\widehat{R}\tilde{S}-{\beta }^2\widehat{R}\tilde{S}\tilde{Q}+\alpha \beta \widehat{R},$$

$$\{\begin{array}{l}\tilde{Q}={(I+\beta \lambda \widehat{S})}^{-1}\widehat{Q}(I+\beta \lambda \widehat{S}),\\ \tilde{S}={(I+\beta \lambda \widehat{S})}^{-1}\widehat{S}(I+\beta \lambda \widehat{S}),\\ \tilde{w}={(I+\beta \lambda \widehat{S})}^{-1}w.\end{array}$$

(21)

Proof. 

In order to analyze the nonzero eigenvalues of $L_{\beta }(\alpha )$, we first need to analyze the nonzero eigenvalues of ${\widehat{L}}_{\beta }(\alpha )$ because the matrices ${\widehat{L}}_{\beta }(\alpha )$ and $L_{\beta }(\alpha )$ are similar. Obviously, the conditions (a) and (b) guarantee the nonsingularity of the matrix $A(\beta )$, $\lambda =1$ is not an eigenvalue of ${\widehat{L}}_{\beta }(\alpha )$.

Assume $\lambda$ is a nonunit and nonzero eigenvalue of the matrix ${\widehat{L}}_{\beta }(\alpha )$, $\mathrm{u}=(u^T,v^T,w^T{)}^T\in {ℂ}^{n+m+p}$, with $u\in {ℂ}^n$, $v\in {ℂ}^m$ and $w\in {ℂ}^p$ is the corresponding eigenvector, i.e., ${\widehat{L}}_{\beta }(\alpha )\mathrm{u}=\lambda \mathrm{u}$. Then, it holds that ${\widehat{N}}_{\beta }(\alpha )\mathrm{u}=\lambda {\widehat{M}}_{\beta }(\alpha )\mathrm{u}$, or equivalently,

$$\{\begin{array}{l}(P-\lambda \widehat{F})u=\beta {\widehat{A}}^T\widehat{B}v-{\widehat{A}}^{T}w,\\ (T-\lambda \widehat{G})v=\lambda \beta {\widehat{B}}^T\widehat{A}u-\widehat{B}w,\\ \frac{1-\lambda }{\alpha \beta }\widehat{Q}w=\lambda (\widehat{A}u+\widehat{B}v).\end{array}$$

(22)

Since the matrices $I+\beta \lambda \widehat{S}$, $P-\lambda \widehat{F}$ and $T-\lambda \widehat{G}$ are nonsingular, by simplifying the first and second equations in (21), we can get

$$\{\begin{array}{l}\widehat{A}u=\widehat{R}(\beta \widehat{B}v-w),\\ \widehat{B}v=\widehat{S}(\lambda \beta \widehat{A}u-w).\end{array}$$

(23)

Obviously, $\widehat{B}v={\displaystyle \frac{1-\lambda }{\alpha \beta \lambda }}\widehat{Q}w-\widehat{A}u$. Substituting the equation into the first and second equation in (21), it holds that

$$\{\begin{array}{c}(I+\beta \widehat{R})\widehat{A}u=\widehat{R}({\displaystyle \frac{1-\lambda }{\alpha \lambda }}\widehat{Q}-I)w,\\ (I+\beta \lambda \widehat{S})\widehat{A}u=({\displaystyle \frac{1-\lambda }{\alpha \beta \lambda }}\widehat{Q}+\widehat{S})w.\end{array}$$

(24)

We can get the following simplified equation

$$(I+\beta \widehat{R})[(1-\lambda )\tilde{Q}+\alpha \beta \lambda \tilde{S}]\tilde{w}=\beta \widehat{R}(I+\beta \lambda \widehat{S})[(1-\lambda )\tilde{Q}-\alpha \lambda I]\tilde{w}.$$

(25)

By combining the same terms in (25) according to the power exponents of $\lambda$, we can get the following equation

$$\{{\lambda }^2{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\lambda [(I+\beta \widehat{R})(\alpha \beta \tilde{S}-\tilde{Q})-{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\beta \widehat{R}\tilde{Q}+\alpha \beta \widehat{R}]+\tilde{Q}\}\tilde{w}=0.$$

(26)

As a result of

$$(I+\beta \widehat{R})(\alpha \beta \tilde{S}-\tilde{Q})-{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\beta \widehat{R}\tilde{Q}+\alpha \beta \widehat{R}=\alpha \beta \tilde{S}-\tilde{Q}+\alpha {\beta }^2\widehat{R}\tilde{S}-{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\alpha \beta \widehat{R},$$

(27)

we can rewrite (26) as $({\lambda }^2{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\lambda E+\tilde{Q})\tilde{w}=0$. □

In accordance with Theorem 2, we can instantly get the following sufficient condition, which guarantees the globally asymptotic convergence of the PPADMM.

Theorem 3.

Let the conditions of Theorem 2 be satisfied.For any non-zero vector$\tilde{w}\in {ℂ}^n$, define

$${\mu }_s=\frac{{\tilde{w}}^{*}\tilde{S}\tilde{w}}{{\tilde{w}}^{*}\widehat{R}\widehat{S}\tilde{Q}\tilde{w}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mu }_q=\frac{{\tilde{w}}^{*}\tilde{Q}\tilde{w}}{{\tilde{w}}^{*}\widehat{R}\widehat{S}\tilde{Q}\tilde{w}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\chi =\frac{{\tilde{w}}^{*}\widehat{R}\tilde{S}\tilde{w}}{{\tilde{w}}^{*}\widehat{R}\widehat{S}\tilde{Q}\tilde{w}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mu }_r=\frac{{\tilde{w}}^{*}\widehat{R}\tilde{w}}{{\tilde{w}}^{*}\widehat{R}\widehat{S}\tilde{Q}\tilde{w}}.$$

(28)

If $|\kappa -\overline{\kappa }\eta |+|\eta {|}^2<1$ is satisfied, then the iteration sequence ${\left\{x^{(k)}\right\}}_{k=0}^{\infty }$ generated by the iteration scheme of the PPADMM in (10) converges to the optimal solution of the problem (1)–(2), where

$$\{\begin{array}{l}\kappa ={\displaystyle \frac{\alpha \beta {\mu }_s-{\mu }_q+\alpha {\beta }^2\chi -{\beta }^2+\alpha \beta {\mu }_r}{{\beta }^2}},\\ \eta ={\displaystyle \frac{{\mu }_q}{{\beta }^2}}.\end{array}$$

(29)

Moreover, the convergence factor of the PPADMM is given by $\sigma (\alpha ,\beta )=\max \{{\lambda }_{+}^{(\max )},{\lambda }_{-}^{(\max )}\}$, where

$${\lambda }_{\pm }^{(\max )}=\underset{\tilde{w}\in {ℂ}^p\backslash \left\{0\right\}}{\max }\left\{{\displaystyle \frac{|-\kappa \pm \sqrt{{\kappa }^2-4\eta }|}{2}}\right\}.$$

Proof. 

According to Theorem 2, we know that the PPADMM generates the following rational eigenvalue problem

$$({\lambda }^2{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\lambda E+\tilde{Q})\tilde{w}=0.$$

The two sides of the equation are multiplied by ${\tilde{w}}^{*}\in {ℂ}^p\backslash \left\{0\right\}$ at the same time, and we can get

$${\lambda }^2{\beta }^2{\tilde{w}}^{*}\widehat{R}\widehat{S}\tilde{Q}\tilde{w}+\lambda {\tilde{w}}^{*}E\tilde{w}+{\tilde{w}}^{*}\tilde{Q}\tilde{w}=0$$

(30)

where $E=\alpha \beta \tilde{S}-\tilde{Q}+\alpha {\beta }^2\widehat{R}\tilde{S}-{\beta }^2\widehat{R}\widehat{S}\tilde{Q}+\alpha \beta \widehat{R}$, we can rewrite the Equation (30) as

$${\beta }^2{\lambda }^2+\lambda (\alpha \beta {\mu }_s-{\mu }_q+\alpha {\beta }^2\chi -{\beta }^2+\alpha \beta {\mu }_r)+{\mu }_q=0$$

(31)

With the notation in (28), Equation (30) is organized into

$${\lambda }^2+\lambda \kappa +\eta =0.$$

(32)

According to the Lemma 1, if the root of the quadratic function about the eigenvalue is less than one, the coefficient must satisfy the following conditions:

$$|\kappa -\overline{\kappa }\eta |+|\eta {|}^2<1.$$

Last but not least, it is easy to acquire two roots of the quadratic Equation (32). The two roots are listed:

$${\lambda }_{\pm }^{(\max )}=\underset{\tilde{w}\in {ℂ}^p\backslash \left\{0\right\}}{\max }\left\{{\displaystyle \frac{\left|-\kappa \pm \sqrt{{\kappa }^2-4\eta }\right|}{2}}\right\}.$$

So, we can get the convergence factor of the PPADMM as follows $\sigma (\alpha ,\beta )=\max \{{\lambda }_{+}^{(\max )},{\lambda }_{-}^{(\max )}\}$. □

4. Numerical Results

In this section, numerical examples are adopted to illustrate the performance of the PPADMM for the image deblurring problem. In [26], the PRADMM was shown to be more efficient than the PADMM. Hence, we only need to compare the PPADMM that we proposed with the PRADMM and ADMM in this section.

Image deblurring is a classical and significant subject study in image processing, which is usually an inverse problem. It is comprehensively concerned by many scholars engaged in computer vision. Normally, the goal of image deblurring is to recover an unknown original image $u\in {ℝ}^n$ from a noisy image $y\in {ℝ}^n$ that is often modeled as $y=Bu+n$, where B is a linear operator, and n is a white Gaussian noise with variance ${\sigma }^2$.

In general, the blurring matrix B is a highly ill-conditioned matrix, so the image deblurring problem is an ill-conditioned inverse problem. A common method for solving the ill-conditioned problems is the regularization methods. Therefore, in practical applications, the image deblurring problem is converted into the following optimization problem

$$\underset{x}{\min }\{{\displaystyle \frac{1}{2}}{\Vert Ax-c\Vert }_2^2+{\displaystyle \frac{{\epsilon }^2}{2}}{\Vert Kx\Vert }_2^2\}$$

(33)

where A is a blurring operator, K is a regularization operator, $\epsilon$ is a regularization parameter, and c is the observed image (see [16,29,30,31,32,33]). The mathematical expression (33) can be equivalently reformulated into the equality-constraint quadratic programming problem (1)–(2) as follows:

$$\{\begin{array}{l}\underset{x,y}{\min }\hspace{0.17em}\frac{{\epsilon }^2}{2}{\Vert Kx\Vert }_{}^2+{\displaystyle \frac{1}{2}}{\Vert y\Vert }_{}^2,\\ s.t\hspace{0.17em}Ax-y=b.\end{array}.$$

(34)

As it was mentioned above, the blurring image c can be described as $b=A{x}^t+\omega r$, where $x^t$ is the true image, r is the noise generated by the random vector, and $\omega$ is the level of noise. In our simulations of image deblurring, in order to generate blurred images, first, the original images are blurred by a blurring kernel, and then followed by an additive Gaussian white noise. As in [15], we set $\epsilon =0.1$, $\beta =0.1$, and $\omega =3$.

In our simulations, K takes the Laplacian operator. Under the periodic boundary conditions, K and A are the block circulant matrices with circulant blocks. Therefore, both $K^{T}K$ and $A^{T}A$ are also block circulant matrices with circulant blocks, so that they can be diagonalized by the FFT2 (2 Dimension Fast Fourier Transform). See, e.g., [34]. Therefore, the three methods experimented require $o(n\log n)$ operations per iteration.

The PRADMM is used to solve the problem (34), and its corresponding iteration scheme is as follows:

$$\{\begin{array}{l}({\epsilon }^2{K}^{T}K+\beta A^T{W}^{-1}A){\widehat{x}}^{(k+1)}=A^T{W}^{-1}[\beta (c+y^{(k)})+{\lambda }^{(k)}),\\ x^{(k+1)}=\omega {\widehat{x}}^{(k+1)}+(1-\omega )x^{(k)},\\ (I+\beta W^{-1}){\widehat{y}}^{(k+1)}=W^{-1}[\beta (A{x}^{(k+1)}-c)+{\lambda }^{(k)}],\\ y^{(k+1)}=\tau {\widehat{y}}^{(k+1)}+(1-\tau )y^{(k)}.\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\alpha Q^{-1}{W}^{-1}(A{x}^{(k+1)}-y^{(k+1)}-c).\end{array}$$

(35)

Applying the PPADMM to (33), we obtain the following iterative scheme:

$$\{\begin{array}{l}({\epsilon }^2{K}^{T}K+\beta A^T{W}^{-1}A+P)x^{(k+1)}=P{x}^{(k)}+A^T{W}^{-1}[\beta (c+y^{(k)})+{\lambda }^{(k)}),\\ (I+\beta W^{-1}+T)y^{(k+1)}=W^{-1}[\beta (A{x}^{(k+1)}-c)+{\lambda }^{(k)}]+T{y}^{(k)},\\ {\lambda }^{(k+1)}={\lambda }^{(k)}-\alpha \beta Q^{-1}{W}^{-1}(A{x}^{(k+1)}-y^{(k+1)}-c).\end{array}$$

(36)

In order to reduce the computational cost of iterative schemes (35) and (36), we choose the following method to generate the positive definite proximal term matrices P and T as $P={\displaystyle \frac{\beta }{{\tau }_1}}I-\beta A^{T}A$, $T=({\displaystyle \frac{\beta }{{\tau }_2}}-\beta )I$, where ${\tau }_1\in (0,{\displaystyle \frac{1}{{\lambda }_{\max }(A^{T}A)}})$, ${\tau }_2\in (0,1)$. For the PRADMM and PPADMM, $W=({\displaystyle \frac{\beta }{{\gamma }_1}}I-\beta A^{T}A{)}^{-1}$, $Q=({\displaystyle \frac{\beta }{{\gamma }_2}}-\beta )I$, where ${\gamma }_1\in (0,{\displaystyle \frac{1}{{\lambda }_{\max }(A^{T}A)}})$, ${\gamma }_2\in (0,1)$. In our simulations, ${\tau }_1=0.9$, ${\tau }_2=0.04$, and ${\gamma }_1={\gamma }_2=0.1$. It is obvious that the matrices $({\epsilon }^2{K}^{T}K+\beta A^T{W}^{-1}A)$, $(I+\beta W^{-1})$, and $(I+\beta W^{-1}+T)$, $({\epsilon }^2{K}^{T}K+\beta A^T{W}^{-1}A+P)$ are the block circulant matrices with circulant blocks so that their inverse matrices can be easily obtained by the FFT2. It will greatly reduce the computational cost of schemes (35) and (36).

In our experiments, the methods have been implemented in MATLAB (version R2014a) on a PC with Core i5-4590, 3.30 GHz CPU, and 8.00 GB RAM. We test the 256-by-256 Grayscale images Capsule, Baboon, House, and Cameraman [35], which are shown in Figure 1. We state that the image Capsule is a .jpg file and the other three images are all .pgm files. For 8-bit images, their pixel values are all integers in [0, 255]. Hence, for these four images, we have $m=n=p=256\times 256=\mathrm{65,536}$ in the problem (34). In our experiments of deblurring, we test two types of blurring kernels: Type I (fspecial(average, 13)) and TypeII (fspecial(Gaussian, [9, 9], 3)). For the three methods in the experiment, all initial values are chosen as the blurred image $c$, i.e., $x^{(0)}=c$, which is terminated if the current iterates satisfy $\frac{\Vert x^{(k)}-x^{(k-1)}\Vert }{\Vert x^{(k-1)}\Vert }\le {10}^{-5}$ and are performed in MATLAB with a machine precision 10⁻¹⁶.

The peak signal-to-noise ratio (PSNR) is the most common and widely used objective measurement method for evaluating the quality of an image. It is widely used in the literature to investigate the performance of the methods solving the image restoration problem. The PSNR is defined as follows:

$$PSNR(x)=20{\log }_{10}\frac{255}{\sqrt{\mathrm{var}(x,x^t)}} \mathrm{with} \mathrm{var}(x,x^t)=\frac{1}{n}{\displaystyle \sum _{j=0}^{n-1}{(x_j^t-x_j)}^2}$$

where $x_j^t$ is the j-th pixel value of the original image $x^t$, and $x_j$ is the j-th pixel values of the corresponding estimated image $y=Bu+n$. PSNR is a very important index in image deblurring, which can measure the difference between the restored image and the original image. In general, the PSNR value increases with the quality of restoration. In image deblurring, if the PSNR is larger, the effect of image deblurring is better. In order to obtain the high value of PSNR, it will be worth taking a long time.

Table 1. The optimal iteration parameters computed by experiment for the preconditioned and proximal alternating direction method of multipliers (PPADMM) and preconditioned and relaxed alternating direction method of multipliers (PRADMM).

Image		Capsule		Baboon		House		Cameraman
blur type		I	II	I	II	I	II	I	II
PPADMM	${\alpha }_{*}$	2.0	2.1	2.1	2.1	2.1	2.1	2.1	2.1
PRADMM	${\omega }_{*}$	0.8	0.8	0.8	0.8	0.8	0.8	0.8	0.8
	${\tau }_{*}$	0.6	0.6	0.6	0.6	0.6	0.6	0.6	0.6
	${\alpha }_{*}$	0.23	0.23	0.23	0.23	0.23	0.26	0.25	0.26

shows the optimal parameters for the PPADMM and the PRADMM in the experiment, respectively. These optimal parameters are obtained experimentally by maximizing the corresponding PSNR.

In order to illustrate the effectiveness and robustness of the PPADMM to the different images, we show the experimental results for the PPADMM, PRADMM, and ADMM to solve the image deblurring problem in

Table 2. The elapsed CPU time (CPU), peak signal-to-noise ratio (PSNR) and valofun for the PPAD (PPADMM), PRAD (PRADMM) and AD (alternating direction method of multipliers, or ADMM) with respect to optimal parameters.

Image	Blur	Cpu			Psnr
	Type	Ppad	Prad	Ad	Ppad	Prad	Ad
Capsule	I	1.018558	0.211038	0.449924	25.0501	24.4955	24.4942
	II	0.732373	0.198781	0.425788	26.5942	26.0038	26.0026
Baboon	I	0.670735	0.190145	0.428676	20.8321	20.4507	20.4500
	II	0.625869	0.180112	0.411398	21.0639	20.6318	20.6311
House	I	0.604927	0.188013	0.433318	26.6212	26.1505	26.1484
	II	0.544012	0.161778	0.404065	27.3147	26.3739	26.3706
Cameraman	I	0.677334	0.169664	0.421255	23.7530	23.2583	23.2566
	II	0.538381	0.168758	0.416063	24.4705	23.6654	23.6632

. Table 2 shows the elapsed CPU time in seconds (denoted as “CPU”) and the peak signal-to-noise ratio measured in the described (dB) (denoted as “PSNR”) image deblurring problem (34). In order to be accurate, the experimental results are the average value of the three repeatedly executed results. In short, the PPADMM, PRADMM, and ADMM are denoted as PPAD, PRAD, and AD. From Table 2, we can see that the PPADMM acquires the highest value of PSNR for the four images of two types of the kernel. The difference with respect to the PSNR ranges from 0.38 to 0.94 dB. In order to know the relationship between PSNR and iteration, Figure 2 takes the capsule blurred by Type I as an example, which shows the PSNR versus iterations for the PPADMM, PRADMM, and ADMM. We can see that for the image capsule blurred by Type I, the PPADMM was able to solve the problem (34) in 83 iterations and achieved a PSNR value of 25.05 dB. The PRADMM and ADMM took 18 iterations and 47 iterations to solve the problem (33) and obtained a PSNR value of 24.5 dB. Clearly, although the PPADMM took more iterations than the PRADMM and ADMM, the value of PSNR that the PPADMM calculated is much larger than those of the other two methods. Above all, the PPADMM is much more effective than the PRADMM and the ADMM in the qualities of the deblurred images, and it is robust.

Finally, for the blurred (Type I) and noised (B&N) image capsule, we deblur it by the ADMM, PRADMM, and PPADMM, respectively, and we show the deblurred images in Figure 3. We can see that the PRADMM and ADMM produced over-smoothed results and eliminated many images details. Instead, the PPADMM ensured that the restored images have better visual quality. On the one hand, it can remove effectively the blurring effects and noise. On the other hand, it is also able to reconstruct more image edges than the two other methods.

In summary, it can be concluded from Table 2 and Figure 1 and Figure 2 that for the image deblurring problem (33), the proposed PPADMM is clearly more effective than the PRADMM and ADMM in obtaining comparable qualities of the deblurred images.

5. Conclusions

In this paper, we have proposed an efficient PPADMM for solving the linear constraint quadratic programming problem (1)–(2). This algorithm is a proximal generalization of the PADMM, and it extrapolates the block variables and the block multiplier in each new iteration. In fact, the PPADMM is naturally a generalized and modified block SOR iterative method in the viewpoint of matrix computation. Its theoretical properties such as global convergence and convergence factor are established. Meanwhile, our numerical results verify the efficiency of the proposed method in comparison with PRADMM and ADMM. In addition, it is easy to be applied to construct the similar method for solving convex optimization problems of several block variables and of several equality and inequality constraints. However, how to choose the optimal parameters is still a challenging problem that deserves further discussion.