A Fixed-Point Subgradient Splitting Method for Solving Constrained Convex Optimization Problems

Abstract

In this work, we consider a bilevel optimization problem consisting of the minimizing sum of two convex functions in which one of them is a composition of a convex function and a nonzero linear transformation subject to the set of all feasible points represented in the form of common fixed-point sets of nonlinear operators. To find an optimal solution to the problem, we present a fixed-point subgradient splitting method and analyze convergence properties of the proposed method provided that some additional assumptions are imposed. We investigate the solving of some well known problems by using the proposed method. Finally, we present some numerical experiments for showing the effectiveness of the obtained theoretical result.

1. Introduction

Many applications in science and engineering have shown a huge interest in solving an inverse problem of finding $x\in {\mathbb{R}}^n$ satisfying

$$Bx=b,$$

(1)

where $b\in {\mathbb{R}}^r$ is the observed data and $B^{r\times n}$ is the corresponding nonzero matrix. Actually, the inverse problem is typically facing the ill-condition of the matrix B so that it may have no solution. Then, an approach for finding an approximate solution by minimizing the squared norm of the residual term has been considered:

$$\begin{array}{ll}\mathrm{minimize}& {\parallel Bx-b\parallel }^2\\ \mathrm{subject}\mathrm{to}& x\in {\mathbb{R}}^n.\hfill \end{array}$$

(2)

Observe that the problem (2) has several optimal solutions; in this situation, it is not clear which of these solutions should be considered. One strategy for pursuing the best optimal solution among these many solutions is to add a regularization term to the objective function. The classical technique is to consider the celebrated Tikhonov regularization [1] of the form

$$\begin{array}{ll}\mathrm{minimize}& {\parallel Bx-b\parallel }^2+\lambda {\parallel x\parallel }^2\\ \mathrm{subject}\mathrm{to}& x\in {\mathbb{R}}^n,\hfill \end{array}$$

(3)

where $\lambda >0$ is a regularization parameter. In this setting, the uniqueness of the solution to (3) is acquired. However, note that from a practical point of view, the shortcoming of this strategy is that the unique solution to the regularization problem (3) may probably not optimal in the original sense as in (2), see [2,3] for further discussions.

To over come this, we should consider the strategy of selecting a specific solution among optimal solutions to (2) by minimizing an additional prior function over these optimal solutions. This brings the framework of the following bilevel optimization problem,

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\\ \mathrm{subject}\mathrm{to}& x\in \underset{u\in {\mathbb{R}}^n}{argmin}{\parallel Bu-b\parallel }^2,\hfill \end{array}$$

(4)

where $f:{\mathbb{R}}^n\to \mathbb{R}$ and $h:{\mathbb{R}}^m\to \mathbb{R}$ are convex functions, and $A:{\mathbb{R}}^n\to {\mathbb{R}}^m$ is a nonzero linear transformation. It is very important to point out that many problems can be formulated into this form. For instance, if $m:=n-1$, $f:={\parallel ·\parallel }_1$, $h:={\parallel ·\parallel }_1$, and

$$A:=\left[\begin{array}{cccccc}-1& 1& 0& \cdots & 0& 0\\ 0& -1& 1& \cdots & 0& 0\\ \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\ 0& 0& 0& \cdots & -1& 1\end{array}\right]\in {\mathbb{R}}^{(n-1)\times n}$$

the problem (4) becomes the fused lasso [4] solution to the problem (2). This situation also occurs in image denoising problems ($r=n$ and B is the identity matrix), and in image inpainting problems ($r=n$ and B is a symmetric diagonal matrix), where the term ${\parallel Ax\parallel }_1$ is known as an 1D total variation [5]. When $m=n$, $f={\parallel ·\parallel }^2$, $h={\parallel ·\parallel }_1$, and A is the identity matrix, the problem (4) becomes the elastic net [6] solution to the problem (2). Moreover, in wavelet-based image restoration problems, the matrix A is given by an inverse wavelet transform [7].

Let us consider the constrained set of (4), it is known that introducing the Landweber operator $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ of the form

$$Tx:=x-\frac{1}{{\parallel B\parallel }^2}{B}^{\top }(Bx-b),$$

yields T is firmly nonexpansive and the set of all fixed points of T is nothing else that the set ${argmin}_{x\in {\mathbb{R}}^n}{\parallel Bu-b\parallel }^2$, see [8] for more details. Motivated by this observation, the bilevel problem (4) can be considered in the general setting as

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in FixT,\hfill \end{array}$$

(5)

where $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is a nonlinear operator with $FixT:=\{x\in {\mathbb{R}}^n:Tx=x\}\ne \varnothing$. Note that problem (5) encompasses not only the problem (4), but also many problems in the literature, for instance, the minimization over the intersection of a finite number of sublevel sets of convex nonsmooth functions (see Section 5.2), the minimization over the intersection of many convex sets in which the metric projection on such intersection can not be computed explicitly, see [9,10,11] for more details.

There are some existing methods for solving convex optimization problems over fixed-point set in the form of (5), but the celebrated one is due to the hybrid steepest descent method, which was firstly investigated in [12]. Note that the algorithm proposed by Yamada [12] goes on with the hypotheses that the objective functions are strongly convex and smooth and the operator T is nonexpansive. Several variants and generalizations of this well-known method are, for instance, Yamada and Ogura [11] considered the same scheme for solving the problem (5) when T belongs to a class of so called quasi-shrinking operator. Cegielski [10] proposed a generalized hybrid steepest descent method by using the sequence of quasi-nonexpansive operators. Iiduka [13,14] considered a nonsmooth convex optimization problem (5) with fixed-point constraints of certain quasi-nonexpansive operators.

On the other hand, in the recent decade, the split common fixed point problem [15,16] turns out to be one of the attractions among several nonlinear problems due to its widely applications in many image and signal processing problems. Actually, for given a nonzero linear transformation $A:{\mathbb{R}}^n\to {\mathbb{R}}^m$, and two nonlinear operators $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ and $S:{\mathbb{R}}^m\to {\mathbb{R}}^m$ with $\mathrm{Fix}\left(T\right)\ne \varnothing$, and $\mathcal{R}\left(A\right)\cap \mathrm{Fix}\left(S\right)\ne \varnothing$, the split common fixed point problem is to find a point $x^{*}\in {\mathbb{R}}^n$ in which

$$x^{*}\in FixT\hspace{1em}\mathrm{such}\mathrm{that}\hspace{1em}A{x}^{*}\in FixS.$$

The key idea of this problem is to find a point in the fixed point of a nonlinear operator in a primal space, and subsequently its image under an appropriate linear transformation forms a fixed point of another nonlinear operator in another space. This situation appears, for instance, in dynamic emission tomographic image reconstruction [17] and in the intensity-modulated radiation therapy treatment planning, see [18] for more details. Of course, there are many authors that have investigated the study of iterative algorithms for split common fixed point problems and proposed their generalizations in several aspects, see, for example, [9,19,20,21,22] and references therein.

The aim of this paper is to present a nonsmooth and non-strongly convex version of the hybrid steepest descent method for minimizing the sum of two convex functions over the fixed-point constraints of the form:

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in X\cap FixT,Ax\in FixS,\hfill \end{array}$$

where $f:{\mathbb{R}}^n\to \mathbb{R}$ and $h:{\mathbb{R}}^m\to \mathbb{R}$ are convex nonsmooth functions, $A:{\mathbb{R}}^n\to {\mathbb{R}}^m$ is a nonzero linear transformation, $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ and $S:{\mathbb{R}}^m\to {\mathbb{R}}^m$ are certain quasi-nonexpansive operators with $\mathrm{Fix}\left(T\right)\ne \varnothing$, and $\mathcal{R}\left(A\right)\cap \mathrm{Fix}\left(S\right)\ne \varnothing$, and $X\subset {\mathbb{R}}^n$ is a simple closed convex bounded set. We prove the convergence of function value to the minimum value where some control conditions on a stepsize sequence and a parameter are imposed.

The paper is organized as follows. After recalling and introducing some useful notions and tools in Section 2, we present our algorithm and discuss the convergence analysis in Section 3. Furthermore, in Section 4, we discuss an important implication of our problem and algorithm to the minimizing sum of convex functions over coupling constraints. In Section 5, we discuss in detail some remarkably practical applications, and Section 6 describes the results of numerical experiments on fused lasso like problem. Finally, the conclusions are given in Section 7.

2. Preliminaries

We summarize some useful notations, definitions, and properties, which we will utilize later.

For further details, the reader can consult the well-known books, for instance, in [8,23,24,25].

Let ${\mathbb{R}}^n$ be an n-dimensional Euclidean space with an inner product $⟨·,·⟩$ and the corresponding norm $\parallel ·\parallel$.

Let $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ be an operator. We denote the set of all fixed points of T by $FixT$, that is,

$$FixT:=\{x\in {\mathbb{R}}^n:Tx=x\}.$$

We say that T is $\rho$-strongly quasi-nonexpansive ($\rho$-SQNE), where $\rho \ge 0$, if $FixT\ne \varnothing$ and

$${\parallel Tx-z\parallel }^2\le {\parallel x-z\parallel }^2-\rho {\parallel Tx-x\parallel }^2,$$

for all $x\in {\mathbb{R}}^n$ and $z\in FixT$. If $\rho >0$, then T is called strongly quasi-nonexpansive (SQNE). If $\rho =0$, then T is called quasi-nonexpansive (QNE), that is,

$$\parallel Tx-z\parallel \le \parallel x-z\parallel ,$$

for all $x\in {\mathbb{R}}^n$ and $z\in FixT$. Clearly, if T is SQNE, then it is QNE. We say that T is cutter if $FixT\ne \varnothing$ and

$$⟨x-Tx,z-Tx⟩\le 0,$$

for all $x\in {\mathbb{R}}^n$ and all $z\in FixT$. We say that T is firmly nonexpansive (FNE) if

$$⟨Tx-Ty,x-y⟩\ge {\parallel Tx-Ty\parallel }^2,$$

for all $x,y\in {\mathbb{R}}^n$.

The following properties will be applied in the next sections.

Fact 1

(Lemma 2.1.21 [8]). If $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is $\mathrm{QNE}$, then $FixT$ is closed and convex.

Fact 2

([Theorem 2.2.5 [8]). If $T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is $\mathrm{FNE}$ with $FixT\ne \varnothing$, then T is a cutter.

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a function and $x\in {\mathbb{R}}^n$. The subdifferential of f at x is the set

$$\partial f\left(x\right):=\{x^{*}\in {\mathbb{R}}^n:f\left(y\right)\ge f\left(x\right)+⟨x^{*},y-x⟩\mathrm{for}\mathrm{all}y\in {\mathbb{R}}^n\}.$$

If $\partial f\left(x\right)\ne \varnothing$, then an element $f^{\prime }\left(x\right)\in \partial f\left(x\right)$ is called a subgradient of f at x.

Fact 3

(Corollary 16.15 [24]). Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a convex function. Then, the subdifferential $\partial f\left(x\right)\ne \varnothing$ for all $x\in {\mathbb{R}}^n$.

Fact 4

(Proposition 16.17 [24]). Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a convex function. Then, the subdifferential $\partial f$ maps every bounded subset of ${\mathbb{R}}^n$ to a bounded set.

As we work on the n-dimensional Euclidean space, we will use the notion of matrix instead of the notion of linear transformation throughout this work. Denote ${\mathbb{R}}^{m\times n}$ by the set of all real-valued $m\times n$ matrices. Let $A\in {\mathbb{R}}^{m\times n}$ be given. We denote by $\mathcal{R}\left(A\right):=\{y\in {\mathbb{R}}^m:y=Ax\mathrm{for}\mathrm{some}x\in \mathbb{R}{}^n\}$ its range, and $A^{\top }$ its (conjugate) transpose. We denote the induced norm of A by $\parallel A\parallel$, which is given by $\parallel A\parallel =\sqrt{{\lambda }_{max}\left(A^{\top }A\right)}$, where ${\lambda }_{max}\left(A^{\top }A\right)$ is the maximum eigenvalue of the matrix $A^{\top }A$.

3. Method and its Convergence

Now, we formulate the composite nonsmooth convex minimization problem over the intersections of fixed-point sets which we aim to investigate throughout this paper.

Problem 1.

Let ${\mathbb{R}}^n$ and ${\mathbb{R}}^m$ be two Euclidean spaces. Assume that

$\left(\mathrm{A}1\right)$

$f:{\mathbb{R}}^n\to \mathbb{R}$ and $h:{\mathbb{R}}^m\to \mathbb{R}$ are convex functions.

$\left(\mathrm{A}2\right)$

$A\in {\mathbb{R}}^{m\times n}$ is a nonzero matrix.

$\left(\mathrm{A}3\right)$

$T:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is $\mathrm{QNE}$, and $S:{\mathbb{R}}^m\to {\mathbb{R}}^m$ is cutter with $\mathcal{R}\left(A\right)\cap \mathrm{Fix}\left(S\right)\ne \varnothing$.

$\left(\mathrm{A}4\right)$

X is a nonempty convex closed bounded simple subset of ${\mathbb{R}}^n$.

Our objective is to solve

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in X\cap FixT,Ax\in FixS.\hfill \end{array}$$

Throughout this work, we denote the solution set of Problem 1 by $\Gamma$ and assume that it is nonempty.

Problem 1 can be viewed as a bilevel problem in which data given from two sources in a system. Actually, let us consider the system of two users in different sources (they can have differently a number of factors n and m) in which they can communicate to each other via the transformation A. The first user aims to find the best solutions with respect to criterion f among many feasible points represented in the form of fixed point set of an appropriate operator T. Similarly, the second user has its own objective in the same fashion of finding the best solutions among feasible points in $FixS$ with priori criterion h. Now, to find the best solutions of this system, we consider the fixed-point subgradient splitting method (in short, FSSM) as follows, see Algorithm 1.

Algorithm 1: Fixed-Point Subgradient Splitting Method.

Initialization: The positive sequence ${\left\{{\alpha }_k\right\}}_{k\ge 1}$ and the parameter $\gamma \in \left(0,+\infty \right)$, and an arbitrary $x_1\in {\mathbb{R}}^n$. Iterative Step: For given $x_k\in {\mathbb{R}}^n$, compute $$\begin{array}{ccc}\hfill z_k& :=& SA{x}_k-\frac{{\alpha }_k}{\gamma }{h}^{\prime }\left(SA{x}_k\right),\mathrm{where}{h}^{\prime }\left(SA{x}_k\right)\in \partial h\left(SA{x}_k\right),\hfill \\ \hfill y_k& :=& x_k+\gamma A^{\top }(z_k-A{x}_k),\hfill \\ \hfill x_{k+1}& :=& P_X\left(T{y}_k-{\alpha }_k{f}^{\prime }\left(T{y}_k\right)\right),\mathrm{where}{f}^{\prime }\left(T{y}_k\right)\in \partial f\left(T{y}_k\right).\hfill \end{array}$$

Remark 1.

Actually, this algorithm has simultaneously the following features; (i) splitting computation, (ii) simple scheme, and (iii) boundedness of iterates. Concerning the first feature, we present the iterative scheme by allowing the process of a subgradient of f and a use of operator T in the space ${\mathbb{R}}^n$, and a subgradient of h and a use of operator S in the space ${\mathbb{R}}^m$ separately. Regarding the simplicity of iterative scheme, we need not to compute the inverse of the matrix A; in this case, the transpose of A is enough. Finally, the third feature is typically required when performing the convergence of subgradient type method. Of course, the boundedness is often considered in image processing and machine learning in the form of a (big) box constraint or a big Euclidean ball.

To study the convergence properties of a function values of a sequence generated by Algorithm 1, we start with the following technical result.

Lemma 1.

Let ${\left\{x_k\right\}}_{k\ge 1}$ be a sequence generated by Algorithm 1. Then, for every $k\ge 1$ and $u\in X\cap FixT\cap A^{-1}(FixS)$, it holds that

$$\begin{array}{ccc}\hfill \parallel x_{k+1}{-u\parallel }^2& \le & \parallel x_k{-u\parallel }^2-\left(1-{\gamma \parallel A\parallel }^2\right)\gamma {\parallel z_k-A{x}_k\parallel }^2\hfill \\ & & +{\alpha }_k^2\parallel f^{\prime }\left(T{y}_k\right){\parallel }^2+\left(\frac{{\gamma }^2+2\gamma +2}{\gamma }\right){\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\hfill \\ & & +2{\alpha }_k\left[(f\left(u\right)+h\left(Au\right))-(f\left(T{y}_k\right)+h\left(SA{x}_k\right))\right].\hfill \end{array}$$

Proof. 

Let $k\ge 1$ be arbitrary. By the definition of ${\left\{y_k\right\}}_{k\ge 1}$, we have

$$\begin{array}{ccc}\hfill \parallel y_k{-u\parallel }^2& =& \parallel x_k{-u\parallel }^2+{\gamma }^2{\parallel A^{\top }(z_k-A{x}_k)\parallel }^2+2\gamma ⟨x_k-u,A^{\top }(z_k-A{x}_k)⟩\hfill \\ & \le & \parallel x_k{-u\parallel }^2+{\gamma }^2{\parallel A\parallel }^2{\parallel z_k-A{x}_k\parallel }^2+2\gamma ⟨A{x}_k-Au,z_k-A{x}_k⟩.\hfill \end{array}$$

(6)

Now, by using the definition of ${\left\{z_k\right\}}_{k\ge 1}$ and the cutter property of S, we derive

$$\begin{array}{ccc}\hfill ⟨A{x}_k-Au,z_k-A{x}_k⟩& =& ⟨A{x}_k-z_k,z_k-A{x}_k⟩+⟨z_k-Au,z_k-A{x}_k⟩\hfill \\ & =& -\parallel z_k-A{x}_k{\parallel }^2+⟨SA{x}_k-Au,z_k-A{x}_k⟩\hfill \\ & & -\frac{{\alpha }_k}{\gamma }⟨h^{\prime }\left(SA{x}_k\right),z_k-A{x}_k⟩\hfill \\ & =& -\parallel z_k-A{x}_k{\parallel }^2+⟨SA{x}_k-Au,SA{x}_k-A{x}_k⟩\hfill \\ & & -\frac{{\alpha }_k}{\gamma }⟨SA{x}_k-Au,h^{\prime }\left(SA{x}_k\right)⟩+\frac{{\alpha }_k^2}{{\gamma }^2}{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\hfill \\ & & -\frac{{\alpha }_k}{\gamma }⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-A{x}_k⟩\hfill \\ & \le & -\parallel z_k-A{x}_k{\parallel }^2+\frac{{\alpha }_k^2}{{\gamma }^2}{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\hfill \\ & & -\frac{{\alpha }_k}{\gamma }⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-Au⟩\hfill \\ & & -\frac{{\alpha }_k}{\gamma }⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-A{x}_k⟩,\hfill \end{array}$$

which in turn implies that (6) becomes

$$\begin{array}{ccc}\hfill \parallel y_k{-u\parallel }^2& \le & \parallel x_k{-u\parallel }^2+\left({\gamma \parallel A\parallel }^2-2\right)\gamma \parallel z_k-A{x}_k{\parallel }^2+2\frac{{\alpha }_k^2}{\gamma }{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\hfill \\ & & -2{\alpha }_k⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-Au⟩-2{\alpha }_k⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-A{x}_k⟩.\hfill \end{array}$$

(7)

We now focus on the last two terms of the right-hand side of (7).

Observe that

$$\begin{array}{ccc}\hfill 0& \le & \frac{1}{2\gamma }{∥\gamma (SA{x}_k-A{x}_k)+2{\alpha }_k{h}^{\prime }\left(SA{x}_k\right)∥}^2\hfill \\ & =& \frac{\gamma }{2}\parallel SA{x}_k-A{x}_k{\parallel }^2+2{\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2+2{\alpha }_k⟨SA{x}_k-A{x}_k,h^{\prime }\left(SA{x}_k\right)⟩,\hfill \end{array}$$

thus, by the definition of ${\left\{z_k\right\}}_{k\ge 1}$, we obtain

$$\begin{array}{ccc}\hfill -2{\alpha }_k⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-A{x}_k⟩& \le & \frac{\gamma }{2}\parallel SA{x}_k-A{x}_k{\parallel }^2+2{\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\hfill \\ & \le & \gamma \parallel z_k-A{x}_k{\parallel }^2+(2+\gamma ){\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2.\hfill \end{array}$$

(8)

Now, inequalities (6)–(8) together give

$$\begin{array}{ccc}\hfill \parallel y_k{-u\parallel }^2& \le & \parallel x_k{-u\parallel }^2-\left(1-{\gamma \parallel A\parallel }^2\right)\gamma {\parallel z_k-A{x}_k\parallel }^2\hfill \\ & & +\left(\frac{2}{\gamma }+2+\gamma \right){\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2-2{\alpha }_k⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-Au⟩.\hfill \end{array}$$

(9)

On the other hand, using the definition of ${\left\{x_k\right\}}_{k\ge 1}$ and the assumption that T is QNE, we obtain

$$\begin{array}{ccc}\hfill \parallel x_{k+1}{-u\parallel }^2& \le & \parallel T{y}_k-{\alpha }_k{f}^{\prime }\left(T{y}_k\right){-u\parallel }^2\hfill \\ & =& \parallel T{y}_k{-u\parallel }^2+{\alpha }_k^2{\parallel f^{\prime }\left(T{y}_k\right)\parallel }^2-2{\alpha }_k⟨f^{\prime }\left(T{y}_k\right),T{y}_k-u⟩\hfill \\ & \le & \parallel y_k{-u\parallel }^2+{\alpha }_k^2{\parallel f^{\prime }\left(T{y}_k\right)\parallel }^2-2{\alpha }_k⟨f^{\prime }\left(T{y}_k\right),T{y}_k-u⟩.\hfill \end{array}$$

(10)

Replacing (9) in (10), we obtain

$$\begin{array}{ccc}\hfill \parallel x_{k+1}{-u\parallel }^2& \le & \parallel x_k{-u\parallel }^2-\left(1-{\gamma \parallel A\parallel }^2\right)\gamma {\parallel z_k-A{x}_k\parallel }^2\hfill \\ & & +{\alpha }_k^2\parallel f^{\prime }\left(T{y}_k\right){\parallel }^2+\left(\frac{{\gamma }^2+2\gamma +2}{\gamma }\right){\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\hfill \\ & & -2{\alpha }_k⟨f^{\prime }\left(T{y}_k\right),T{y}_k-u⟩-2{\alpha }_k⟨h^{\prime }\left(SA{x}_k\right),SA{x}_k-Au⟩.\hfill \end{array}$$

(11)

Next, the convexities of f and g give

$$\begin{array}{c}\hfill ⟨f^{\prime }\left(T{y}_k\right),u-T{y}_k⟩\le f\left(u\right)-f\left(T{y}_k\right)\end{array}$$

and

$$\begin{array}{c}\hfill ⟨h^{\prime }\left(SA{x}_k\right),Au-SA{x}_k⟩\le h\left(Au\right)-h\left(SA{x}_k\right).\end{array}$$

By making use of these two inequalities in (11), we obtain

$$\begin{array}{ccc}\hfill \parallel x_{k+1}{-u\parallel }^2& \le & \parallel x_k{-u\parallel }^2-\left(1-{\gamma \parallel A\parallel }^2\right)\gamma {\parallel z_k-A{x}_k\parallel }^2\hfill \\ & & +{\alpha }_k^2\parallel f^{\prime }\left(T{y}_k\right){\parallel }^2+\left(\frac{{\gamma }^2+2\gamma +2}{\gamma }\right){\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\hfill \\ & & +2{\alpha }_k\left[(f\left(u\right)+h\left(Au\right))-(f\left(T{y}_k\right)+h\left(SA{x}_k\right))\right],\hfill \end{array}$$

which is the required inequality and the proof is completed. ☐

The following lemma is very useful for the convergence result.

Lemma 2.

Let ${\left\{x_k\right\}}_{k\ge 1}$ be a sequence generated by Algorithm 1. Then, ${\left\{x_k\right\}}_{k\ge 1}$ is bounded. Furthermore, if $0<\gamma <\frac{1}{{\parallel A\parallel }^2}$ and ${\left\{{\alpha }_k\right\}}_{k\ge 1}$ is bounded, then the sequences ${\left\{SA{x}_k\right\}}_{k\ge 1}$, ${\left\{h^{\prime }\left(SA{x}_k\right)\right\}}_{k\ge 1}$, ${\left\{z_k\right\}}_{k\ge 1}$, ${\left\{y_k\right\}}_{k\ge 1}$, ${\left\{T{y}_k\right\}}_{k\ge 1}$, and ${\left\{f^{\prime }\left(T{y}_k\right)\right\}}_{k\ge 1}$ are bounded.

Proof. 

As X is a bounded set, it is clear that the sequence ${\left\{x_k\right\}}_{k\ge 1}$ is bounded. Now, let $u\in \Gamma$ be given. The linearity of A and quasi-nonexpansiveness of S yield

$$\begin{array}{c}\hfill \parallel SA{x}_k\parallel \le \parallel SA{x}_k-Au\parallel +\parallel Au\parallel \le \parallel A\parallel \parallel x_k-u\parallel +\parallel Au\parallel .\end{array}$$

This implies that ${\left\{SA{x}_k\right\}}_{k\ge 1}$ is bounded. Consequently, applying Fact 4, we obtain that ${\left\{h^{\prime }\left(SA{x}_k\right)\right\}}_{k\ge 1}$ is also bounded.

By the triangle inequality, we have

$$\parallel z_k\parallel \le \parallel SA{x}_k\parallel +\frac{{\alpha }_k}{\gamma }\parallel h^{\prime }\left(SA{x}_k\right)\parallel .$$

Therefore, the boundedness of ${\left\{{\alpha }_k\right\}}_{k\ge 1}$ implies that ${\left\{z_k\right\}}_{k\ge 1}$ is bounded. Consequently, the triangle inequality and the linearity of $A^{\top }$ yields the boundedness of ${\left\{y_k\right\}}_{k\ge 1}$. As T is QNE, we have ${\left\{T{y}_k\right\}}_{k\ge 1}$ is bounded. Thus, ${\left\{f^{\prime }\left(T{y}_k\right)\right\}}_{k\ge 1}$ is bounded by Fact 4. ☐

For the sake of simplicity, we let

$${(f+h\circ A)}^{*}:=\underset{z\in X\cap FixT\cap A^{-1}(FixS)}{inf}(f\left(z\right)+h\left(Az\right))$$

and assume that ${(f+h\circ A)}^{*}>-\infty$.

We consider a convergence property in objective values with diminishing stepsize as the following theorem.

Theorem 1.

Let ${\left\{x_k\right\}}_{k\ge 1}$ be a sequence generated by Algorithm 1. If the following control conditions hold,

(i) 

$0<\gamma <\frac{1}{{\parallel A\parallel }^2}$;

(ii) 

${\sum }_{k=1}^{\infty }{\alpha }_k=+\infty$ and ${\sum }_{k=1}^{\infty }{\alpha }_k^2<+\infty$;

then

$$\begin{array}{c}\hfill \underset{k\to +\infty }{lim\; inf}(f\left(T{y}_k\right)+h\left(SA{x}_k\right))\le {(f+h\circ A)}^{*}.\end{array}$$

Proof. 

Let $z\in \Gamma$ be given. We note from Lemma 1 that for every $k\ge 1$

$$\begin{array}{ccc}\hfill 2{\alpha }_k[(f\left(T{y}_k\right)+h\left(SA{x}_k\right))-(f\left(z\right)+h\left(Az\right))]& \le & \parallel x_k{-z\parallel }^2-\parallel x_{k+1}{-z\parallel }^2+{\alpha }_k^2{\parallel f^{\prime }\left(T{y}_k\right)\parallel }^2\hfill \\ & & +\left(\frac{{\gamma }^2+2\gamma +2}{\gamma }\right){\alpha }_k^2{\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2,\hfill \end{array}$$

(12)

this is true because $-\left(1-{\gamma \parallel A\parallel }^2\right)\gamma {\parallel z_k-A{x}_k\parallel }^2\le 0$ via the assumption (i). Summing up (12) for $1,\dots ,k$ we obtain that

$$\begin{array}{c}\hfill 2\sum _{i=1}^k{\alpha }_k[(f\left(T{y}_k\right)+h\left(SA{x}_k\right))-{(f+h\circ A)}^{*})]\le \parallel x^1{-z\parallel }^2+M\sum _{i=1}^k{\alpha }_k^2,\end{array}$$

where

$$M:=\underset{k\ge 1}{sup}\left\{\parallel f^{\prime }\left(T{y}_k\right){\parallel }^2+\left(\frac{{\gamma }^2+2\gamma +2}{\gamma }\right){\parallel h^{\prime }\left(SA{x}_k\right)\parallel }^2\right\}.$$

This implies that

$$\begin{array}{c}\hfill \sum _{i=1}^{\infty }{\alpha }_k[(f\left(T{y}_k\right)+h\left(SA{x}_k\right))-{(f+h\circ A)}^{*})]<+\infty .\end{array}$$

Next, we show that ${lim\; inf}_{k\to +\infty }\left((f\left(T{y}_k\right)+h\left(SA{x}_k\right))-{(f+h\circ A)}^{*})\right)\le 0$. By supposing a contradiction that

$$\underset{k\to +\infty }{lim\; inf}\left((f\left(T{y}_k\right)+h\left(SA{x}_k\right))-{(f+h\circ A)}^{*})\right)>0,$$

there exist $k_0\ge 1$ and $\gamma >0$ such that

$$\left((f\left(T{y}_k\right)+h\left(SA{x}_k\right))-{(f+h\circ A)}^{*})\right)\ge \gamma$$

for all $k\ge k_0$. Thus, we have

$$+\infty =\gamma \sum _{k=k_0}^{\infty }{\alpha }_k\le \sum _{k=k_0}^{\infty }{\alpha }_k\left((f\left(T{y}_k\right)+h\left(SA{x}_k\right))-{(f+h\circ A)}^{*})\right)<+\infty ,$$

which is a contradiction. Therefore, we can conclude that

$$\begin{array}{c}\hfill \underset{k\to +\infty }{lim\; inf}(f\left(T{y}_k\right)+h\left(SA{x}_k\right))\le {(f+h\circ A)}^{*}.\end{array}$$

 ☐

Remark 2.

The convergence results obtained in Theorem 1 are slightly different from the convergence results obtained by the classical gradient method or even projected gradient method, namely, ${lim\; inf}_{k\to +\infty }(f\left(T{y}_k\right)+h\left(SA{x}_k\right))={(f+h\circ A)}^{*}$. This is because, in each iteration, we can not ensure whether the estimate $T{x}^k$ is belonging to the constrained set $Fix\left(T\right)$ or not, this means that the property $f\left(T{y}_k\right)\ge f^{*}$ may not be true in general. Similarly, we can not ensure that $h\left(SA{x}_k\right)\ge {(h\circ A)}^{*}$.

Remark 3.

A step size sequence satisfies the assumption that ${\sum }_{k=1}^{\infty }{\alpha }_k=+\infty$ and ${\sum }_{k=1}^{\infty }{\alpha }_k^2<+\infty$ is, for instance, ${\left\{\frac{a}{k}\right\}}_{k\ge 1}$ where $a>0$.

4. Convex Minimization Involving Sum of Composite Functions

The aim of this section is to show that Algorithm 1 and their convergence properties can be employed when solving a convex minimization involving sum of a finite number of composite functions.

Let us take a look the composite convex minimization problem:

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+\sum _{i=1}^l{h}_i\left(A_{i}x\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in X\cap FixT,A_{i}x\in Fix{S}_i,i=1,\dots ,l.\hfill \end{array}$$

(13)

where, we assume further that, for all $i=1,\dots ,l$, there hold

(I)

] $A_i\in {\mathbb{R}}^{m_i\times n}$ are nonzero matrices,

(II)

$S_i:{\mathbb{R}}^{m_i}\to {\mathbb{R}}^{m_i}$ are cutter operators with $\mathcal{R}\left(A_i\right)\cap Fix{S}_i\ne \varnothing$, and

(III)

$h_i:{\mathbb{R}}^{m_i}\to \mathbb{R}$ is a convex function.

In this section, we assume that the solution set of (13) is denoted by $\Omega$ and asuume that it is nonempty.

Denote the product of spaces

$${\mathbb{R}}^m:={\mathbb{R}}^{m_1}\times {\mathbb{R}}^{m_2}\times \cdots \times {\mathbb{R}}^{m_l}$$

equipped with the addition

$$\mathbf{x}+\mathbf{y}:=(x_1+y_1,x_2+y_2,\dots ,x_l+y_l),$$

the scalar multiplication

$$\alpha \mathbf{x}:=(\alpha x_1,\alpha x_2,\dots ,\alpha x_l)$$

with the inner product defined by

$${⟨⟨\mathbf{x},\mathbf{y}⟩⟩}_{{\mathbb{R}}^m}:=\sum _{i=1}^l{⟨x_i,y_i⟩}_{{\mathbb{R}}^{m_i}},$$

and the norm by

$${\parallel \mathbf{x}\parallel }_{{\mathbb{R}}^m}=\sqrt{{⟨⟨\mathbf{x},\mathbf{x}⟩⟩}_{{\mathbb{R}}^m}},$$

for all $\mathbf{x}=(x_1,x_2,\dots ,x_l)$, $\mathbf{y}=(y_1,y_2,\dots ,y_l)\in {\mathbb{R}}^m$, is again a Euclidean space (see [24], Example 2.1). Define a matrix $\mathbf{A}\in {\mathbb{R}}^{n\times m}$ by

$$\mathbf{A}:=[A_1^{\top }|A_2^{\top }|\dots |A_m^{\top }{]}^{\top },$$

and an operator $\mathbf{S}:{\mathbb{R}}^m\to {\mathbb{R}}^m$ by

$$\mathbf{S}\left(\mathbf{y}\right):=(S_1{y}_1,S_2{y}_2,\dots ,S_l{y}_l),$$

for all $\mathbf{y}=(y_1,y_2,\dots ,y_l)\in {\mathbb{R}}^m$. Note that the operator $\mathbf{S}$ is cutter with

$$\mathrm{Fix}\left(\mathbf{S}\right)=\mathrm{Fix}\left(S_1\right)\times \cdots \times \mathrm{Fix}\left(S_l\right).$$

Furthermore, defining a function $\mathbf{h}:{\mathbb{R}}^m\to \mathbb{R}$ by

$$\mathbf{h}\left(\mathbf{x}\right):=\sum _{i=1}^l{h}_i\left(x_i\right),$$

for all $\mathbf{x}=(x_1,x_2,\dots ,x_l)\in {\mathbb{R}}^m$, we also have that the function $\mathbf{h}$ is a convex function (see [24], Proposition 8.25). By hte above setting, we can rewrite the problem (13) as

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+\mathbf{h}\left(\mathbf{A}x\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in X\cap FixT,\mathbf{A}x\in Fix\mathbf{S},\hfill \end{array}$$

which is nothing else than Problem 1.

Here, to investigate the solving of the problem (13), we state the Algorithm 2 as follow.

Algorithm 2: Distributed Fixed-Point Subgradient Splitting Method.

Initialization: The positive sequence ${\left\{{\alpha }_k\right\}}_{k\ge 1}$ and the parameter $\gamma \in \left(0,+\infty \right)$, and an arbitrary $x_1\in {\mathbb{R}}^n$. Iterative Step: For given $x_k\in {\mathbb{R}}^n$, compute $$\begin{array}{ccc}\hfill z_{k,i}& :=& S_i{A}_i{x}_k-\frac{{\alpha }_k}{\gamma }{d}_{k,i},d_{k,i}\in \partial h_i\left(S_i{A}_i{x}_k\right),\mathrm{for}\mathrm{all}i=1,\dots ,l,\hfill \\ \hfill y_k& :=& x_k+\gamma \sum _{i=1}^l{A}_i^{\top }(z_{k,i}-A_i{x}_k),\hfill \\ \hfill x_{k+1}& :=& P_X\left(T{y}_k-{\alpha }_k{f}^{\prime }\left(T{y}_k\right)\right),f^{\prime }\left(T{y}_k\right)\in \partial f\left(T{y}_k\right).\hfill \end{array}$$

As an above consequence, we note that

$${\mathbf{A}}^{\top }\mathbf{x}=\sum _{i=1}^l{A}_i^{\top }{x}_i,$$

for all $\mathbf{x}=(x_1,x_2,\dots ,x_l)\in {\mathbb{R}}^m$. Furthermore, we know that

$$\partial \mathbf{h}\left(\mathbf{SA}{x}_k\right)=\partial h_1\left(S_1{A}_1{x}_k\right)\times \cdots \times \partial h_l\left(S_l{A}_l{x}_k\right),$$

see ([25], Corollary 2.4.5). Putting ${\mathbf{d}}_k:=(d_{k,1},\dots ,d_{k,l})$ where $d_{k,i}\in \partial h_i\left(S_i{A}_i{x}_k\right),i=1,\dots ,l$, for all $k\ge 1$, we obtain that

$${\mathbf{z}}_k:=(z_{k,1},\dots ,z_{k,m})=\mathbf{S}\mathbf{A}{x}_k-{\alpha }_k{\mathbf{d}}_k$$

for all $k\ge 1$. Notice that

$${\mathbf{A}}^{\top }({\mathbf{z}}_k-\mathbf{A}{x}_k)=\sum _{i=1}^l{A}_i^{\top }\left(z_{k,i}-A_i{x}_k\right)$$

for all $k\ge 1.$ Thus, Algorithm 2 can be rewrite as

$$\begin{array}{ccc}\hfill {\mathbf{z}}_k& =& \mathbf{S}\mathbf{A}{x}_k-\frac{{\alpha }_k}{\gamma }{\mathbf{d}}_k,{\mathbf{d}}_k\in \partial \mathbf{h}\left(\mathbf{SA}{x}_k\right)\hfill \\ \hfill y_k& =& x_k+\gamma {\mathbf{A}}^{*}({\mathbf{z}}_k-\mathbf{A}{x}_k),\hfill \\ \hfill x_{k+1}& =& P_X\left(T{y}_k-{\alpha }_k{f}^{\prime }\left(T{y}_k\right)\right),f^{\prime }\left(T{y}_k\right)\in \partial f\left(T{y}_k\right).\hfill \end{array}$$

for all $k\ge 1$. Since ${\parallel \mathbf{A}\parallel }^2\le {\sum }_{i=1}^l{\parallel A_i\parallel }^2$, the convergence result therefore follows from Theorem 1 and can be stated as the following corollary.

Corollary 1.

Let ${\left\{x_k\right\}}_{k\ge 1}$ be a sequence generated by Algorithm 2. If the following control conditions hold:

(i) 

$0<\gamma <\frac{1}{{\sum }_{i=1}^l{\parallel A_i\parallel }^2}$;

(ii) 

${\sum }_{k=1}^{\infty }{\alpha }_k=+\infty$ and ${\sum }_{k=1}^{\infty }{\alpha }_k^2<+\infty$;

then

$$\begin{array}{c}\hfill \underset{k\to +\infty }{lim\; inf}\left(f\left(T{y}_k\right)+\sum _{i=1}^l{h}_i\left(S_i{A}_i{x}_k\right)\right)\le {\left(f+\sum _{i=1}^l{h}_i\circ A_i\right)}^{*}.\end{array}$$

5. Related Problems

The aim of this section is to show that Algorithm 1 and their convergence results can be employed when solving some well-known problems. Furthermore, we also present some noticeable applications relate to Algorithm 1 and its convergence results. For simplicity, we assume here that $S=I$.

5.1. Convex Minimization with Least Square Constraint

Let us discuss the composite minimization problem over the set of all minimizers of the proximity function of a system of linear equations:

$$\begin{array}{c}\hfill \begin{array}{cc}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in X\cap argming:=\frac{1}{2}{\parallel B·-b\parallel }^2,\hfill \end{array}\end{array}$$

(14)

where B is a nonzero $p\times n$ matrix and b is an $p\times 1$ vector. Now, we define the Landweber operator by

$$L\left(x\right):=x-\frac{1}{{\parallel B\parallel }^2}{B}^{\top }\left(Bx-b\right).$$

Suppose that B is nonzero, we have ${\parallel B\parallel }^2={\lambda }_{max}\left(B^{\top }B\right)\ne 0$, which yields that the Landweber operator L is well defined. Furthermore, if $argming\ne \varnothing$, then it holds that L is FNE with $FixL=argming$ [8] (Lemma 4.6.2, Theorem 4.6.3). In view of $T:=L$, the problem (14) is a special case of Problem 1 so that the problem (14) can be solved by Algorithm 1. Furthermore, if the set $\{u\in {\mathbb{R}}^n:Bu=b\}\ne \varnothing$, the problem (14) is equivalent to

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& Bx=b\hfill \\ & x\in X.\hfill \end{array}$$

5.2. Convex Minimization with Nonsmooth Functional Constraints

Let us discuss a convex minimization with nonsmooth functional constraints

$$\begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& g_i\left(x\right)\le 0,i=1,\dots ,l,\hfill \\ & x\in X,\hfill \end{array}$$

(15)

where $g_i:{\mathbb{R}}^n\to \mathbb{R}$ are convex function, $i=1,\dots ,l$. Denote

$$\Xi (g_i,0):=\{x\in {\mathbb{R}}^n:g_i\left(x\right)\le 0\}$$

and assume that

$$\bigcap _{i=1}^l\Xi (g_i,0)\ne \varnothing .$$

Furthermore, define the subgradient projection related to $g_i$ by

$$P_{i}x:=\left\{\begin{array}{cc}x-\frac{g_i\left(x\right)}{\parallel g_i^{\prime }{\left(x\right)\parallel }^2}{g}_i^{\prime }\left(x\right),\hfill & \mathrm{if}{g}_i\left(x\right)>0\hfill \\ x,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where $g_i^{\prime }\left(x\right)$ is a (fixed) subgradient of $g_i$ at $x\in {\mathbb{R}}^n$. It is well known that $P_i$ is a cutter (a 1-SQNE operator) with $Fix{P}_i=\Xi (g_i,0)$. Note that, if ${\bigcap }_{i=1}^l\Xi (g_i,0)\ne \varnothing$, the cyclic subgradient projection operator

$$P:=P_m{P}_{m-1}\dots P_1,$$

which is a composition of SQNE operators $P_i$, is SQNE with

$$FixP=\bigcap _{i=1}^{l}Fix{P}_i=\bigcap _{i=1}^l\Xi (g_i,0).$$

(Theorem 2.1.50, [8]). In 2018, Cegielski and Nimana [26] proposed the following extrapolated operator,

$$P_{\lambda ,\sigma }\left(x\right)=x+\lambda \sigma \left(x\right)(Px-x),$$

where $\lambda \in (0,2)$ and $\sigma :{\mathbb{R}}^n\to \mathbb{R}$ is a step size function defined by

$$\sigma \left(x\right)={\sigma }_{max}\left(x\right):=\left\{\begin{array}{cc}{\displaystyle \frac{{\sum }_{i=1}^p⟨Px-U_{i-1}x,U_{i}x-U_{i-1}x⟩}{{\parallel Px-x\parallel }^2},}\hfill & \mathrm{for}x\notin C,\hfill \\ 1,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where $U_i:=P_i{P}_{i-1}\dots P_1$ for $i=1,2,\dots ,m$ and $U_0:=I$. Note that the operator $P_{\lambda ,\sigma }$ is SQNE with $Fix{P}_{\lambda ,\sigma }=FixP\ne \varnothing$ ([26], Theorem 3.2).

By means of $T:=P_{\lambda ,\sigma }$ or $T:=P$, the problem (15) is nothing else than Problem 1 and Algorithm 1 is applicable for the problem (15).

5.3. Convex Minimization with Complex Constraints

Let $B:{\mathbb{R}}^n$⇉${\mathbb{R}}^n$ be a set-valued operator. We denote by

$$\mathrm{Gr}\left(B\right):=\{(x,u)\in {\mathbb{R}}^n\times {\mathbb{R}}^n:u\in Bx\}$$

its graph, and

$$\mathrm{zer}\left(B\right):=\{z\in {\mathbb{R}}^n:0\in B\left(z\right)\}$$

the set of all zeros of the operator B. The set-valued operator B is said to be monotone if

$$⟨x-y,u-v⟩\ge 0,$$

for all $(x,u),(y,v)\in \mathrm{Gr}\left(B\right)$, and it is called maximally monotone if its graph is not properly contained in the graph of any other monotone operators. For a set-valued operator $B:{\mathbb{R}}^n\rightrightarrows {\mathbb{R}}^n$, we define the resolvent of B, $J_B:{\mathbb{R}}^n\rightrightarrows {\mathbb{R}}^n$, by

$$J_B:={(Id+B)}^{-1}.$$

It is well known that if B is maximally monotone and $r>0$, then the resolvent of $rB$ is (single-valued) FNE with

$$\mathrm{zer}\left(B\right)=Fix{J}_{rB},$$

see ([24], Corollary 23.31, Proposition 23.38). Now, let us consider the minimal norm-like solution of the classical monotone inclusion problem:

$$\begin{array}{c}\hfill \begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in X\cap \mathrm{zer}\left(B\right),\hfill \end{array}\end{array}$$

(16)

where $B:{\mathbb{R}}^n\rightrightarrows {\mathbb{R}}^n$ is a maximally monotone operator such that $\mathrm{zer}\left(B\right)\ne \varnothing$. For given $r>0$, and putting $T:=J_{rB}$, the problem (15) is nothing else than Problem 1 and Algorithm 1 is also applicable for the problem (16).

In particular, let us consider the following minimal norm-like solution of minimization problem,

$$\begin{array}{c}\hfill \begin{array}{ll}\mathrm{minimize}& f\left(x\right)+h\left(Ax\right)\hfill \\ \mathrm{subject}\mathrm{to}& x\in X\cap argmin\phi .\hfill \end{array}\end{array}$$

(17)

The problem (17) has been considered by many authors, for instance [2,27,28,29,30] and references therein. Recall that for given $r>0$ and a proper convex lower semicontinuous function $\phi :{\mathbb{R}}^n\to (-\infty ,+\infty ]$, we denote by ${\mathrm{prox}}_{r\phi }\left(x\right)$ the proximal point of parameter r of $\phi$ at x, which is the unique optimal solution of the optimization problem

$$min\left\{\phi \left(u\right)+\frac{1}{2r}{\parallel u-x\parallel }^2:u\in {\mathbb{R}}^n\right\}.$$

Note that ${\mathrm{prox}}_{r\phi }=J_{r\partial \phi }$. Therefore, putting $T:={\mathrm{prox}}_{r\phi }$, Algorithm 1 is also applicable for the problem (17).

6. Numerical Experiments

In this section, to demonstrate the effectiveness of the fixed-point subgradient splitting method (Algorithm 1), we apply the proposed method to solve the fused lasso like problem. All the experiments were performed under MATLAB 9.6 (R2019a) running on a MacBook Pro 13-inch, 2019 with a 2.4 GHz Intel Core i5 processor and 8 GB 2133 MHz LPDDR3 memory.

For a given design matrix $\mathbf{A}:=[{\mathbf{a}}_1|\cdots |{\mathbf{a}}_r{]}^{\top }$ in ${\mathbb{R}}^{r\times s}$ where ${\mathbf{a}}_i=(a_{1i},\dots ,a_{si})\in {\mathbb{R}}^s$ and a response vector $b=(b_1,\dots ,b_r)\in {\mathbb{R}}^r$. We consider the fused lasso like problem of the form

$$\begin{array}{ll}\mathrm{minimize}& {\parallel x\parallel }_1+{\parallel Dx\parallel }_1\\ \mathrm{subject}\mathrm{to}& x\in {[-1,1]}^s\cap argmin\frac{1}{2}{\parallel \mathbf{A}·-b\parallel }^2,\hfill \end{array}$$

(18)

where

$$D:=\left[\begin{array}{cccccc}-1& 1& 0& \cdots & 0& 0\\ 0& -1& 1& \cdots & 0& 0\\ \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\ 0& 0& 0& \cdots & -1& 1\end{array}\right]\in {\mathbb{R}}^{(s-1)\times s}$$

Observe that by setting the functions $f\left(x\right)={\parallel x\parallel }_1$, $h\left(x\right):={\parallel x\parallel }_1$, $A:=D$, the Landweber operator

$$T\left(x\right):=x-\frac{1}{{\parallel \mathbf{A}\parallel }^2}{\mathbf{A}}^{\top }\left(\mathbf{A}x-b\right),$$

$S:=Id$; the identity matrix; and the constrained set $X:={[-1,1]}^s$, we obtain that the problem (18) is a special case of Problem 1 so that the problem (14) can be solved by Algorithm 1 (See Section 5.1) for more details.

We generate the matrix $\mathbf{A}$ with normally distributed random chosen in $(-10,10)$ with a given percentage $p_{\mathbf{A}}$ of nonzero elements. We generate vectors $b=(b_1,\dots ,b_r)\in {\mathbb{R}}^r$ corresponding to $\mathbf{A}$ by the linear model $b=\mathbf{A}{x}_0+ϵ$, where $ϵ\sim N(0,\parallel \mathbf{A}{x}_0{\parallel }^2)$ and the vector $x_0$ has $10\%$ of nonzero components with normally distributed random generating. The initial point is a vector whose coordinates are chosen randomly in $(-1,1)$. In the numerical experiment, denoting the estimate

$$F_k:=f\left(T{y}_k\right)+h\left(SA{x}_k\right)$$

for all $k\ge 1$, we consider the behavior of the average of the relative changes

$$\frac{1}{k}\sum _{j=1}^k\frac{|F_{j+1}-F_j|}{|F_j|+1}$$

with the optimality tolerance ${10}^{-3}$. We performed 10 independent tests for any collection of dimensions $(r,s)$ and the percentages of nonzero elements of $\mathbf{A}$ for various step size parameters ${\alpha }_k$. The results are showed in Table 1, where the average number of iterations (#Iters) and average CPU time (Time) to reach the optimality tolerance for any collection of parameters are presented.

Table 1. Influences of step sizes ${\left\{{\alpha }_k\right\}}_{k\ge 1}$ for different sizes and different percentages of nonzero elements of a random matrix $\mathbf{A}$.

${\mathit{\alpha }}_{\mathit{k}}$→	${\mathit{p}}_{\mathit{A}}$	$0.1/\mathit{k}$		$0.3/\mathit{k}$		$0.5/\mathit{k}$		$0.7/\mathit{k}$		$0.9/\mathit{k}$
$(\mathit{r},\mathit{s})$↓		#Iters	Time	#Iters	Time	#Iters	Time	#Iters	Time	#Iters	Time
$(20,50)$	10%	54,253	6.33	87,529	13.56	87,416	13.30	87,347	13.35	87,455	13.45
	20%	58,085	6.99	71,969	9.85	71,813	9.72	71,831	9.77	71,928	9.85
	50%	55,414	6.38	63,113	8.13	62,962	8.00	62,914	8.02	62,969	8.10
	80%	56,190	6.46	57,780	6.97	57,459	6.82	57,170	6.80	56,949	6.81
	100%	56,960	6.60	54,084	6.32	53,800	6.19	53,630	6.18	53,538	6.22
$(50,100)$	10%	111,358	25.63	137,970	42.04	137,347	42.44	137,099	42.86	137,197	42.49
	20%	108,019	25.21	128,821	34.51	128,621	34.85	128,536	35.51	128,496	35.50
	50%	108,760	25.54	117,274	28.93	116,965	28.77	116,818	29.01	116,751	28.80
	80%	108,409	25.30	105,886	24.94	105,621	24.81	105,529	25.00	105,726	24.86
	100%	107,967	25.42	105,388	24.81	104,766	24.55	104,582	24.70	104,519	24.49
$(100,200)$	10%	209,683	117.26	365,219	347.96	364,925	353.51	364,907	354.07	364,951	354.03
	20%	210,069	116.14	285,174	193.44	285,100	198.02	285,048	197.57	285,049	197.01
	50%	208,808	116.02	281,421	185.71	281,167	188.84	280,913	190.41	280,756	189.61
	80%	213,331	118.26	212,045	119.80	211,676	122.81	211,647	122.87	211,749	127.19
	100%	213,904	114.29	197,259	104.91	197,104	107.81	197,023	108.38	196,954	108.52

We see that the method performed with parameter ${\alpha }_k=0.1/k$ behaves significantly better than other in the sense of the average number of iterations as well as the averaged CPU time for all dimensions and percentages of nonzero elements of $\mathbf{A}$. Moreover, in the case $p_{\mathbf{A}}=10\%$, we observed much bigger number of the averaged iterations and CPU time for the choice of the combinations of parameters ${\alpha }_k=0.3/k,0.5/k,0.7/k$ and $0.9/k$ with all the problem sizes.

7. Conclusions

In this paper, we introduced a simple fixed-point subgradient splitting method, whose main feature is the combination of the subgradient method with the hybrid steepest descent method relating to a nonlinear operator. We performed the convergence analysis of the method by proving the convergence of the function values to the minimum value. The result is obtained by adopting some specific suitable assumptions on the step sizes. We discussed in detail the applications of the proposed scheme to some remarkable problems in the literature. Numerical experiments on fused lasso like problem show evidence of the performance of our work.