A New Accelerated Fixed-Point Algorithm for Classification and Convex Minimization Problems in Hilbert Spaces with Directed Graphs

Abstract

A new accelerated algorithm for approximating the common fixed points of a countable family of G-nonexpansive mappings is proposed, and the weak convergence theorem based on our main results is established in the setting of Hilbert spaces with a symmetric directed graph G. As an application, we apply our results for solving classification and convex minimization problems. We also apply our proposed algorithm to estimate the weight connecting the hidden layer and output layer in a regularized extreme learning machine. For numerical experiments, the proposed algorithm gives a higher performance of accuracy of the testing set than that of FISTA-S, FISTA, and nAGA.

1. Introduction

Let H be a real Hilbert space with the norm $\parallel ·\parallel$ and C be a nonempty closed convex subset of $H.$ A mapping $T:C\to C$ is said to be nonexpansive if it satisfies the following symmetric contractive-type condition:

$$\parallel Tx-Ty\parallel \le \parallel x-y\parallel ,$$

for all $x,y\in C$; see [1].

The element $x\in C$ is a fixed point of T if $Tx=x$ and $F\left(T\right):=\{x\in C:x=Tx\}$ stands for the set of all fixed points of T.

Fixed point theory, i.e., the study of the conditions under which a map admits a fixed point, is an extensive area of research due to its numerous applications in many fields. It started with Banach’s work, which established the existence of a unique fixed point for a contraction using a classical theorem known as the Banach contraction principle; see [2]. The contraction principle of Banach has been expanded and generalized in various directions due to its applications in mathematics and other fields. One of the more recent generalizations is due to Jachymski.

Jachymski [3] introduced the structure of the graph on metric spaces using fixed point theory and obtained certain conditions for self-mapping to be a Picard operator. Several authors [4,5,6,7] proved fixed point theorems for a new type of contraction on a metric space endowed with graphs. Aleomraninejad et al. [8] used the idea of Reich et al. [9] and proved a strong convergence theorem for G-contractive and G-nonexpansive mappings. On hyperbolic metric spaces, Alfuraidan and Khamsi [10] gave a definition of G-monotone nonexpansive multivalued mappings and proved the existence of a fixed point for multivalued contraction and monotone single-valued mappings. Later on, Alfuraidan [11] presented and study the existence of fixed points for G-monotone nonexpansive and extended the results of Jachymski [3]. For approximating common fixed points of a finite family of G-nonexpansive mappings, Suantai et al. [12] used the shrinking projection with the parallel monotone hybrid method. They also used a graph to prove a strong convergence theorem in Hilbert spaces under specific conditions, and they then applied their iterative scheme to signal recovery.

In the past decade, algorithms for approximating fixed points of G-nonexpansive mappings without inertial techniques have been proposed by many researchers; see [3,8,10,11,12,13,14,15,16,17,18]. We require more efficient algorithms for solving such problems. As a result, some accelerated fixed-point algorithms using inertial techniques have been proposed to improve convergence behavior; see [19,20,21,22,23,24,25,26,27]. Recently, Janngam et al. [28] proved the weak convergence theorem for a countable family of G-nonexpansive mappings in a Hilbert space by using a coordinate affine structure with an inertial technique. They also applied their method to image recovery.

Inspired by previous research described above, we introduce a new accelerated algorithm based on the concept of the inertial technique for finding a common fixed point of a family of G-nonexpansive mappings in Hilbert spaces. We employ our result to solve data classification and convex minimization problems and also compare our algorithm efficiency to that of FISTA-S, FISTA, and nAGA.

This paper is classified as follows: in Section 2, we give certain terminology as well as some facts that will be useful in later sections. We investigate and prove our algorithm’s weak convergence in Section 3. For application, we apply our method for solving convex minimization and data classification problems in Section 4 and provide numerical experiments of classification problems in Section 5. The last section of our paper, Section 6, is a summary.

2. Preliminaries

Let C be a nonempty subset of a real Banach space X. Let $\Delta =\left\{\right(u,u):u\in C\}$, where Δ stands for the diagonal of the Cartesian product $C\times C$. Consider a directed graph G in which the set $V\left(G\right)$ of its vertices corresponds to C, and the set $E\left(G\right)$ of its edges contains all loops. A directed graph G is said to have parallel edges if two or more edges with both the same tail vertex and the same head vertex.

Assume that G does not have parallel edges. Then, $G=\left(V\right(G),E(G\left)\right)$. The conversion of a graph G is denoted by $G^{-1}$. Thus, we have

$$E\left(G^{-1}\right)=\{(u,v)\in C\times C:(v,u)\in E\left(G\right)\}.$$

Let us give some definitions of basic graph properties which are used in this paper (see [29] for more details).

Definition 1. 

A graph G is said to be

(i)

symmetric if for all $(x,y)\in E\left(G\right);$ we have $(y,x)\in E\left(G\right);$

(ii)

transitive if for any $u,v,w\in V\left(G\right)$ with $(u,v),(v,w)\in E\left(G\right);$ then, $(u,w)\in E\left(G\right);$

(iii)

connected if there is a path between any two vertices of the graph $G.$

The definition of G-contraction [3] and G-nonexpansive mappings [13] are given as follows.

Definition 2. 

A mapping $T:C\to C$ is said to be

(i)

G-contraction if

(a)

T is edge-preserving, i.e., $(Tu,Tv)\in E\left(G\right)$ for all $(u,v)\in E\left(G\right).$

(b)

There exists $\rho \in [0,1)$ such that $\parallel Tu-Tv\parallel \le \rho \parallel u-v\parallel$ for all $(u,v)\in E\left(G\right),$ where ρ is called a contraction factor.

(ii)

G-nonexpansive if

(a)

T is edge-preserving.

(b)

$\parallel Tu-Tv\parallel \le \parallel u-v\parallel$ for all $(u,v)\in E\left(G\right).$

Example 1. 

Let $C=[0,2]\subset \mathbb{R}$. Suppose that $(u,v)\in E\left(G\right)$ if and only if $0.4\le u,v\le 1.6$ or $u=v$, where $u,v\in \mathbb{R}$. Let $S,T:C\to C$ be defined by

$$\begin{array}{c}\hfill Su=cos\left(tan(u-1)\right)sin\left(\frac{\pi }{2}\right)andTu=1+\frac{ln\left(u\right)}{3},\end{array}$$

for all $u\in C$. Then, both S and T are G-nonexpansive but not nonexpansive (see [30] for more details).

Definition 3. 

A mapping $T:C\to C$ is called G-demiclosed at 0 if $u_n\rightharpoonup u$ and $T{u}_n\to 0$, then $Tu=0$ for all sequence $\left\{u_n\right\}\subseteq C$ with $(u_n,u_{n+1})\in E\left(G\right)$.

To prove our main result, we need the definition of the coordinate affine of the graph G as follows.

Definition 4 

([28]). Assume that $\Lambda :={\cap }_{n=1}^{\infty }F\left(T_n\right)\ne \varnothing$ and $\Lambda \times \Lambda \subseteq E\left(G\right)$. Then, $E\left(G\right)$ is said to be

(i)

left coordinate affine if $\alpha (x,y)+\beta (u,y)\in E\left(G\right)$ for all $(x,y)$, $(u,y)\in E\left(G\right)$ and all α, $\beta \in \mathbb{R}$ with $\alpha +\beta =1.$

(ii)

right coordinate affine if $\alpha (x,y)+\beta (x,z)\in E\left(G\right)$ for all $(x,y)$, $(x,z)\in E\left(G\right)$ and all α, $\beta \in \mathbb{R}$ with $\alpha +\beta =1.$

We say that $E\left(G\right)$ is coordinate affine if $E\left(G\right)$ is both left and right coordinate affine.

The results of the following lemmas can be used to prove our main theorem; see also [19,31,32].

Lemma 1 

([31]). Let $\left\{{\eta }_n\right\},\left\{{\nu }_n\right\}$ and $\left\{{\vartheta }_n\right\}$ be sequences of nonnegative real numbers such that ${\sum }_{n=1}^{\infty }{\vartheta }_n<\infty$ and ${\sum }_{n=1}^{\infty }{\nu }_n<\infty$. Suppose that

$$\begin{array}{c}\hfill {\eta }_{n+1}\le (1+{\vartheta }_n){\eta }_n+{\nu }_{n}foralln\ge 1.\end{array}$$

Then, ${lim}_{n\to \infty }{\eta }_n$ exists.

Lemma 2 

([32]). For a real Hilbert space H, the following results hold:

(i) For any $u,\upsilon \in H$ and $\gamma \in [0,1],$

$${\parallel \gamma u+(1-\gamma )\upsilon \parallel }^2={\gamma \parallel u\parallel }^2+{(1-\gamma )\parallel \upsilon \parallel }^2-\gamma (1-\gamma ){\parallel u-\upsilon \parallel }^2.$$

(ii) For any $u,\upsilon \in H,$

$${\parallel u\pm \upsilon \parallel }^2={\parallel u\parallel }^2\pm 2⟨u,\upsilon ⟩+{\parallel \upsilon \parallel }^2.$$

Lemma 3 

([19]). Let $\left\{{\upsilon }_n\right\}$ and $\left\{{\mu }_n\right\}$ be sequences of nonnegative real numbers such that

$$\begin{array}{c}\hfill {\upsilon }_{n+1}\le (1+{\mu }_n){\upsilon }_n+{\mu }_n{\upsilon }_{n-1}for alln\ge 1.\end{array}$$

Then,

$${\upsilon }_{n+1}\le M·{\displaystyle \prod _{j=1}^n}(1+2{\mu }_j),$$

where $M=max\{{\upsilon }_1,{\upsilon }_2\}.$ If ${\sum }_{n=1}^{\infty }{\mu }_n<\infty ,$ then $\left\{{\upsilon }_n\right\}$ is bounded.

Let $\left\{u_n\right\}$ be a sequence in $X.$ We write $u_n\rightharpoonup u$ to indicate that a sequence $\left\{u_n\right\}$ converges weakly to a point $u\in H.$ Similarly, $u_n\to u$ will symbolize the strong convergence. For  $v\in C,$ if there is a subsequence $\left\{u_{n_k}\right\}$ of $\left\{u_n\right\}$ such that $u_{n_k}\rightharpoonup v,$ then v is called a weak cluster point of $\left\{u_n\right\}.$ The set of all weak cluster points of $\left\{u_n\right\}$ is denoted by ${\omega }_w\left(u_n\right)$.

The following lemma was proved by Moudafi and Al-Shemas; see [33].

Lemma 4 

([33]). Let $\left\{u_n\right\}$ be a sequence in a real Hilbert space H such that there exists $\varnothing \ne \Lambda \subset H$ satisfying:

(i) For any $p\in \Lambda ,{lim}_{n\to \infty }\parallel u_n-p\parallel$ exists.

(ii) Any weak cluster point of $\left\{u_n\right\}\in \Lambda .$

Then, there exists $x^{*}\in \Lambda$ such that $u_n\rightharpoonup x^{*}.$

Let $\left\{T_n\right\}$ and $\psi$ be families of nonexpansive mappings of C into itself such that $\varnothing \ne F\left(\psi \right)\subset {\cap }_{n=1}^{\infty }F\left(T_n\right),$ where $F\left(\psi \right)$ is the set of all common fixed points of each $T\in \psi .$ A sequence $\left\{T_n\right\}$ satisfies the NST-condition (I) with $\psi$ if, for any bounded sequence $\left\{u_n\right\}$ in $C,$

$$\underset{n\to \infty }{lim}\parallel T_n{u}_n-u_n\parallel =0\mathrm{implies}\underset{n\to \infty }{lim}\parallel T{u}_n-u_n\parallel =0,$$

for all $T\in \psi$; see [34]. If $\psi =\left\{T\right\},$ then $\left\{T_n\right\}$ satisfies the NST-condition (I) with $T.$

Example 2 

([30]). Let $T\in \psi$. Define $T_n={\beta }_{n}I+(1-{\beta }_n)T$, where $0<s\le {\beta }_n\le t<1$ for all $n\in \mathbb{N}$. Therefore, $\left\{T_n\right\}$ is a family of G-nonexpansive mappings and satisfies the NST-condition.

Let $A:H\to 2^H$ be a maximal monotone operator and $c>0$. The resolvent of A is defined by $J_{cA}={(I+cA)}^{-1}$, where I is an identity operator. If $A=\partial f$ for some $f\in {\Gamma }_0\left(H\right)$, where ${\Gamma }_0\left(H\right)$ stands for the set of proper lower semicontinuous convex functions from $H\to (-\infty ,+\infty ]$, then $J_{cA}=pro{x}_{cf}$. The forward-backward operator of lower semi-continuous and convex functions of $f,g:{\mathbb{R}}^n\to (-\infty ,+\infty ]$ has the following definition:

A forward-backward operator T is defined by $T:=pro{x}_{\lambda g}(I-\lambda \nabla f)$ for $\lambda >0$, where $\nabla f$ is the gradient operator of function f and $pro{x}_{\lambda g}x:=argmi{n}_{y\in H}\left\{g\left(y\right)+\frac{1}{2\lambda }{\parallel y-x\parallel }^2\right\}$ (see [35,36]). Moreau [37] defined the operator $pro{x}_{\lambda g}$ as the proximity operator with respect to $\lambda$ and g. If $\lambda \in (0,2/L)$, then T is nonexpansive and L is a Lipschitz constant of $\nabla f$.

We have the following remark for the definition of the proximity operator; see [38].

Remark 1. 

Let $g:{\mathbb{R}}^n\to \mathbb{R}$ be given by $g\left(x\right)={\lambda \parallel x\parallel }_1$. The proximity operator of g is evaluated by the following formula

$$pro{x}_{{\lambda \parallel ·\parallel }_1}\left(x\right)=(sign\left(x_i\right)max\left(\right|x_i{|-\lambda ,0))}_{i=1}^n,$$

where $x=(x_1,x_2,\cdots ,x_n)$ and ${\parallel x\parallel }_1={\sum }_{i=1}^n\left|x_i\right|$.

The following lemma was proved by Bussaban et al.; see [20].

Lemma 5. 

Let H be a real Hilbert space and T be the forward-backward operator of f and g, where g is a proper lower semi-continuous convex function from H into $\mathbb{R}\cup \{\infty \}$, and f is a convex differentiable function from H into $\mathbb{R}$ with gradient $\nabla f$ being L-Lipschitz constant for some $L>0$. If $\left\{T_n\right\}$ is the forward-backward operator of f and g such that $a_n\to a$ with a, $a_n\in (0,2/L)$, then $\left\{T_n\right\}$ satisfies the $NST$-condition (I) with T.

3. Main Results

Let C be a nonempty closed and convex subset of a real Hilbert space H with a directed graph $G=\left(V\right(G),E(G\left)\right)$ such that $V\left(G\right)=C$. Let $\left\{T_n\right\}$ be a family of G-nonexpansive mappings of C into itself such that $\varnothing \ne {\cap }_{n=1}^{\infty }F\left(T_n\right)$.

The following proposition is useful for our main theorem.

Proposition 1. 

Let $x^{*}\in {\cap }_{n=1}^{\infty }F\left(T_n\right)$ and $y_0,x_1\in C$ be such that $(x^{*},y_0)$, $(x^{*},x_1)\in E\left(G\right)$. Let $\left\{x_n\right\}$ be a sequence generated by Algorithm 1. Suppose $E\left(G\right)$ is symmetric, transitive, and a right coordinate affine. Then, $(x^{*},x_n),$ $(x^{*},y_n),$ $(x^{*},z_n)$ and $(x_n,x_{n+1})\in E\left(G\right)$ for all $n\in \mathbb{N}.$

Algorithm 1: (ASA) An Accelerated S-algorithm

1: Initial. Take $y_0$, $x_1\in C$ are arbitrary, $n=1$, ${\beta }_n\in [a,b]\subset (0,1)$, ${\theta }_n\ge 0$ and ${\sum }_{n=1}^{\infty }{\theta }_n<\infty$ and ${\alpha }_n\to 1$. 2: Step 1. Compute $y_n,z_n$ and $x_{n+1}$ by $$\begin{array}{c}{z}_n=(1-{\beta }_n)x_n+{\beta }_n{T}_n{x}_n,\hfill \\ y_n=(1-{\alpha }_n)T_n{x}_n+{\alpha }_n{T}_n{z}_n,\hfill \\ x_{n+1}=y_n+{\theta }_n(y_n-y_{n-1}).\hfill \end{array}$$ Then, $n:=n+1$ and go to Step 1.

Proof. 

We shall prove the results by using strong mathematical induction. From Algorithm 1, we obtain

$$\begin{array}{cc}\hfill (x^{*},z_1)& =\left(x^{*},(1-{\beta }_1)x_1+{\beta }_1{T}_1{x}_1\right)\hfill \\ \hfill & =(1-{\beta }_1)(x^{*},x_1)+{\beta }_1(x^{*},T_1{x}_1).\hfill \end{array}$$

Since $(x^{*},x_1)\in E\left(G\right)$ and $T_n$ is edge preserving, we obtain $(x^{*},z_1)\in E\left(G\right).$ Again, by Algorithm 1, we obtain

$$\begin{array}{cc}\hfill (x^{*},y_1)& =(x^{*},(1-{\alpha }_1)T_1{x}_1+{\alpha }_1{T}_1{z}_1))\hfill \\ \hfill & =(1-{\alpha }_1)(x^{*},T_1{x}_1)+{\alpha }_1(x^{*},T_1{z}_1).\hfill \end{array}$$

Since $(x^{*},z_1)\in E\left(G\right)$ and $T_n$ is edge preserving, we obtain $(x^{*},y_1)\in E\left(G\right).$ Next, we assume that $(x^{*},z_k),(x^{*},y_k)$ and $(x^{*},x_k)\in E\left(G\right)$ for all $k<n.$ By Algorithm 1, we obtain

$$\begin{array}{cc}\hfill (x^{*},z_{k+1})& =(x^{*},(1-{\beta }_{k+1})x_{k+1}+{\beta }_{k+1}\left(T_{k+1}{x}_{k+1}\right))\hfill \\ \hfill & =(1-{\beta }_{k+1})(x_k,x_{k+1})+{\beta }_{k+1}(x^{*},T_{k+1}{x}_{k+1}),\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill (x^{*},y_{k+1})& =(x^{*},(1-{\alpha }_{k+1})T_{k+1}{y}_{k+1}+{\alpha }_{k+1}{T}_{k+1}{z}_{k+1}\hfill \\ \hfill & =(1-{\alpha }_{k+1})(x^{*},T_{k+1}{y}_{k+1}+{\alpha }_{k+1}(x^{*},T_{k+1}{z}_{k+1})\hfill \end{array}$$

(2)

and

$$\begin{array}{cc}\hfill (x^{*},x_{k+1})& =(x^{*},y_k+{\theta }_k(y_k-y_{k+1}))\hfill \\ \hfill & =(x^{*},(1+{\theta }_k)y_k-{\theta }_k{y}_{k-1})\hfill \\ \hfill & =(1+{\theta }_k)(x^{*},y_k)-{\theta }_k(x^{*},y_{k-1}).\hfill \end{array}$$

(3)

By (1)–(3) and since $E\left(G\right)$ is right coordinate affine and $T_n$ is edge preserving, we obtain $(x^{*},x_{k+1}),(x^{*},y_{k+1})$ and $(x^{*},z_{k+1})$ are in $E\left(G\right).$ By strong mathematical induction, we conclude that $(x^{*},x_n),(x^{*},y_n),$ $(x^{*},z_n)\in E\left(G\right)$ for all $n\in \mathbb{N}.$ Since $E\left(G\right)$ is symmetric, we obtain $(x_n,x^{*})\in E\left(G\right).$ Since $(x_n,x^{*}),(x^{*},x_{n+1})\in E\left(G\right)$ and $E\left(G\right)$ is transitive, we obtain $(x_n,x_{n+1})\in E\left(G\right)$ as required. □

We now prove the weak convergence of G-nonexpansive mapping in a real Hilbert space by using Algorithm 1.

Theorem 1. 

Let C be a nonempty closed and convex subset of a real Hilbert space H with a directed graph $G=\left(V\right(G),E(G\left)\right)$ with $V\left(G\right)=C$ and $E\left(G\right)$ is symmetric, transitive, and right coordinate affine. Let $y_0$, $x_1\in C$ and $\left\{x_n\right\}$ be a sequence in H defined by Algorithm 1. Suppose $\left\{T_n\right\}$ satisfies NST-condition (I) with T such that $\varnothing \ne F\left(T\right)\subset {\cap }_{n=1}^{\infty }F\left(T_n\right)$ and $(x^{*},y_0),(x^{*},x_1)\in E\left(G\right)$ for all $x^{*}\in {\cap }_{n=1}^{\infty }F\left(T_n\right).$ Then, $x_n$ $\rightharpoonup x^{*}\in F\left(T\right).$

Proof. 

Let $x^{*}\in {\cap }_{n=1}^{\infty }F\left(T_n\right).$ By the definition of $z_n$ and $y_n,$ we obtain

$$\begin{array}{cc}\hfill \parallel x^{*}-z_n\parallel & \le (1-{\beta }_n)\parallel x^{*}-x_n\parallel +{\beta }_n\parallel x^{*}-T_n{x}_n\parallel \hfill \\ \hfill & =\parallel x^{*}-x_n\parallel \hfill \end{array}$$

(4)

and

$$\begin{array}{cc}\hfill \parallel x^{*}-y_n\parallel & \le (1-{\alpha }_n)\parallel x^{*}-T_n{x}_n\parallel +{\alpha }_n\parallel x^{*}-T_n{z}_n\parallel \hfill \\ \hfill & \le (1-{\alpha }_n)\parallel x^{*}-x_n\parallel +{\alpha }_n\parallel x^{*}-z_n\parallel \hfill \end{array}$$

(5)

which implies that

$$\begin{array}{c}\hfill \parallel x^{*}-y_n\parallel \le \parallel x^{*}-x_n\parallel .\end{array}$$

(6)

By the definition of $x_n,$ we obtain

$$\begin{array}{c}\hfill \parallel x^{*}-x_n\parallel \le \parallel x^{*}-y_{n-1}\parallel +{\theta }_{n-1}\parallel y_{n-2}-y_{n-1}\parallel .\end{array}$$

(7)

From (6) and (7), we obtain

$$\begin{array}{c}\hfill \parallel x^{*}-y_n\parallel \le \parallel x^{*}-y_{n-1}\parallel +{\theta }_{n-1}\parallel y_{n-2}-y_{n-1}\parallel .\end{array}$$

(8)

It follows from (8) that

$$\begin{array}{c}\hfill \parallel x^{*}-y_n\parallel \le (1+{\theta }_{n-1})\parallel x^{*}-y_{n-1}\parallel +{\theta }_{n-1}\parallel x^{*}-y_{n-2}\parallel .\end{array}$$

(9)

Applying Lemma 3, we obtain $\parallel x^{*}-y_n\parallel \le M·{\prod }_{j=1}^n(1+2{\theta }_j),$ where $M=max\{\parallel x^{*}-y_1\parallel ,\parallel x^{*}-y_2\parallel \}.$ Since ${\sum }_{n=1}^{\infty }{\theta }_n<\infty ,$ we obtain that $\left\{y_n\right\}$ is bounded and so are $\left\{z_n\right\}$ and $\left\{x_n\right\}$. Thus,

$$\begin{array}{c}\hfill {\displaystyle \sum _{n=1}^{\infty }}{\theta }_n\parallel y_n-y_{n-1}\parallel <\infty \end{array}$$

(10)

By Lemma 1 and (9), we obtain ${lim}_{n\to \infty }\parallel x^{*}-y_n\parallel$ exists. By Lemma 2(i) and the definition of $z_n,$ we obtain

$$\begin{array}{cc}\hfill \parallel x^{*}-z_n{\parallel }^2& =\parallel (1-{\beta }_n)(x^{*}-x_n)+{\beta }_n(x^{*}-T_n{x}_n){\parallel }^2\hfill \\ \hfill & =(1-{\beta }_n)\parallel x^{*}-x_n{\parallel }^2+{\beta }_n\parallel x^{*}-T_n{x}_n{\parallel }^2-(1-{\beta }_n){\beta }_n{\parallel T_n{x}_n-x_n\parallel }^2\hfill \\ \hfill & \le \parallel x^{*}-x_n{\parallel }^2-(1-{\beta }_n){\beta }_n{\parallel T_n{x}_n-x_n\parallel }^2.\hfill \end{array}$$

(11)

Let ${lim}_{n\to \infty }\parallel x^{*}-y_n\parallel =a.$ From the boundedness of $\left\{x_n\right\}$ and (6), we obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; inf}\parallel x^{*}-x_n\parallel \ge a.\end{array}$$

(12)

Since $\parallel x^{*}-x_{n-1}\parallel \le \parallel x^{*}-y_n\parallel +{\theta }_n\parallel y_n-y_{n-1}\parallel$ and (10), we obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}\parallel x^{*}-x_n\parallel \le a.\end{array}$$

(13)

It follows from (12) and (13) that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\parallel x^{*}-x_n\parallel =a.\end{array}$$

(14)

From (4), one can easily see that ${lim\; sup}_{n\to \infty }\parallel x^{*}-z_n\parallel \le a.$ By (5) with ${\alpha }_n\to 1,$ we obtain that $a\le {lim\; inf}_{n\to \infty }\parallel x^{*}-z_n\parallel .$ Thus,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\parallel x^{*}-z_n\parallel =a.\end{array}$$

(15)

Use the facts that (11), (14), and (15) yield

$$\begin{array}{c}\hfill \parallel T_n{x}_n-x_n\parallel \to 0.\end{array}$$

According to $\left\{T_n\right\}$ satisfying the NST-condition (I) with $T,$ we obtain that $\parallel T{x}_n-x_n\parallel \to 0$ as $n\to \infty .$ Let ${\omega }_w\left(x_n\right)$ be the set of all weak cluster point of $\left\{x_n\right\}.$ Thus, ${\omega }_w\left(x_n\right)\subset F\left(T\right)$ by demicloseness of $I-T$ at $0.$ From Lemma 4, we conclude that $x_n\rightharpoonup x^{*}$ with $x^{*}\in F\left(T\right)$ as required. □

Corollary 1. 

Let C be a nonempty closed and convex subset of a real Hilbert space H and let $\left\{T_n\right\}$ be a family of nonexpansive mappings of C into itself. Let $y_0$, $x_1\in C$, and $\left\{x_n\right\}$ be a sequence in H defined by Algorithm 1. Suppose that $\left\{T_n\right\}$ satisfies NST-condition (I) with T such that $\varnothing \ne F\left(T\right)\subset {\cap }_{n=1}^{\infty }F\left(T_n\right)$. Then, $\left\{x_n\right\}$ converges weakly to a point in $F\left(T\right)$.

4. Applications

In the past decade, extreme learning machine (ELM) [39], a new learning algorithm for single-hidden layer feedforward networks (SLFNs), has been extensively studied in various research topics for machine learning and artificial intelligence such as face classification, image segmentation, regression, and data classification problems. ELM was theoretically proven to have extremely fast learning speed and good performance better than the gradient-based learning such as backpropagation in most of the cases. The target of this model is to find the parameter $\beta$ that solves the following minimization problem, called ordinary least square (OLS),

$$\begin{array}{c}\hfill \underset{\beta }{min}\parallel \mathbf{H}\beta -\mathbf{T}\parallel ,\end{array}$$

(16)

where $\parallel ·\parallel$ is the $l_2$-norm defined by ${\parallel x\parallel }_2=\sqrt{{\sum }_{i=1}^n{\left|x_i\right|}^2}$, $\mathbf{T}\in {\mathbb{R}}^{N\times m}$ is the target of data, $\beta \in {\mathbb{R}}^{M\times m}$ is a weight which connects hidden layer and output layer, and $H\in {\mathbb{R}}^{N\times M}$ is the hidden layer output matrix. In general mathematical modeling, there are several methods to estimate the solution of (16); in this case, the solution $\beta$ obtained by $\beta ={\mathbf{H}}^{†}\mathbf{T}$, where ${\mathbf{H}}^{†}$ is the Moore–Penrose generalized inverse of $\mathbf{H}$. However, in a real situation, the number of unknown variable M is much more than the number of training data N, which causes the network to possibly lead to overfitting. On the other hand, the accuracy is low while the number of hidden nodes M is small. Thus, in order to improve (16), several regularization methods were introduced. The classical two standard techniques for improving (16) are subset selection and ridge regression (sometimes called Tikhonov regularization) [40].

In this paper, we focus on the following problem, called least absolute shrinkage and selection operator (LASSO) [41],

$$\begin{array}{c}\hfill \underset{\beta }{min}{\parallel \mathbf{H}\beta -\mathbf{T}\parallel }_2^2+\lambda {\parallel \beta \parallel }_1,\end{array}$$

(17)

where $\lambda$ is a regularization parameter. LASSO tries to retain the good features of both subset selection and ridge regression [41]. After the regularization methods and the original ELM were introduced for improving performance of OLS, five years later, the regularized extreme learning machine [42] was proposed and applied to solve regression problems. In general, the (17) can be rewritten as minimization of $f+g$, that is,

$$\begin{array}{c}\hfill \underset{x}{min}F\left(x\right):=f\left(x\right)+g\left(x\right),\end{array}$$

(18)

where f is a smooth convex function with gradient having Lipschitz constant L and g is a convex smooth (or possible non-smooth) function. The solution of (18) can be rewritten into $\tilde{x}$ is a minimizer of $(f+g)$ if and only if $0\in \nabla f\left(\tilde{x}\right)+\partial g\left(\tilde{x}\right)$, where $\nabla f\left(\tilde{x}\right)$ is the gradient of f and $\partial g\left(\tilde{x}\right)$ is a subdifferential of g by using Fermat’s rule (see [35] for more details). In fixed point theory, Parikh et al. [43] characterized (18) as follows: $\tilde{x}$ is a minimizer of $f+g$ if and only if

$$\begin{array}{c}\hfill \tilde{x}=pro{x}_{\lambda g}(I-\lambda \nabla f)\left(\tilde{x}\right)=J_{\lambda \partial g}(I-\lambda \nabla f)\left(\tilde{x}\right),\end{array}$$

(19)

where $pro{x}_{\lambda g}$ is the proximity operator of $\lambda g$, $\lambda >0$ and $J_{\partial g}$ is defined by $J_{\partial g}={(I+\partial g)}^{-1}$, $J_{\partial g}$ is the resolvent of $\partial g$ and I is an identity operator. The problem (18) can be rewritten into a general problem, called a zero of sum of two operators problem, by finding $\tilde{x}$ such that

$$\tilde{x}\in zer(A+B),$$

where $A,B:H\to 2^H$ are two set-valued operators and $zer(A+B):=\{x:0\in Ax+Bx\}$. In this case, we assume that $A:H\to 2^H$ is a maximal monotone operator and $B:H\to H$ is an L-Lipschitz operator. For convenience, (19) also can be rewritten as:

$$\tilde{x}=T\tilde{x},$$

where $T=pro{x}_{\lambda g}(I-\lambda \nabla f)$. It is also known that T is nonexpansive if $\lambda \in (0,2/L)$ when L is a Lipschitz constant of $\nabla f$.

We are interested in applying our proposed method for solving a convex minimization problem and compared the convergence behavior of our proposed algorithm with the others and give some applications to solve classification problems. Our proposed method will be used to solve (18). Over the past two decades, several algorithms have been introduced for solving the problem (18). A simple and classical algorithm is the forward-backward algorithm (FBA), which was introduced by Lions and Mercier [21].

The forward-backward algorithm (FBA) is defined by

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{y}_n=x_n-\gamma \nabla f{x}_n,\hfill \\ x_{n+1}=x_n+{\rho }_n(J_{\gamma \partial g}{y}_n-x_n),\hfill \end{array}\right.\end{array}$$

(20)

where $n\ge 1$, $x_0\in H$ and L is a Lipschitz constant of $\nabla f$, $\gamma \in (0,2/L)$, $\delta =2-(\gamma L/2)$ and $\left\{{\rho }_n\right\}$ is a sequence in $[0,\delta ]$ such that ${\sum }_{n\in \mathbb{N}}{\rho }_n(\delta -{\rho }_n)=+\infty$. A technique for improving speed and giving a better convergence behavior of the algorithms was introduced firstly by Polyak [44] by adding an inertial step called inertial techniques. Since then, many authors have employed the inertial technique to accelerate their algorithms for various kinds of problems; see [19,20,22,23,24,25,26]. The performance of FBA can be improved using an iterative method with the inertial steps described below.

A fast iterative shrinkage-thresholding algorithm (FISTA) [25] is defined by

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{y}_n=T{x}_n,\hfill \\ s_{n+1}=\frac{1+\sqrt{1+4s_n^2}}{2},\hfill \\ {\mu }_n=\frac{s_n-1}{s_{n+1}},\hfill \\ x_{n+1}=y_n+{\mu }_n(y_n-y_{n-1}),\hfill \end{array}\right.\end{array}$$

(21)

where $n\ge 1$, $s_1=1$, $x_1=y_0\in {\mathbb{R}}^n$, $T:=pro{x}_{\frac{1}{L}g}(I-\frac{1}{L}\nabla f)$ and ${\mu }_n$ is the inertial step size. Beck and Teboulle [25] solved the image recovery and proved the convergence rate using FISTA. The inertial step size ${\mu }_n$ of the FISTA was firstly introduced by Nesterov [45].

A fast iterative shrinkage-thresholding algorithm-Siteration (FISTA-S) [27] is defined by

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{s}_{n+1}=\frac{1+\sqrt{1+4s_n^2}}{2},{\mu }_n=\frac{s_n-1}{s_{n+1}},\hfill \\ y_n=x_n+{\mu }_n(x_n-x_{n-1}),\hfill \\ z_n=(1-{\beta }_n)y_n+{\beta }_n{T}_n{y}_n,\hfill \\ x_{n+1}=(1-{\alpha }_n)T_n{y}_n+{\alpha }_n{T}_n{z}_n,\hfill \end{array}\right.\end{array}$$

(22)

where $x_0$, $x_1\in H$, ${\alpha }_n$, ${\beta }_n\in [a,b]\subset (0,1)$ $T_n=pro{x}_{c_{n}g}(I-c_n\nabla f)$ and $T=pro{x}_{cg}(I-c\nabla f)$. Bussaban et al. [27] solved the image recovery and proved the weak convergence theorem using FISTA-S.

A new accelerated proximal gradient algorithm (nAGA) [26] is defined by

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{y}_n=x_n+{\mu }_n(x_n-x_{n-1}),\hfill \\ x_{n+1}=T_n[(1-{\rho }_n)y_n+{\rho }_n{T}_n{y}_n],\hfill \end{array}\right.\end{array}$$

(23)

where $n\ge 1$, $T_n=pro{x}_{a_{n}g}(I-a_n\nabla f)$ with $a_n\in (0,2/L)$ and $\left\{{\mu }_n\right\},\left\{{\rho }_n\right\}$ are sequences in $(0,1)$ and $\frac{\parallel x_n-x_{n-1}{\parallel }_2}{{\mu }_n}\to 0$. The nAGA was introduced for proving a convergence theorem by Verma and Shukla [26]. The nonsmooth convex minimization problem with sparsity, including regularizers, was solved using this method for the multitask learning framework.

Theorem 2. 

Let H be a Hilbert space, $A:H\to 2^H$ be maximal monotone operator and $B:H\to H$ be an L-Lipschitz operator. Let $a\in (0,2/L)$ and $\left\{a_n\right\}\subset (0,2/L)$ such that $a_n\to a$. Define $T_n=J_{c_{n}A}(I-c_{n}B)$ and $T=J_{cA}(I-cB)$. Suppose that $zer(A+B)\ne \varnothing$. Let $\left\{x_n\right\}$ be a sequence in H defined by Algorithm 1. Then, $\left\{x_n\right\}$ converges weakly to a point in $zer(A+B)$.

Proof. 

Using Proposition 26.1(iv) (see [35]), we have $\left\{T_n\right\}$ and T are nonexpansive mappings such that $F\left(T\right)=F\left(T_n\right)=zer(A+B).$ Then, the proof is completed by Theorem 1 and Lemma 5. □

The convergence of Algorithm 2 is obtained by using our main result.

Algorithm 2: (FBASA) A forward-backward accelerated S-algorithm

1: Initial. Take $y_0$, $x_1\in C$ are arbitary, $n=1$, ${\beta }_n\in [a,b]\subset (0,1),$ ${\theta }_n\ge 0,$ ${\sum }_{n=1}^{\infty }{\theta }_n<\infty$ and ${\alpha }_n\to 1.$ 2: Step 1. Compute $y_n,$$z_n$ and $x_{n+1}$ by using $$\begin{array}{c}{z}_n=(1-{\beta }_n)x_n+{\beta }_{n}pro{x}_{a_{n}g}(I-a_n\nabla f)y_n,\hfill \\ y_n=(1-{\alpha }_n)pro{x}_{a_{n}g}(I-a_n\nabla f)+{\alpha }_{n}pro{x}_{a_{n}g}(I-a_n\nabla f)z_n,\hfill \\ x_{n+1}=y_n+{\theta }_n(y_n-y_{n-1}).\hfill \end{array}$$ Then, $n:=n+1$ and go to Step 1.

Theorem 3. 

For $f,g:{\mathbb{R}}^n\to (-\infty ,\infty ]$, f is a smooth convex function with a gradient having a Lipschitz constant L and g is a convex function. Let $a_n\in (0,2/L)$ be such that $\left\{a_n\right\}$ converges to a and let $T:=pro{x}_{ag}(I-a\nabla f)$ and $T_n:=pro{x}_{a_{n}g}(I-a_n\nabla f)$ and let $\left\{x_n\right\}$ be a sequence generated by Algorithm 2, where ${\beta }_n,{\alpha }_n$ and ${\theta }_n$ are the same as in Algorithm 1. Then,

(i) $\parallel x^{*}-x_{n+1}\parallel \le M·{\prod }_{j=1}^n(2{\theta }_j+1)$, where $M=max\{\parallel x^{*}-x_1\parallel ,\parallel x^{*}-x_2\parallel \}$ and $x^{*}\in Argmin(f+g);$

(ii) $\left\{x_n\right\}$ converges weakly to a point in Argmin $(f+g)$.

Proof. 

We know that T and $\left\{T_n\right\}$ are nonexpansive operators, and $F\left(T\right)={\cap }_{n=1}^{\infty }F\left(T_n\right)=Argmin(f+g)$ for all n; see [35]. Then, $\left\{T_n\right\}$ satisfies the NST-condition (I) with T by using Lemma 5. We get the desired result immediately from Theorem 1 by putting $G={\mathbb{R}}^n\times {\mathbb{R}}^n$, the complete graph, on ${\mathbb{R}}^n$. □

5. Numerical Experiments

The classification problem is one of the most important problems in the convex minimization problem. We illustrate the process of reformulating the data classification problem in machine learning.

We first present a basic idea of extreme learning machines for data classification problem, and use our algorithm to find this problem through numerical experiments. Moreover, the performance of Algorithm 2, FISTA-S, FISTA, and nAGA are compared.

Extreme learning machine (ELM). Let R:= $\{(x_k,t_k):x_k\in {\mathbb{R}}^n$, $t_k\in {\mathbb{R}}^m$, $k=1,2,\cdots ,N\}$ be a training set of different samples N, where $x_k$ is input data and $t_k$ is a target. A standard SLFNs with activation function $\Psi \left(x\right)$ (for instance sigmoid), and M hidden nodes can be rewritten as

$${\displaystyle \sum _{j=1}^M}{\beta }_j\Psi (⟨w_j,x_i⟩+c_j)=o_i,i=1,\cdots ,N,$$

where ${\beta }_j$ is the weight vector connecting the j-th the output node and hidden node, $w_j$ is the weight vector connecting the j-th the input node and hidden node, and $c_j$ is the threshold of the j-th hidden node. The objective of standard SLFNs is to estimate these N different samples with ${\sum }_{i=1}^N\parallel t_i-o_i\parallel =0$, that is, there exist ${\beta }_j$, $w_j$, $c_j$ such that

$${\displaystyle \sum _{j=1}^M}{\beta }_j\Psi (⟨w_j,x_i⟩+c_j)=t_i,i=1,\cdots ,N.$$

We can derive a simple equation from the above N equations as follows:

$$\mathbf{H}\beta =\mathbf{T},$$

$$\mathbf{H}=\left[\begin{array}{ccc}\Psi (⟨w_1,x_1⟩+c_1)& \cdots & \Psi (⟨w_M,x_1⟩+c_M)\\ ⋮& \ddots & ⋮\\ \Psi (⟨w_j,x_N⟩+c_1)& \cdots & \Psi (⟨w_N,x_N⟩+c_M)\end{array}\right],$$

$$\beta ={[{\beta }_1^T,\cdots ,{\beta }_{m\times M}^T]}_{m\times M}^T,\mathbf{T}={[t_1^T,\cdots ,t_N^T]}_{m\times N}^T.$$

A standard SLFN goal is to estimate ${\beta }_j$, $w_j$, and $c_j$ to solve (18), whereas an ELM goal is to find only ${\beta }_j$ with $w_j$ and $c_j$ chosen at random.

In an experiment on classification problems, we employ the model (17) to solve the convex minimization problem. We set $f\left(x\right)={\parallel \mathbf{H}\beta -\mathbf{T}\parallel }_2^2$ and $g\left(x\right)={\lambda \parallel \beta \parallel }_1$. Next, we use the Iris dataset to classify iris plant types, and the Heart Disease UCI dataset to identify heart patients which are detailed as follows:

Iris dataset [46]. This dataset has three classes of 50 examples, each of which represents a different variety of iris plant. The purpose is to identify each iris plant species based on the length of its sepals and petals.

Heart Disease UCI dataset [47]. Although there are 76 attributes in the original dataset, all published experiments only use 14 of them. Data on patients with heart disease are provided in this dataset. We divide the data into two classes based on the predicted attributes.

The dataset was graciously provided by https://archive.ics.uci.edu (accessed on April 2020).

All control parameters are set to the values shown in Table 1, $L=2\parallel {\mathbf{H}}_{\mathbf{1}}{\parallel }^2$, where ${\mathbf{H}}_{\mathbf{1}}$ is a hidden layer output matrix of a training matrix, $M=100$, and $\Psi \left(x\right)$ is sigmoid. Each dataset is given a training set, as indicated in Table 2. We evaluated the output data’s accuracy by

$$\begin{array}{c}\hfill \mathrm{accuracy}=\frac{\mathrm{correct} \mathrm{predicted} \mathrm{data}}{\mathrm{all} \mathrm{data}}\times 100\end{array}$$

Table 1. Parameters of each methods.

Methods	Setting
Algorithm 2	${\alpha }_n={\beta }_n=0.5$, $c=1/L$, ${\theta }_n=0.9$ if $1\le n\le 400$, and $1/2^n$ otherwise
FISTA-S	${\alpha }_n={\beta }_n=0.5$, $c=1/L$, $s_1=1$, $s_{n+1}=(1+\sqrt{1+4s_n^2})/2$, ${\varrho }_n=(s_n-1)/s_{n+1}$ if $1\le n\le 400$, and $1/2^n$ otherwise
FISTA	$s_1=1$, $s_{n+1}=(1+\sqrt{1+4s_n^2})/2$, ${\varrho }_n=(s_n-1)/s_{n+1}$
nAGA	${\rho }_n=0.9$, $\gamma =1/L$, $s_1=1$, $s_{n+1}=(1+\sqrt{1+4s_n^2})/2$, ${\varrho }_n=(s_n-1)/s_{n+1}$

Table 2. Training and testing sets of the Iris and Heart Disease UCI datasets.

Dataset	Attributes	Sample
		Train	Test
Heart Disease UCI	14	213	90
Iris	4	105	45

From the results in Table 3, we conclude that the proposed learning algorithm under selection with the identical number of hidden nodes M has high performance in terms of the accuracy. The weight computed by Algorithm 2 converges faster to the optimal weight and performs accuracy better than those computed by FISTA-S, FISTA, and nAGA.

Table 3. Performance comparison using different methods at the 400th iteration.

Dataset	Algorithm 2		FISTA-S		FISTA		nAGA
	Train	Test	Train	Test	Train	Test	Train	Test
Heart Disease UCI	68.26	62.37	39.96	37.63	53.76	50.50	67.83	60.22
Iris	100	98.10	95.56	94.29	95.56	94.29	94.29	93.33

6. Conclusions

We generate and study an algorithm for approximating the common fixed points of a countable family of G-nonexpansive mappings and prove the weak convergence of our algorithm. In addition, we give an application of our result for solving data classification and convex minimization problems. Finally, our numerical experiments assert that our proposed algorithm provides more accuracy than FISTA-S, FISTA, and nAGA.

In future work, we aim to find new models and methods for data prediction and classification of real datasets in medical science and create new innovations for health care service.